Details
-
Bug
-
Status: Resolved
-
Normal
-
Resolution: Fixed
-
None
-
None
-
Red hat linux 5.4, Hadoop 1.0.3, pig 0.11.1
-
Normal
Description
We have got very strange beheviour of hadoop cluster after upgrading
Cassandra from 1.1.5 to Cassandra 1.2.1. We have 5 nodes cluster of Cassandra, where three of them are hodoop slaves. Now when we are submitting job through Pig script, only one map assigns in task running on one of the hadoop slaves regardless of
volume of data (already tried with more than million rows).
Configure of pig as follows:
export PIG_HOME=/oracle/pig-0.10.0
export PIG_CONF_DIR=${HADOOP_HOME}/conf
export PIG_INITIAL_ADDRESS=192.168.157.103
export PIG_RPC_PORT=9160
export PIG_PARTITIONER=org.apache.cassandra.dht.Murmur3Partitioner
Also we have these following properties in hadoop:
<property>
<name>mapred.tasktracker.map.tasks.maximum</name>
<value>10</value>
</property>
<property>
<name>mapred.map.tasks</name>
<value>4</value>
</property>
Attachments
Attachments
Issue Links
- relates to
-
CASSANDRA-5604 Vnodes decrease Hadoop performances cause it creates too many small splits
- Resolved