Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-5544

Hadoop jobs assigns only one mapper in task

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Normal
    • Resolution: Fixed
    • 1.2.6
    • None
    • None
    • Red hat linux 5.4, Hadoop 1.0.3, pig 0.11.1

    • Normal

    Description

      We have got very strange beheviour of hadoop cluster after upgrading
      Cassandra from 1.1.5 to Cassandra 1.2.1. We have 5 nodes cluster of Cassandra, where three of them are hodoop slaves. Now when we are submitting job through Pig script, only one map assigns in task running on one of the hadoop slaves regardless of
      volume of data (already tried with more than million rows).
      Configure of pig as follows:
      export PIG_HOME=/oracle/pig-0.10.0
      export PIG_CONF_DIR=${HADOOP_HOME}/conf
      export PIG_INITIAL_ADDRESS=192.168.157.103
      export PIG_RPC_PORT=9160
      export PIG_PARTITIONER=org.apache.cassandra.dht.Murmur3Partitioner

      Also we have these following properties in hadoop:
      <property>
      <name>mapred.tasktracker.map.tasks.maximum</name>
      <value>10</value>
      </property>
      <property>
      <name>mapred.map.tasks</name>
      <value>4</value>
      </property>

      Attachments

        1. 5544.txt
          0.7 kB
          Alex Liu
        2. 5544-1.txt
          2 kB
          Alex Liu
        3. 5544-2.txt
          4 kB
          Alex Liu
        4. 5544-3.txt
          3 kB
          Alex Liu
        5. Screen Shot 2013-05-26 at 4.49.48 PM.png
          82 kB
          Shamim Ahmed

        Issue Links

          Activity

            People

              alexliu68 Alex Liu
              shamim_ru Shamim Ahmed
              Alex Liu
              Brandon Williams
              Votes:
              4 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: