Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-1961 HBase EC2 scripts
  3. HBASE-2300

[EC2] On demand monitoring and auto scaling

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Minor
    • Resolution: Not A Problem
    • None
    • None
    • None
    • None

    Description

      Modify the scripts to install monitoring and auto-scaling rules to launch new slaves on demand according to CPU, disk, or I/O (over)use.

      http://docs.amazonwebservices.com/AmazonCloudWatch/latest/DeveloperGuide/
      http://docs.amazonwebservices.com/AutoScaling/latest/DeveloperGuide/

      CloudWatch metrics are provided as statistics ['Minimum', 'Maximum', 'Sum', 'Average', 'Samples'] in units of ['Seconds', 'Percent', 'Bytes', 'Bits', 'Count', 'Bytes/Second', 'Bits/Second', 'Count/Second']. Some useful metrics:

      • NetworkIn
      • NetworkOut
      • CPUUtilization
      • DiskWriteBytes
      • DiskWriteOps
      • DiskReadBytes
      • DiskReadOps

      We can be agnostic about statistics and scaling triggers and just pass them through to EC2.

      This would be especially easy if we port our scripts to Python like Hadoop has on trunk and use boto: http://code.google.com/p/boto/.

      Attachments

        Activity

          People

            Unassigned Unassigned
            apurtell Andrew Kyle Purtell
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: