Uploaded image for project: 'Apache Whirr (retired)'
  1. Apache Whirr (retired)
  2. WHIRR-729

Support default value of whirr.blobstore-provider when it comes to the Openstack swift

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Critical
    • Resolution: Unresolved
    • 0.8.2
    • 0.9.0
    • core
    • None
    • Ubuntu,Openstack

    Description

      I have try to launch a cluster based on the environment of Openstack and found it will be failed when we didn't specify the value of whirr.blobstore-provider. Here's some of the failed informations:

      2013-06-21 19:36:52,428 ERROR [org.apache.whirr.ClusterController] (main) Unable to start the cluster. Terminating all nodes.
      com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: id must be defined
      	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2258)
      	at com.google.common.cache.LocalCache.get(LocalCache.java:3990)
      	at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3994)
      	at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4878)
      	at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4884)
      	at org.apache.whirr.service.BlobStoreContextBuilder$Cache.apply(BlobStoreContextBuilder.java:72)
      	at org.apache.whirr.service.BlobStoreContextBuilder.build(BlobStoreContextBuilder.java:101)
      	at org.apache.whirr.util.BlobCache.<init>(BlobCache.java:95)
      	at org.apache.whirr.util.BlobCache.getInstance(BlobCache.java:69)
      	at org.apache.whirr.service.ClusterActionHandlerSupport.prepareRemoteFileUrl(ClusterActionHandlerSupport.java:239)
      	at org.apache.whirr.service.zookeeper.ZooKeeperClusterActionHandler.beforeBootstrap(ZooKeeperClusterActionHandler.java:71)
      	at org.apache.whirr.service.ClusterActionHandlerSupport.beforeAction(ClusterActionHandlerSupport.java:55)
      	at org.apache.whirr.actions.ScriptBasedClusterAction.execute(ScriptBasedClusterAction.java:123)
      	at org.apache.whirr.ClusterController.bootstrapCluster(ClusterController.java:137)
      	at org.apache.whirr.ClusterController.launchCluster(ClusterController.java:113)
      	at org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:69)
      	at org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:59)
      	at org.apache.whirr.cli.Main.run(Main.java:69)
      	at org.apache.whirr.cli.Main.main(Main.java:102)
      Caused by: java.lang.IllegalArgumentException: id must be defined
      	at com.google.common.base.Preconditions.checkArgument(Preconditions.java:92)
      	at org.jclouds.util.Preconditions2.checkNotEmpty(Preconditions2.java:50)
      	at org.jclouds.providers.ProviderPredicates.id(ProviderPredicates.java:107)
      	at org.jclouds.providers.Providers.withId(Providers.java:103)
      	at org.jclouds.ContextBuilder.newBuilder(ContextBuilder.java:164)
      	at org.apache.whirr.service.BlobStoreContextBuilder$Cache$1.load(BlobStoreContextBuilder.java:83)
      	at org.apache.whirr.service.BlobStoreContextBuilder$Cache$1.load(BlobStoreContextBuilder.java:78)
      	at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3589)
      	at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2374)
      	at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2337)
      	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2252)
      	... 18 more
      

      Here's my property file:

      whirr.cluster-name=myhadoopcluster
      #whirr.instance-templates=1 hadoop-jobtracker+hadoop-namenode,1 hadoop-datanode+hadoop-tasktracker
      whirr.instance-templates=1 zookeeper
      
      whirr.provider=openstack-nova
      whirr.location-id=RegionOne
      whirr.identity=admin:admin
      whirr.credential=hastexo
      whirr.image-id=RegionOne/7cf21d93-d04a-4fb4-a09b-f546a9461726
      whirr.endpoint = http://10.167.157.86:5000/v2.0
      #whirr.blobstore-provider=swift-keystone
      whirr.blobstore-endpoint=http://10.167.157.86:5000/v2.0/
      whirr.blobstore-identity=service:swift
      whirr.blobstore-credential=hastexo
      #whirr.blobstore-cache-container=swift
      #whirr.blobstore-location-id=swift-keystone
      whirr.zookeeper.tarball.url=file:///home/Jeremy/test_sample1.war
      jclouds.openstack-nova.auto-generate-keypairs = true
      #jclouds.openstack-nova.auto-create-floating-ips = true
      
      whirr.private-key-file=${sys:user.home}/.ssh/id_rsa
      whirr.public-key-file=${sys:user.home}/.ssh/id_rsa.pub
      

      After detailed investigation about the whirr project, I found it is becasue the whirr-core component didn't specify the default value which will Computed from whirr.provider as documented in the offical specification:
      http://whirr.apache.org/docs/0.8.2/configuration-guide.html

      Here's my modification to support the specification which is documented in the official website

      ClusterSpec.java
      Index: ClusterSpec.java
      ===================================================================
      --- ClusterSpec.java	(revision 1488828)
      +++ ClusterSpec.java	(working copy)
      @@ -576,6 +576,7 @@
           mappings.put("cloudservers", "cloudfiles-us");
           mappings.put("cloudservers-us", "cloudfiles-us");
           mappings.put("cloudservers-uk", "cloudfiles-uk");
      +    mappings.put("openstack-nova", "swift-keystone");
       
           if (!mappings.containsKey(provider)) {
             return null;
      

      Thanks

      -Jeremy Lv

      Attachments

        1. ClusterSpec.java.patch
          0.5 kB
          Jeremy Lv

        Activity

          People

            Unassigned Unassigned
            jeremy_lv Jeremy Lv
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: