Description
I have try to launch a cluster based on the environment of Openstack and found it will be failed when we didn't specify the value of whirr.blobstore-provider. Here's some of the failed informations:
2013-06-21 19:36:52,428 ERROR [org.apache.whirr.ClusterController] (main) Unable to start the cluster. Terminating all nodes. com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: id must be defined at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2258) at com.google.common.cache.LocalCache.get(LocalCache.java:3990) at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3994) at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4878) at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4884) at org.apache.whirr.service.BlobStoreContextBuilder$Cache.apply(BlobStoreContextBuilder.java:72) at org.apache.whirr.service.BlobStoreContextBuilder.build(BlobStoreContextBuilder.java:101) at org.apache.whirr.util.BlobCache.<init>(BlobCache.java:95) at org.apache.whirr.util.BlobCache.getInstance(BlobCache.java:69) at org.apache.whirr.service.ClusterActionHandlerSupport.prepareRemoteFileUrl(ClusterActionHandlerSupport.java:239) at org.apache.whirr.service.zookeeper.ZooKeeperClusterActionHandler.beforeBootstrap(ZooKeeperClusterActionHandler.java:71) at org.apache.whirr.service.ClusterActionHandlerSupport.beforeAction(ClusterActionHandlerSupport.java:55) at org.apache.whirr.actions.ScriptBasedClusterAction.execute(ScriptBasedClusterAction.java:123) at org.apache.whirr.ClusterController.bootstrapCluster(ClusterController.java:137) at org.apache.whirr.ClusterController.launchCluster(ClusterController.java:113) at org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:69) at org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:59) at org.apache.whirr.cli.Main.run(Main.java:69) at org.apache.whirr.cli.Main.main(Main.java:102) Caused by: java.lang.IllegalArgumentException: id must be defined at com.google.common.base.Preconditions.checkArgument(Preconditions.java:92) at org.jclouds.util.Preconditions2.checkNotEmpty(Preconditions2.java:50) at org.jclouds.providers.ProviderPredicates.id(ProviderPredicates.java:107) at org.jclouds.providers.Providers.withId(Providers.java:103) at org.jclouds.ContextBuilder.newBuilder(ContextBuilder.java:164) at org.apache.whirr.service.BlobStoreContextBuilder$Cache$1.load(BlobStoreContextBuilder.java:83) at org.apache.whirr.service.BlobStoreContextBuilder$Cache$1.load(BlobStoreContextBuilder.java:78) at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3589) at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2374) at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2337) at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2252) ... 18 more
Here's my property file:
whirr.cluster-name=myhadoopcluster #whirr.instance-templates=1 hadoop-jobtracker+hadoop-namenode,1 hadoop-datanode+hadoop-tasktracker whirr.instance-templates=1 zookeeper whirr.provider=openstack-nova whirr.location-id=RegionOne whirr.identity=admin:admin whirr.credential=hastexo whirr.image-id=RegionOne/7cf21d93-d04a-4fb4-a09b-f546a9461726 whirr.endpoint = http://10.167.157.86:5000/v2.0 #whirr.blobstore-provider=swift-keystone whirr.blobstore-endpoint=http://10.167.157.86:5000/v2.0/ whirr.blobstore-identity=service:swift whirr.blobstore-credential=hastexo #whirr.blobstore-cache-container=swift #whirr.blobstore-location-id=swift-keystone whirr.zookeeper.tarball.url=file:///home/Jeremy/test_sample1.war jclouds.openstack-nova.auto-generate-keypairs = true #jclouds.openstack-nova.auto-create-floating-ips = true whirr.private-key-file=${sys:user.home}/.ssh/id_rsa whirr.public-key-file=${sys:user.home}/.ssh/id_rsa.pub
After detailed investigation about the whirr project, I found it is becasue the whirr-core component didn't specify the default value which will Computed from whirr.provider as documented in the offical specification:
http://whirr.apache.org/docs/0.8.2/configuration-guide.html
Here's my modification to support the specification which is documented in the official website
ClusterSpec.java
Index: ClusterSpec.java =================================================================== --- ClusterSpec.java (revision 1488828) +++ ClusterSpec.java (working copy) @@ -576,6 +576,7 @@ mappings.put("cloudservers", "cloudfiles-us"); mappings.put("cloudservers-us", "cloudfiles-us"); mappings.put("cloudservers-uk", "cloudfiles-uk"); + mappings.put("openstack-nova", "swift-keystone"); if (!mappings.containsKey(provider)) { return null;
Thanks
-Jeremy Lv