Details
Major Description
I found it will be blocked when create a virtual machine based on the Openstack(It is because the whirr project can't remove the swift container after created the new instance)
Here's my reproduced steps:
First of all, you must configrate the Openstack environments including the openstack swift
1>. Configurate my properties file called zookeeper.properties(In my environment, I have put the file in the directory of "/home/Jeremy/zookeeper.properties")
whirr.cluster-name=myhadoopcluster #whirr.instance-templates=1 hadoop-jobtracker+hadoop-namenode,1 hadoop-datanode+hadoop-tasktracker whirr.instance-templates=1 zookeeper whirr.provider=openstack-nova whirr.location-id=RegionOne whirr.identity=admin:admin whirr.credential=hastexo whirr.image-id=RegionOne/7cf21d93-d04a-4fb4-a09b-f546a9461726 whirr.endpoint = http://10.167.157.86:5000/v2.0 whirr.blobstore-provider=swift-keystone whirr.blobstore-endpoint=http://10.167.157.86:5000/v2.0/ whirr.blobstore-identity=service:swift whirr.blobstore-credential=hastexo #whirr.blobstore-cache-container=swift #whirr.blobstore-location-id=sdb1 whirr.zookeeper.tarball.url=file:///home/Jeremy/test_sample1.war jclouds.openstack-nova.auto-generate-keypairs = true #jclouds.openstack-nova.auto-create-floating-ips = true whirr.private-key-file=${sys:user.home}/.ssh/id_rsa whirr.public-key-file=${sys:user.home}/.ssh/id_rsa.pub
2>. Download the whirr-0.8.2.zip to my local platform(/home/Jeremy/whirr-0.8.2) and unzip it
3>. cd /home/Jeremy/whirr-0.8.2 and execute the following command:
./bin/whirr launch-cluster --config ../zookeeper.properties
4>.You will found the swift blob container can be created successfully but can't be removed automatically after create the new instance.(It will be blocked during the process of launch-cluster and can't be canced).
After a further investigation about the whirr project, I found it is because of the following code:
static { /* Ensure that all created containers are removed when the JVM stops */ Runtime.getRuntime().addShutdownHook(new Thread() { @Override public void run() { BlobCache.dropAndCloseAll(); } }); }
It is because the default value of the user thread in jclouds side is "0", even we create a new thread in the whirr side, we can't delete the container here because of the following Exception
Exception in thread "Thread-0" java.util.concurrent.RejectedExecutionException: Task [delegate=[method=SwiftKeystoneAsyncClient.listContainers, request=GET http://10.167.157.86:8080/v1/AUTH_60d91f3ee474439e8edbbf9f15f424c1/?format=json HTTP/1.1], executionList=com.google.common.util.concurrent.ExecutionList@1b8633e] rejected from java.util.concurrent.ThreadPoolExecutor@7b7e61[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 8] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2013) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:816) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1337) at org.jclouds.concurrent.config.DescribingExecutorService.execute(DescribingExecutorService.java:105) at org.jclouds.concurrent.Futures$FutureListener.addListener(Futures.java:122) at org.jclouds.concurrent.Futures$ListenableFutureAdapter.addListener(Futures.java:153) at com.google.common.util.concurrent.Futures.transform(Futures.java:269) at com.google.common.util.concurrent.Futures.transform(Futures.java:380) at org.jclouds.concurrent.Futures.compose(Futures.java:270) at org.jclouds.http.TransformingHttpCommandExecutorServiceImpl.submit(TransformingHttpCommandExecutorServiceImpl.java:54) at org.jclouds.http.TransformingHttpCommandImpl.execute(TransformingHttpCommandImpl.java:73) at org.jclouds.rest.internal.AsyncRestClientProxy.createListenableFutureForHttpRequestMappedToMethodAndArgs(AsyncRestClientProxy.java:254) at org.jclouds.rest.internal.AsyncRestClientProxy.invoke(AsyncRestClientProxy.java:148) at $Proxy70.listContainers(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.jclouds.concurrent.internal.SyncProxy.invoke(SyncProxy.java:170) at $Proxy71.listContainers(Unknown Source) at com.jclouds.test.JCloudsSwift.deleteConatiner(JCloudsSwift.java:63) at com.jclouds.test.JCloudsSwift.access$0(JCloudsSwift.java:60) at com.jclouds.test.JCloudsSwift$1.run(JCloudsSwift.java:34)
All in all, I have also write a sample related to the jcloud and found if we define the PROPERTY_USER_THREADS in the jclouds side, it will be successful to remove the swift container though the Runtime.getRuntime().addShutdownHook(new Thread(). If we didn't define the PROPERTY_USER_THREADS in the jclouds side and it will be thrown the RejectedExecutionException in calling the Blobstore.deleteContainer method because of the default user thread is "0"
So I think we need to define a new option to support this situation.