Details
-
New Feature
-
Status: Open
-
P3
-
Resolution: Unresolved
-
None
-
None
-
None
Description
Azure Blob Storage can be accessed by using the wasb:// and wasbs:// protocols. This should be quite similar to the hdfs:// implementations already there.
We should just be able to use it like this:
( p | ReadFromText("yourcontainer@youraccount.blob.core.windows.net/test/sample.txt") | WriteToText("output.txt", shard_name_template="", num_shards=0) )
See:
[1] https://docs.microsoft.com/en-us/azure/hdinsight/hdinsight-hadoop-use-blob-storage
[2] https://hadoop.apache.org/docs/current/hadoop-azure/index.html