[BEAM-1497] Refactor Hadoop/HDFS IO - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Resolved
Priority: P2
Resolution: Fixed
Affects Version/s: 0.5.0
Fix Version/s: 0.6.0
Component/s: io-java-hadoop-file-system
Labels:
- backward-incompatible

Description

Scio includes refactored version of Hadoop/HDFS IO, primarily:

single source/sink for all hadoop formats (no separate avro specific io)
no separate simple authentication - UGI in sink
use builder pattern instead of CTORs

This issue is to report push of this refactor to upstream/beam. Also tracked in scio#416

Attachments

Issue Links

links to

GitHub Pull Request #2031

Activity

People

Assignee:: Rafal Wojdyla

Reporter:: Rafal Wojdyla

Votes:: 0 Vote for this issue

Watchers:: 7 Start watching this issue

Dates

Created:: 16/Feb/17 06:10

Updated:: 24/Jul/20 20:14

Resolved:: 18/May/17 19:01