[BEAM-12072] WriteToFiles with a concrete sink can shunt data to a single window - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Open
Priority: P3
Resolution: Unresolved
Affects Version/s: None
Fix Version/s: None
Component/s: sdk-py-core
Labels:
None

Description

When given a concrete `FileSink`, `WriteToFiles` will re-use the same sink across windows:

This can lead to data for one window being written to the sink for another window.

See discussion: https://github.com/apache/beam/pull/14374#discussion_r604320333

Attachments

Issue Links

causes

BEAM-12071 DataFrame IO sinks do not correctly partition by window

Triage Needed

Activity

People

Assignee:: Unassigned

Reporter:: Brian Hulette

Votes:: 0 Vote for this issue

Watchers:: 4 Start watching this issue

Dates

Created:: 30/Mar/21 18:11

Updated:: 04/Jun/22 19:54