Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Incomplete
-
None
-
None
Description
We need to explore how to cache DataFrames during the execution of Pipelines. It's a hard problem in general to handle automatically or manually, so we should start with some design discussions about:
- How to control it manually
- Whether & how to handle it automatically
- API changes needed for each