Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-12064

[Rust] [DataFusion] Make DataFrame extensible

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Invalid
    • None
    • None
    • Rust - DataFusion
    • None

    Description

      The DataFrame implementation currently has two types of logic:

      1. Logic for building a logical query plan
      2. Logic for executing a query using the DataFusion context

      We can make DataFrame more extensible by having it always delegate to the context for execution, allowing the same DataFrame logic to be used for local and distributed execution.

      We will likely need to introduce a new ExecutionContext trait with different implementations for DataFusion and Ballista.

       

       

      Attachments

        Activity

          People

            andygrove Andy Grove
            andygrove Andy Grove
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: