Uploaded image for project: 'ManifoldCF'
  1. ManifoldCF
  2. CONNECTORS-993

Pipeline does not handle "no document" case properly

    XMLWordPrintableJSON

Details

    Description

      The incremental ingester and its pipeline code, and indeed transformation connectors themselves, do not handle the "no document" case properly.

      What should happen is that whenever a document is not retransmitted into a branch of the pipeline, the document version should nevertheless be recorded in the ingeststatus table for the corresponding output. But transformation connectors can interfere with that transmission if they decide simply to not index an incoming document. They may also receive a null RepositoryDocument into the transformation method, and will be unequipped to handle that.

      Instead, I propose the following:
      (1) An explicit IIncrementalIngester method needs to be invented for handling this case
      (2) The activity class used for transformation methods should have a "noDocument()" method
      (3) IncrementalIngester needs to propagate "noDocument" method calls down the pipeline without calling transformations on them

      Attachments

        Activity

          People

            kwright@metacarta.com Karl Wright
            kwright@metacarta.com Karl Wright
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: