Uploaded image for project: 'SystemDS'
  1. SystemDS
  2. SYSTEMDS-3545

Image pre-processing vectorized

    XMLWordPrintableJSON

Details

    • Task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None

    Description

      SystemDS supports image pre-processing on single images. If images are materialized as linear rows in matrices,
      the operations no longer work without materializing individual rows back into images.
      This hinder neural network implementations and could be improved by providing image processing techniques in linear algebra
      that allows tensor like access on matrices.
      The project is to implement as many of the image processing techniques on row linearized images as time allows.

      To make it concrete, the operations that work on images in SystemDS can be found in:

      https://github.com/apache/systemds/tree/main/scripts/builtin

      In specific look at the ones with pre-prended "img".

      Not all of these are currently supporting the linearized version containing multiple images.

      An example that support both:

      https://github.com/apache/systemds/blob/main/scripts/builtin/img_brightness.dml

      This method takes an input image and scale the brightness of an images, and caps it at a given specific value.

      An example that only works on single images:

      https://github.com/apache/systemds/blob/main/scripts/builtin/img_cutout.dml

      This method adds an rectangle of a specific value on the image.

      The task would be to add a new builtin here called:

      img_cutout_linearized.dml

      that instead take linearized images, and add the rectangle to all images at the same time.

      Note that currently the methods also only handle single chanel images, (aka gray scale)
      so you can also add RGB versions if you want to.

      img_cutout_rgb.dml and img_cutout_rgb_linearized.dml

      We can also debate what to call the functions instead of "linearized". Perhaps batched ?
      or the existing methods can be modified with a boolean argument for batched.

      Attachments

        Activity

          People

            Unassigned Unassigned
            mboehm7 Matthias Boehm
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: