Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-5980

Add tests to guard against repeated dag trigger using spark event listeners

    XMLWordPrintableJSON

Details

    Description

      as of now, we don't have a good way to guard repeated dag trigger. all of our existing tests only checks for data. but w/ reconcile strategy, the extra files will be removed if some dag was repeated. So, we might need to add more tests to catch them if incase someone changes the dag in future. 

       

      Eg test: https://github.com/apache/hudi/pull/8275/files

       

       

      Attachments

        Issue Links

          Activity

            People

              ljain Lokesh Jain
              shivnarayan sivabalan narayanan
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: