Uploaded image for project: 'Airavata'
  1. Airavata
  2. AIRAVATA-3326

Improved database migration script creation

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • None
    • Registry API
    • None

    Description

      Improve automatic generation of database migration scripts. Work on this was started in AIRAVATA-3126. In that issue the OpenJPA MappingTool was integrated to run and generate a migration script whenever the schema validation fails.

      Continuing on that work, I'd like to

      • create a migration script generation tool (MappingTool) that can be run directly from the command line
      • integrate starting a MariaDB, perhaps via docker-compose, and run MappingTool against that to generate the mysql scripts
      • finish the documentation on how to take the generated scripts and integrate them in the various places in the code base where they belong
      • change ide-migration docker-compose database server to mariadb. MariaDB has some good additions to mysql for creating migration scripts that can be re-runnable (e.g., IF NOT EXISTS).
      • clean up remove generate-sql-persistence.xml and maven goal that used it

      Questions

      • when there are new/changed tables in more than one schema, what migration script gets created? I'm guessing that they will clobber each other so the name of the file needs to be unique to the schema.
      TODO
      • [x] Run derby db and migration script creation in ./target directory
      • [x] derby db not shutting down cleanly?
      • [x] run MappingTool directly, don't bother too much with checking if it validates, just run it
        • maybe run it in a finally block?
      • [x] generate schema for all of the registry databases
      • [x] separate logging configuration from test logging and separate airavata-server.properties files (one for derby and one for MariaDB)
      • [ ] have a mode to generate the entire schema and another mode to generate a migration script
        • appears createDB works better than build since it generates FKs
      • [x] integrate running MariaDB via Docker and generating migration scripts for that
      • [x] update ide-integration to use MariaDB
        • also integrate automatically running migration scripts at start up, if possible (scripts in /docker-entrypoint-initdb.d are only executed the first time the container starts
      • [x] update documentation
      • [ ] refactor schema migration generation code to common so that it can be used in sharing catalog and credential store, etc.
      • [x] undo the changes to JPAUtils.java to automatically run schema migration generation when schema fails to validate
      Issues

      Attachments

        Issue Links

          Activity

            People

              marcuschristie Marcus Christie
              marcuschristie Marcus Christie
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: