Uploaded image for project: 'Atlas'
  1. Atlas
  2. ATLAS-492

Hive Hook Improvements

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Done
    • 0.7-incubating
    • None
    • None
    • None

    Description

      Requirements:

      • Support ALTER statements (add/replace/rename cols)
      • Support DROP statements ( support drop database, table, view)
      • Support LOAD statements
      • Remodelling changes

      Attachments

        Issue Links

          contains

          Sub-task - The sub-task of the issue ATLAS-653 insert into local dir, load from local dir does not show up in lineage

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. ATLAS-876 No lineage for a re-created ctas table

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. ATLAS-877 CreateTime of an entity(for example hive table) changes for every update

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. ATLAS-619 Queries should be normalized when tracking lineage at table

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. ATLAS-635 Process showing old entity name where as actual entity is renamed.

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. ATLAS-636 Tags/Traits associated with partitionkeys should be retained after rename table.

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. ATLAS-652 Delete on a non-existing table/view/db will result in 404

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. ATLAS-782 Hive Hook : Append to list of recentQueries instead of replacing

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. ATLAS-725 UI : Filter out or highlight deleted entities in search result outputs(dsl, text) , schema view, and lineage graph

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. ATLAS-596 UI : Bucket cols and sortCols within storage descriptor are not displayed correctly

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. ATLAS-685 UI : HDFS datasets are not displayed correctly

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. ATLAS-724 UI: Lineage view should display datasets as the source and target view ( more intuitive )

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. ATLAS-726 UI : dataset lineage process should be sorted according to process start time

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. ATLAS-752 Column renames should retain traits/tags

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. ATLAS-804 For hive_db, "locationUri" field is renamed to "location" but the hive data model json is still showing as "locationUri"

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. ATLAS-638 UI : Recognize and display HDFS Paths with a different notation

          • Major - Major loss of function.
          • Resolved
          relates to

          Sub-task - The sub-task of the issue ATLAS-702 "ALTER TABLE .. NOT STORED AS DIRECTORIES" metadata update is not sent to Atlas.

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. ATLAS-648 Alter view rename does not work across databases

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. ATLAS-758 hdfs location of hive table is pointing to old location even after rename.

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. ATLAS-760 Missing table parameters(last_modified_by and last_modified_time)

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. ATLAS-626 Hive temporary table metadata is captured in atlas.

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. ATLAS-627 Change column name should retain traits/tags assigned to columns

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. ATLAS-636 Tags/Traits associated with partitionkeys should be retained after rename table.

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. ATLAS-641 Lineage for a view created from a view seems to be confusing(from user's perspective).

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. ATLAS-703 No entity notification updates are sent for hive table protection feature. Seems like table protection changes are not yet modelled.

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. ATLAS-772 Ordering of columns is not maintained in schema query response, where as hive table entity response maintains the ordering

          • Major - Major loss of function.
          • Resolved
          1.
          		 Support Alter Table column commands Sub-task Resolved Suma Shivaprasad
          2.
          		 Support Alter table commands Sub-task Resolved Suma Shivaprasad
          3.
          		 Support alter view Sub-task Resolved Suma Shivaprasad
          4.
          		 Support alter database Sub-task Resolved Suma Shivaprasad
          5.
          		 Drop support for partitions, select query lineage, roles, principals, resource, hive_type... Sub-task Resolved Suma Shivaprasad
          6.
          		 Verify insert into select results in table lineage Sub-task Resolved Suma Shivaprasad
          7.
          		 Support lineage for load table, import, export Sub-task Resolved Suma Shivaprasad
          8.
          		 support drop table, view Sub-task Resolved Suma Shivaprasad
          9.
          		 support drop database Sub-task Resolved Suma Shivaprasad
          10.
          		 Add table information to column class Sub-task Resolved Suma Shivaprasad
          11.
          		 Remove isComposite on storageDescriptor.cols Sub-task Resolved Suma Shivaprasad
          12.
          		 Change Data types of all timestamps in Hive model(currently long) Sub-task Resolved Suma Shivaprasad
          13.
          		 Support drop database with cascade Sub-task Resolved Unassigned
          14.
          		 Verify free text search is able to surface search on table parameters Sub-task Resolved Suma Shivaprasad
          15.
          		 Support delete cascade efficiently Sub-task Resolved Unassigned
          16.
          		 Make qualifiedName and name consistent across all Datasets and Process Sub-task Resolved Suma Shivaprasad
          17.
          		 alter table rename should modify the list of columns and storage descriptor Sub-task Resolved Suma Shivaprasad
          18.
          		 Parallelize Hive hook operations Sub-task Open Unassigned
          19.
          		 Rename table should retain traits/tags assigned to columns/storage descriptors Sub-task Resolved Suma Shivaprasad
          20.
          		 alter table rename doesnt work across databases Sub-task Resolved Suma Shivaprasad
          21.
          		 'Description/Comment' attribute needs to be normalized hive_db, hive_table Sub-task Resolved Unassigned
          22.
          		 Handle Cluster renaming Sub-task Open Unassigned
          23.
          		 Change from Hive thrift interface to Hive MetastoreClient Sub-task Resolved Suma Shivaprasad
          24.
          		 import-hive should create the lineage for external tables Sub-task Resolved Vimal Sharma
          25.
          		 Truncate table, insert into local dir, load from local dir , update, delete commands should be easily trackable from a table Sub-task Open Unassigned
          26.
          		 ALTER TABLE TOUCH is not modelled Sub-task Open Unassigned
          27.
          		 "ALTER TABLE .. NOT STORED AS DIRECTORIES" metadata update is not sent to Atlas. Sub-task Open Unassigned
          28.
          		 insert into local dir, load from local dir does not show up in lineage Sub-task Resolved Unassigned

          Activity

            People

              Unassigned Unassigned
              bergenholtz Erik Bergenholtz
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: