Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
Description
Currently, hive hook executes in background thread pool and is an best effort approach to register entities. But this needs to be reliable for data governance to be effective
One way is - in hive hook, add the entities to some messaging framework and atlas server can read the entities from the message and register in atlas. Since, posting message is faster, we can do it synchronously and hence reliable entity registration.
We can start with kafka for messaging, but any other messaging framework should be pluggable
Attachments
Attachments
Issue Links
- links to