Description
The SegmentMK now has two backends: one in-memory and the other based on MongoDB. Since MongoDB isn't readily available everywhere and the in-memory backend obviously isn't suited for production use, it would be useful to have an additional SegmentMK backend based on just the local file system.
Inspired by, though not directly based on, the proprietary TarPM in CRX, I propose to implement such a backend using the tar file format as the basis.
Unlike the MongoDB backend or the original TarPM, such a "TarMK" backend would be designed primarily for optimum performance on a single-node deployment, i.e. without support for clustering. A typical deployment could be replicated set of independent copies of a repository, designed for massively parallel read-only or mostly-read workloads.
Attachments
Issue Links
- is related to
-
OAK-1423 SegmentMK: Add index(es) to tar file backend
- Closed