Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-2856 TestAcidGuarantee broken on trunk
  3. HBASE-4345

Ensure that Scanners that read from the storefiles respect MVCC

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.89.20100924, 0.94.0
    • None
    • None

    Description

      Currently, the key-values written to the disk do not include the MVCC (RWCC) version information. Once we add that
      information, and make it persistent to disk; let us make the scanners respect the MVCC mechanism by ignoring
      "newer" writes.

      Attachments

        1. 4345-v2.txt
          2 kB
          Amitanand Aiyer
        2. patch-3
          8 kB
          Amitanand Aiyer

        Activity

          To enforce the read-write consistency mechanism, we need to ignore all key-values that have a memstoreTS greater than the read point for the get/scan operation.

          Filters provide an excellent mechanism to ignore values based on any desired condition. We implement the RWCC mechanism for the key-values read from the disk by including a filter for the scan object that ignores "newer" reads.

          amitanand Amitanand Aiyer added a comment - To enforce the read-write consistency mechanism, we need to ignore all key-values that have a memstoreTS greater than the read point for the get/scan operation. Filters provide an excellent mechanism to ignore values based on any desired condition. We implement the RWCC mechanism for the key-values read from the disk by including a filter for the scan object that ignores "newer" reads.
          stack Michael Stack added a comment -

          Interesting take, enforcing we only read at the read point or before by doing the functionality as a filter that is always used. Intellectually I'd say that this should be core and not be done via a filter but if filter gets the job done, lets go for it. Nice.

          stack Michael Stack added a comment - Interesting take, enforcing we only read at the read point or before by doing the functionality as a filter that is always used. Intellectually I'd say that this should be core and not be done via a filter but if filter gets the job done, lets go for it. Nice.
          yuzhihong@gmail.com Ted Yu added a comment -

          Minor comment:

          +    @Override
          +    public boolean hasFilterRow() {
          +      if (existingFilter == null)
          +        return false;
          +      else
          +        return existingFilter.hasFilterRow();
          +    }
          

          else is not needed above.

          What's the plan for all the sub-tasks of HBASE-2856 ?

          yuzhihong@gmail.com Ted Yu added a comment - Minor comment: + @Override + public boolean hasFilterRow() { + if (existingFilter == null ) + return false ; + else + return existingFilter.hasFilterRow(); + } else is not needed above. What's the plan for all the sub-tasks of HBASE-2856 ?
          stack Michael Stack added a comment -

          @Amit You have any revisions on this patch since you wrote it? Should we commit?

          stack Michael Stack added a comment - @Amit You have any revisions on this patch since you wrote it? Should we commit?

          @stack: I'm going to redo this patch. I gave walk through of this code last week, internally, and one of the
          comments I got was that doing the filter approach is not going to ignore delete-markers that occur after
          the read point. i.e. they will take effect, even though they should not.

          I've been in austin for a recruiting trip, so don't have the patch ready. Will try to get it and update asap.

          amitanand Amitanand Aiyer added a comment - @stack: I'm going to redo this patch. I gave walk through of this code last week, internally, and one of the comments I got was that doing the filter approach is not going to ignore delete-markers that occur after the read point. i.e. they will take effect, even though they should not. I've been in austin for a recruiting trip, so don't have the patch ready. Will try to get it and update asap.

          I have modified the ScanQueryMatcher to take care of ignoring "newer" KV's. If done by the
          Scan query matcher, it is also going to disregard the deletes so it will help us with other
          issues.

          4344-v8 includes this change.

          amitanand Amitanand Aiyer added a comment - I have modified the ScanQueryMatcher to take care of ignoring "newer" KV's. If done by the Scan query matcher, it is also going to disregard the deletes so it will help us with other issues. 4344-v8 includes this change.

          fix included in 2856-v6.txt

          amitanand Amitanand Aiyer added a comment - fix included in 2856-v6.txt

          People

            amitanand Amitanand Aiyer
            amitanand Amitanand Aiyer
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: