Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
Hide
HBASE-7171adds 2 new pages to the region server Web UI to ease debugging and provide greater insight into the physical data layout.
Region names in UI table listing all regions (on the RS status page) are now hyperlinks leading to region detail page which shows some aggregate memstore information (currently just memory used) along with the list of all Store Files (HFiles) in the region. Names of Store Files are also hyperlinks leading to Store File detail page, which currently runs 'hbase hfile' command behind the scene and displays statistics about store file.
ShowHBASE-7171 adds 2 new pages to the region server Web UI to ease debugging and provide greater insight into the physical data layout. Region names in UI table listing all regions (on the RS status page) are now hyperlinks leading to region detail page which shows some aggregate memstore information (currently just memory used) along with the list of all Store Files (HFiles) in the region. Names of Store Files are also hyperlinks leading to Store File detail page, which currently runs 'hbase hfile' command behind the scene and displays statistics about store file. -
beginner
Description
Click on a region in UI and get a listing of hfiles in HDFS and summary of memstore content; click on an HFile and see its content
Attachments
Attachments
- region_details.png
- 165 kB
- Mikhail Antonov
- region_list.png
- 140 kB
- Mikhail Antonov
- storefile_details.png
- 148 kB
- Mikhail Antonov
- HBASE-7171.patch
- 14 kB
- Mikhail Antonov
Issue Links
- is related to
-
HBASE-14534 Bump yammer/coda/dropwizard metrics dependency version
- Closed
Activity
This came up again recently. Will be especially useful when layout in fs changes in 2.0.
stack yeah, for debugging also. I assume the code should be about the same for both 1.2 and master though.
Yessir. Could you steal the hdfs navigation servlet code and just add the bit that shows hfile meta info....when you click on one (or a reference...)
Do we have a WAL equivalent for this? I needed that recently where going to a RS can show the WAL files (clickable, etc).
I don't think we do, at least I havent' seen one. That sounds like it's be quite useful too.
stack Looking at HBASE-14090 (which I assume is the way to go, or at least one of the favorable ways to go forward?), I'm recollecting we're going to have structure like /hbase/data/abc/def/ghi/abcdefghijklmnopqrstuvwxyz, which means it's easy to display as a column (list) of full paths (hyperlinked to pages displaying store file stats/content), but HDFS nav code in servlet running inside NN would be of little value, since it's showing one 'current' folder at a time?)
Also, thinking about hfile content I see 2 nuances here. One is that store file might be huge so it's not likely user would want to see all of that data.. show some stats, or show output similar to what 'hbase hfile' shows?
And secondly, security...our UI doesn't have it, so not sure we want to display actual user data from tables in it?
What do you think?
but HDFS nav code in servlet running inside NN would be of little value, since it's showing one 'current' folder at a time
You are right.
I think we need two 'views'. One view would be a table of:
/hbase/data/abc/def/ghi/abcdefghijklmnopqrstuvwxyz
Another view would be tables/familes/regions view (though what if the table has a million regions?).
For the latter view, currently you could do listing of HDFS content but when we moved over to the new fs layout, this view would be gotten by querying the master for what are the tables, families, etc., and ultimately what files are in a Store. It would become a pretty pivotal piece since you won't be able to get it anymore doing a dfs -lsr /hbase
show some stats, or show output similar to what 'hbase hfile' shows?...
For sure. For a start this would be a million miles ahead of what we have now.
...so not sure we want to display actual user data from tables in it?
There is that concern, yes. Would have to default not showing anything. Would be sweet though if could browse... but that could come later.
Here're some basic screenshots to take a look at, just to give general idea. Obviously formatting and exact additional data to be displayed are subject to change. Tested on local cluster after running 'hbase pe...'.
Here's draft patch. Added two new pages, region details and storefile details. Some notes (cc stack, enis):
- regarding using "logical" api to traverse store files (think region.getStores().get('cfname').getStoreFiles() ) vs "physical" (traversing fs assuming certain physical layout and linking to NN-style printout) I think first one would be easier to maintain and cleaner going forward? What do we need to display on region page. Memstore params (besides current size, what else? timestamps, snapshot info? File information - size, lmd, what else is of interest given the limited horizontal space.
- for now I just used quick and dirty approach when clicking at store file name link simply invokes another java process (via 'hbase hfile...') and redirects the output to the page. Is this information what we want to see at this store file details page (assuming that browsing individual cells is not something I'm doing now, that needs more thinking)? If this information is kind of what we want here, then to avoid having to call another process I can change HFilePrettyPrinter, pull KeyValueStatsCollector out of it, and a bit more refactoring would be needed there.
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12775907/HBASE-7171.patch
against master branch at commit 4f36632d7b7f987e91327f9ce1673a842f4eb864.
ATTACHMENT ID: 12775907
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 hadoop versions. The patch compiles with all supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0 2.7.1)
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 protoc. The applied patch does not increase the total number of protoc compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
-1 checkstyle. The applied patch generated new checkstyle errors. Check build console for list of new errors.
+1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 lineLengths. The patch introduces the following lines longer than 100:
+ <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-collapse">
+ <a class="navbar-brand" href="/rs-status"><img src="/static/hbase_logo_small.png" alt="HBase Logo"/></a>
+ <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-collapse">
+ <a class="navbar-brand" href="/rs-status"><img src="/static/hbase_logo_small.png" alt="HBase Logo"/></a>
+1 site. The mvn post-site goal succeeds with this patch.
+1 core tests. The patch passed unit tests in .
+1 zombies. No zombie tests found running at the end of the build.
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/16774//testReport/
Release Findbugs (version 2.0.3) warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/16774//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/16774//artifact/patchprocess/checkstyle-aggregate.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/16774//console
This message is automatically generated.
Getting info as json would be a crazy ask? We do that in a few places currently....No harm if a later.
stack not crazy, why.There's related handy jira for that - HBASE-4040. Let me take a look at it, should be similar to how it's done in WALPrettyPrinter (and then could do similar web UI for it). The refactoring needed is that HFilePrettyPrinter is a tool, invocable from shell, which outputs to hardcoded System.out, so need to make it invocable from regular code (need to make sure it's backward compatible).
Meanwhile any thoughts on the screenshots? Some more info to add in header/columns...?
Seems like yammer json metrics is not a thing, and json metrics (whatever they look like) exist in io.dropwizard. Once we bump the version may need to revise rendering here.
After looking into it I'm inclined to propose to make it as separate efforts.
HFilePrettyPrinter prints out metrics when certain cli options are used. Metrics in json seems to be not something supported well in the version of yammer we're using (see HBASE-14534). So I started migrating things to new yammer (now called dropwizard 3.1.2), and turned out it's not a matter of changing couple pom files, as in some pieces they changed API quite a bit.
So I thought to make this as following:
- get this one in with current 'console-style pretty-printing'
- complete migration to new yammer/dropwizard
- complete refactoring of HFilePrettyPrinter to be like WALPrettyPrinter (support both cli and json, as well as programmatic invocation)
Screenshots are great. My thought is getting them in earlier rather than later and we can refine as we go. Want me to commit what is here?
I can commit if I have your +1 Thought to wait might be one more day though in case someone has some feedback (like add column etc, though this could easily be added/removed in another jira)
Next ask: make the rendering extensible such that, ie, phoenix can plug in it's schema info and render typed values.
Looks good. Can we invoke HFilePrettyPrinter without building it through an OS process. We can add static methods if needed to HFPP.
+ ProcessBuilder pb=new ProcessBuilder("hbase", "hfile", "-s", "-f", storeFile); + pb.redirectErrorStream(true); + Process pr = pb.start(); + BufferedReader in = new BufferedReader(new InputStreamReader(pr.getInputStream()));
enis sure, that piece needs to change. I'm refactoring HFPP (see HBASE-4040) anyway to support json printout, so want to make this change there I think.
I think I'll commit what's here in next hour unless objections and move on to bigger refactoring of HFPP.
You mean - rendering of regions or store files? Or more generic rendering?
Committed to master - thanks for review and feedback! (should be very easy to port to any branch - busbey, ndimiduk, enis - do you want it?
Will move to refactoring HFPP to print json and to be invocable programmatically.
Thanks Mikhail. Doing the HFPP in HBASE-4040 is fine as long as we are fixing the servlet. It would be good to have this in branch-1 and if possible 1.2 as well. It helps with debugging and support.
Sure, I'll get it to other branches as well. I'll create another jira to change this web-ui once we figure out things with yammer metrics upgrade.
The ship has sailed on 1.0 and 1.1, but as enis said would be nice to slip in for 1.2.
SUCCESS: Integrated in HBase-1.3-IT #361 (See https://builds.apache.org/job/HBase-1.3-IT/361/)
HBASE-7171 Initial web UI for region/memstore/storefiles details (antonov: rev 9f39d356c404d07b5826a147d29c700da8b7489a)
- hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/RegionListTmpl.jamon
- hbase-server/src/main/resources/hbase-webapps/regionserver/storeFile.jsp
- hbase-server/src/main/resources/hbase-webapps/regionserver/region.jsp
FAILURE: Integrated in HBase-1.2-IT #330 (See https://builds.apache.org/job/HBase-1.2-IT/330/)
HBASE-7171 Initial web UI for region/memstore/storefiles details (antonov: rev ded97582063ca16ccc5d0ab4cd93ae7afa66bdad)
- hbase-server/src/main/resources/hbase-webapps/regionserver/storeFile.jsp
- hbase-server/src/main/resources/hbase-webapps/regionserver/region.jsp
- hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/RegionListTmpl.jamon
FAILURE: Integrated in HBase-Trunk_matrix #540 (See https://builds.apache.org/job/HBase-Trunk_matrix/540/)
HBASE-7171 Initial web UI for region/memstore/storefiles details (antonov: rev 2430efc2b2f2ec43e6ae1f88b6ebcb04f4e116c4)
- hbase-server/src/main/resources/hbase-webapps/regionserver/region.jsp
- hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/RegionListTmpl.jamon
- hbase-server/src/main/resources/hbase-webapps/regionserver/storeFile.jsp
SUCCESS: Integrated in HBase-1.3 #424 (See https://builds.apache.org/job/HBase-1.3/424/)
HBASE-7171 Initial web UI for region/memstore/storefiles details (antonov: rev 9f39d356c404d07b5826a147d29c700da8b7489a)
- hbase-server/src/main/resources/hbase-webapps/regionserver/storeFile.jsp
- hbase-server/src/main/resources/hbase-webapps/regionserver/region.jsp
- hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/RegionListTmpl.jamon
FAILURE: Integrated in HBase-1.2 #431 (See https://builds.apache.org/job/HBase-1.2/431/)
HBASE-7171 Initial web UI for region/memstore/storefiles details (antonov: rev ded97582063ca16ccc5d0ab4cd93ae7afa66bdad)
- hbase-server/src/main/resources/hbase-webapps/regionserver/region.jsp
- hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/RegionListTmpl.jamon
- hbase-server/src/main/resources/hbase-webapps/regionserver/storeFile.jsp
(off topic - just noticed branch-2 came in after fetch; what's current policy regarding cherry picks, should any commit going to master go there as well? mbertozzi?)
branch-2 isn't supposed to exist yet. I guess was a wrong push from someone
I've seen it here - https://github.com/apache/hbase/tree/branch-2
Yep, will add a release note.
StoreFile details page seems to be broken. Also it needs to be re-implemented anyway to not spawn external process. I updated HBASE-4040 to reflect that.
What happened to this? Spawning a process from servlet is just awful. mantonov do you need help in HBASE-4040? We don't need the JSON support for this, should we do just the changes needed to invoke HFPP statically?
without JSON support it's actually should be lot easier. Let me see if I could get it in this weekend. in this case could get it in 1.3 before the increment time stamp filter is fixed...
This idea came from a coworker Jesse Anderson. He thought that if a user could click through all the ways down to seeing the cells in hdfs that it might make hbase easier to understand. The 'region' page might show list of hfiles. It might then let you click say compact or flush and you'd see the memstore stats change or the list of files go from many to one, etc.