Details
-
Sub-task
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
-
HBaseTestingUtility now assumes that all clusters will use local storage until a MiniDFSCluster is started or assigned.
Description
The following can be observed running against hadoop3:
java.io.IOException: cannot get log writer at org.apache.hadoop.hbase.regionserver.TestCompactingMemStore.compactingSetUp(TestCompactingMemStore.java:107) at org.apache.hadoop.hbase.regionserver.TestCompactingMemStore.setUp(TestCompactingMemStore.java:89) Caused by: org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException: hflush and hsync at org.apache.hadoop.hbase.regionserver.TestCompactingMemStore.compactingSetUp(TestCompactingMemStore.java:107) at org.apache.hadoop.hbase.regionserver.TestCompactingMemStore.setUp(TestCompactingMemStore.java:89)
This was due to hbase-server/src/test/resources/hbase-site.xml not being picked up by Configuration object. Among the configs from this file, the value for "hbase.unsafe.stream.capability.enforce" relaxes check for presence of hflush and hsync. Without this config entry, StreamLacksCapabilityException is thrown.
Attachments
Attachments
- 19841.007.patch
- 10 kB
- Mike Drob
- 19841.06.patch
- 29 kB
- Mike Drob
- 19841.v0.txt
- 0.8 kB
- Ted Yu
- 19841.v1.txt
- 4 kB
- Ted Yu
- HBASE-19841.v10.patch
- 19 kB
- Mike Drob
- HBASE-19841.v11.patch
- 20 kB
- Michael Stack
- HBASE-19841.v11.patch
- 20 kB
- Mike Drob
- HBASE-19841.v2.patch
- 1 kB
- Mike Drob
- HBASE-19841.v3.patch
- 9 kB
- Mike Drob
- HBASE-19841.v4.patch
- 28 kB
- Mike Drob
- HBASE-19841.v5.patch
- 29 kB
- Mike Drob
- HBASE-19841.v7.patch
- 30 kB
- Mike Drob
- HBASE-19841.v8.patch
- 14 kB
- Michael Stack
- HBASE-19841.v8.patch
- 14 kB
- Michael Stack
- HBASE-19841.v8.patch
- 14 kB
- Mike Drob
- HBASE-19841.v9.patch
- 21 kB
- Mike Drob
Issue Links
- relates to
-
HBASE-21735 Port HBASE-18784 (Use of filesystem that requires hflush / hsync / append / etc should query outputstream capabilities) to branch-1
- Closed
Activity
The test does this in TestCompactingMemStore.compactingSetUp():
Configuration conf = new Configuration();
HBaseConfiguration is not involved.
You are inside in HBaseTestingUtility – THE HBase Testing tool... Under what circumstance is the passed in Configuation NOT an HBaseConfiguration (and it does not have hbase-site.xml loaded from test resources?)?
+1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 8s | Docker mode activated. |
0 | patch | 0m 2s | The patch file was not named according to hbase's naming conventions. Please see https://yetus.apache.org/documentation/0.6.0/precommit-patchnames for instructions. |
Prechecks | |||
0 | findbugs | 0m 0s | Findbugs executables are not available. |
+1 | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
master Compile Tests | |||
+1 | mvninstall | 4m 37s | master passed |
+1 | compile | 0m 44s | master passed |
+1 | checkstyle | 1m 12s | master passed |
+1 | shadedjars | 6m 2s | branch has no errors when building our shaded downstream artifacts. |
+1 | javadoc | 0m 28s | master passed |
Patch Compile Tests | |||
+1 | mvninstall | 4m 34s | the patch passed |
+1 | compile | 0m 42s | the patch passed |
+1 | javac | 0m 42s | the patch passed |
+1 | checkstyle | 1m 10s | the patch passed |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | shadedjars | 4m 36s | patch has no errors when building our shaded downstream artifacts. |
+1 | hadoopcheck | 21m 22s | Patch does not cause any errors with Hadoop 2.6.5 2.7.4 or 3.0.0. |
+1 | javadoc | 0m 29s | the patch passed |
Other Tests | |||
+1 | unit | 110m 35s | hbase-server in the patch passed. |
+1 | asflicense | 0m 19s | The patch does not generate ASF License warnings. |
151m 23s |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12907195/19841.v1.txt |
Optional Tests | asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
uname | Linux 8f795239c2c5 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9 14:43:09 UTC 2018 x86_64 GNU/Linux |
Build tool | maven |
Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh |
git revision | master / d49357f265 |
maven | version: Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
Default Java | 1.8.0_151 |
Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/11159/testReport/ |
modules | C: hbase-server U: hbase-server |
Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/11159/console |
Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
TestCompactingMemStore#compactingSetUp calls HBaseTestingUtility.createWal(Configuration, Path, RegionInfo) which is passed the conf (not instance of HBaseConfiguration) :
HBaseTestingUtility hbaseUtility = HBaseTestingUtility.createLocalHTU(conf);
createLocalHTU() doesn't currently create instance of HBaseConfiguration.
Hence the hbase-site.xml under src/test isn't picked up.
Looking at similar test failures, e.g. https://builds.apache.org/job/HBase-2.0-hadoop3-tests/org.apache.hbase$hbase-server/19/testReport/junit/org.apache.hadoop.hbase.regionserver/TestCompactingToCellFlatMapMemStore/testFlatteningToBigCellChunkMap_0_/ , it is not just this one test which uses Configuration.
Since every test in hbase-server module is supposed to pick up ./hbase-server/src/test/resources/hbase-site.xml , I think the attached patch is proper fix.
HTU is the HBase testing utility. If it requires hbase-site.xml test resource AND HBaseConfiguration does this anyways, why dupe the HBC op in HTU?
Why is hbase.unsafe.stream.capability.enforce in hbase-site test only anyways?
Refreshed patch which creates HBaseConfiguration in the setup method.
See HBASE-19289 for the introduction of hbase.unsafe.stream.capability.enforce
See
HBASE-19289for the introduction of hbase.unsafe.stream.capability.enforce
That was not my question. Why is it on in test hbase-site only?
That does not answer my question. My question is why is this config not set in hbase-default? Why only in hbase-site in tests?
Mike was the author of the final patch. Josh gave +1 to it.
They would be in better position to answer the above.
If you're suggesting adding the config to hbase-default (with value of true), that can be an addendum to HBASE-19289
Sounds like you do not understand the full implications of your patch then. You punt to others. To me it looks like you are just trying to hack in a change to suppress a particular complaint. No effort at trying to understand the broader problem.
This was due to "hbase.unsafe.stream.capability.enforce" config from hbase-server/src/test/resources/hbase-site.xml not being picked up.
hbase-site has a bunch of config in it beyond the one you want. Why weren't these being set when we go this code path? Was there a reason?
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 10s | Docker mode activated. |
0 | patch | 0m 2s | The patch file was not named according to hbase's naming conventions. Please see https://yetus.apache.org/documentation/0.6.0/precommit-patchnames for instructions. |
Prechecks | |||
0 | findbugs | 0m 0s | Findbugs executables are not available. |
+1 | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. |
master Compile Tests | |||
+1 | mvninstall | 5m 11s | master passed |
+1 | compile | 0m 51s | master passed |
+1 | checkstyle | 1m 11s | master passed |
+1 | shadedjars | 6m 8s | branch has no errors when building our shaded downstream artifacts. |
+1 | javadoc | 0m 35s | master passed |
Patch Compile Tests | |||
+1 | mvninstall | 5m 27s | the patch passed |
+1 | compile | 1m 5s | the patch passed |
+1 | javac | 1m 5s | the patch passed |
-1 | checkstyle | 1m 14s | hbase-server: The patch generated 1 new + 35 unchanged - 0 fixed = 36 total (was 35) |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | shadedjars | 5m 8s | patch has no errors when building our shaded downstream artifacts. |
+1 | hadoopcheck | 22m 50s | Patch does not cause any errors with Hadoop 2.6.5 2.7.4 or 3.0.0. |
+1 | javadoc | 0m 31s | the patch passed |
Other Tests | |||
+1 | unit | 106m 26s | hbase-server in the patch passed. |
+1 | asflicense | 0m 18s | The patch does not generate ASF License warnings. |
150m 59s |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12907322/19841.v1.txt |
Optional Tests | asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
uname | Linux bbdcc50b621c 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9 14:43:09 UTC 2018 x86_64 GNU/Linux |
Build tool | maven |
Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh |
git revision | master / d589b72382 |
maven | version: Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
Default Java | 1.8.0_151 |
checkstyle | https://builds.apache.org/job/PreCommit-HBASE-Build/11169/artifact/patchprocess/diff-checkstyle-hbase-server.txt |
Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/11169/testReport/ |
modules | C: hbase-server U: hbase-server |
Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/11169/console |
Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
tedyu stack - The first patch was removed so I don't know what the fix was, but this v2 seems reasonable to me. WDYT?
Hopefully, it will fix other failures related to WALs as well, instead of having to chase them down individually.
Why only create HBaseConfiguration for WAL, instead of the HBaseTestingUtility ?
Why only create HBaseConfiguration for WAL, instead of the HBaseTestingUtility ?
Because the WAL is what needs that config, and it was already making a copy of the configuration.
If we change createLocalHTU to create a new configuration then we have to propagate it back everywhere, and that gets much messier.
For patch v0, we can also create HBaseConfiguration at the beginning of createLocalHTU() .
we have to propagate it back everywhere
Then it seems the current v1 patch is better - no need to propagate back anywhere
+1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 1m 59s | Docker mode activated. |
0 | patch | 0m 1s | The patch file was not named according to hbase's naming conventions. Please see https://yetus.apache.org/documentation/0.6.0/precommit-patchnames for instructions. |
Prechecks | |||
0 | findbugs | 0m 0s | Findbugs executables are not available. |
+1 | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
master Compile Tests | |||
+1 | mvninstall | 4m 39s | master passed |
+1 | compile | 0m 42s | master passed |
+1 | checkstyle | 1m 9s | master passed |
+1 | shadedjars | 6m 1s | branch has no errors when building our shaded downstream artifacts. |
+1 | javadoc | 0m 32s | master passed |
Patch Compile Tests | |||
+1 | mvninstall | 5m 6s | the patch passed |
+1 | compile | 0m 48s | the patch passed |
+1 | javac | 0m 48s | the patch passed |
+1 | checkstyle | 1m 15s | the patch passed |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | shadedjars | 4m 57s | patch has no errors when building our shaded downstream artifacts. |
+1 | hadoopcheck | 20m 11s | Patch does not cause any errors with Hadoop 2.6.5 2.7.4 or 3.0.0. |
+1 | javadoc | 0m 29s | the patch passed |
Other Tests | |||
+1 | unit | 99m 33s | hbase-server in the patch passed. |
+1 | asflicense | 0m 17s | The patch does not generate ASF License warnings. |
141m 44s |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12907390/19841.v0.txt |
Optional Tests | asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
uname | Linux 028b5487334a 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19 15:49:21 UTC 2017 x86_64 GNU/Linux |
Build tool | maven |
Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh |
git revision | master / 0b6422658c |
maven | version: Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
Default Java | 1.8.0_151 |
Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/11174/testReport/ |
modules | C: hbase-server U: hbase-server |
Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/11174/console |
Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 10s | Docker mode activated. |
0 | patch | 0m 2s | The patch file was not named according to hbase's naming conventions. Please see https://yetus.apache.org/documentation/0.6.0/precommit-patchnames for instructions. |
Prechecks | |||
0 | findbugs | 0m 0s | Findbugs executables are not available. |
+1 | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
master Compile Tests | |||
+1 | mvninstall | 5m 5s | master passed |
+1 | compile | 0m 43s | master passed |
+1 | checkstyle | 1m 9s | master passed |
+1 | shadedjars | 5m 52s | branch has no errors when building our shaded downstream artifacts. |
+1 | javadoc | 0m 35s | master passed |
Patch Compile Tests | |||
+1 | mvninstall | 4m 36s | the patch passed |
+1 | compile | 0m 42s | the patch passed |
+1 | javac | 0m 42s | the patch passed |
+1 | checkstyle | 1m 9s | the patch passed |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | shadedjars | 4m 38s | patch has no errors when building our shaded downstream artifacts. |
+1 | hadoopcheck | 19m 58s | Patch does not cause any errors with Hadoop 2.6.5 2.7.4 or 3.0.0. |
+1 | javadoc | 0m 28s | the patch passed |
Other Tests | |||
-1 | unit | 112m 44s | hbase-server in the patch failed. |
+1 | asflicense | 0m 19s | The patch does not generate ASF License warnings. |
152m 34s |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12907390/19841.v0.txt |
Optional Tests | asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
uname | Linux 7ed14a21fc0e 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9 14:43:09 UTC 2018 x86_64 GNU/Linux |
Build tool | maven |
Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh |
git revision | master / 0b6422658c |
maven | version: Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
Default Java | 1.8.0_151 |
unit | https://builds.apache.org/job/PreCommit-HBASE-Build/11175/artifact/patchprocess/patch-unit-hbase-server.txt |
Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/11175/testReport/ |
modules | C: hbase-server U: hbase-server |
Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/11175/console |
Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
Then it seems the current v1 patch is better - no need to propagate back anywhere
How do you figure this?
v1 patch has to change return types and also doesn't fix other tests that would fail the same way.
Thanks for the reminder about other tests.
For patch v2, the Configuration instance used by the WAL and the instance used by testing util have different views of config entries.
This inconsistency may lead to issue down the road.
When I ran TestDefaultMemStore with patch v2 against hadoop3:
[ERROR] testShouldFlushMeta(org.apache.hadoop.hbase.regionserver.TestDefaultMemStore) Time elapsed: 1.184 s <<< ERROR! java.io.IOException: cannot get log writer at org.apache.hadoop.hbase.regionserver.TestDefaultMemStore.testShouldFlushMeta(TestDefaultMemStore.java:950) Caused by: org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException: hflush and hsync at org.apache.hadoop.hbase.regionserver.TestDefaultMemStore.testShouldFlushMeta(TestDefaultMemStore.java:950) [ERROR] testShouldFlush(org.apache.hadoop.hbase.regionserver.TestDefaultMemStore) Time elapsed: 0.117 s <<< ERROR! java.io.IOException: cannot get log writer at org.apache.hadoop.hbase.regionserver.TestDefaultMemStore.checkShouldFlush(TestDefaultMemStore.java:919) at org.apache.hadoop.hbase.regionserver.TestDefaultMemStore.testShouldFlush(TestDefaultMemStore.java:906) Caused by: org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException: hflush and hsync at org.apache.hadoop.hbase.regionserver.TestDefaultMemStore.checkShouldFlush(TestDefaultMemStore.java:919) at org.apache.hadoop.hbase.regionserver.TestDefaultMemStore.testShouldFlush(TestDefaultMemStore.java:906)
With patch v0, TestDefaultMemStore and TestCompactingMemStore pass against hadoop3.
What is interesting about TestDefaultMemStore is that some methods use HBaseConfiguration.create() and some use new Configuration()
Probably best to just rip off the band-aid quickly and properly set up the tests instead of continuing down this path of hacks and rabbit holes that I inadvertently started.
Actually, just had an idea... Maybe it's better to go with something similar to v0 of the patch, except instead of adding the resource we explicitly set the property there. Then we still have to go through and update usage from new Configuration to HTU.createLocalHTU.getConfiguration, but it will be consistent then. I'm not sure if that will actually work or not, but will give it a try.
Attached a new v3 that demonstrates the direction that I think we should be going in. It removes the hbase-site.xml change from earlier in favor of only having it as part of the local HTU. Then we make sure that we're using the correct conf in places. This is a WIP, since I haven't run a full test suite yet against H3 with the patch.
This is a good intermediate step because it will need to identify all of the places where we rely on WALs with a localFS and then will be able to update them later when we have a real fix.
cc: elserj, since you were in the original discussion.
+1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 10s | Docker mode activated. |
0 | patch | 0m 1s | The patch file was not named according to hbase's naming conventions. Please see https://yetus.apache.org/documentation/0.6.0/precommit-patchnames for instructions. |
Prechecks | |||
0 | findbugs | 0m 0s | Findbugs executables are not available. |
+1 | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
master Compile Tests | |||
+1 | mvninstall | 4m 47s | master passed |
+1 | compile | 0m 41s | master passed |
+1 | checkstyle | 1m 7s | master passed |
+1 | shadedjars | 5m 42s | branch has no errors when building our shaded downstream artifacts. |
+1 | javadoc | 0m 26s | master passed |
Patch Compile Tests | |||
+1 | mvninstall | 4m 34s | the patch passed |
+1 | compile | 0m 40s | the patch passed |
+1 | javac | 0m 40s | the patch passed |
+1 | checkstyle | 1m 7s | the patch passed |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | shadedjars | 4m 35s | patch has no errors when building our shaded downstream artifacts. |
+1 | hadoopcheck | 18m 49s | Patch does not cause any errors with Hadoop 2.6.5 2.7.4 or 3.0.0. |
+1 | javadoc | 0m 27s | the patch passed |
Other Tests | |||
+1 | unit | 102m 26s | hbase-server in the patch passed. |
+1 | asflicense | 0m 18s | The patch does not generate ASF License warnings. |
140m 21s |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12907390/19841.v0.txt |
Optional Tests | asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
uname | Linux 6b177a271e2c 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19 15:49:21 UTC 2017 x86_64 GNU/Linux |
Build tool | maven |
Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh |
git revision | master / 0b6422658c |
maven | version: Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
Default Java | 1.8.0_151 |
Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/11177/testReport/ |
modules | C: hbase-server U: hbase-server |
Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/11177/console |
Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
+1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 14s | Docker mode activated. |
0 | patch | 0m 2s | The patch file was not named according to hbase's naming conventions. Please see https://yetus.apache.org/documentation/0.6.0/precommit-patchnames for instructions. |
Prechecks | |||
0 | findbugs | 0m 0s | Findbugs executables are not available. |
+1 | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
master Compile Tests | |||
+1 | mvninstall | 5m 46s | master passed |
+1 | compile | 0m 56s | master passed |
+1 | checkstyle | 1m 21s | master passed |
+1 | shadedjars | 6m 40s | branch has no errors when building our shaded downstream artifacts. |
+1 | javadoc | 0m 38s | master passed |
Patch Compile Tests | |||
+1 | mvninstall | 5m 34s | the patch passed |
+1 | compile | 0m 52s | the patch passed |
+1 | javac | 0m 52s | the patch passed |
+1 | checkstyle | 1m 18s | the patch passed |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | shadedjars | 5m 11s | patch has no errors when building our shaded downstream artifacts. |
+1 | hadoopcheck | 22m 31s | Patch does not cause any errors with Hadoop 2.6.5 2.7.4 or 3.0.0. |
+1 | javadoc | 0m 34s | the patch passed |
Other Tests | |||
+1 | unit | 103m 29s | hbase-server in the patch passed. |
+1 | asflicense | 0m 16s | The patch does not generate ASF License warnings. |
149m 1s |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12907390/19841.v0.txt |
Optional Tests | asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
uname | Linux 18426aaa9c89 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19 15:49:21 UTC 2017 x86_64 GNU/Linux |
Build tool | maven |
Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh |
git revision | master / 0b6422658c |
maven | version: Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
Default Java | 1.8.0_151 |
Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/11178/testReport/ |
modules | C: hbase-server U: hbase-server |
Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/11178/console |
Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
v4 passed all tests against an older master, I'm rebased and running tests again locally now.
It removes the property from hbase-site.xml and moves it to HTU.createLocalHTU. This is good because it actually exposes it for downstream consumers who may be using our test code to do their own thing and will just do the right thing for them most of the time. A further improvement would be to remove the distinction between a local and cluster based HTU, make it local until somebody starts a DFS cluster then reconfigure everything. shouldn't be much of a performance impact and will be much easier for folks to use.
+1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 10s | Docker mode activated. |
0 | patch | 0m 2s | The patch file was not named according to hbase's naming conventions. Please see https://yetus.apache.org/documentation/0.6.0/precommit-patchnames for instructions. |
Prechecks | |||
0 | findbugs | 0m 0s | Findbugs executables are not available. |
+1 | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
master Compile Tests | |||
+1 | mvninstall | 4m 47s | master passed |
+1 | compile | 0m 45s | master passed |
+1 | checkstyle | 1m 8s | master passed |
+1 | shadedjars | 6m 49s | branch has no errors when building our shaded downstream artifacts. |
+1 | javadoc | 0m 43s | master passed |
Patch Compile Tests | |||
+1 | mvninstall | 6m 36s | the patch passed |
+1 | compile | 0m 58s | the patch passed |
+1 | javac | 0m 58s | the patch passed |
+1 | checkstyle | 1m 33s | the patch passed |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | shadedjars | 5m 44s | patch has no errors when building our shaded downstream artifacts. |
+1 | hadoopcheck | 20m 24s | Patch does not cause any errors with Hadoop 2.6.5 2.7.4 or 3.0.0. |
+1 | javadoc | 0m 28s | the patch passed |
Other Tests | |||
+1 | unit | 109m 8s | hbase-server in the patch passed. |
+1 | asflicense | 0m 29s | The patch does not generate ASF License warnings. |
153m 9s |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12907390/19841.v0.txt |
Optional Tests | asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
uname | Linux 5a279e00f621 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19 15:49:21 UTC 2017 x86_64 GNU/Linux |
Build tool | maven |
Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh |
git revision | master / 548426edc0 |
maven | version: Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
Default Java | 1.8.0_151 |
Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/11185/testReport/ |
modules | C: hbase-server U: hbase-server |
Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/11185/console |
Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
Also, taking out of patch available until we get a fixed Yetus so that we don't keep rerunning the same patch...
tedyu - my proposed approach is very different from your approach, does what I am trying to do make sense to you? I don't want us to be talking past each other here and would like to make sure that you are on board with this plan.
diff --git a/hbase-common/src/main/java/org/apache/hadoop/hbase/util/CommonFSUtils.java b/hbase-common/src/main/java/org/apache/hadoop/hbase/util/CommonFSUtils.java index 9efec07915..bb98c407b4 100644 --- a/hbase-common/src/main/java/org/apache/hadoop/hbase/util/CommonFSUtils.java +++ b/hbase-common/src/main/java/org/apache/hadoop/hbase/util/CommonFSUtils.java @@ -394,7 +394,13 @@ public abstract class CommonFSUtils { public static FileSystem getWALFileSystem(final Configuration c) throws IOException { Path p = getWALRootDir(c); - return p.getFileSystem(c); + FileSystem fs = p.getFileSystem(c); + // Need to copy this to the new filesystem we are returning in case it is localFS + String enforceStreamCapabilities = c.get(CommonFSUtils.UNSAFE_STREAM_CAPABILITY_ENFORCE); + if (enforceStreamCapabilities != null) { + fs.getConf().set(CommonFSUtils.UNSAFE_STREAM_CAPABILITY_ENFORCE, enforceStreamCapabilities); + } + return fs; }
I'm surprised/confused by this. I would have thought that the FS's configuration would have been the Configuration which you provided when constructing it from the Path. I assume that must not be the case?
There's a file system cache, so when you call get FileSystem, you get the copy of the file system that was created before we modified the configuration to add our property.
v6: hopefully this patch is named correctly to work around yetus' limitations
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 8s | Docker mode activated. |
Prechecks | |||
0 | findbugs | 0m 0s | Findbugs executables are not available. |
+1 | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 25 new or modified test files. |
master Compile Tests | |||
0 | mvndep | 0m 11s | Maven dependency ordering for branch |
+1 | mvninstall | 4m 15s | master passed |
+1 | compile | 0m 58s | master passed |
+1 | checkstyle | 1m 40s | master passed |
+1 | shadedjars | 6m 20s | branch has no errors when building our shaded downstream artifacts. |
+1 | javadoc | 0m 44s | master passed |
Patch Compile Tests | |||
0 | mvndep | 0m 13s | Maven dependency ordering for patch |
+1 | mvninstall | 4m 17s | the patch passed |
+1 | compile | 0m 58s | the patch passed |
+1 | javac | 0m 58s | the patch passed |
-1 | checkstyle | 1m 17s | hbase-server: The patch generated 2 new + 782 unchanged - 0 fixed = 784 total (was 782) |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | xml | 0m 2s | The patch has no ill-formed XML file. |
+1 | shadedjars | 4m 41s | patch has no errors when building our shaded downstream artifacts. |
+1 | hadoopcheck | 18m 10s | Patch does not cause any errors with Hadoop 2.6.5 2.7.4 or 3.0.0. |
+1 | javadoc | 0m 44s | the patch passed |
Other Tests | |||
+1 | unit | 2m 14s | hbase-common in the patch passed. |
+1 | unit | 103m 22s | hbase-server in the patch passed. |
+1 | asflicense | 0m 37s | The patch does not generate ASF License warnings. |
145m 14s |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12907763/19841.06.patch |
Optional Tests | asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile xml |
uname | Linux 5e759db9c627 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9 14:43:09 UTC 2018 x86_64 GNU/Linux |
Build tool | maven |
Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh |
git revision | master / aeffca497b |
maven | version: Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
Default Java | 1.8.0_151 |
checkstyle | https://builds.apache.org/job/PreCommit-HBASE-Build/11197/artifact/patchprocess/diff-checkstyle-hbase-server.txt |
Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/11197/testReport/ |
modules | C: hbase-common hbase-server U: . |
Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/11197/console |
Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
v7 is what I pushed to master. It's just like v6 but removes unused imports. Running tests locally before pushing to branch-2 as well; will resolve when that's done.
There's a lot of failure on branch-2, I'm going to revert this from master until I figure out why there are such huge differences. Sorry for the noise.
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4473 (See https://builds.apache.org/job/HBase-Trunk_matrix/4473/)
HBASE-19841 LocalHTU to not enforce stream capabilities (mdrob: rev 9ea152d2351cfb2607a5a861358470969ce24548)
- (edit) hbase-server/src/test/resources/hbase-site.xml
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksScanned.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestCase.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestWALMonotonicallyIncreasingSeqId.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHStore.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestIntraRowPagination.java
- (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/util/CommonFSUtils.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverStacking.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestResettingCounters.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDefaultMemStore.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestWALMethods.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/asyncfs/TestLocalAsyncOutput.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/encoding/TestSeekBeforeWithReverseScan.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingMemStore.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionInfoBuilder.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestColumnPrefixFilter.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFileRefresherChore.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestDependentColumnFilter.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactionPolicy.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALActionsListener.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFilterFromRegionSide.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFilter.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestInvocationRecordFilter.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestMultipleColumnPrefixFilter.java
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4474 (See https://builds.apache.org/job/HBase-Trunk_matrix/4474/)
Revert "HBASE-19841 LocalHTU to not enforce stream capabilities" (stack: rev a5a8c4f3f2ae42eda7d35e1e4410d40674cedb47)
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestInvocationRecordFilter.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionInfoBuilder.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHStore.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestIntraRowPagination.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingMemStore.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestDependentColumnFilter.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALActionsListener.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestCase.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/encoding/TestSeekBeforeWithReverseScan.java
- (edit) hbase-server/src/test/resources/hbase-site.xml
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactionPolicy.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestResettingCounters.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFileRefresherChore.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFilter.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksScanned.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDefaultMemStore.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestWALMonotonicallyIncreasingSeqId.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestWALMethods.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/asyncfs/TestLocalAsyncOutput.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFilterFromRegionSide.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestColumnPrefixFilter.java
- (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/util/CommonFSUtils.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestMultipleColumnPrefixFilter.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverStacking.java
new patch, much slimmer.
Let's let HTU act like a local cluster until we know for sure that it isn't. Setting a few extra properties on a conf isn't going to make much difference for test runtime, and we don't have to make developers remember the difference between localHTU and regular HTU.
+1 on patch.
I like the clean exposition on diff between local and hdfs fs and when either is in effect. I like the purge of the hbase-site config.
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 2m 27s | Docker mode activated. |
Prechecks | |||
0 | findbugs | 0m 0s | Findbugs executables are not available. |
+1 | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. |
master Compile Tests | |||
+1 | mvninstall | 7m 15s | master passed |
+1 | compile | 0m 41s | master passed |
+1 | checkstyle | 1m 2s | master passed |
+1 | shadedjars | 5m 21s | branch has no errors when building our shaded downstream artifacts. |
+1 | javadoc | 0m 34s | master passed |
Patch Compile Tests | |||
+1 | mvninstall | 4m 10s | the patch passed |
+1 | compile | 0m 42s | the patch passed |
+1 | javac | 0m 42s | the patch passed |
+1 | checkstyle | 1m 1s | the patch passed |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | xml | 0m 1s | The patch has no ill-formed XML file. |
+1 | shadedjars | 4m 29s | patch has no errors when building our shaded downstream artifacts. |
+1 | hadoopcheck | 18m 11s | Patch does not cause any errors with Hadoop 2.6.5 2.7.4 or 3.0.0. |
+1 | javadoc | 0m 28s | the patch passed |
Other Tests | |||
-1 | unit | 83m 27s | hbase-server in the patch failed. |
+1 | asflicense | 0m 31s | The patch does not generate ASF License warnings. |
124m 59s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.hbase.regionserver.wal.TestAsyncWALReplay |
hadoop.hbase.regionserver.TestCompactSplitThread | |
hadoop.hbase.io.TestFileLink | |
hadoop.hbase.regionserver.TestRegionReplicas | |
hadoop.hbase.security.visibility.TestVisibilityLabelsWithDefaultVisLabelService | |
hadoop.hbase.regionserver.wal.TestWALReplayCompressed | |
hadoop.hbase.security.visibility.TestVisibilityLabelsWithDeletes | |
hadoop.hbase.regionserver.wal.TestSecureAsyncWALReplay | |
hadoop.hbase.wal.TestSecureWAL | |
hadoop.hbase.regionserver.TestScannerWithBulkload | |
hadoop.hbase.regionserver.throttle.TestCompactionWithThroughputController | |
hadoop.hbase.regionserver.wal.TestAsyncLogRolling | |
hadoop.hbase.regionserver.TestFSErrorsExposed | |
hadoop.hbase.tool.TestSecureLoadIncrementalHFilesSplitRecovery | |
hadoop.hbase.regionserver.TestPerColumnFamilyFlush | |
hadoop.hbase.TestIOFencing | |
hadoop.hbase.master.procedure.TestSafemodeBringsDownMaster | |
hadoop.hbase.regionserver.wal.TestSecureWALReplay | |
hadoop.hbase.regionserver.TestClusterId | |
hadoop.hbase.regionserver.wal.TestAsyncWALReplayCompressed | |
hadoop.hbase.regionserver.TestSplitTransactionOnCluster | |
hadoop.hbase.TestZooKeeper | |
hadoop.hbase.regionserver.TestHRegionFileSystem | |
hadoop.hbase.TestHBaseTestingUtility | |
hadoop.hbase.regionserver.wal.TestWALReplay | |
hadoop.hbase.security.access.TestCoprocessorWhitelistMasterObserver | |
hadoop.hbase.master.balancer.TestFavoredStochasticBalancerPickers | |
hadoop.hbase.mob.compactions.TestPartitionedMobCompactor | |
hadoop.hbase.regionserver.wal.TestLogRollAbort | |
hadoop.hbase.wal.TestFSHLogProvider | |
hadoop.hbase.master.procedure.TestDeleteColumnFamilyProcedureFromClient | |
hadoop.hbase.regionserver.TestRegionFavoredNodes | |
hadoop.hbase.tool.TestLoadIncrementalHFilesSplitRecovery | |
hadoop.hbase.regionserver.TestHdfsSnapshotHRegion | |
hadoop.hbase.security.visibility.TestVisibilityLabelsWithSLGStack | |
hadoop.hbase.wal.TestBoundedRegionGroupingStrategy | |
hadoop.hbase.regionserver.wal.TestWALReplayBoundedLogWriterCreation | |
hadoop.hbase.coprocessor.TestWALObserver | |
hadoop.hbase.regionserver.wal.TestLogRolling | |
hadoop.hbase.regionserver.TestCompactionInDeadRegionServer | |
hadoop.hbase.mob.compactions.TestMobCompactor | |
hadoop.hbase.master.cleaner.TestSnapshotFromMaster | |
hadoop.hbase.security.visibility.TestVisibilityLabelsOnNewVersionBehaviorTable |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12908199/19841.007.patch |
Optional Tests | asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile xml |
uname | Linux eaf9335afad7 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 GNU/Linux |
Build tool | maven |
Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh |
git revision | master / 0b9a0dc951 |
maven | version: Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
Default Java | 1.8.0_151 |
unit | https://builds.apache.org/job/PreCommit-HBASE-Build/11239/artifact/patchprocess/patch-unit-hbase-server.txt |
Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/11239/testReport/ |
modules | C: hbase-server U: hbase-server |
Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/11239/console |
Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 10s | Docker mode activated. |
Prechecks | |||
0 | findbugs | 0m 1s | Findbugs executables are not available. |
+1 | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 6 new or modified test files. |
master Compile Tests | |||
0 | mvndep | 0m 11s | Maven dependency ordering for branch |
+1 | mvninstall | 4m 19s | master passed |
+1 | compile | 0m 59s | master passed |
+1 | checkstyle | 1m 31s | master passed |
+1 | shadedjars | 6m 15s | branch has no errors when building our shaded downstream artifacts. |
+1 | javadoc | 0m 43s | master passed |
Patch Compile Tests | |||
0 | mvndep | 0m 14s | Maven dependency ordering for patch |
+1 | mvninstall | 4m 20s | the patch passed |
+1 | compile | 0m 59s | the patch passed |
+1 | javac | 0m 59s | the patch passed |
-1 | checkstyle | 1m 9s | hbase-server: The patch generated 1 new + 342 unchanged - 0 fixed = 343 total (was 342) |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | xml | 0m 1s | The patch has no ill-formed XML file. |
+1 | shadedjars | 4m 41s | patch has no errors when building our shaded downstream artifacts. |
+1 | hadoopcheck | 18m 56s | Patch does not cause any errors with Hadoop 2.6.5 2.7.4 or 3.0.0. |
+1 | javadoc | 0m 52s | the patch passed |
Other Tests | |||
+1 | unit | 2m 33s | hbase-common in the patch passed. |
-1 | unit | 54m 14s | hbase-server in the patch failed. |
0 | asflicense | 0m 54s | ASF License check generated no output? |
97m 35s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.hbase.regionserver.wal.TestLogRollPeriod |
hadoop.hbase.master.cleaner.TestSnapshotFromMaster | |
hadoop.hbase.master.procedure.TestTruncateTableProcedure | |
hadoop.hbase.regionserver.wal.TestSecureAsyncWALReplay | |
hadoop.hbase.master.TestMasterRestartAfterDisablingTable | |
hadoop.hbase.master.procedure.TestProcedureAdmin | |
hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures | |
hadoop.hbase.regionserver.TestHRegionFileSystem | |
hadoop.hbase.regionserver.TestParallelPut | |
hadoop.hbase.regionserver.TestCompoundBloomFilter | |
hadoop.hbase.regionserver.TestRegionOpen | |
hadoop.hbase.master.locking.TestLockProcedure | |
hadoop.hbase.master.procedure.TestDeleteColumnFamilyProcedureFromClient | |
hadoop.hbase.regionserver.TestClearRegionBlockCache | |
hadoop.hbase.master.procedure.TestModifyTableProcedure | |
hadoop.hbase.trace.TestHTraceHooks | |
hadoop.hbase.regionserver.TestCompactionState | |
hadoop.hbase.regionserver.TestHRegionServerBulkLoadWithOldClient | |
hadoop.hbase.regionserver.wal.TestWALReplayBoundedLogWriterCreation | |
hadoop.hbase.regionserver.TestEndToEndSplitTransaction | |
hadoop.hbase.regionserver.wal.TestAsyncFSWAL | |
hadoop.hbase.master.procedure.TestEnableTableProcedure | |
hadoop.hbase.regionserver.TestHdfsSnapshotHRegion | |
hadoop.hbase.regionserver.TestSplitTransactionOnCluster | |
hadoop.hbase.tool.TestLoadIncrementalHFilesSplitRecovery | |
hadoop.hbase.master.procedure.TestCloneSnapshotProcedure | |
hadoop.hbase.regionserver.wal.TestAsyncWALReplayCompressed | |
hadoop.hbase.master.procedure.TestSafemodeBringsDownMaster | |
hadoop.hbase.regionserver.wal.TestLogRollingNoCluster | |
hadoop.hbase.regionserver.TestHRegionServerBulkLoad | |
hadoop.hbase.ipc.TestRpcClientLeaks | |
hadoop.hbase.regionserver.TestScannerWithBulkload | |
hadoop.hbase.master.TestMasterOperationsForRegionReplicas | |
hadoop.hbase.regionserver.TestMultiLogThreshold | |
hadoop.hbase.regionserver.TestRegionReplicas | |
hadoop.hbase.tool.TestSecureLoadIncrementalHFiles | |
hadoop.hbase.regionserver.wal.TestAsyncLogRollPeriod | |
hadoop.hbase.regionserver.wal.TestDurability | |
hadoop.hbase.TestAcidGuaranteesWithAdaptivePolicy | |
hadoop.hbase.regionserver.TestCompactionInDeadRegionServer | |
hadoop.hbase.master.normalizer.TestSimpleRegionNormalizerOnCluster | |
hadoop.hbase.regionserver.TestDeleteMobTable | |
hadoop.hbase.regionserver.wal.TestWALReplayCompressed | |
hadoop.hbase.regionserver.wal.TestLogRollAbort | |
hadoop.hbase.tool.TestCanaryTool | |
hadoop.hbase.regionserver.TestRegionServerAbort | |
hadoop.hbase.regionserver.wal.TestAsyncWALReplay | |
hadoop.hbase.TestHBaseTestingUtility | |
hadoop.hbase.TestInfoServers | |
hadoop.hbase.regionserver.TestScannerRetriableFailure | |
hadoop.hbase.master.procedure.TestTableDescriptorModificationFromClient | |
hadoop.hbase.regionserver.TestMajorCompaction | |
hadoop.hbase.tool.TestSecureLoadIncrementalHFilesSplitRecovery | |
hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush | |
hadoop.hbase.regionserver.throttle.TestFlushWithThroughputController | |
hadoop.hbase.regionserver.TestClusterId | |
hadoop.hbase.regionserver.TestRegionServerMetrics | |
hadoop.hbase.master.procedure.TestRestoreSnapshotProcedure | |
hadoop.hbase.regionserver.wal.TestProtobufLog | |
hadoop.hbase.regionserver.TestRegionReplicasWithModifyTable | |
hadoop.hbase.regionserver.wal.TestSecureWALReplay | |
hadoop.hbase.regionserver.TestTimestampFilterSeekHint | |
hadoop.hbase.regionserver.TestCompactionFileNotFound | |
hadoop.hbase.tool.TestLoadIncrementalHFiles | |
hadoop.hbase.master.procedure.TestCreateNamespaceProcedure | |
hadoop.hbase.master.procedure.TestServerCrashProcedure | |
hadoop.hbase.regionserver.TestPerColumnFamilyFlush |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12908360/HBASE-19841.v8.patch |
Optional Tests | asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile xml |
uname | Linux 9f7ce25d286b 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9 14:43:09 UTC 2018 x86_64 GNU/Linux |
Build tool | maven |
Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh |
git revision | master / 194890b694 |
maven | version: Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
Default Java | 1.8.0_151 |
checkstyle | https://builds.apache.org/job/PreCommit-HBASE-Build/11257/artifact/patchprocess/diff-checkstyle-hbase-server.txt |
unit | https://builds.apache.org/job/PreCommit-HBASE-Build/11257/artifact/patchprocess/patch-unit-hbase-server.txt |
Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/11257/testReport/ |
Max. process+thread count | 876 (vs. ulimit of 1000) |
modules | C: hbase-common hbase-server U: . |
Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/11257/console |
Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
Retry. Patch failed run on H0 with OOME, can't create threads. Looking at processing listing on machine, seems calm-enough – just OS processes (will study more). Chia-ping patch just ran successfully on H15. Retry to see what happens.
I'm still getting some errors locally, will need to iterate a bit more. I've been a bit too optimistic in posting earlier.
Retry. Added count of open fds, uptime, and ulimit -l to test output... under machine.... Lets see what it says.
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 8s | Docker mode activated. |
Prechecks | |||
0 | findbugs | 0m 0s | Findbugs executables are not available. |
+1 | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 6 new or modified test files. |
master Compile Tests | |||
0 | mvndep | 0m 13s | Maven dependency ordering for branch |
+1 | mvninstall | 4m 15s | master passed |
+1 | compile | 0m 57s | master passed |
+1 | checkstyle | 1m 32s | master passed |
+1 | shadedjars | 6m 9s | branch has no errors when building our shaded downstream artifacts. |
+1 | javadoc | 0m 44s | master passed |
Patch Compile Tests | |||
0 | mvndep | 0m 12s | Maven dependency ordering for patch |
+1 | mvninstall | 4m 21s | the patch passed |
+1 | compile | 0m 57s | the patch passed |
+1 | javac | 0m 57s | the patch passed |
-1 | checkstyle | 1m 8s | hbase-server: The patch generated 1 new + 342 unchanged - 0 fixed = 343 total (was 342) |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | xml | 0m 1s | The patch has no ill-formed XML file. |
+1 | shadedjars | 4m 34s | patch has no errors when building our shaded downstream artifacts. |
+1 | hadoopcheck | 18m 40s | Patch does not cause any errors with Hadoop 2.6.5 2.7.4 or 3.0.0. |
+1 | javadoc | 0m 46s | the patch passed |
Other Tests | |||
+1 | unit | 2m 19s | hbase-common in the patch passed. |
-1 | unit | 21m 43s | hbase-server in the patch failed. |
+1 | asflicense | 0m 20s | The patch does not generate ASF License warnings. |
63m 35s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.hbase.io.hfile.bucket.TestBucketCache |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12908406/HBASE-19841.v8.patch |
Optional Tests | asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile xml |
uname | Linux db082b0bc9f5 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19 15:49:21 UTC 2017 x86_64 GNU/Linux |
Build tool | maven |
Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh |
git revision | master / 194890b694 |
maven | version: Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
Default Java | 1.8.0_151 |
checkstyle | https://builds.apache.org/job/PreCommit-HBASE-Build/11261/artifact/patchprocess/diff-checkstyle-hbase-server.txt |
unit | https://builds.apache.org/job/PreCommit-HBASE-Build/11261/artifact/patchprocess/patch-unit-hbase-server.txt |
Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/11261/testReport/ |
Max. process+thread count | 551 (vs. ulimit of 1000) |
modules | C: hbase-common hbase-server U: . |
Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/11261/console |
Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 2m 22s | Docker mode activated. |
Prechecks | |||
0 | findbugs | 0m 0s | Findbugs executables are not available. |
+1 | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 6 new or modified test files. |
master Compile Tests | |||
0 | mvndep | 1m 24s | Maven dependency ordering for branch |
+1 | mvninstall | 6m 48s | master passed |
+1 | compile | 1m 0s | master passed |
+1 | checkstyle | 1m 34s | master passed |
+1 | shadedjars | 6m 21s | branch has no errors when building our shaded downstream artifacts. |
+1 | javadoc | 0m 48s | master passed |
Patch Compile Tests | |||
0 | mvndep | 0m 14s | Maven dependency ordering for patch |
+1 | mvninstall | 4m 35s | the patch passed |
+1 | compile | 1m 3s | the patch passed |
+1 | javac | 1m 3s | the patch passed |
-1 | checkstyle | 1m 9s | hbase-server: The patch generated 1 new + 342 unchanged - 0 fixed = 343 total (was 342) |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | xml | 0m 1s | The patch has no ill-formed XML file. |
+1 | shadedjars | 4m 51s | patch has no errors when building our shaded downstream artifacts. |
+1 | hadoopcheck | 18m 45s | Patch does not cause any errors with Hadoop 2.6.5 2.7.4 or 3.0.0. |
+1 | javadoc | 0m 44s | the patch passed |
Other Tests | |||
+1 | unit | 2m 19s | hbase-common in the patch passed. |
-1 | unit | 20m 23s | hbase-server in the patch failed. |
+1 | asflicense | 0m 26s | The patch does not generate ASF License warnings. |
69m 5s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.hbase.master.balancer.TestRegionLocationFinder |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12908409/HBASE-19841.v8.patch |
Optional Tests | asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile xml |
uname | Linux 26020d73ad06 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18 11:55:51 UTC 2017 x86_64 GNU/Linux |
Build tool | maven |
Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh |
git revision | master / 194890b694 |
maven | version: Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
Default Java | 1.8.0_151 |
checkstyle | https://builds.apache.org/job/PreCommit-HBASE-Build/11263/artifact/patchprocess/diff-checkstyle-hbase-server.txt |
unit | https://builds.apache.org/job/PreCommit-HBASE-Build/11263/artifact/patchprocess/patch-unit-hbase-server.txt |
Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/11263/testReport/ |
Max. process+thread count | 551 (vs. ulimit of 1000) |
modules | C: hbase-common hbase-server U: . |
Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/11263/console |
Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 9s | Docker mode activated. |
Prechecks | |||
0 | findbugs | 0m 1s | Findbugs executables are not available. |
+1 | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 6 new or modified test files. |
master Compile Tests | |||
0 | mvndep | 0m 13s | Maven dependency ordering for branch |
+1 | mvninstall | 4m 12s | master passed |
+1 | compile | 0m 56s | master passed |
+1 | checkstyle | 1m 21s | master passed |
+1 | shadedjars | 5m 32s | branch has no errors when building our shaded downstream artifacts. |
+1 | javadoc | 0m 41s | master passed |
Patch Compile Tests | |||
0 | mvndep | 0m 6s | Maven dependency ordering for patch |
+1 | mvninstall | 4m 7s | the patch passed |
+1 | compile | 0m 55s | the patch passed |
+1 | javac | 0m 55s | the patch passed |
-1 | checkstyle | 0m 57s | hbase-server: The patch generated 1 new + 342 unchanged - 0 fixed = 343 total (was 342) |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | xml | 0m 1s | The patch has no ill-formed XML file. |
-1 | shadedjars | 4m 13s | patch has 7 errors when building our shaded downstream artifacts. |
-1 | hadoopcheck | 7m 47s | The patch causes 7 errors with Hadoop v2.6.5. |
-1 | hadoopcheck | 8m 16s | The patch causes 7 errors with Hadoop v2.7.4. |
-1 | hadoopcheck | 8m 24s | The patch causes 7 errors with Hadoop v3.0.0. |
+1 | javadoc | 0m 41s | the patch passed |
Other Tests | |||
-1 | unit | 1m 26s | hbase-common in the patch failed. |
-1 | unit | 60m 28s | hbase-server in the patch failed. |
+1 | asflicense | 0m 30s | The patch does not generate ASF License warnings. |
90m 2s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.hbase.TestChoreService |
hadoop.hbase.client.TestIllegalTableDescriptor | |
hadoop.hbase.replication.TestReplicationWithTags | |
hadoop.hbase.regionserver.TestSCVFWithMiniCluster | |
hadoop.hbase.quotas.TestSpaceQuotasWithSnapshots | |
hadoop.hbase.regionserver.TestRegionServerAbort | |
hadoop.hbase.client.TestScanWithoutFetchingData | |
hadoop.hbase.regionserver.wal.TestAsyncProtobufLog | |
hadoop.hbase.regionserver.TestEndToEndSplitTransaction | |
hadoop.hbase.replication.TestReplicationKillMasterRSCompressed | |
hadoop.hbase.replication.TestMultiSlaveReplication | |
hadoop.hbase.client.TestLimitedScanWithFilter | |
hadoop.hbase.client.replication.TestReplicationAdmin | |
hadoop.hbase.client.TestAsyncBufferMutator | |
hadoop.hbase.client.TestAsyncSnapshotAdminApi | |
hadoop.hbase.client.TestAsyncRegionAdminApi | |
hadoop.hbase.client.TestAsyncClusterAdminApi | |
hadoop.hbase.client.TestCIPutRpcTimeout | |
hadoop.hbase.util.TestConnectionCache | |
hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpointNoMaster | |
hadoop.hbase.replication.regionserver.TestGlobalThrottler | |
hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpoint | |
hadoop.hbase.client.TestCIDeleteOperationTimeout | |
hadoop.hbase.regionserver.TestMobStoreCompaction | |
hadoop.hbase.procedure.TestFailedProcCleanup | |
hadoop.hbase.client.TestScannerTimeout | |
hadoop.hbase.namespace.TestNamespaceAuditor | |
hadoop.hbase.replication.TestReplicationDroppedTables | |
hadoop.hbase.replication.TestReplicationKillSlaveRS | |
hadoop.hbase.client.TestAdmin2 | |
hadoop.hbase.replication.TestReplicationStatus | |
hadoop.hbase.regionserver.TestMutateRowsRecovery | |
hadoop.hbase.TestServerSideScanMetricsFromClientSide | |
hadoop.hbase.regionserver.TestRegionServerHostname | |
hadoop.hbase.regionserver.wal.TestWALReplayBoundedLogWriterCreation | |
hadoop.hbase.regionserver.wal.TestWALReplayCompressed | |
hadoop.hbase.util.TestCoprocessorScanPolicy | |
hadoop.hbase.client.TestRestoreSnapshotFromClientWithRegionReplicas | |
hadoop.hbase.replication.regionserver.TestWALEntryStream | |
hadoop.hbase.client.TestAsyncRegionLocatorTimeout | |
hadoop.hbase.replication.TestReplicationSyncUpTool | |
hadoop.hbase.quotas.TestQuotaThrottle | |
hadoop.hbase.replication.TestSerialReplication | |
hadoop.hbase.client.TestAppendFromClientSide | |
hadoop.hbase.client.TestServerLoadDurability | |
hadoop.hbase.client.TestAsyncTableNoncedRetry | |
hadoop.hbase.util.TestMiniClusterLoadEncoded | |
hadoop.hbase.mob.TestDefaultMobStoreFlusher | |
hadoop.hbase.replication.TestReplicationSmallTests | |
hadoop.hbase.regionserver.TestWALLockup | |
hadoop.hbase.quotas.TestQuotaObserverChoreWithMiniCluster | |
hadoop.hbase.client.TestAsyncSingleRequestRpcRetryingCaller | |
hadoop.hbase.client.TestResultSizeEstimation | |
hadoop.hbase.client.TestLeaseRenewal | |
hadoop.hbase.replication.TestReplicationChangingPeerRegionservers | |
hadoop.hbase.snapshot.TestSnapshotClientRetries | |
hadoop.hbase.client.TestAsyncToolAdminApi | |
hadoop.hbase.client.TestAsyncNonMetaRegionLocatorConcurrenyLimit | |
hadoop.hbase.fs.TestBlockReorder | |
hadoop.hbase.quotas.TestQuotaAdmin | |
hadoop.hbase.regionserver.TestCompaction | |
hadoop.hbase.wal.TestWALSplitBoundedLogWriterCreation | |
hadoop.hbase.io.asyncfs.TestFanOutOneBlockAsyncDFSOutput | |
hadoop.hbase.procedure.TestProcedureManager | |
hadoop.hbase.replication.TestMasterReplication | |
hadoop.hbase.client.TestCIBadHostname | |
hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush | |
hadoop.hbase.regionserver.TestScannerRetriableFailure | |
hadoop.hbase.client.TestDropTimeoutRequest | |
hadoop.hbase.client.TestAlwaysSetScannerId | |
hadoop.hbase.regionserver.TestHdfsSnapshotHRegion | |
hadoop.hbase.replication.TestReplicationEndpoint | |
hadoop.hbase.client.replication.TestReplicationAdminWithTwoDifferentZKClusters | |
hadoop.hbase.client.TestTimestampsFilter | |
hadoop.hbase.regionserver.TestRowTooBig | |
hadoop.hbase.mob.compactions.TestMobCompactor | |
hadoop.hbase.client.TestFromClientSideWithCoprocessor | |
hadoop.hbase.replication.multiwal.TestReplicationEndpointWithMultipleWAL | |
hadoop.hbase.replication.TestReplicationDisableInactivePeer | |
hadoop.hbase.client.TestEnableTable | |
hadoop.hbase.util.TestFSUtils | |
hadoop.hbase.client.TestIncrementFromClientSideWithCoprocessor | |
hadoop.hbase.util.TestMiniClusterLoadSequential | |
hadoop.hbase.client.TestScannersFromClientSide2 | |
hadoop.hbase.client.TestAsyncResultScannerCursor | |
hadoop.hbase.regionserver.TestFSErrorsExposed | |
hadoop.hbase.client.TestAsyncTable | |
hadoop.hbase.client.TestCloneSnapshotFromClient | |
hadoop.hbase.client.TestFromClientSideScanExcpetion | |
hadoop.hbase.ipc.TestRpcServerSlowConnectionSetup |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12908409/HBASE-19841.v8.patch |
Optional Tests | asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile xml |
uname | Linux 780babdddf5e 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 GNU/Linux |
Build tool | maven |
Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh |
git revision | master / 194890b694 |
maven | version: Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
Default Java | 1.8.0_151 |
checkstyle | https://builds.apache.org/job/PreCommit-HBASE-Build/11262/artifact/patchprocess/diff-checkstyle-hbase-server.txt |
unit | https://builds.apache.org/job/PreCommit-HBASE-Build/11262/artifact/patchprocess/patch-unit-hbase-common.txt |
unit | https://builds.apache.org/job/PreCommit-HBASE-Build/11262/artifact/patchprocess/patch-unit-hbase-server.txt |
Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/11262/testReport/ |
Max. process+thread count | 921 (vs. ulimit of 1000) |
modules | C: hbase-common hbase-server U: . |
Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/11262/console |
Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 11s | Docker mode activated. |
Prechecks | |||
0 | findbugs | 0m 0s | Findbugs executables are not available. |
+1 | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 9 new or modified test files. |
master Compile Tests | |||
0 | mvndep | 1m 26s | Maven dependency ordering for branch |
+1 | mvninstall | 7m 5s | master passed |
+1 | compile | 1m 2s | master passed |
+1 | checkstyle | 1m 19s | master passed |
+1 | shadedjars | 5m 30s | branch has no errors when building our shaded downstream artifacts. |
+1 | javadoc | 0m 45s | master passed |
Patch Compile Tests | |||
0 | mvndep | 0m 14s | Maven dependency ordering for patch |
+1 | mvninstall | 4m 14s | the patch passed |
+1 | compile | 0m 56s | the patch passed |
+1 | javac | 0m 56s | the patch passed |
-1 | checkstyle | 0m 54s | hbase-server: The patch generated 3 new + 361 unchanged - 0 fixed = 364 total (was 361) |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | xml | 0m 1s | The patch has no ill-formed XML file. |
+1 | shadedjars | 4m 4s | patch has no errors when building our shaded downstream artifacts. |
+1 | hadoopcheck | 16m 26s | Patch does not cause any errors with Hadoop 2.6.5 2.7.4 or 3.0.0. |
+1 | javadoc | 0m 47s | the patch passed |
Other Tests | |||
+1 | unit | 2m 29s | hbase-common in the patch passed. |
-1 | unit | 24m 24s | hbase-server in the patch failed. |
+1 | asflicense | 1m 52s | The patch does not generate ASF License warnings. |
69m 9s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.hbase.TestClientClusterStatus |
hadoop.hbase.TestCheckTestClasses |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12908444/HBASE-19841.v10.patch |
Optional Tests | asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile xml |
uname | Linux 1677f221bd0e 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 GNU/Linux |
Build tool | maven |
Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh |
git revision | master / 08bcd64c25 |
maven | version: Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
Default Java | 1.8.0_151 |
checkstyle | https://builds.apache.org/job/PreCommit-HBASE-Build/11271/artifact/patchprocess/diff-checkstyle-hbase-server.txt |
unit | https://builds.apache.org/job/PreCommit-HBASE-Build/11271/artifact/patchprocess/patch-unit-hbase-server.txt |
Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/11271/testReport/ |
Max. process+thread count | 838 (vs. ulimit of 5000) |
modules | C: hbase-common hbase-server U: . |
Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/11271/console |
Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
v11: fixed checkstyle issues. failing tests passed locally for me, will see what happens on retry.
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 11s | Docker mode activated. |
Prechecks | |||
0 | findbugs | 0m 0s | Findbugs executables are not available. |
+1 | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 9 new or modified test files. |
master Compile Tests | |||
0 | mvndep | 0m 12s | Maven dependency ordering for branch |
+1 | mvninstall | 4m 34s | master passed |
+1 | compile | 1m 3s | master passed |
+1 | checkstyle | 1m 34s | master passed |
+1 | shadedjars | 6m 39s | branch has no errors when building our shaded downstream artifacts. |
+1 | javadoc | 0m 48s | master passed |
Patch Compile Tests | |||
0 | mvndep | 0m 14s | Maven dependency ordering for patch |
+1 | mvninstall | 4m 39s | the patch passed |
+1 | compile | 1m 5s | the patch passed |
+1 | javac | 1m 5s | the patch passed |
-1 | checkstyle | 1m 12s | hbase-server: The patch generated 4 new + 360 unchanged - 1 fixed = 364 total (was 361) |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | xml | 0m 1s | The patch has no ill-formed XML file. |
+1 | shadedjars | 4m 53s | patch has no errors when building our shaded downstream artifacts. |
+1 | hadoopcheck | 19m 16s | Patch does not cause any errors with Hadoop 2.6.5 2.7.4 or 3.0.0. |
+1 | javadoc | 0m 47s | the patch passed |
Other Tests | |||
+1 | unit | 2m 24s | hbase-common in the patch passed. |
-1 | unit | 270m 18s | hbase-server in the patch failed. |
+1 | asflicense | 0m 56s | The patch does not generate ASF License warnings. |
315m 20s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.hbase.client.TestAsyncProcedureAdminApi |
hadoop.hbase.master.procedure.TestTruncateTableProcedure | |
hadoop.hbase.master.TestDeadServer | |
hadoop.hbase.regionserver.TestRegionReplicasWithRestartScenarios | |
hadoop.hbase.client.TestMobRestoreSnapshotFromClient | |
hadoop.hbase.regionserver.wal.TestWALReplay | |
hadoop.hbase.regionserver.TestJoinedScanners | |
hadoop.hbase.regionserver.TestScannerHeartbeatMessages | |
hadoop.hbase.replication.multiwal.TestReplicationSyncUpToolWithMultipleAsyncWAL | |
hadoop.hbase.client.TestAsyncRegionLocatorTimeout | |
hadoop.hbase.regionserver.TestHRegionFileSystem | |
hadoop.hbase.client.TestAsyncTableGetMultiThreaded | |
hadoop.hbase.master.TestMaster | |
hadoop.hbase.client.TestResultFromCoprocessor | |
hadoop.hbase.security.access.TestAccessController | |
hadoop.hbase.io.encoding.TestEncodedSeekers | |
hadoop.hbase.regionserver.TestRegionOpen | |
hadoop.hbase.regionserver.TestMobStoreScanner | |
hadoop.hbase.regionserver.TestClearRegionBlockCache | |
hadoop.hbase.master.TestMasterFileSystemWithWALDir | |
hadoop.hbase.replication.TestReplicationKillMasterRS | |
hadoop.hbase.replication.TestReplicationStatus | |
hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpoint | |
hadoop.hbase.master.procedure.TestModifyTableProcedure | |
hadoop.hbase.replication.multiwal.TestReplicationKillMasterRSCompressedWithMultipleAsyncWAL | |
hadoop.hbase.client.TestRawAsyncTableScan | |
hadoop.hbase.regionserver.TestHRegionServerBulkLoadWithOldClient | |
hadoop.hbase.regionserver.wal.TestWALReplayBoundedLogWriterCreation | |
hadoop.hbase.quotas.TestSnapshotQuotaObserverChore | |
hadoop.hbase.regionserver.TestSplitWalDataLoss | |
hadoop.hbase.client.TestTableSnapshotScanner | |
hadoop.hbase.master.procedure.TestEnableTableProcedure | |
hadoop.hbase.regionserver.TestHdfsSnapshotHRegion | |
hadoop.hbase.regionserver.TestSplitTransactionOnCluster | |
hadoop.hbase.replication.TestReplicationEmptyWALRecovery | |
hadoop.hbase.io.hfile.TestCacheOnWrite | |
hadoop.hbase.master.procedure.TestCloneSnapshotProcedure | |
hadoop.hbase.regionserver.throttle.TestCompactionWithThroughputController | |
hadoop.hbase.client.TestAvoidCellReferencesIntoShippedBlocks | |
hadoop.hbase.regionserver.TestAtomicOperation | |
hadoop.hbase.regionserver.wal.TestAsyncWALReplayCompressed | |
hadoop.hbase.replication.TestReplicationSource | |
hadoop.hbase.client.TestAsyncTableBatch | |
hadoop.hbase.namespace.TestNamespaceAuditor | |
hadoop.hbase.client.TestFromClientSideScanExcpetion | |
hadoop.hbase.replication.multiwal.TestReplicationEndpointWithMultipleAsyncWAL | |
hadoop.hbase.TestRegionMetrics | |
hadoop.hbase.TestZooKeeper | |
hadoop.hbase.util.TestRegionSplitter | |
hadoop.hbase.regionserver.TestHRegionServerBulkLoad | |
hadoop.hbase.client.TestAsyncResultScannerCursor | |
hadoop.hbase.security.visibility.TestVisibilityWithCheckAuths | |
hadoop.hbase.filter.TestFilterWithScanLimits | |
hadoop.hbase.master.assignment.TestAssignmentOnRSCrash | |
hadoop.hbase.regionserver.TestRegionReplicas | |
hadoop.hbase.replication.multiwal.TestReplicationSyncUpToolWithMultipleWAL | |
hadoop.hbase.tool.TestSecureLoadIncrementalHFiles | |
hadoop.hbase.snapshot.TestFlushSnapshotFromClient | |
hadoop.hbase.client.TestSnapshotFromClientWithRegionReplicas | |
hadoop.hbase.replication.TestReplicationDroppedTables | |
hadoop.hbase.TestAcidGuaranteesWithNoInMemCompaction | |
hadoop.hbase.client.TestResultScannerCursor | |
hadoop.hbase.client.TestFromClientSideWithCoprocessor | |
hadoop.hbase.util.TestRegionMover | |
hadoop.hbase.TestAcidGuaranteesWithAdaptivePolicy | |
hadoop.hbase.regionserver.TestCompactionInDeadRegionServer | |
hadoop.hbase.master.TestRollingRestart | |
hadoop.hbase.regionserver.TestFlushLifeCycleTracker | |
hadoop.hbase.util.TestFromClientSide3WoUnsafe | |
hadoop.hbase.client.TestSnapshotMetadata | |
hadoop.hbase.regionserver.wal.TestAsyncWALReplay | |
hadoop.hbase.TestHBaseTestingUtility | |
hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy | |
hadoop.hbase.client.TestAsyncReplicationAdminApiWithClusters | |
hadoop.hbase.io.encoding.TestChangingEncoding | |
hadoop.hbase.regionserver.TestRegionServerNoMaster | |
hadoop.hbase.regionserver.TestMajorCompaction | |
hadoop.hbase.master.TestRestartCluster | |
hadoop.hbase.replication.regionserver.TestWALEntryStream | |
hadoop.hbase.replication.TestNamespaceReplication | |
hadoop.hbase.client.TestCloneSnapshotFromClientWithRegionReplicas | |
hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush | |
hadoop.hbase.replication.TestReplicationEndpoint | |
hadoop.hbase.client.TestCloneSnapshotFromClient | |
hadoop.hbase.regionserver.throttle.TestFlushWithThroughputController | |
hadoop.hbase.util.TestConnectionCache | |
hadoop.hbase.regionserver.TestClusterId | |
hadoop.hbase.client.TestBlockEvictionFromClient | |
hadoop.hbase.master.TestSplitLogManager | |
hadoop.hbase.client.TestFlushFromClient | |
hadoop.hbase.util.TestHBaseFsckReplication | |
hadoop.hbase.replication.multiwal.TestReplicationEndpointWithMultipleWAL | |
hadoop.hbase.client.TestAsyncClusterAdminApi2 | |
hadoop.hbase.regionserver.TestRegionServerMetrics | |
hadoop.hbase.snapshot.TestRegionSnapshotTask | |
hadoop.hbase.client.TestScannersFromClientSide | |
hadoop.hbase.master.procedure.TestRestoreSnapshotProcedure | |
hadoop.hbase.client.TestAsyncClusterAdminApi | |
hadoop.hbase.master.TestRegionPlacement2 | |
hadoop.hbase.TestMetaTableAccessor | |
hadoop.hbase.client.TestFromClientSide3 | |
hadoop.hbase.regionserver.TestRegionReplicasWithModifyTable | |
hadoop.hbase.client.TestCISleep | |
hadoop.hbase.client.TestRawAsyncScanCursor | |
hadoop.hbase.client.TestAsyncDecommissionAdminApi | |
hadoop.hbase.quotas.TestQuotaThrottle | |
hadoop.hbase.regionserver.wal.TestSecureWALReplay | |
hadoop.hbase.TestIOFencing | |
hadoop.hbase.client.TestRawAsyncTablePartialScan | |
hadoop.hbase.regionserver.TestFSErrorsExposed | |
hadoop.hbase.regionserver.TestCompactionFileNotFound | |
hadoop.hbase.filter.TestFilterListOrOperatorWithBlkCnt | |
hadoop.hbase.util.TestMiniClusterLoadSequential | |
hadoop.hbase.client.TestAdmin1 | |
hadoop.hbase.tool.TestLoadIncrementalHFiles | |
hadoop.hbase.client.TestMobSnapshotCloneIndependence | |
hadoop.hbase.replication.TestMasterReplication | |
hadoop.hbase.master.procedure.TestServerCrashProcedure | |
hadoop.hbase.regionserver.TestPerColumnFamilyFlush | |
hadoop.hbase.replication.multiwal.TestReplicationKillMasterRSCompressedWithMultipleWAL |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12908441/HBASE-19841.v9.patch |
Optional Tests | asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile xml |
uname | Linux 987efae83869 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9 14:43:09 UTC 2018 x86_64 GNU/Linux |
Build tool | maven |
Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh |
git revision | master / 08bcd64c25 |
maven | version: Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
Default Java | 1.8.0_151 |
checkstyle | https://builds.apache.org/job/PreCommit-HBASE-Build/11270/artifact/patchprocess/diff-checkstyle-hbase-server.txt |
unit | https://builds.apache.org/job/PreCommit-HBASE-Build/11270/artifact/patchprocess/patch-unit-hbase-server.txt |
Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/11270/testReport/ |
Max. process+thread count | 3797 (vs. ulimit of 5000) |
modules | C: hbase-common hbase-server U: . |
Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/11270/console |
Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
Starts out with "java.lang.OutOfMemoryError: unable to create new native thread".. thereafter can't start minhbasecluster.
Its on node H0
Yetus limit is
3797 (vs. ulimit of 5000) |
I added ulmit -a and it dumps this:
core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 386176 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 60000 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 10240 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited
Two java proceses on the box at start time only. No Geode running that I can see.
I think that v9 run was one that started before my v10 run, we should disregard it.
https://builds.apache.org/job/PreCommit-HBASE-Build/11276/console is the one that ran v11
test output at https://builds.apache.org/job/PreCommit-HBASE-Build/11276/artifact/patchprocess/patch-unit-hbase-server.txt
something bad happened early in the secondpart execution and the rest of the tests never recovered from it.
rerunning at https://builds.apache.org/job/PreCommit-HBASE-Build/11293/ to see if it's environmental or patch failure.
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 10s | Docker mode activated. |
Prechecks | |||
0 | findbugs | 0m 0s | Findbugs executables are not available. |
+1 | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. |
master Compile Tests | |||
+1 | mvninstall | 5m 6s | master passed |
+1 | compile | 0m 53s | master passed |
+1 | checkstyle | 1m 14s | master passed |
+1 | shadedjars | 6m 24s | branch has no errors when building our shaded downstream artifacts. |
+1 | javadoc | 0m 36s | master passed |
Patch Compile Tests | |||
+1 | mvninstall | 5m 13s | the patch passed |
+1 | compile | 0m 51s | the patch passed |
+1 | javac | 0m 51s | the patch passed |
+1 | checkstyle | 1m 21s | the patch passed |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | xml | 0m 2s | The patch has no ill-formed XML file. |
+1 | shadedjars | 5m 32s | patch has no errors when building our shaded downstream artifacts. |
+1 | hadoopcheck | 21m 27s | Patch does not cause any errors with Hadoop 2.6.5 2.7.4 or 3.0.0. |
+1 | javadoc | 0m 35s | the patch passed |
Other Tests | |||
-1 | unit | 86m 42s | hbase-server in the patch failed. |
+1 | asflicense | 0m 28s | The patch does not generate ASF License warnings. |
130m 2s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.hbase.master.cleaner.TestSnapshotFromMaster |
hadoop.hbase.regionserver.wal.TestSecureAsyncWALReplay | |
hadoop.hbase.io.TestFileLink | |
hadoop.hbase.regionserver.wal.TestWALReplay | |
hadoop.hbase.regionserver.TestHRegionFileSystem | |
hadoop.hbase.quotas.TestSpaceQuotas | |
hadoop.hbase.wal.TestSecureWAL | |
hadoop.hbase.master.procedure.TestDeleteColumnFamilyProcedureFromClient | |
hadoop.hbase.mob.compactions.TestMobCompactor | |
hadoop.hbase.mob.compactions.TestPartitionedMobCompactor | |
hadoop.hbase.regionserver.wal.TestLogRolling | |
hadoop.hbase.regionserver.wal.TestWALReplayBoundedLogWriterCreation | |
hadoop.hbase.coprocessor.TestWALObserver | |
hadoop.hbase.regionserver.TestHdfsSnapshotHRegion | |
hadoop.hbase.regionserver.TestSplitTransactionOnCluster | |
hadoop.hbase.tool.TestLoadIncrementalHFilesSplitRecovery | |
hadoop.hbase.regionserver.throttle.TestCompactionWithThroughputController | |
hadoop.hbase.regionserver.wal.TestAsyncWALReplayCompressed | |
hadoop.hbase.master.procedure.TestSafemodeBringsDownMaster | |
hadoop.hbase.regionserver.TestScannerWithBulkload | |
hadoop.hbase.wal.TestFSHLogProvider | |
hadoop.hbase.regionserver.TestRegionReplicas | |
hadoop.hbase.regionserver.TestRegionFavoredNodes | |
hadoop.hbase.regionserver.TestCompactionInDeadRegionServer | |
hadoop.hbase.regionserver.wal.TestWALReplayCompressed | |
hadoop.hbase.regionserver.wal.TestLogRollAbort | |
hadoop.hbase.regionserver.wal.TestAsyncWALReplay | |
hadoop.hbase.regionserver.TestCompactSplitThread | |
hadoop.hbase.TestHBaseTestingUtility | |
hadoop.hbase.security.visibility.TestVisibilityLabelsWithSLGStack | |
hadoop.hbase.tool.TestSecureLoadIncrementalHFilesSplitRecovery | |
hadoop.hbase.regionserver.TestClusterId | |
hadoop.hbase.master.balancer.TestFavoredStochasticBalancerPickers | |
hadoop.hbase.regionserver.wal.TestAsyncLogRolling | |
hadoop.hbase.security.visibility.TestVisibilityLabelsWithDefaultVisLabelService | |
hadoop.hbase.regionserver.wal.TestSecureWALReplay | |
hadoop.hbase.TestIOFencing | |
hadoop.hbase.regionserver.TestFSErrorsExposed | |
hadoop.hbase.wal.TestBoundedRegionGroupingStrategy | |
hadoop.hbase.regionserver.TestPerColumnFamilyFlush |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12908199/19841.007.patch |
Optional Tests | asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile xml |
uname | Linux 2f9c900b6396 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9 14:43:09 UTC 2018 x86_64 GNU/Linux |
Build tool | maven |
Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh |
git revision | master / 7c318cead9 |
maven | version: Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
Default Java | 1.8.0_151 |
unit | https://builds.apache.org/job/PreCommit-HBASE-Build/11293/artifact/patchprocess/patch-unit-hbase-server.txt |
Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/11293/testReport/ |
modules | C: hbase-server U: hbase-server |
Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/11293/console |
Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 0m 9s | Docker mode activated. |
Prechecks | |||
0 | findbugs | 0m 1s | Findbugs executables are not available. |
+1 | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 9 new or modified test files. |
master Compile Tests | |||
0 | mvndep | 0m 12s | Maven dependency ordering for branch |
+1 | mvninstall | 4m 24s | master passed |
+1 | compile | 1m 2s | master passed |
+1 | checkstyle | 1m 24s | master passed |
+1 | shadedjars | 5m 36s | branch has no errors when building our shaded downstream artifacts. |
+1 | javadoc | 0m 40s | master passed |
Patch Compile Tests | |||
0 | mvndep | 0m 12s | Maven dependency ordering for patch |
+1 | mvninstall | 4m 18s | the patch passed |
+1 | compile | 0m 59s | the patch passed |
+1 | javac | 0m 59s | the patch passed |
+1 | checkstyle | 0m 22s | The patch hbase-common passed checkstyle |
+1 | checkstyle | 1m 1s | hbase-server: The patch generated 0 new + 360 unchanged - 1 fixed = 360 total (was 361) |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | xml | 0m 1s | The patch has no ill-formed XML file. |
+1 | shadedjars | 4m 16s | patch has no errors when building our shaded downstream artifacts. |
+1 | hadoopcheck | 17m 25s | Patch does not cause any errors with Hadoop 2.6.5 2.7.4 or 3.0.0. |
+1 | javadoc | 0m 43s | the patch passed |
Other Tests | |||
+1 | unit | 2m 26s | hbase-common in the patch passed. |
-1 | unit | 101m 53s | hbase-server in the patch failed. |
+1 | asflicense | 0m 29s | The patch does not generate ASF License warnings. |
142m 9s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.hbase.TestZooKeeper |
hadoop.hbase.TestFullLogReconstruction | |
hadoop.hbase.replication.TestReplicationDroppedTables |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12908483/HBASE-19841.v11.patch |
Optional Tests | asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile xml |
uname | Linux 3e85f5b570da 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 GNU/Linux |
Build tool | maven |
Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh |
git revision | master / 9272f40a5c |
maven | version: Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
Default Java | 1.8.0_151 |
unit | https://builds.apache.org/job/PreCommit-HBASE-Build/11333/artifact/patchprocess/patch-unit-hbase-server.txt |
Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/11333/testReport/ |
Max. process+thread count | 5454 (vs. ulimit of 20000) |
modules | C: hbase-common hbase-server U: . |
Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/11333/console |
Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
Retry. At least one of the test failures should be fixed now and log reconstruction seems to be a flakey.
+1 on latest patch. How did you find those test cases where they were creating their own Configuration rather than use the utilities's one?
I found them based on which tests were failing!
Excellent.
+1 on commit.... though I suppose no harm waiting on build to come in...
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 4m 2s | Docker mode activated. |
Prechecks | |||
0 | findbugs | 0m 1s | Findbugs executables are not available. |
+1 | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 9 new or modified test files. |
master Compile Tests | |||
0 | mvndep | 0m 22s | Maven dependency ordering for branch |
+1 | mvninstall | 5m 44s | master passed |
+1 | compile | 1m 21s | master passed |
+1 | checkstyle | 2m 8s | master passed |
+1 | shadedjars | 8m 7s | branch has no errors when building our shaded downstream artifacts. |
+1 | javadoc | 1m 2s | master passed |
Patch Compile Tests | |||
0 | mvndep | 0m 16s | Maven dependency ordering for patch |
+1 | mvninstall | 5m 0s | the patch passed |
+1 | compile | 1m 9s | the patch passed |
+1 | javac | 1m 9s | the patch passed |
+1 | checkstyle | 0m 24s | The patch hbase-common passed checkstyle |
+1 | checkstyle | 1m 11s | hbase-server: The patch generated 0 new + 360 unchanged - 1 fixed = 360 total (was 361) |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | xml | 0m 2s | The patch has no ill-formed XML file. |
+1 | shadedjars | 5m 31s | patch has no errors when building our shaded downstream artifacts. |
+1 | hadoopcheck | 21m 46s | Patch does not cause any errors with Hadoop 2.6.5 2.7.4 or 3.0.0. |
+1 | javadoc | 0m 59s | the patch passed |
Other Tests | |||
+1 | unit | 2m 37s | hbase-common in the patch passed. |
-1 | unit | 132m 18s | hbase-server in the patch failed. |
+1 | asflicense | 0m 37s | The patch does not generate ASF License warnings. |
187m 21s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.hbase.security.visibility.TestVisibilityLabelsWithDefaultVisLabelService |
hadoop.hbase.regionserver.TestMajorCompaction | |
hadoop.hbase.io.hfile.TestCacheOnWrite | |
hadoop.hbase.namespace.TestNamespaceAuditor | |
hadoop.hbase.master.procedure.TestRestoreSnapshotProcedure |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12908870/HBASE-19841.v11.patch |
Optional Tests | asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile xml |
uname | Linux 88cd9d5c1130 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18 11:55:51 UTC 2017 x86_64 GNU/Linux |
Build tool | maven |
Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh |
git revision | master / d472422475 |
maven | version: Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
Default Java | 1.8.0_151 |
unit | https://builds.apache.org/job/PreCommit-HBASE-Build/11343/artifact/patchprocess/patch-unit-hbase-server.txt |
Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/11343/testReport/ |
Max. process+thread count | 4906 (vs. ulimit of 10000) |
modules | C: hbase-common hbase-server U: . |
Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/11343/console |
Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4512 (See https://builds.apache.org/job/HBase-Trunk_matrix/4512/)
HBASE-19841 Every HTU should be local until DFS starts (mdrob: rev 99b9fff07bb2669792f9c1c8a796605971d02592)
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestCase.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverStacking.java
- (edit) hbase-common/src/test/java/org/apache/hadoop/hbase/HBaseCommonTestingUtility.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHStore.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/wal/WALPerformanceEvaluation.java
- (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/util/CommonFSUtils.java
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/asyncfs/TestLocalAsyncOutput.java
- (edit) hbase-server/src/test/resources/hbase-site.xml
- (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverScannerOpenHook.java
Reopening. Breaks launching of MR jobs on a cluster. Here is what a good launch looks like:
... 18/02/05 17:11:33 INFO impl.YarnClientImpl: Submitted application application_1517369646236_0009 18/02/05 17:11:33 INFO mapreduce.Job: The url to track the job: http://ve0524.halxg.cloudera.com:10134/proxy/application_1517369646236_0009/ 18/02/05 17:11:33 INFO mapreduce.Job: Running job: job_1517369646236_0009 18/02/05 17:11:40 INFO mapreduce.Job: Job job_1517369646236_0009 running in uber mode : false 18/02/05 17:11:40 INFO mapreduce.Job: map 0% reduce 0% 18/02/05 17:11:57 INFO mapreduce.Job: map 14% reduce 0% ...
... but now it does this....
18/02/05 17:17:54 INFO mapreduce.Job: The url to track the job: http://ve0524.halxg.cloudera.com:10134/proxy/application_1517369646236_0011/ 18/02/05 17:17:54 INFO mapreduce.Job: Running job: job_1517369646236_0011 18/02/05 17:17:56 INFO mapreduce.Job: Job job_1517369646236_0011 running in uber mode : false 18/02/05 17:17:56 INFO mapreduce.Job: map 0% reduce 0% 18/02/05 17:17:56 INFO mapreduce.Job: Job job_1517369646236_0011 failed with state FAILED due to: Application application_1517369646236_0011 failed 2 times due to AM Container for appattempt_1517369646236_0011_000002 exited with exitCode: -1000 Failing this attempt.Diagnostics: File file:/tmp/stack/.staging/job_1517369646236_0011/job.splitmetainfo does not exist java.io.FileNotFoundException: File file:/tmp/stack/.staging/job_1517369646236_0011/job.splitmetainfo does not exist at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:635) at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:861) at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:625) at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:442) at org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:253) at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:63) at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:361) at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:359) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836) at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:358) at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:62) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) For more detailed output, check the application tracking page: http://ve0524.halxg.cloudera.com:8188/applicationhistory/app/application_1517369646236_0011 Then click on links to logs of each attempt. . Failing the application. 18/02/05 17:17:56 INFO mapreduce.Job: Counters: 0
If I revert this patch, the submit runs again.
I'd made the staging dir /tmp/stack and seemed to get further... The job staging was made in the local fs... but it seems like we are then looking for it up in hdfs. My guess is our stamping the fs as local until minihdfscluster starts works for the unit test case but it messes up the inference of fs that allows the above submission to work.
I'd like to revert this if thats ok.
Thanks for reporting, stack. I'm confused as to how this related though... What are we doing with HTU during job submission? Seems completely orthogonal to me.
(Took a while to track down...). The job that fails is the launch of ITBLL Generate. IntegrationTestingUtility subclasses HTU. There is then some interesting dancing around this ITU with Configuration objects. If an existing Configuration, we'll use it when we create an ITU. It will have read in the config files and have HDFS as fs set. Along comes this patch and in the construction of ITU which calls through to an HTU Constructor, we stamp local fs as our fs
351 // Every cluster is a local cluster until we start DFS
352 // Note that conf could be null, but this.conf will not be
353 String dataTestDir = getDataTestDir().toString();
354 this.conf.set("fs.defaultFS","file:///");
355 this.conf.set(HConstants.HBASE_DIR, "file://" + dataTestDir);
356 LOG.debug("Setting {} to {}", HConstants.HBASE_DIR, dataTestDir);
357 this.conf.setBoolean(CommonFSUtils.UNSAFE_STREAM_CAPABILITY_ENFORCE,false);
The launch gets confused on which fs to use.
Ok, I understand what you're describing.
Integration tests can run either with a minicluster (which I think will be fine, since it starts MiniDFS) or with a distributed cluster. I'm thinking that we need to set the defaultFS back to hdfs in the ITU.createDistributedCluster call, question is, can we get that info back out?
I'd rather not revert this, as I think there is some gymnastics we can do to make ITU work and I feel like the changes are a huge improvement to HTU. Let me try some testing locally and get back to you today. Sorry for causing you troubles..
I'd rather not revert this, as I think there is some gymnastics we can do to make ITU work and I feel like the changes are a huge improvement to HTU.
Agree. Revert suggestion was made before analysis on what was causing the job launch fail. It is specific to IT tests and yeah, a bit of hackery should do it (and I like the HTU changes).
Agree. Revert suggestion was made before analysis on what was causing the job launch fail. It is specific to IT tests and yeah, a bit of hackery should do it (and I like the HTU changes).
I'll close this one again then and we can discuss in a follow on. HBASE-19947
Why is it not being picked-up? The HBaseConfiguration creation does the one-liner you are adding here.