[HADOOP-14445] Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances - ASF JIRA

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: 2.8.0, 3.0.0-alpha1
Fix Version/s: 3.2.0, 3.0.4, 3.1.2
Component/s: kms
Labels:
None
Environment:

CDH5.7.4, Kerberized, SSL, KMS-HA, at rest encryption

Target Version/s:

2.8.4, 3.2.0
Hadoop Flags:

Reviewed
Release Note:

Hide


This patch improves the KMS delegation token issuing and authentication logic, to enable tokens to authenticate with a set of KMS servers. The change is backport compatible, in that it keeps the existing authentication logic as a fall back.

Historically, KMS delegation tokens have ip:port as service, making KMS clients only able to use the token to authenticate with the KMS server specified as ip:port, even though the token is shared among all KMS servers at server-side. After this patch, newly created tokens will have the KMS URL as service.

A `DelegationTokenIssuer` interface is introduced for token creation.

Show
 This patch improves the KMS delegation token issuing and authentication logic, to enable tokens to authenticate with a set of KMS servers. The change is backport compatible, in that it keeps the existing authentication logic as a fall back. Historically, KMS delegation tokens have ip:port as service, making KMS clients only able to use the token to authenticate with the KMS server specified as ip:port, even though the token is shared among all KMS servers at server-side. After this patch, newly created tokens will have the KMS URL as service. A `DelegationTokenIssuer` interface is introduced for token creation.

Description

As discovered in ~~HADOOP-14441~~, KMS HA using LoadBalancingKMSClientProvider do not share delegation tokens. (a client uses KMS address/port as the key for delegation token)

DelegationTokenAuthenticatedURL#openConnection

if (!creds.getAllTokens().isEmpty()) {
        InetSocketAddress serviceAddr = new InetSocketAddress(url.getHost(),
            url.getPort());
        Text service = SecurityUtil.buildTokenService(serviceAddr);
        dToken = creds.getToken(service);

But KMS doc states:

Delegation Tokens

Similar to HTTP authentication, KMS uses Hadoop Authentication for delegation tokens too.

Under HA, A KMS instance must verify the delegation token given by another KMS instance, by checking the shared secret used to sign the delegation token. To do this, all KMS instances must be able to retrieve the shared secret from ZooKeeper.

We should either update the KMS documentation, or fix this code to share delegation tokens.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

HADOOP-14445-branch-2.8.patch
26/May/17 20:43
31 kB
Rushabh Shah
HADOOP-14445-branch-2.8.002.patch
26/Dec/17 03:38
38 kB
Rushabh Shah
HADOOP-14445.revert.patch
07/May/18 18:28
86 kB
Xiao Chen
HADOOP-14445.compat.patch
24/Aug/18 19:16
84 kB
Daryn Sharp
HADOOP-14445.branch-3.0.001.patch
12/Oct/18 16:45
88 kB
Xiao Chen
HADOOP-14445.branch-2.8.revert.patch
07/May/18 18:29
85 kB
Xiao Chen
HADOOP-14445.branch-2.8.006.patch
10/Apr/18 06:35
82 kB
Xiao Chen
HADOOP-14445.branch-2.8.005.patch
10/Apr/18 05:09
82 kB
Xiao Chen
HADOOP-14445.branch-2.8.004.patch
10/Apr/18 03:41
82 kB
Xiao Chen
HADOOP-14445.branch-2.8.003.patch
09/Apr/18 22:44
82 kB
Xiao Chen
HADOOP-14445.branch-2.06.patch
10/Apr/18 16:32
82 kB
Xiao Chen
HADOOP-14445.branch-2.05.patch
10/Apr/18 04:51
82 kB
Xiao Chen
HADOOP-14445.branch-2.04.patch
10/Apr/18 03:38
82 kB
Xiao Chen
HADOOP-14445.branch-2.03.patch
09/Apr/18 21:56
83 kB
Xiao Chen
HADOOP-14445.branch-2.02.patch
09/Apr/18 17:41
82 kB
Xiao Chen
HADOOP-14445.branch-2.01.patch
05/Apr/18 22:26
82 kB
Xiao Chen
HADOOP-14445.branch-2.001.precommit.patch
09/Apr/18 20:54
0.4 kB
Xiao Chen
HADOOP-14445.branch-2.000.precommit.patch
09/Apr/18 19:58
0.4 kB
Xiao Chen
HADOOP-14445.addemdum.patch
15/Oct/18 03:27
12 kB
Xiao Chen
HADOOP-14445.20.patch
12/Oct/18 02:06
87 kB
Xiao Chen
HADOOP-14445.19.patch
09/Oct/18 17:50
87 kB
Xiao Chen
HADOOP-14445.18.patch
06/Sep/18 21:15
87 kB
Xiao Chen
HADOOP-14445.17.patch
06/Sep/18 04:45
87 kB
Xiao Chen
HADOOP-14445.16.patch
31/Jul/18 05:44
64 kB
Xiao Chen
HADOOP-14445.15.patch
26/Jul/18 03:35
64 kB
Xiao Chen
HADOOP-14445.14.patch
19/Jun/18 17:33
64 kB
Xiao Chen
HADOOP-14445.13.patch
09/Apr/18 17:35
84 kB
Xiao Chen
HADOOP-14445.12.patch
05/Apr/18 22:30
84 kB
Xiao Chen
HADOOP-14445.11.patch
02/Apr/18 23:13
78 kB
Xiao Chen
HADOOP-14445.10.patch
29/Mar/18 18:15
78 kB
Xiao Chen
HADOOP-14445.09.patch
27/Mar/18 05:43
81 kB
Xiao Chen
HADOOP-14445.08.patch
23/Mar/18 05:58
81 kB
Xiao Chen
HADOOP-14445.07.patch
14/Mar/18 22:59
71 kB
Xiao Chen
HADOOP-14445.06.patch
09/Mar/18 07:04
72 kB
Xiao Chen
HADOOP-14445.05.patch
08/Mar/18 07:51
73 kB
Xiao Chen
HADOOP-14445.004.patch
07/Mar/18 07:55
71 kB
Xiao Chen
HADOOP-14445.003.patch
10/Jan/18 00:52
52 kB
Rushabh Shah
HADOOP-14445.002.patch
26/Dec/17 21:46
38 kB
Rushabh Shah

Issue Links

breaks

HADOOP-15408 HADOOP-14445 broke Spark.

Resolved

HADOOP-15431 KMSTokenRenewer should work with KMS_DELEGATION_TOKEN which has ip:port as service

Resolved

HADOOP-15861 Move DelegationTokenIssuer to the right path

Resolved

HADOOP-15997 KMS client uses wrong UGI after HADOOP-14445

Resolved

HADOOP-16199 KMSLoadBlanceClientProvider does not select token correctly

Resolved

HDFS-13430 Fix TestEncryptionZonesWithKMS failure due to HADOOP-14445

Resolved

is duplicated by

HADOOP-14134 KMSTokenRenewer should renew token from the service address in token.

Resolved

HADOOP-14441 LoadBalancingKMSClientProvider#addDelegationTokens should add delegation tokens from all KMS instances

Resolved

relates to

HADOOP-15414 Job submit not work well on HDFS Federation with Transparent Encryption feature

Resolved

(1 breaks, 2 is duplicated by, 1 relates to)

Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances

Details

Description

Attachments

Attachments

Issue Links

Activity

People

Dates