Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
Impala 4.1.1
-
None
-
**Host:**
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.1", GitCommit:"632ed300f2c34f6d6d15ca4cef3d3c7073412212", GitTreeState:"clean", BuildDate:"2021-08-19T15:45:37Z", GoVersion:"go1.16.7", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.6", GitCommit:"f59f5c2fda36e4036b49ec027e556a15456108f0", GitTreeState:"clean", BuildDate:"2022-01-19T17:26:47Z", GoVersion:"go1.16.12", Compiler:"gc", Platform:"linux/amd64"}
**Container:**
impala@coordinator:/opt/impala$ cat /etc/os-release
NAME="Ubuntu"
VERSION="18.04.6 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.6 LTS"
The container image was built from source on Ubuntu 18.04 LTS with the following commands:
./buildall.sh -release -ninja -notests
ninja docker_images
**Host:** $ kubectl version Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.1", GitCommit:"632ed300f2c34f6d6d15ca4cef3d3c7073412212", GitTreeState:"clean", BuildDate:"2021-08-19T15:45:37Z", GoVersion:"go1.16.7", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.6", GitCommit:"f59f5c2fda36e4036b49ec027e556a15456108f0", GitTreeState:"clean", BuildDate:"2022-01-19T17:26:47Z", GoVersion:"go1.16.12", Compiler:"gc", Platform:"linux/amd64"} **Container:** impala@coordinator :/opt/impala$ cat /etc/os-release NAME="Ubuntu" VERSION="18.04.6 LTS (Bionic Beaver)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 18.04.6 LTS" The container image was built from source on Ubuntu 18.04 LTS with the following commands: ./buildall.sh -release -ninja -notests ninja docker_images
-
ghx-label-12
Description
I am trying to set up a Kerberized Impala cluster in Kubernetes (K8S). In K8S deployments, hostnames of pods (containers) are unknown in advance (random suffix), therefore service principals (SPN) have to be used, which are fixed ahead. Impala has a `-hostname` flag that could be used for this, but it seems, that it's not respected by Kudu RPC (KRPC) and it's still using the original hostname returned by the system. Therefore it won't look for the right principal, and won't find a matching keytab entry, and finally it will fail to set up a KRPC connection.
See the stack trace for the error message:
I1125 11:22:12.005645 2949 exec-env.cc:483] Starting KRPC service E1125 11:22:12.006127 2949 authentication.cc:239] (stacktrace: @ 0x116cd54 impala::SaslLogCallback() @ 0x7f173c893a71 sasl_seterror @ 0x7f1737499024 (unknown) @ 0x7f173749b9ae (unknown) @ 0x7f173c89255d sasl_server_step @ 0x7f173c892b17 sasl_server_start @ 0x1a73486 kudu::rpc::WrapSaslCall() @ 0x1a78b78 kudu::rpc::ServerNegotiation::PreflightCheckGSSAPI() @ 0x1a54cb7 kudu::rpc::Messenger::AddAcceptorPool() @ 0x11a1fd0 impala::RpcMgr::StartServices() @ 0x11e0ded impala::ExecEnv::StartKrpcService() @ 0x141e3d5 impala::ImpalaServer::Start() @ 0x1402ada ImpaladMain() @ 0xbdbf7a main @ 0x7f17392fbc86 __libc_start_main @ 0xc82279 _start ) SASL message (General): GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (No key table entry found matching impala/coordinator.impala.hadoop.svc.cluster.local@) E1125 11:22:12.045207 2949 impalad-main.cc:90] Impalad services did not start correctly, exiting. Error: Failed to add acceptor pool: Runtime error: GSSAPI/Kerberos not properly configured: No key table entry found matching impala/coordinator.impala.hadoop.svc.cluster.local@
In the case above, the hostname for the service pricipal (SPN) is "impala-coordinator.hadoop.svc.cluster.local", and the pod's FQDN is "coordinator.impala.hadoop.svc.cluster.local".