SSL/TLS with IMAP & SMTP

I can't make JAMES work with SSL/TLS configured. When I use JAMES without SSL/TLS everything works as expected, but when I switch to SSL/TLS nothing works. I am using self signed certificate which I created.

Output from the keytool:

Keystore type: JKS
Keystore provider: SUN

Your keystore contains 1 entry

Alias name: james
Creation date: Jan 6, 2021
Entry type: PrivateKeyEntry
Certificate chain length: 1
Certificate[1]:
Owner: CN=VMUbuntu, OU=me, O=org, C=HR
Issuer: CN=VMUbuntu, OU=me, O=org, C=HR
Serial number: 630c2cd7
Valid from: Wed Jan 06 15:12:47 CET 2021 until: Tue Apr 06 16:12:47 CEST 2021
Certificate fingerprints:
SHA1: ED:22:F8:A7:C4:5C:EA:C9:10:04:7C:FD:3E:CE:7E:7E:5C:CD:94:9F
SHA256: F4:9F:F5:11:1A:7B:8D:A2:A7:42:FF:5F:41:64:2B:D2:58:85:3E:11:F4:C1:82:9B:91:9A:E5:92:CA:F4:B9:1E
Signature algorithm name: SHA384withRSA
Subject Public Key Algorithm: 4096-bit RSA key
Version: 3

Extensions:

#1: ObjectId: 2.5.29.17 Criticality=false
SubjectAlternativeName [
IPAddress: 127.0.0.1
]

#2: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: ED 16 4A 36 E6 DA 28 3A F1 DB A9 A0 5A 24 21 A2 ..J6..(:....Z$!.
0010: 01 5E 78 00 .^x.
]
]

************************************************************************************

When I try to connect to smtp server from the openssl, openssl just "hangs":

OpenSSL> s_client -connect VMUbuntu:465 -starttls smtp
CONNECTED(00000003)
Can't use SSL_get_servername
depth=0 C = HR, O = org, OU = me, CN = VMUbuntu
verify error:num=18:self signed certificate
verify return:1
depth=0 C = HR, O = org, OU = me, CN = VMUbuntu
verify return:1

Thunderbird also can't connect (sending/receiving), and the wrapper.log is full of errors

************************************************************************************

and this is the error from the wrapper.log:

INFO | jvm 1 | 2021/01/06 15:18:22 | 06-Jan-2021 15:18:22.864 INFO [smtpserver-executor-16] org.apache.james.protocols.netty.BasicChannelUpstreamHandler.channelConnected:93 - Connection established from 127.0.0.1
INFO | jvm 1 | 2021/01/06 15:18:22 | 06-Jan-2021 15:18:22.878 ERROR [smtpserver-executor-22] org.apache.james.protocols.netty.BasicChannelUpstreamHandler.exceptionCaught:228 - Unable to process request
INFO | jvm 1 | 2021/01/06 15:18:22 | java.lang.NullPointerException: null
INFO | jvm 1 | 2021/01/06 15:18:22 | at org.bouncycastle.crypto.signers.PSSSigner.generateSignature(Unknown Source) ~[bcprov-jdk15on-1.62.jar:1.62.0]
INFO | jvm 1 | 2021/01/06 15:18:22 | at org.bouncycastle.jcajce.provider.asymmetric.rsa.PSSSignatureSpi.engineSign(Unknown Source) ~[bcprov-jdk15on-1.62.jar:1.62.0]
INFO | jvm 1 | 2021/01/06 15:18:22 | at java.security.Signature$Delegate.engineSign(Signature.java:1404) ~[?:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at java.security.Signature.sign(Signature.java:713) ~[?:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at sun.security.ssl.CertificateVerify$T13CertificateVerifyMessage.<init>(CertificateVerify.java:932) ~[?:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at sun.security.ssl.CertificateVerify$T13CertificateVerifyProducer.onProduceCertificateVerify(CertificateVerify.java:1106) ~[?:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at sun.security.ssl.CertificateVerify$T13CertificateVerifyProducer.produce(CertificateVerify.java:1099) ~[?:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at sun.security.ssl.SSLHandshake.produce(SSLHandshake.java:436) ~[?:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at sun.security.ssl.ClientHello$T13ClientHelloConsumer.goServerHello(ClientHello.java:1234) ~[?:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at sun.security.ssl.ClientHello$T13ClientHelloConsumer.consume(ClientHello.java:1170) ~[?:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at sun.security.ssl.ClientHello$ClientHelloConsumer.onClientHello(ClientHello.java:852) ~[?:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at sun.security.ssl.ClientHello$ClientHelloConsumer.consume(ClientHello.java:813) ~[?:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392) ~[?:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444) ~[?:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1074) ~[?:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1061) ~[?:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at java.security.AccessController.doPrivileged(Native Method) ~[?:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:1008) ~[?:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at org.jboss.netty.handler.ssl.SslHandler.runDelegatedTasks(SslHandler.java:1393) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at org.jboss.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1256) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at org.jboss.netty.handler.ssl.SslHandler.decode(SslHandler.java:852) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:425) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
INFO | jvm 1 | 2021/01/06 15:18:22 | at java.lang.Thread.run(Thread.java:834) [?:?]

************************************************************************************

INFO | jvm 1 | 2021/01/06 16:02:54 | 06-Jan-2021 16:02:54.405 ERROR [smtpserver-executor-13] org.apache.james.protocols.netty.BasicChannelUpstreamHandler.exceptionCaught:228 - Unable to process request
INFO | jvm 1 | 2021/01/06 16:02:54 | javax.net.ssl.SSLHandshakeException: Received fatal alert: bad_certificate
INFO | jvm 1 | 2021/01/06 16:02:54 | at sun.security.ssl.Alert.createSSLException(Alert.java:131) ~[?:?]
INFO | jvm 1 | 2021/01/06 16:02:54 | at sun.security.ssl.Alert.createSSLException(Alert.java:117) ~[?:?]
INFO | jvm 1 | 2021/01/06 16:02:54 | at sun.security.ssl.TransportContext.fatal(TransportContext.java:337) ~[?:?]
INFO | jvm 1 | 2021/01/06 16:02:54 | at sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:293) ~[?:?]
INFO | jvm 1 | 2021/01/06 16:02:54 | at sun.security.ssl.TransportContext.dispatch(TransportContext.java:186) ~[?:?]
INFO | jvm 1 | 2021/01/06 16:02:54 | at sun.security.ssl.SSLTransport.decode(SSLTransport.java:171) ~[?:?]
INFO | jvm 1 | 2021/01/06 16:02:54 | at sun.security.ssl.SSLEngineImpl.decode(SSLEngineImpl.java:681) ~[?:?]
INFO | jvm 1 | 2021/01/06 16:02:54 | at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:636) ~[?:?]
INFO | jvm 1 | 2021/01/06 16:02:54 | at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:454) ~[?:?]
INFO | jvm 1 | 2021/01/06 16:02:54 | at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:433) ~[?:?]
INFO | jvm 1 | 2021/01/06 16:02:54 | at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:637) ~[?:?]
INFO | jvm 1 | 2021/01/06 16:02:54 | at org.jboss.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1219) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 16:02:54 | at org.jboss.netty.handler.ssl.SslHandler.decode(SslHandler.java:852) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 16:02:54 | at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:425) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 16:02:54 | at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 16:02:54 | at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 16:02:54 | at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 16:02:54 | at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 16:02:54 | at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 16:02:54 | at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 16:02:54 | at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 16:02:54 | at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 16:02:54 | at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 16:02:54 | at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 16:02:54 | at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 16:02:54 | at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 16:02:54 | at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) ~[netty-3.10.6.Final.jar:?]
INFO | jvm 1 | 2021/01/06 16:02:54 | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
INFO | jvm 1 | 2021/01/06 16:02:54 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
INFO | jvm 1 | 2021/01/06 16:02:54 | at java.lang.Thread.run(Thread.java:834) [?:?]

************************************************************************************************************