JDBCTokenStateService is not HA-compatible

In case of Knox HA deployments, the JDBC token state service cannot guarantee that expiration time and metadata-related information (e.g. the enabled flag) is up-to-date.

For instance:

1. a token is created on node 1 -> the in-memory storage in DefaultTokenStateService will have all information and the DB will also contain everything properly
2. the token is used on node 2 for authentication purposes -> since token metadata is not yet available in-memory then first we'll look-up the missing piece of information in the DB and then update the in-memory cache in DefaultTokenStateService
3. the token disable request arrives on node 1 -> the in-memory storage in DefaultTokenStateService will be updated and the DB will also contain everything properly. Please note, at this time the token is disabled
4. the token is used on node 2 for authentication purposes -> since the in-memory cache already has metadata information about this token, the DB will not be checked -> the token is considered enabled

I did research and found out that we could use PostgreSQL's NOTIFY and LISTEN mechanism to implement the observer pattern on our end. Unfortunately, this only works for Postgres.

Instead of going down that way, I'd make the JDBC token state service DB vendor-independent by skipping the in-memory lookup for data that can be changed:

• expiration time
• metadata

We may have to think about adding connection pooling as well.