FST Linking Engine / Linkable Token Filter should consider Chunks

The LinkableTokenFilter a Solr TokenFilter is used by the FST linking engine to add the TaggingAttribute (supported by the Solr Text Tagger library) to tokens that should be looked up in the FST - the vocabulary.

This implementation can be improved by taking chunks into consideration that are

• chunks representing named entities
• processable (typically Noun phrases but no Verb phrases ...) AND
• have a linkable token in the chunk OR
• have two or more matchable tokens in the chunk

All tokens in such chunks should be classified as tagable by setting the TaggingAttribute to true.