Downloadable incremental repository indexes and common embedder API

Index I build for ibiblio is nearly 12Mb (with class names) and it is not really good idea to download whole index on every update.

We discussed this with jason and agreed on the following structure.

– since lucene index is actually a folder it would make sense to have zip it
– we need some metadata in a common repository place. That metadata will enumerate all existing index zips, their type (full or incremental) and also specify date interval for incremental indexes
– full index for those who don't have anything yet
– incremental indexes for last N months, M last weeks and K last days.

We also need this stuff exposed trough Embedder API, e.g. get index updates for given repository and put them into given location (or perhaps somewhere under local repository) and probably merge incremental chunks into the local index copy

This is quite critical since there is no direct access to centrl repository and local indexer can't be used...