I’d like to use Fess as a component in a RAG system. I’ve installed fess-webapp-semantic-search and played with it a bit, and it looks like it generates embeddings for the whole document. For long PDFs, it doesn’t really match the pertinent points in the document. Is it possible to configure the crawler/indexer to chunk the text while maintaining metadata pointing to the original doc?
In a future release, text chunking will be supported.