FESS - How to Crawl 200 MB .txt file?

English:Fess
1

Hi team,

Is the crawler capable of crawling a 100/200mb .txt file? is it possible to upgrade the crawler? or it is a hard system limitation.

Currently it displays an error - org.codelibs.fess.crawler.exception.MaxLengthExceededException: The content length (32636005 byte) is over 10485760 byte.

What configuration is needed to change? Thank you!

2

This limit is set in the contentlength.xml file, which contains a defaultMaxLength setting. By modifying this setting, you can increase the maximum allowable file size that the crawler can process.

3

Thanks Shinsuke - I was able to index the document, however there was an error when I tried to open it.

Document was only 30mb - does it not open in the browser?

image
image729×97 2.65 KB

action:ACCESS user:admin path:/admin/failureurl/ execute:index ip:0:0:0:0:0:0:0:1 time:2024-05-02T09:41:46.506123100Z

4

I think an exception is logged in the fess.log file.