(from github.com/charles-pinkston)
I have a site with roughly 200k pages. The crawler has not completed crawling the site after a couple of days. I have increased the number of threads on the web crawler configuration to 30, and updated the Simultaneous Crawler Config to 10. Is there anything else I can do to improve the speed of the crawler?
I am running Fess 11.4.5
(from github.com/marevol)
Try to change the number of shards for .crawler index.
(from github.com/charles-pinkston)
I’ve bumped up the shards for the .crawler index. Is there any place to make that change as part of the configuration? (Something like doc.json)
(from github.com/marevol)
It’s in fess_config.properties.
index.number_of_shards=5
(from github.com/charles-pinkston)
Thank you. That setting appears to be Fess 12 related. I’m having issues upgrading to that, but I’ve raised in in a separate ticket and will close this one.