Crawl Speed

(from github.com/charles-pinkston)
I have a site with roughly 200k pages. The crawler has not completed crawling the site after a couple of days. I have increased the number of threads on the web crawler configuration to 30, and updated the Simultaneous Crawler Config to 10. Is there anything else I can do to improve the speed of the crawler?

I am running Fess 11.4.5

(from github.com/marevol)
Try to change the number of shards for .crawler index.

(from github.com/charles-pinkston)
I’ve bumped up the shards for the .crawler index. Is there any place to make that change as part of the configuration? (Something like doc.json)

(from github.com/marevol)
It’s in fess_config.properties.

index.number_of_shards=5

(from github.com/charles-pinkston)
Thank you. That setting appears to be Fess 12 related. I’m having issues upgrading to that, but I’ve raised in in a separate ticket and will close this one.