OS：Winddows Server 2012 64ビット
メモリ：4G
Fess：13.6.3
elasticsearch：7.6.2
JDK：14
■クローラー
・ファイルシステム
パス：smbで別途ファイルサーバー指定
クロール対象から除外するパス：exe等もろもろ
深さ：指定なし
→指定先のディレクトリの階層はかなり上位
ディレクトリの深さはかなり深い
ファイル数、数十万件
最大アクセス数：6000
間隔：1000 ミリ秒
数十万件のディレクトリに1回でクロールを掛けると、out o fmemoryが発生したため
上記のように最大アクセスを6000にして、スケジューラで30分毎に起動するよう設定しました。
途中までうまくいっていたようですが、ある時点からエラーが出続けるようになりました。
具体的には以下のエラーが発生します。
■fess-crawler.log
2020-05-12 07:30:20,043 [main] ERROR Crawler does not work correctly.
org.lastaflute.di.exception.ContainerInitFailureException: Look! Read the message below.
－－－－－－－－－－－－－－－－－－－－－－－－－
Failed to initialize the container.[Path]
esclient.xml[Namespace]
null[Included by]
esflute_config.xml
esflute_user.xml
esflute_log.xml
－－－－－－－－－－－－－－－－－－－－－－－－－
at org.lastaflute.di.core.meta.impl.LaContainerImpl.throwContainerInitFailureException(LaContainerImpl.java:463) ~[lasta-di-0.8.2.jar:?]
at org.lastaflute.di.core.meta.impl.LaContainerImpl.init(LaContainerImpl.java:415) ~[lasta-di-0.8.2.jar:?]
at org.lastaflute.di.core.meta.impl.LaContainerImpl.doInit(LaContainerImpl.java:431) ~[lasta-di-0.8.2.jar:?]
at org.lastaflute.di.core.meta.impl.LaContainerImpl.init(LaContainerImpl.java:413) ~[lasta-di-0.8.2.jar:?]
at org.lastaflute.di.core.meta.impl.LaContainerImpl.doInit(LaContainerImpl.java:431) ~[lasta-di-0.8.2.jar:?]
at org.lastaflute.di.core.meta.impl.LaContainerImpl.init(LaContainerImpl.java:413) ~[lasta-di-0.8.2.jar:?]
at org.lastaflute.di.core.meta.impl.LaContainerImpl.doInit(LaContainerImpl.java:431) ~[lasta-di-0.8.2.jar:?]
at org.lastaflute.di.core.meta.impl.LaContainerImpl.init(LaContainerImpl.java:413) ~[lasta-di-0.8.2.jar:?]
at org.lastaflute.di.core.factory.SingletonLaContainerFactory.init(SingletonLaContainerFactory.java:66) ~[lasta-di-0.8.2.jar:?]
at org.codelibs.fess.exec.Crawler.main(Crawler.java:221) [classes/:?]
Caused by: org.lastaflute.di.core.exception.IllegalMethodRuntimeException: [ESSR0060]Can not configure method(open) of org.codelibs.fess.es.client.FessEsClient, because [fess] IndexNotFoundException[no such index [fess]]
at org.lastaflute.di.core.assembler.AbstractMethodAssembler.invoke(AbstractMethodAssembler.java:61) ~[lasta-di-0.8.2.jar:?]
at org.lastaflute.di.core.assembler.DefaultInitMethodAssembler.assemble(DefaultInitMethodAssembler.java:40) ~[lasta-di-0.8.2.jar:?]
at org.lastaflute.di.core.deployer.SingletonComponentDeployer.assemble(SingletonComponentDeployer.java:57) ~[lasta-di-0.8.2.jar:?]
at org.lastaflute.di.core.deployer.SingletonComponentDeployer.deploy(SingletonComponentDeployer.java:41) ~[lasta-di-0.8.2.jar:?]
at org.lastaflute.di.core.deployer.SingletonComponentDeployer.init(SingletonComponentDeployer.java:35) ~[lasta-di-0.8.2.jar:?]
at org.lastaflute.di.core.meta.impl.ComponentDefImpl.init(ComponentDefImpl.java:85) ~[lasta-di-0.8.2.jar:?]
at org.lastaflute.di.core.meta.impl.LaContainerImpl.doInit(LaContainerImpl.java:434) ~[lasta-di-0.8.2.jar:?]
at org.lastaflute.di.core.meta.impl.LaContainerImpl.init(LaContainerImpl.java:413) ~[lasta-di-0.8.2.jar:?]
… 8 more
Caused by: org.elasticsearch.index.IndexNotFoundException: no such index [fess]
at org.codelibs.elasticsearch.client.action.HttpGetMappingsAction.lambda$execute$0(HttpGetMappingsAction.java:47) ~[elasticsearch-httpclient-7.6.2.jar:?]
at org.codelibs.curl.CurlRequest.lambda$execute$4(CurlRequest.java:220) ~[curl4j-1.2.4.jar:?]
at org.codelibs.curl.CurlRequest.lambda$connect$3(CurlRequest.java:199) ~[curl4j-1.2.4.jar:?]
at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1429) ~[?:?]
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) ~[?:?]
at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1016) ~[?:?]
at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1665) ~[?:?]
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1598) ~[?:?]
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177) ~[?:?]
2020-05-12 07:30:20,067 [main] INFO Destroyed LaContainer.
■fess.log
2020-05-12 07:30:20,161 [job_default_crawler] WARN Failed to execute default_crawler: return container.getComponent(“crawlJob”).logLevel(“info”).gcLogging().execute(executor);
org.codelibs.fess.exception.JobProcessingException: Exit Code: 1
Output:at org.codelibs.fess.job.CrawlJob.executeCrawler(CrawlJob.java:357) ~[classes/:?]
at org.codelibs.fess.job.CrawlJob.execute(CrawlJob.java:152) ~[classes/:?]
at org.codelibs.fess.job.ExecJob.execute(ExecJob.java:65) ~[classes/:?]
at org.codelibs.fess.job.ExecJob$execute$1.call(Unknown Source) ~[?:?]
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47) ~[groovy-3.0.0.jar:3.0.0]
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:115) ~[groovy-3.0.0.jar:3.0.0]
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:127) ~[groovy-3.0.0.jar:3.0.0]
at Script1.run(Script1.groovy:1) ~[?:?]
at groovy.lang.GroovyShell.evaluate(GroovyShell.java:431) ~[groovy-3.0.0.jar:3.0.0]
at groovy.lang.GroovyShell.evaluate(GroovyShell.java:465) ~[groovy-3.0.0.jar:3.0.0]
at groovy.lang.GroovyShell.evaluate(GroovyShell.java:440) ~[groovy-3.0.0.jar:3.0.0]
at org.codelibs.fess.util.GroovyUtil.evaluate(GroovyUtil.java:42) ~[classes/:?]
at org.codelibs.fess.job.impl.GroovyExecutor.execute(GroovyExecutor.java:31) ~[classes/:?]
at org.codelibs.fess.app.job.ScriptExecutorJob.process(ScriptExecutorJob.java:101) [classes/:?]
at org.codelibs.fess.app.job.ScriptExecutorJob.run(ScriptExecutorJob.java:42) [classes/:?]
at org.lastaflute.job.LaJobRunner.actuallyRun(LaJobRunner.java:265) [lasta-job-0.5.4.jar:?]
at org.lastaflute.job.LaJobRunner.doRun(LaJobRunner.java:232) [lasta-job-0.5.4.jar:?]
at org.lastaflute.job.LaJobRunner.run(LaJobRunner.java:182) [lasta-job-0.5.4.jar:?]
at org.lastaflute.job.cron4j.Cron4jTask.runJob(Cron4jTask.java:371) [lasta-job-0.5.4.jar:?]
at org.lastaflute.job.cron4j.Cron4jTask.actuallyExecute(Cron4jTask.java:355) [lasta-job-0.5.4.jar:?]
at org.lastaflute.job.cron4j.Cron4jTask.lambda$doExecute$6(Cron4jTask.java:258) [lasta-job-0.5.4.jar:?]
at org.lastaflute.job.cron4j.Cron4jTask.synchronizedNeighborRunning(Cron4jTask.java:328) [lasta-job-0.5.4.jar:?]
at org.lastaflute.job.cron4j.Cron4jTask.doExecute(Cron4jTask.java:246) [lasta-job-0.5.4.jar:?]
at org.lastaflute.job.cron4j.Cron4jTask.execute(Cron4jTask.java:151) [lasta-job-0.5.4.jar:?]
at it.sauronsoftware.cron4j.RomanticCron4jNativeTaskExecutor$RomanticRunner.run(RomanticCron4jNativeTaskExecutor.java:121) [lasta-job-0.5.4.jar:?]
at java.lang.Thread.run(Thread.java:832) [?:?]
このエラーは何が原因と考えられるでしょうか。
ちなみにElasticsearchの方にはエラーログは出力されておりませんでした。
また、このエラーが発生した後にFessとElasticのサービスを落として、再度起動すると
Fessを開くことが出来なくなり、
org.elasticsearch.index.IndexNotFoundException: no such index [fess]
のエラーが出続けます。
これまで2，3回ほど同じ症状となっており、あまり良くないかもしれませんが、
Elasticsearch側の「elasticsearch-7.6.2\data」を丸ごと移動させることで、解消しております。