File Crawling no results

English:Fess
1

I’m trying to use docker-fess to deploy Fess with Elasticsearch cluster and Kibana, and use local file storage in file crawling config.

My File Crawler Settings:

Name: test
Paths: file:/rawdata/reports/test/
Included Path for Crawling: file:/rawdata/reports/test/.*
Depth: 5
The Number of Thread: 5
Interval Time: 1000ms
Boost: 1.0
Permissions: {role}guest
Status: Enabled

I put a test pdf file (which is used in fess-testdata) in a host machine folder, which mounted as /rawdata/reports/test/ folder in all running container (fess, es01, es02).

After I run the scheduler and the job finished, I try to search file name ‘test’ but find nothing, neither can I search with any other keywords.

The log with INFO level:

2020-07-29 07:50:43,610 [main] INFO  Starting Crawler..
2020-07-29 07:50:44,243 [WebFsCrawler] INFO  Created .crawler.data index.
2020-07-29 07:50:44,352 [WebFsCrawler] INFO  Created .crawler.data mapping.
2020-07-29 07:50:44,926 [WebFsCrawler] INFO  Created .crawler.queue index.
2020-07-29 07:50:45,032 [WebFsCrawler] INFO  Created .crawler.queue mapping.
2020-07-29 07:50:45,633 [WebFsCrawler] INFO  Created .crawler.filter index.
2020-07-29 07:50:45,749 [WebFsCrawler] INFO  Created .crawler.filter mapping.
2020-07-29 07:50:45,841 [WebFsCrawler] INFO  Target Path: file:/rawdata/reports/test
2020-07-29 07:50:45,843 [WebFsCrawler] INFO  Included Path: file:/rawdata/reports/test/.*
2020-07-29 07:50:55,885 [IndexUpdater] INFO  Processing no docs in indexing queue (Doc:{access 11ms}, Mem:{used 156MB, heap 376MB, max 512MB})
2020-07-29 07:51:05,867 [IndexUpdater] INFO  Processing no docs in indexing queue (Doc:{access 9ms}, Mem:{used 160MB, heap 376MB, max 512MB})
2020-07-29 07:51:15,867 [IndexUpdater] INFO  Processing no docs in indexing queue (Doc:{access 9ms}, Mem:{used 147MB, heap 376MB, max 512MB})
2020-07-29 07:51:16,205 [WebFsCrawler] INFO  [EXEC TIME] crawling time: 32533ms
2020-07-29 07:51:25,870 [IndexUpdater] INFO  Processing no docs in indexing queue (Doc:{access 10ms}, Mem:{used 147MB, heap 376MB, max 512MB})
2020-07-29 07:51:25,870 [IndexUpdater] INFO  [EXEC TIME] index update time: 56ms
2020-07-29 07:51:26,014 [main] INFO  Finished Crawler

And then I re-run the crawler with DEBUG level, the crawler log still show nothing helpful.

Here’s my fess-crawler.log

2

file:/rawdata/reports/test does not match file:/rawdata/reports/test/.* because of missing / at the end of the target path.
So, you need to set file:/rawdata/reports/test/ to Paths.