(from github.com/jay-cyrb)
I download newest FESS version 12.4.0 and run it in Windows 8.1 x64. I switch on “Check Last Modified” paramenter on General configuration page. I found issue - crawler always crawl ALL files in destination folder despite the files modification date.
If I use FESS 11.2.1 and second time run crawler I have no problems. fess-crawler.log contains:
2018-12-14 22:46:10,042 [Crawler-AWes9U2_kL2BkAhv803W-1-1] INFO Crawling URL: file:/D:/dd/%D0%92%D0%BE%D0%BF%D1%80%D0%BE%D1%81%20%20___.rtf
2018-12-14 22:46:10,069 [Crawler-AWes9U2_kL2BkAhv803W-1-1] INFO Not modified URL: file:/D:/dd/%D0%92%D0%BE%D0%BF%D1%80%D0%BE%D1%81%20%20___.rtf
2018-12-14 22:46:18,934 [IndexUpdater] INFO Processing 1/1 docs (Doc:{access 11ms}, Mem:{used 141MB, heap 227MB, max 494MB})
2018-12-14 22:46:18,955 [IndexUpdater] INFO Processing no docs (Doc:{access 2ms, cleanup 19ms}, Mem:{used 141MB, heap 227MB, max 494MB})
But if I use FESS 12.4.0 and second time run crawler fess-crawler.log contains:
2018-12-14 22:23:47,448 [Crawler-wozerGcB-e1p5mRFLkFB-1-4] INFO Crawling URL: file:/D:/dd/%D0%92%D0%BE%D0%BF%D1%80%D0%BE%D1%81%20%20___.rtf
2018-12-14 22:23:47,466 [Crawler-wozerGcB-e1p5mRFLkFB-1-4] WARN Unknown FileOwnerAttributeView: sun.nio.fs.FileOwnerAttributeViewImpl@d5e6e15
2018-12-14 22:23:56,320 [IndexUpdater] INFO Processing 1/1 docs (Doc:{access 9ms}, Mem:{used 151MB, heap 220MB, max 494MB})
2018-12-14 22:23:56,376 [IndexUpdater] INFO Processing no docs (Doc:{access 2ms, cleanup 22ms}, Mem:{used 152MB, heap 220MB, max 494MB})
2018-12-14 22:23:56,442 [IndexUpdater] INFO Sent 1 docs (Doc:{process 32ms, send 66ms, size 132KB}, Mem:{used 155MB, heap 220MB, max 494MB})
2018-12-14 22:24:06,306 [IndexUpdater] INFO Processing no docs (Doc:{access 4ms, cleanup 22ms}, Mem:{used 155MB, heap 220MB, max 494MB})
I reproduced this Issue many times. If use FESS 12.4.0 crawler always update almost all document fields including “content”, “created”. The crawled file did not change.