META robots

English:Fess
1

(from github.com/Anders-Bergqvist)
On a page we have: <meta name="ROBOTS" CONTENT="NOINDEX, FOLLOW">

The expected result would be that the crawler follows the links on the page but excludes the page itself from the index.

The Fess-crawler log says:

2018-12-21 00:54:24,541 [Crawler-20181220160844-2-1] INFO META(robots=noindex): https://www.oru.se/nyheter/nyhetsarkiv/nyhetsarkiv-2016/ 2018-12-21 00:54:31,237 [IndexUpdater] INFO Processing no docs (Doc:{access 2ms, cleanup 12ms}, Mem:{used 193MB, heap 495MB, max 1GB})

It does not follow the links. Why?

2

(from github.com/marevol)
It works. Links included in the site were crawled. FOLLOW is not logged.

3

(from github.com/MajidSafari)
hi ,

NOFOLLOW in fess working ?

4

(from github.com/marevol)
It works.