(from github.com/MajidSafari)
hi how to config , no Crawling page if noindex ???
(from github.com/MajidSafari)
i want when page has meta noindex Crawling skip page
but now in ver 10.2 not working
(from github.com/marevol)
noindex of meta tag is not supported at the moment.
Our current release supports robots.txt.
For meta tag, we will support it in the future release.
© 2020. All Rights Reserved - CodeLibs, Inc.