Hi, I’ve installed FESS to index a web site. This web site is a software documentation web site with table of content on the left and text on the right. We do not use IFRAME, we inject the html into a div dynamically based on an ID in the URL. When I add the root url to FESS crawlers, it browses the web site but include in all the pages and index the table of content. So if we search for one work of the table of content, we retrieve all the pages of the web site. I’ve done the test crawling the mysql developer web site (https://dev.mysql.com/doc/refman/5.7/en/) and the behavior is not the same. The table of content is not parsed and indexed. What is the trick ? Do I have to use specific class name in the elements I want to ignore ?
No trick. I checked it but the toc of MySQL site was indexed.
Although Fess is able to ignore the specific elements by specifying it in a configuration file, it is not documented in English at the moment…
Is it linked to the HTMLMapper of Tika ?
No. nekohtml is used.
Hi, I have a similar problem.
I am indexing a website with a common header and footer in all of the pages, when I search for a word that is included on the header the results are not good.
Can I config the crawler for ignoring the header a footer html tags?
To ignore tags by name, modify the following value in fess_config.properties
Thanks @marevol !!
And to ignore a div with a concrete id or class? Is it just as easy?
id/class attributes are not supported at the moment.
I think that it’s better to specify indexed content by XPATH in fess_config.properties as below.