No synonyms

On our system (fess 13.6.0), System > Dictionaries does not have any synonym entry:

What do I need to do in order to add synonyms to our system?

To upload a dictionary file, see configsync.

$ curl -XPOST -H 'Content-Type:application/json' localhost:9200/_configsync/file?path=en/synonym.txt --data-binary @synonym.txt

and then update english_analyzer.

@shinsuke thank you.

What format does the dictionary file need to be in for synonyms? Do you have an example?

To update english_analyzer, what do I add? Do I add synonym to the filter list or synonym_filter or something else?

Thanks.

To edit analysis settings, knowledge of Text Analysis is required.

I don’t understand. Your original reply said that I had to do two things:

  1. Upload a dictionary file and
  2. Update english_analyzer

I think that all I need to be told is:

  1. What format the dictionary file needs to be in, preferably with an example and
  2. What changes I need to make to english_analyzer.

The documentation says that synonyms are supported but fess doesn’t have any built in, so I’m struggling to get that feature working.

Thank you.

As for English, you can edit synonym.txt.
If you want to add it to english_analyzer, 1 and 2 are needed.

The synonym information is included in the text analysis.
So, you can add a synonym graph token filter to filter, and then also add the name to filter of english_analyzer.

If you need more supports, please contact Commercial Support .

@shinsuke Thank you for the additional information.

It looks like fess supports the Solr format of synonyms (https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-synonym-graph-tokenfilter.html#_solr_synonyms_2) and I’ve successfully added a synonym file to Elasticsearch. I’ve also added the synonym filter to english_analyzer.

One final question: when the synonym list changes, do I have to wait for fess to recrawl everything or is there a way to speed up the process of getting the synonyms to work?

To apply changes on dictionary files, reopening index is needed.
To reopen an index, please click Reload button on System Info > Maintainace page.

I still don’t have synonyms working on my system :frowning:

I want to share multiple screenshots to make it clear what the system is doing at the moment but I can only post one image at a time, so it will require multiple posts, I’m afraid.

I’ve got the synonyms defined:

Here are the actual synonyms:

If I search using Bennée, it finds 29 results:
image

If I search using Bennee, it finds 6 results:
image

We want to use synonyms to handle the wider differences in US and UK language (e.g. virtualise vs virtualize) but using Bennee and Bennée is an easy test of the system.

I would expect all of the search results to be the same, regardless of which spelling I use to search with.

What can I do next to either troubleshoot this or resolve it, please?

Thank you.

en/*.txt is for English documents.
A language which documents has is detected automatically.
You can check it on Admin Search page.

I can confirm that the first result for Bennée and the first result for Bennee are both classified as “en”.

Did you delete docs in the index and crawl them again?

I haven’t. So far, I have followed these instructions:

I have also tried Maintenance > Reindex but that doesn’t seem to have made any difference. It is difficult to tell as I cannot find any logs or other indication that the reindexing has finished.

Should I be deleting the indexes? If so, which ones?

or

and do I use the “delete index” menu option on the Dashboard?
image

Previously, I had asked:

to which you replied

Does your question:

mean that if the synonyms change but the documents don’t, the documents have got to be removed from the index (somehow) and re-crawled?

Hopefully your answers to my screenshots will help me to better understand how to get this working.

Thank you.

I’ve got it working.

I made a mistake in the modifications to the fess.json file. I thought that because there was already a synonym filter there, I could point the english analyzer at that, but that then wasn’t picking up the english synonym file.

Thank you for your assistance.