I have two fess instance work on one elasticsearch cluster,
I want to crawling data into different index in the same time like:
FessA >> IndexA
FessB >> IndexB
I change fess_config.properties as below:
index.document.update.index=alias1 (binding with IndexA)
index.document.update.index=alias2 (binding with IndexB)
and fess.update is binding with IndexC.
But when I restart fess sever and do some crawling job,I see all the data is in IndexC.
I want to know is it possible to crawling data into different index in the same time with two instance?
If it is possible,how to setting it?
more information added:
index.document.search.index for searching.
You need to change all the following settings.
Thanks your answer,
If I use one fess instance for data search only(for example: FessZ),
use FessA ,FessB for data crawling only(into IndexA & IndexB).
set fess.search binding both IndexA & IndexB,
FessZ,FessA ,FessB use the same .fess_user,fess.search in elastic search cluster,
In this way,if it is possible ,could you tell me the minimal modification to set FessA & FessB just let them crawling data in to different index?
Why do you want to split an index?
I want to crawling multiple domains.
(crawling the data of file server A into IndexA ,
and crawling the data of file server B into IndexB .)
And there are batch job modify the data in different indexes every evening,
(modify methods are different)
Thank you for the answer.
I change instance A’s setting like this:
scheduler.target.name =node4 (this is one of my es-cluster nodes)
and restart my instance,
then my setting is like below:
FESS instance A with alias fessA.update binding indexA, with scheduler that target to node4
FESS instance B with alias fess.update binding indexB, with scheduler that target to all
I run scheduler of instance A , but data is crawling into indexB(fess.update binding).
Could you tell me is there anything I missed ?
- Do not change
- Set a target name to Target
Thank you for the answer.
Do you mean we can make jobs crawling data into different indexes in parallel from multiple instances only by set the target name as a node？ I don’t understand how to set it.
In my view,if I do not change index.document.*.index,I have to use fess.update in all of my multiple Fess instance,All of my job started in one time will crawling data into the same index which fess.update is binding .
I also try to set target name as a Index(alias) name in fess_config.properties,
and edit Scheduled job’s target with the same Index(alias) name,
but it doesn’t work. crawler job still mapping with fess.update.
I stop trying to use scheduler.target.name and turn to try this:
1.modify in fess_config.properties:
2.rename json file :
then I restart fess and found an error that .fess_user185.user was not found.
I reindex an other fess instance’s .fess_user.user to .fess_user185.user and then restart successful with no error log.
Alias [fess185.search] and [fess185.update] was created and bind with fess.20191123.
I set crawler like this:
Set scheduler like this:
And the crawler log show "No crawling target urls"
I check my file path and make sure its right:
Can you give me some advise?