Search failed in datastore

(from github.com/luigigf49)
Hi all,
I have configured the crawling for some folders and search works correctly.
Then I added the crawling of an Oracle table called CO_CLIFO:

PARAMETER
driver = oracle.jdbc.OracleDriver
url = jdbc: oracle: thin: @localhost: 1521: ORA9ZV
username = “AZ09”
password = “AZ09”
sql = select * from CO_CLIFO

SCRIPT
name = cliforagi1
address = clifoindir
location = clifolocal

Crawling of datastore ends succesfully.

I then tried to do a search in different ways (CORRIDONIA is the name of a city in the clifolocal field):

  • CORRIDONIA
    or:
  • location: CORRIDONIA

but I find only some documents contained in the folders and no reference to the records of the CO_CLIFO table.

Many thanks
Luigi

(from github.com/marevol)
A type of location field is geo_point.
See doc.json.

(from github.com/luigigf49)
Hi marevol,
Thank you for your help but I still have problems (certainly due to my lack of knowledge of the instrument).
Now my script is:
content = clifolocal
digest = clifolocal
My searches however do not work.
Is there a way to see the contents of the indexes that were created for the datastore by the crawler?

Many thanks
Luigi

(from github.com/luigigf49)
Hi marevol,
Thank you for your help but I still have problems (certainly due to my lack of knowledge of the instrument).
Now my script is:
content = clifolocal
digest = clifolocal
My searches however do not work.
Is there a way to see the contents of the indexes that were created for the datastore by the crawler?

Many thanks
Luigi

(from github.com/marevol)
See Admin Search.

(from github.com/luigigf49)
Already tried without success.

(from github.com/marevol)
Are these values indexed?
Did you check log files?
Need more info…

(from github.com/luigigf49)
Hi marevol,
this is a sample message generated by the datastore crawler:

ID 3W3mtWMB4hOHaSFZhk0j

select * from CO_CLIFO:1360
L1-HsmMByoyqlpSzApha-1
org.codelibs.fess.exception.DataStoreException
org.codelibs.fess.exception.DataStoreException: url is null. dataMap={expires=Sun Jun 03 12:57:43 CEST 2018, role=[Rguest], config_id=DL1-HsmMByoyqlpSzApha, created=Thu May 31 12:57:45 CEST 2018, segment=L1-HsmMByoyqlpSzApha, boost=9.9999998E17, mimetype=application/datastore, label=[]}at org.codelibs.fess.ds.impl.IndexUpdateCallbackImpl.store(IndexUpdateCallbackImpl.java:69)at org.codelibs.fess.ds.impl.DatabaseDataStoreImpl.storeData(DatabaseDataStoreImpl.java:122)at org.codelibs.fess.ds.impl.AbstractDataStoreImpl.store(AbstractDataStoreImpl.java:106)at org.codelibs.fess.helper.DataIndexHelper$DataCrawlingThread.process(DataIndexHelper.java:236)at org.codelibs.fess.helper.DataIndexHelper$DataCrawlingThread.run(DataIndexHelper.java:222)
1
1527765173794

(from github.com/marevol)
?

(from github.com/luigigf49)
Hi marevol,
on the page ‘Failure URL’ for each record to crawl (1360 records/1360 messages) there is a message like this:
org.codelibs.fess.exception.DataStoreException: url is null. dataMap={expires=Sun Jun 03 16:10:23 CEST 2018, role=[Rguest], config_id=DL1-

Many thanks

(from github.com/luigigf49)
Hi marevol,

attached a document that contains some information about my problem.

Many thanks

DataStoreCrawlingProblem.zip

(from github.com/marevol)
url field is missing.
You need to put the following fields:

  • url
  • host
  • site
  • title
  • content
  • digest
  • anchor
  • content_length
  • last_modified

(from github.com/luigigf49)
Hi marevol,
thanks to your help I have made some progress and now I have no more problems like ‘Failure URL’.
However, the database search does not work yet.
In the file ‘fess-crawler.log’ I noticed some error reports that I reported in the attached document.

Many thanks
Luigi
DataStoreCrawlingProblem_2.zip

(from marevol (Shinsuke Sugaya) · GitHub)

MissingPropertyException: No such property: cliforagi1 for class: Script1

cliforagi1 column does not exist in a result of sql.

(from github.com/luigigf49)
But cliforagi1, clifoindir etc. are all columns of table CO_CLIFO and sql command is ‘slq * from CO_CLIFO’

(from github.com/marevol)

  1. Change logLevel to debug for Default Crawler in Scheduler
  2. Run Default Crawler
  3. Check fess-crawler.log

(from github.com/luigigf49)
Default Crawler or Crawler that I defined for my DataStore ?

(from github.com/luigigf49)
Hi marevol,

fess-crawler.log contains this message for each record crawled:

2018-06-01 22:13:08,902 [L1-HsmMByoyqlpSzApha-1] WARN Failed to evalue groovy script: “http://localhost/” + (clifocazie + clifoclfor + clifocodic) => org.codelibs.fess.ds.impl.DatabaseDataStoreImpl$ResultSetParamMap@6c75ebbd
groovy.lang.MissingPropertyException: No such property: clifocazie for class: Script1

I think that this assignment form the script :

Url string “http://localhost/” + (clifocazie + clifoclfor + clifocodic)

(where clifocazie,clifoclfor,clifocodic are columns of my Oracle table CO_CLIFO) is not correct.
I thought that the names of the fields were replaced with their value in the table.
Can you suggest me a possible correct form?

Many thanks
Luigi

(from github.com/marevol)
need more info such as fess-crawler.log…

(from github.com/luigigf49)
Attached some documents.
Logs_2018_06_01.zip