Output all URLs in index

English:Fess
1

Is it possible to get a result or file with all the URLs in the index? I want to use this in another system so it must be the actual count after each crawl. Is there a way via the api?

2

You can use Scroll API. To use it, the step is as below:

  1. Set api.search.scroll to true in fess_config.properties
  2. Restart Fess
  3. Access http://localhost:8080/json/scroll?q=Search_Word
3

I only get this:

"Elasticsearch exception [type=action_request_validation_exception, reason=Validation Failed: 1: disabling [track_total_hits] is not allowed in a scroll context;] [ ElasticsearchStatusException[Elasticsearch exception [type=action_request_validation_exception, reason=Validation Failed: 1: disabling [track_total_hits] is not allowed in a scroll context;]]\n\tat org.elasticsearch.rest.BytesRestResponse.errorFromXContent(BytesRestResponse.java:177)\n\tat org.codelibs.elasticsearch.client.action.HttpAction.toElasticsearchException(HttpAction.java:138)\n\tat org.codelibs.elasticsearch.client.action.HttpSearchAction.lambda$execute$0(HttpSearchAction.java:48)\n\tat org.codelibs.curl.CurlRequest.lambda$execute$4(CurlRequest.java:202)\n\tat org.codelibs.curl.CurlRequest.lambda$connect$3(CurlRequest.java:182)\n\tat java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426)\n\tat java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)\n\tat java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)\n\tat java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)\n\tat java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)\n\tat java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)\n\tSuppressed: ElasticsearchException[hits is null.]\n\t\t... 9 more\n\tSuppressed: org.codelibs.elasticsearch.client.action.HttpAction$CurlResponseException: {\"error\":{\"root_cause\":[{\"type\":\"action_request_validation_exception\",\"reason\":\"Validation Failed: 1: disabling [track_total_hits] is not allowed in a scroll context;\"}],\"type\":\"action_request_validation_exception\",\"reason\":\"Validation Failed: 1: disabling [track_total_hits] is not allowed in a scroll context;\"},\"status\":400}\n ]"

What is the problem?

4

Scroll API doesn’t work… It’ll be fixed in the next release.