invalid query on Fess 11.4.4 with dev elasticsearch 5.6.1

discuss · November 27, 2017, 7:51pm

(from github.com/fanjinfei)
Environment:

Ubuntu 16.04.3,
download and untar elasticsearch 5.6.1 from .tgz
download fess 14.4.4 zip file, unzip , copy plugin to ES 5.6.1
config elasticsearch and fess as the readme suggested here
http://fess.codelibs.org/11.4/install/install.html#using-your-elasticsearch-cluster-on-zip-package
start ES and FESS, create crawler for localhost (empty apache2) and job, it is indexed successfully
in browser, http://localhost:8080/, entry query, it shows ‘invalid query’
Elasticsearch error log:

[2017-11-27T14:30:59,310][DEBUG][o.e.a.s.TransportSearchAction] [hYDf9r7] All shards failed for phase: [query]
org.elasticsearch.index.query.QueryShardException: failed to create query: {
  "function_score" : {
    "query" : {
      "bool" : {
        "should" : [
          {
            "match_phrase" : {
              "title" : {
                "query" : "apache",
                "slop" : 0,
                "boost" : 0.2
              }
            }
          },
          {
            "match_phrase" : {
              "content" : {
                "query" : "apache",
                "slop" : 0,
                "boost" : 0.1
              }
            }
          },
          {
            "match_phrase" : {
              "title_en" : {
                "query" : "apache",
                "slop" : 0,
                "boost" : 1.0
              }
            }
          },
          {
            "match_phrase" : {
              "content_en" : {
                "query" : "apache",
                "slop" : 0,
                "boost" : 0.5
              }
            }
          }
        ],
        "disable_coord" : false,
        "adjust_pure_negative" : true,
        "boost" : 1.0
      }
    },
    "functions" : [
      {
        "filter" : {
          "match_all" : {
            "boost" : 1.0
          }
        },
        "field_value_factor" : {
          "field" : "boost",
          "factor" : 1.0,
          "modifier" : "none"
        }
      }
    ],
    "score_mode" : "multiply",
    "max_boost" : 3.4028235E38,
    "boost" : 1.0
  }
}
	at org.elasticsearch.index.query.QueryShardContext.toQuery(QueryShardContext.java:329) ~[elasticsearch-5.6.1.jar:5.6.1]
	at org.elasticsearch.index.query.QueryShardContext.toQuery(QueryShardContext.java:312) ~[elasticsearch-5.6.1.jar:5.6.1]
	at org.elasticsearch.search.SearchService.parseSource(SearchService.java:617) ~[elasticsearch-5.6.1.jar:5.6.1]
	at org.elasticsearch.search.SearchService.createContext(SearchService.java:485) ~[elasticsearch-5.6.1.jar:5.6.1]
	at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:461) ~[elasticsearch-5.6.1.jar:5.6.1]
	at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:257) ~[elasticsearch-5.6.1.jar:5.6.1]
	at org.elasticsearch.action.search.SearchTransportService$6.messageReceived(SearchTransportService.java:340) ~[elasticsearch-5.6.1.jar:5.6.1]
	at org.elasticsearch.action.search.SearchTransportService$6.messageReceived(SearchTransportService.java:337) ~[elasticsearch-5.6.1.jar:5.6.1]
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) ~[elasticsearch-5.6.1.jar:5.6.1]
	at org.elasticsearch.transport.TransportService$7.doRun(TransportService.java:644) [elasticsearch-5.6.1.jar:5.6.1]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:638) [elasticsearch-5.6.1.jar:5.6.1]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.6.1.jar:5.6.1]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_131]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_131]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
Caused by: java.lang.IllegalArgumentException: Fielddata is disabled on text fields by default. Set fielddata=true on [boost] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.
	at org.elasticsearch.index.mapper.TextFieldMapper$TextFieldType.fielddataBuilder(TextFieldMapper.java:336) ~[elasticsearch-5.6.1.jar:5.6.1]
	at org.elasticsearch.index.fielddata.IndexFieldDataService.getForField(IndexFieldDataService.java:111) ~[elasticsearch-5.6.1.jar:5.6.1]
	at org.elasticsearch.index.query.QueryShardContext.getForField(QueryShardContext.java:166) ~[elasticsearch-5.6.1.jar:5.6.1]
	at org.elasticsearch.index.query.functionscore.FieldValueFactorFunctionBuilder.doToFunction(FieldValueFactorFunctionBuilder.java:155) ~[elasticsearch-5.6.1.jar:5.6.1]
	at org.elasticsearch.index.query.functionscore.ScoreFunctionBuilder.toFunction(ScoreFunctionBuilder.java:137) ~[elasticsearch-5.6.1.jar:5.6.1]
	at org.elasticsearch.index.query.functionscore.FunctionScoreQueryBuilder.doToQuery(FunctionScoreQueryBuilder.java:304) ~[elasticsearch-5.6.1.jar:5.6.1]
	at org.elasticsearch.index.query.AbstractQueryBuilder.toQuery(AbstractQueryBuilder.java:96) ~[elasticsearch-5.6.1.jar:5.6.1]
	at org.elasticsearch.index.query.QueryShardContext.lambda$toQuery$1(QueryShardContext.java:313) ~[elasticsearch-5.6.1.jar:5.6.1]
	at org.elasticsearch.index.query.QueryShardContext.toQuery(QueryShardContext.java:325) ~[elasticsearch-5.6.1.jar:5.6.1]
	... 14 more

discuss · November 27, 2017, 9:17pm

(from github.com/marevol)
Index mapping seems to be wrong.
Could you try to do reindex with Update Aliases on Upgrade page?

discuss · November 27, 2017, 10:18pm

(from github.com/fanjinfei)
thanks. I will try that and report back.

discuss · November 28, 2017, 3:08pm

(from github.com/fanjinfei)
The above error is caused by wrong version of jackson-databind jar. After copy jackson-databind-2.8.6.jar from fess to elasticsearch, there is the following error log(admin gui not working):

[2017-11-28T10:05:56,027][DEBUG][o.e.a.a.i.m.p.TransportPutMappingAction] [node-1] failed to put mappings on indices [[[fess.20171128/S8H_6actTC66mRBqAuRezw]]], type [doc]
org.elasticsearch.ElasticsearchException: profile format error in 'profiles/ar'
	at org.codelibs.elasticsearch.langfield.detect.LangDetectorFactory.lambda$create$1(LangDetectorFactory.java:103) ~[?:?]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_131]
	at org.codelibs.elasticsearch.langfield.detect.LangDetectorFactory.create(LangDetectorFactory.java:94) ~[?:?]
	at org.codelibs.elasticsearch.langfield.index.mapper.LangStringFieldMapper.<init>(LangStringFieldMapper.java:449) ~[?:?]
	at org.codelibs.elasticsearch.langfield.index.mapper.LangStringFieldMapper$Builder.build(LangStringFieldMapper.java:204) ~[?:?]
	at org.codelibs.elasticsearch.langfield.index.mapper.LangStringFieldMapper$Builder.build(LangStringFieldMapper.java:114) ~[?:?]
	at org.elasticsearch.index.mapper.ObjectMapper$Builder.build(ObjectMapper.java:143) ~[elasticsearch-5.6.4.jar:5.6.4]
	at org.elasticsearch.index.mapper.DocumentMapper$Builder.<init>(DocumentMapper.java:69) ~[elasticsearch-5.6.4.jar:5.6.4]
	at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:111) ~[elasticsearch-5.6.4.jar:5.6.4]
	at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:91) ~[elasticsearch-5.6.4.jar:5.6.4]
	at org.elasticsearch.index.mapper.MapperService.parse(MapperService.java:644) ~[elasticsearch-5.6.4.jar:5.6.4]
	at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.applyRequest(MetaDataMappingService.java:264) ~[elasticsearch-5.6.4.jar:5.6.4]
	at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.execute(MetaDataMappingService.java:230) ~[elasticsearch-5.6.4.jar:5.6.4]
	at org.elasticsearch.cluster.service.ClusterService.executeTasks(ClusterService.java:634) ~[elasticsearch-5.6.4.jar:5.6.4]
	at org.elasticsearch.cluster.service.ClusterService.calculateTaskOutputs(ClusterService.java:612) ~[elasticsearch-5.6.4.jar:5.6.4]
	at org.elasticsearch.cluster.service.ClusterService.runTasks(ClusterService.java:571) [elasticsearch-5.6.4.jar:5.6.4]
	at org.elasticsearch.cluster.service.ClusterService$ClusterServiceTaskBatcher.run(ClusterService.java:263) [elasticsearch-5.6.4.jar:5.6.4]
	at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150) [elasticsearch-5.6.4.jar:5.6.4]
	at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188) [elasticsearch-5.6.4.jar:5.6.4]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:569) [elasticsearch-5.6.4.jar:5.6.4]
	at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:247) [elasticsearch-5.6.4.jar:5.6.4]
	at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:210) [elasticsearch-5.6.4.jar:5.6.4]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_131]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_131]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
Caused by: java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "accessDeclaredMembers")
	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:1.8.0_131]
	at java.security.AccessController.checkPermission(AccessController.java:884) ~[?:1.8.0_131]
	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549) ~[?:1.8.0_131]
	at java.lang.Class.checkMemberAccess(Class.java:2348) ~[?:1.8.0_131]
	at java.lang.Class.getDeclaredFields(Class.java:1915) ~[?:1.8.0_131]
	at com.fasterxml.jackson.databind.util.ClassUtil.getDeclaredFields(ClassUtil.java:937) ~[jackson-databind-2.8.6.jar:2.8.6]
	at com.fasterxml.jackson.databind.introspect.AnnotatedClass._findFields(AnnotatedClass.java:817) ~[jackson-databind-2.8.6.jar:2.8.6]
	at com.fasterxml.jackson.databind.introspect.AnnotatedClass.resolveFields(AnnotatedClass.java:583) ~[jackson-databind-2.8.6.jar:2.8.6]
	at com.fasterxml.jackson.databind.introspect.AnnotatedClass.fields(AnnotatedClass.java:353) ~[jackson-databind-2.8.6.jar:2.8.6]
	at com.fasterxml.jackson.databind.introspect.POJOPropertiesCollector._addFields(POJOPropertiesCollector.java:368) ~[jackson-databind-2.8.6.jar:2.8.6]
	at com.fasterxml.jackson.databind.introspect.POJOPropertiesCollector.collectAll(POJOPropertiesCollector.java:301) ~[jackson-databind-2.8.6.jar:2.8.6]
	at com.fasterxml.jackson.databind.introspect.POJOPropertiesCollector.getPropertyMap(POJOPropertiesCollector.java:266) ~[jackson-databind-2.8.6.jar:2.8.6]
	at com.fasterxml.jackson.databind.introspect.POJOPropertiesCollector.getProperties(POJOPropertiesCollector.java:158) ~[jackson-databind-2.8.6.jar:2.8.6]
	at com.fasterxml.jackson.databind.introspect.BasicBeanDescription._properties(BasicBeanDescription.java:144) ~[jackson-databind-2.8.6.jar:2.8.6]
	at com.fasterxml.jackson.databind.introspect.BasicBeanDescription.findProperties(BasicBeanDescription.java:219) ~[jackson-databind-2.8.6.jar:2.8.6]
	at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory._findCreatorsFromProperties(BasicDeserializerFactory.java:332) ~[jackson-databind-2.8.6.jar:2.8.6]
	at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory._constructDefaultValueInstantiator(BasicDeserializerFactory.java:314) ~[jackson-databind-2.8.6.jar:2.8.6]
	at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory.findValueInstantiator(BasicDeserializerFactory.java:253) ~[jackson-databind-2.8.6.jar:2.8.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.buildBeanDeserializer(BeanDeserializerFactory.java:219) ~[jackson-databind-2.8.6.jar:2.8.6]
	at com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.createBeanDeserializer(BeanDeserializerFactory.java:141) ~[jackson-databind-2.8.6.jar:2.8.6]
	at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer2(DeserializerCache.java:403) ~[jackson-databind-2.8.6.jar:2.8.6]
	at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer(DeserializerCache.java:349) ~[jackson-databind-2.8.6.jar:2.8.6]
	at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:264) ~[jackson-databind-2.8.6.jar:2.8.6]
	at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:244) ~[jackson-databind-2.8.6.jar:2.8.6]
	at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142) ~[jackson-databind-2.8.6.jar:2.8.6]
	at com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:476) ~[jackson-databind-2.8.6.jar:2.8.6]
	at com.fasterxml.jackson.databind.ObjectMapper._findRootDeserializer(ObjectMapper.java:3899) ~[jackson-databind-2.8.6.jar:2.8.6]
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3794) ~[jackson-databind-2.8.6.jar:2.8.6]
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2908) ~[jackson-databind-2.8.6.jar:2.8.6]
	at org.codelibs.elasticsearch.langfield.detect.LangDetectorFactory.lambda$create$1(LangDetectorFactory.java:99) ~[?:?]
	... 24 more

discuss · November 28, 2017, 6:07pm

(from github.com/fanjinfei)
Solved the problem:
Cause: The langfield plugin 5.6.1 copied from fess zip release is causing this problem.
Solution: rebuild it from github 5.6.0 source, and install it as zip.

It is not necessary to be 5.6.0.

Thanks @marevol