【GitDataStore】developブランチをクロール対象にしたい

Japanese:Fess
1

GitDataStoreにて、以下のとおりパラメータとスクリプトを指定しましたが、
スケジュールを実行すると「master」ブランチを対象に実行してしまいますが、
「develop」ブランチを対象にするにはどのような設定を行えばよいか、教えていただけないでしょうか。

■パラメータ

uri=http://[サーバ名]/root/fesstest.git
base_url=http://[サーバ名]/root/fesstest/-/blob/develop/
extractors=text/.*:textExtractor,application/xml:textExtractor,application/javascript:textExtractor
delete_old_docs=false
username=[ユーザ名]
password=[パスワード]

■スクリプト

url=url
host="[サーバ名]"
site="[サーバ名]/root/fesstest/-/blob/develop/" + path
title=name
content=content
cache=""
digest=author.toExternalString()
anchor=
content_length=contentLength
last_modified=timestamp
mimetype=mimetype
2

パラメーターにcommit_id=ブランチ名で試してみてください。

3

GitLabのmainブランチをクロール対象にしたくてcommit_idにブランチ名を指定して試してみたのですがエラーが発生しました。
設定に何か不備があったりしますでしょうか？

パラメーター

uri=https://gitlab.com/graphviz/graphviz.git
base_url=https://gitlab.com/graphviz/graphviz/-/tree/main
commit_id=main
extractors=text/.*:textExtractor,application/xml:textExtractor,application/javascript:textExtractor
delete_old_docs=false

スクリプト

url=url
host="gitlab.com"
site="gitlab.com/graphviz/graphviz/" + path
title=name
content=content
cache=""
digest=author.toExternalString()
anchor=
content_length=contentLength
last_modified=timestamp
mimetype=mimetype

クロールログ

2022-06-29 21:10:02,256 [main] INFO  Starting Crawler..
2022-06-29 21:10:02,549 [Ewher4EBom8xRIppAq5u-1] INFO  Git: https://gitlab.com/graphviz/graphviz.git
2022-06-29 21:10:34,662 [Ewher4EBom8xRIppAq5u-1] INFO  Created fess_crawler.data index.
2022-06-29 21:10:34,893 [Ewher4EBom8xRIppAq5u-1] INFO  Created fess_crawler.data mapping.
2022-06-29 21:10:38,358 [Ewher4EBom8xRIppAq5u-1] INFO  Created fess_crawler.queue index.
2022-06-29 21:10:38,576 [Ewher4EBom8xRIppAq5u-1] INFO  Created fess_crawler.queue mapping.
2022-06-29 21:10:42,126 [Ewher4EBom8xRIppAq5u-1] INFO  Created fess_crawler.filter index.
2022-06-29 21:10:42,285 [Ewher4EBom8xRIppAq5u-1] INFO  Created fess_crawler.filter mapping.
2022-06-29 21:10:42,342 [Ewher4EBom8xRIppAq5u-1] WARN  Crawling Access Exception at : {expires=Sat Jul 02 21:10:02 JST 2022, role=[Rguest], config_id=DEwher4EBom8xRIppAq5u, created=Wed Jun 29 21:10:02 JST 2022, segment=Ewher4EBom8xRIppAq5u, boost=1.0, mimetype=application/datastore, virtual_host=[]}
org.eclipse.jgit.api.errors.NoHeadException: No HEAD exists and no explicit starting revision was specified
	at org.eclipse.jgit.api.LogCommand.call(LogCommand.java:131) ~[fess-ds-git-14.2.0.jar:?]
	at org.codelibs.fess.ds.git.GitDataStore.getRevCommit(GitDataStore.java:400) ~[fess-ds-git-14.2.0.jar:?]
	at org.codelibs.fess.ds.git.GitDataStore.processFile(GitDataStore.java:272) ~[fess-ds-git-14.2.0.jar:?]
	at org.codelibs.fess.ds.git.GitDataStore.lambda$storeData$0(GitDataStore.java:189) ~[fess-ds-git-14.2.0.jar:?]
	at java.util.ArrayList.forEach(ArrayList.java:1511) [?:?]
	at org.codelibs.fess.ds.git.GitDataStore.storeData(GitDataStore.java:178) [fess-ds-git-14.2.0.jar:?]
	at org.codelibs.fess.ds.AbstractDataStore.store(AbstractDataStore.java:122) [classes/:?]
	at org.codelibs.fess.helper.DataIndexHelper$DataCrawlingThread.process(DataIndexHelper.java:218) [classes/:?]
	at org.codelibs.fess.helper.DataIndexHelper$DataCrawlingThread.run(DataIndexHelper.java:204) [classes/:?]
2022-06-29 21:10:42,519 [Ewher4EBom8xRIppAq5u-1] WARN  Crawling Access Exception at : {expires=Sat Jul 02 21:10:02 JST 2022, role=[Rguest], config_id=DEwher4EBom8xRIppAq5u, created=Wed Jun 29 21:10:02 JST 2022, segment=Ewher4EBom8xRIppAq5u, boost=1.0, mimetype=application/datastore, virtual_host=[]}
org.eclipse.jgit.api.errors.NoHeadException: No HEAD exists and no explicit starting revision was specified
	at org.eclipse.jgit.api.LogCommand.call(LogCommand.java:131) ~[fess-ds-git-14.2.0.jar:?]
	at org.codelibs.fess.ds.git.GitDataStore.getRevCommit(GitDataStore.java:400) ~[fess-ds-git-14.2.0.jar:?]
	at org.codelibs.fess.ds.git.GitDataStore.processFile(GitDataStore.java:272) ~[fess-ds-git-14.2.0.jar:?]
	at org.codelibs.fess.ds.git.GitDataStore.lambda$storeData$0(GitDataStore.java:189) ~[fess-ds-git-14.2.0.jar:?]
	at java.util.ArrayList.forEach(ArrayList.java:1511) [?:?]
	at org.codelibs.fess.ds.git.GitDataStore.storeData(GitDataStore.java:178) [fess-ds-git-14.2.0.jar:?]
	at org.codelibs.fess.ds.AbstractDataStore.store(AbstractDataStore.java:122) [classes/:?]
	at org.codelibs.fess.helper.DataIndexHelper$DataCrawlingThread.process(DataIndexHelper.java:218) [classes/:?]
	at org.codelibs.fess.helper.DataIndexHelper$DataCrawlingThread.run(DataIndexHelper.java:204) [classes/:?]
2022-06-29 21:10:42,660 [Ewher4EBom8xRIppAq5u-1] WARN  Crawling Access Exception at : {expires=Sat Jul 02 21:10:02 JST 2022, role=[Rguest], config_id=DEwher4EBom8xRIppAq5u, created=Wed Jun 29 21:10:02 JST 2022, segment=Ewher4EBom8xRIppAq5u, boost=1.0, mimetype=application/datastore, virtual_host=[]}
org.eclipse.jgit.api.errors.NoHeadException: No HEAD exists and no explicit starting revision was specified
	at org.eclipse.jgit.api.LogCommand.call(LogCommand.java:131) ~[fess-ds-git-14.2.0.jar:?]
	at org.codelibs.fess.ds.git.GitDataStore.getRevCommit(GitDataStore.java:400) ~[fess-ds-git-14.2.0.jar:?]
	at org.codelibs.fess.ds.git.GitDataStore.processFile(GitDataStore.java:272) ~[fess-ds-git-14.2.0.jar:?]
	at org.codelibs.fess.ds.git.GitDataStore.lambda$storeData$0(GitDataStore.java:189) ~[fess-ds-git-14.2.0.jar:?]
	at java.util.ArrayList.forEach(ArrayList.java:1511) [?:?]
	at org.codelibs.fess.ds.git.GitDataStore.storeData(GitDataStore.java:178) [fess-ds-git-14.2.0.jar:?]
	at org.codelibs.fess.ds.AbstractDataStore.store(AbstractDataStore.java:122) [classes/:?]
	at org.codelibs.fess.helper.DataIndexHelper$DataCrawlingThread.process(DataIndexHelper.java:218) [classes/:?]
	at org.codelibs.fess.helper.DataIndexHelper$DataCrawlingThread.run(DataIndexHelper.java:204) [classes/:?]
2022-06-29 21:10:42,773 [Ewher4EBom8xRIppAq5u-1] WARN  Crawling Access Exception at : {expires=Sat Jul 02 21:10:02 JST 2022, role=[Rguest], config_id=DEwher4EBom8xRIppAq5u, created=Wed Jun 29 21:10:02 JST 2022, segment=Ewher4EBom8xRIppAq5u, boost=1.0, mimetype=application/datastore, virtual_host=[]}
org.eclipse.jgit.api.errors.NoHeadException: No HEAD exists and no explicit starting revision was specified
	at org.eclipse.jgit.api.LogCommand.call(LogCommand.java:131) ~[fess-ds-git-14.2.0.jar:?]
	at org.codelibs.fess.ds.git.GitDataStore.getRevCommit(GitDataStore.java:400) ~[fess-ds-git-14.2.0.jar:?]
	at org.codelibs.fess.ds.git.GitDataStore.processFile(GitDataStore.java:272) ~[fess-ds-git-14.2.0.jar:?]
	at org.codelibs.fess.ds.git.GitDataStore.lambda$storeData$0(GitDataStore.java:189) ~[fess-ds-git-14.2.0.jar:?]
	at java.util.ArrayList.forEach(ArrayList.java:1511) [?:?]
	at org.codelibs.fess.ds.git.GitDataStore.storeData(GitDataStore.java:178) [fess-ds-git-14.2.0.jar:?]
	at org.codelibs.fess.ds.AbstractDataStore.store(AbstractDataStore.java:122) [classes/:?]
	at org.codelibs.fess.helper.DataIndexHelper$DataCrawlingThread.process(DataIndexHelper.java:218) [classes/:?]
	at org.codelibs.fess.helper.DataIndexHelper$DataCrawlingThread.run(DataIndexHelper.java:204) [classes/:?]
2022-06-29 21:10:42,884 [Ewher4EBom8xRIppAq5u-1] WARN  Crawling Access Exception at : {expires=Sat Jul 02 21:10:02 JST 2022, role=[Rguest], config_id=DEwher4EBom8xRIppAq5u, created=Wed Jun 29 21:10:02 JST 2022, segment=Ewher4EBom8xRIppAq5u, boost=1.0, mimetype=application/datastore, virtual_host=[]}
...
4

fess-ds-git 14.2.1でブランチの取得方法を修正したので、それを利用して確認していただければと思います。

5

ありがとうございます。
fess-ds-gitを14.2.1に更新して試したところ、問題なくクロールできるようになりました！

検索結果に表示されるURLがアクセスできないものになっていましたが、base_urlの設定の末尾に/を足して再クロールしたらアクセスできるようになりました。

パラメーター

base_url=https://gitlab.com/graphviz/graphviz/-/tree/main/