Share Pointクロール時の403 Forbiddenについて

Japanese:Fess
1

現在、FESS（v13.8.1）を導入しております。
SharePointのクロールを行いたいのですが、「fess-ds-sharepoint」プラグインが導入済みのFESSバージョンに対応していないようで、最新のFESS（v13.15.1）で動作を検証しております。

以下サイトの手順に従い、SharePointのクロール設定を追加したのですが、ジョブ実行はOKとなるのですが、検索対象がヒットしない状況です。

fess-clawler.logを見るとエラーが発生しておりました。
権限関連のエラーと見受けられるのですが、認証情報等見直してもうまくいかない状況となります。

バージョン情報：
elasticsearch: 7.15.2
fess: 13.15.1
fess-ds-sharepoint: 13.15.0

お手数をおかけしますが、解決方法をご教示いただけないでしょうか？

クロール設定

SharePoint設定１
SharePoint設定１986×785 101 KB

ジョブ実行時のfess-clawler.log

2023-07-14 19:05:59,507 [6XwsTokB6nLRQ4sdYkx_-1] WARN  Error occured: 0  [Retry:{}]Request failure. Cannot deserialize value of type `java.util.LinkedHashMap<java.lang.Object,java.lang.Object>` from Integer value (token `JsonToken.VALUE_NUMBER_INT`)
 at [Source: (String)"403 FORBIDDEN"; line: 1, column: 1]
2023-07-14 19:05:59,509 [6XwsTokB6nLRQ4sdYkx_-1] INFO  [Crawling DocLib Folder] serverRelativeUrl:
2023-07-14 19:05:59,757 [6XwsTokB6nLRQ4sdYkx_-1] WARN  Error occured: 1  [Retry:{}]Request failure. Cannot deserialize value of type `java.util.LinkedHashMap<java.lang.Object,java.lang.Object>` from Integer value (token `JsonToken.VALUE_NUMBER_INT`)
 at [Source: (String)"403 FORBIDDEN"; line: 1, column: 1]
2023-07-14 19:05:59,759 [6XwsTokB6nLRQ4sdYkx_-1] INFO  [Crawling DocLib Folder] serverRelativeUrl:
2023-07-14 19:06:00,000 [6XwsTokB6nLRQ4sdYkx_-1] WARN  Error occured. Request failure. Cannot deserialize value of type `java.util.LinkedHashMap<java.lang.Object,java.lang.Object>` from Integer value (token `JsonToken.VALUE_NUMBER_INT`)
 at [Source: (String)"403 FORBIDDEN"; line: 1, column: 1]
org.codelibs.fess.ds.sharepoint.client.exception.SharePointClientException: Request failure. Cannot deserialize value of type `java.util.LinkedHashMap<java.lang.Object,java.lang.Object>` from Integer value (token `JsonToken.VALUE_NUMBER_INT`)
 at [Source: (String)"403 FORBIDDEN"; line: 1, column: 1]
	at org.codelibs.fess.ds.sharepoint.client.api.SharePointApi.doJsonRequest(SharePointApi.java:93) ~[fess-ds-sharepoint-13.16.1.jar:?]
	at org.codelibs.fess.ds.sharepoint.client.api.doclib.getfolder.GetFolder.execute(GetFolder.java:45) ~[fess-ds-sharepoint-13.16.1.jar:?]
	at org.codelibs.fess.ds.sharepoint.crawl.doclib.FolderCrawl.doCrawl(FolderCrawl.java:61) ~[fess-ds-sharepoint-13.16.1.jar:?]
	at org.codelibs.fess.ds.sharepoint.SharePointCrawler.doCrawl(SharePointCrawler.java:122) [fess-ds-sharepoint-13.16.1.jar:?]
	at org.codelibs.fess.ds.sharepoint.SharePointDataStore.storeData(SharePointDataStore.java:53) [fess-ds-sharepoint-13.16.1.jar:?]
	at org.codelibs.fess.ds.AbstractDataStore.store(AbstractDataStore.java:121) [classes/:?]
	at org.codelibs.fess.helper.DataIndexHelper$DataCrawlingThread.process(DataIndexHelper.java:216) [classes/:?]
	at org.codelibs.fess.helper.DataIndexHelper$DataCrawlingThread.run(DataIndexHelper.java:202) [classes/:?]
Caused by: com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize value of type `java.util.LinkedHashMap<java.lang.Object,java.lang.Object>` from Integer value (token `JsonToken.VALUE_NUMBER_INT`)
 at [Source: (String)"403 FORBIDDEN"; line: 1, column: 1]
	at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59) ~[jackson-databind-2.13.0.jar:2.13.0]
	at com.fasterxml.jackson.databind.DeserializationContext.reportInputMismatch(DeserializationContext.java:1741) ~[jackson-databind-2.13.0.jar:2.13.0]
	at com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1515) ~[jackson-databind-2.13.0.jar:2.13.0]
	at com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1462) ~[jackson-databind-2.13.0.jar:2.13.0]
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:450) ~[jackson-databind-2.13.0.jar:2.13.0]
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:32) ~[jackson-databind-2.13.0.jar:2.13.0]
	at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:322) ~[jackson-databind-2.13.0.jar:2.13.0]
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4675) ~[jackson-databind-2.13.0.jar:2.13.0]
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3630) ~[jackson-databind-2.13.0.jar:2.13.0]
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3598) ~[jackson-databind-2.13.0.jar:2.13.0]
	at org.codelibs.fess.ds.sharepoint.client.api.SharePointApi.doJsonRequest(SharePointApi.java:77) ~[fess-ds-sharepoint-13.16.1.jar:?]
	... 7 more
2023-07-14 19:06:00,025 [6XwsTokB6nLRQ4sdYkx_-1] INFO  Deleted 0 old docs.
2023-07-14 19:06:03,747 [DataStoreCrawler] INFO  [EXEC TIME] crawling time: 5004ms
2023-07-14 19:06:03,754 [main] INFO  Finished Crawler
2023-07-14 19:06:03,794 [main] INFO  [CRAWL INFO] DataCrawlExecTime=5004,DataCrawlEndTime=2023-07-14T19:06:03.754+0900,CrawlerEndTime=2023-07-14T19:06:03.755+0900,DataIndexExecTime=0,CrawlerStatus=true,CrawlerStartTime=2023-07-14T19:05:58.701+0900,DataIndexSize=0,CrawlerExecTime=5054,DataCrawlStartTime=2023-07-14T19:05:58.729+0900
2023-07-14 19:06:03,800 [main] INFO  Destroyed LaContainer.
2

Fess 13はEOLされているので、最新のバージョンで問題が再現するか？を確認してみてください。

3

バージョンを最新のものに更新しましたが、同様のエラーが発生しました。
バージョン情報：
opensearch: 2.8.0
fess: 14.9.1
fess-ds-sharepoint: 14.9.0

Share Point設定
Share Point設定1144×691 83 KB

ジョブ実行時のfess-clawler.log

2023-07-20 18:10:09,382 [main] INFO  ...Loading specified properties and get by the key: fess_env_crawler.properties, lasta_di.smart.deploy.mode
2023-07-20 18:10:09,387 [main] INFO  ...Setting smart deploy mode: warm
2023-07-20 18:10:09,403 [main] INFO  ...Reading lasta_di.xml
2023-07-20 18:10:09,438 [main] INFO  ...Reading   redefiner.xml
2023-07-20 18:10:09,496 [main] INFO  ...Reading   smartdeploy.xml
2023-07-20 18:10:09,504 [main] INFO  ...Reading     smart/warmdeploy.xml
2023-07-20 18:10:09,509 [main] INFO  ...Reading       convention.xml
2023-07-20 18:10:09,512 [main] INFO  ...Reading         embedded_convention.xml
2023-07-20 18:10:09,525 [main] INFO  ...Reading       creator.xml
2023-07-20 18:10:09,528 [main] INFO  ...Reading         convention.xml (recycle)
2023-07-20 18:10:09,528 [main] INFO  ...Reading         customizer.xml
2023-07-20 18:10:09,531 [main] INFO  ...Reading           lastafw_customizer.xml
2023-07-20 18:10:09,544 [main] INFO  ...Reading           embedded_customizer.xml
2023-07-20 18:10:09,547 [main] INFO  ...Reading             tx_customizer.xml
2023-07-20 18:10:09,566 [main] INFO  ...Reading         my_creator.xml
2023-07-20 18:10:09,569 [main] INFO  ...Reading         lastafw_creator.xml
2023-07-20 18:10:09,571 [main] INFO  ...Reading           convention.xml (recycle)
2023-07-20 18:10:09,572 [main] INFO  ...Reading           customizer.xml (recycle)
2023-07-20 18:10:09,592 [main] INFO  ...Reading         embedded_creator.xml
2023-07-20 18:10:09,595 [main] INFO  ...Reading           convention.xml (recycle)
2023-07-20 18:10:09,596 [main] INFO  ...Reading           customizer.xml (recycle)
2023-07-20 18:10:09,604 [main] INFO  ...Reading       customizer.xml (recycle)
2023-07-20 18:10:09,650 [main] INFO  ...Reading app.xml
2023-07-20 18:10:09,656 [main] INFO  ...Reading   convention.xml
2023-07-20 18:10:09,664 [main] INFO  ...Reading     embedded_convention.xml
2023-07-20 18:10:09,668 [main] INFO  ...Reading   lastaflute_core.xml
2023-07-20 18:10:09,672 [main] INFO  ...Reading     lastaflute_assist.xml
2023-07-20 18:10:09,675 [main] INFO  ...Reading       lastaflute_director.xml
2023-07-20 18:10:09,824 [main] INFO  ...Reading   fess.xml
2023-07-20 18:10:09,828 [main] INFO  ...Reading     fess_config.xml
2023-07-20 18:10:09,838 [main] INFO  ...Reading     fess_ds.xml
2023-07-20 18:10:09,845 [main] INFO  ...Reading       jar:file:/C:/fess-14.9.1/app/WEB-INF/plugin/fess-ds-sharepoint-14.9.0.jar!/fess_ds++.xml
2023-07-20 18:10:09,852 [main] INFO  ...Reading     fess_se.xml
2023-07-20 18:10:09,857 [main] INFO  ...Reading       jar:file:/C:/fess-14.9.1/app/WEB-INF/plugin/fess-script-groovy-14.9.0.jar!/fess_se++.xml
2023-07-20 18:10:09,861 [main] INFO  ...Reading     esflute_config.xml
2023-07-20 18:10:09,863 [main] INFO  ...Reading       esclient.xml
2023-07-20 18:10:10,110 [main] INFO  ...Reading     esflute_user.xml
2023-07-20 18:10:10,113 [main] INFO  ...Reading       esclient.xml (recycle)
2023-07-20 18:10:10,134 [main] INFO  ...Reading     esflute_log.xml
2023-07-20 18:10:10,137 [main] INFO  ...Reading       esclient.xml (recycle)
2023-07-20 18:10:10,991 [main] INFO  ...Reading   crawler_es.xml
2023-07-20 18:10:10,994 [main] INFO  ...Reading     crawler/container.xml
2023-07-20 18:10:10,998 [main] INFO  ...Reading     crawler/client.xml
2023-07-20 18:10:11,001 [main] INFO  ...Reading       crawler/container.xml (recycle)
2023-07-20 18:10:11,001 [main] INFO  ...Reading       crawler/robotstxt.xml
2023-07-20 18:10:11,003 [main] INFO  ...Reading         crawler/container.xml (recycle)
2023-07-20 18:10:11,010 [main] INFO  ...Reading       crawler/contentlength.xml
2023-07-20 18:10:11,012 [main] INFO  ...Reading         crawler/container.xml (recycle)
2023-07-20 18:10:11,013 [main] INFO  ...Reading       crawler/mimetype.xml
2023-07-20 18:10:11,015 [main] INFO  ...Reading         crawler/container.xml (recycle)
2023-07-20 18:10:11,076 [main] INFO  ...Reading       jar:file:/C:/fess-14.9.1/app/WEB-INF/lib/fess-crawler-playwright-14.9.0.jar!/crawler/client++.xml
2023-07-20 18:10:11,078 [main] INFO  ...Reading         crawler/container.xml
2023-07-20 18:10:11,091 [main] INFO  ...Reading     crawler/rule.xml
2023-07-20 18:10:11,094 [main] INFO  ...Reading       crawler/container.xml (recycle)
2023-07-20 18:10:11,094 [main] INFO  ...Reading       crawler/transformer.xml
2023-07-20 18:10:11,096 [main] INFO  ...Reading         crawler/transformer_basic.xml
2023-07-20 18:10:11,098 [main] INFO  ...Reading           crawler/container.xml (recycle)
2023-07-20 18:10:11,127 [main] INFO  ...Reading     crawler/filter.xml
2023-07-20 18:10:11,129 [main] INFO  ...Reading       crawler/container.xml (recycle)
2023-07-20 18:10:11,132 [main] INFO  ...Reading     crawler/interval.xml
2023-07-20 18:10:11,134 [main] INFO  ...Reading       crawler/container.xml (recycle)
2023-07-20 18:10:11,136 [main] INFO  ...Reading     crawler/extractor.xml
2023-07-20 18:10:11,138 [main] INFO  ...Reading       crawler/container.xml (recycle)
2023-07-20 18:10:11,143 [main] INFO  ...Reading       crawler/extractor+tikaExtractor.xml
2023-07-20 18:10:11,145 [main] INFO  ...Reading         crawler/container.xml
2023-07-20 18:10:11,179 [main] INFO  ...Reading     crawler/mimetype.xml (recycle)
2023-07-20 18:10:11,179 [main] INFO  ...Reading     crawler/encoding.xml
2023-07-20 18:10:11,181 [main] INFO  ...Reading       crawler/container.xml (recycle)
2023-07-20 18:10:11,183 [main] INFO  ...Reading     crawler/urlconverter.xml
2023-07-20 18:10:11,185 [main] INFO  ...Reading       crawler/container.xml (recycle)
2023-07-20 18:10:11,186 [main] INFO  ...Reading     crawler/log.xml
2023-07-20 18:10:11,188 [main] INFO  ...Reading       crawler/container.xml (recycle)
2023-07-20 18:10:11,191 [main] INFO  ...Reading     crawler/sitemaps.xml
2023-07-20 18:10:11,193 [main] INFO  ...Reading       crawler/container.xml (recycle)
2023-07-20 18:10:11,196 [main] INFO  ...Reading     crawler/es.xml
2023-07-20 18:10:11,211 [main] INFO  ...Reading     crawler_es+crawlerThread.xml
2023-07-20 18:10:11,229 [main] INFO  ...Reading     crawler_es+crawlerConfig.xml
2023-07-20 18:10:11,232 [main] INFO  ...Reading   fess_thumbnail.xml
2023-07-20 18:10:11,242 [main] INFO  ...Reading   fess_ingest.xml
2023-07-20 18:10:11,424 [main] INFO  [Objective Config]
2023-07-20 18:10:11,425 [main] INFO   fess_config.properties extends [fess_env_crawler.properties]
2023-07-20 18:10:11,425 [main] INFO   checkImplicitOverride=true, propertyCount=533
2023-07-20 18:10:11,462 [main] INFO  [Exception Translator]
2023-07-20 18:10:11,463 [main] INFO   exceptionTranslationProvider: null
2023-07-20 18:10:11,463 [main] INFO  [Async Manager]
2023-07-20 18:10:11,463 [main] INFO   defaultConcurrentAsyncOption: {no option}
2023-07-20 18:10:11,464 [main] INFO   primaryExecutorService: ThreadPoolExecutor@e9ef5b6
2023-07-20 18:10:11,464 [main] INFO   secondaryExecutorService: ThreadPoolExecutor@5300cac
2023-07-20 18:10:11,464 [main] INFO   tertiaryExecutorService: ThreadPoolExecutor@4110765e
2023-07-20 18:10:11,464 [main] INFO  [Primary Cipher]
2023-07-20 18:10:11,464 [main] INFO   invertibleCryptographer: {AES, UTF-8}
2023-07-20 18:10:11,464 [main] INFO   oneWayCryptographer: {SHA-256, UTF-8}
2023-07-20 18:10:11,466 [main] INFO  [Time Manager]
2023-07-20 18:10:11,466 [main] INFO   businessTimeHandler: TypicalBusinessTimeHandler:{TypicalTimeResourceProvider$$Lambda$150/0x0000000800f51ba8, TypicalTimeResourceProvider$$Lambda$151/0x0000000800f51fc0}
2023-07-20 18:10:11,466 [main] INFO   currentIgnoreTransaction: false
2023-07-20 18:10:11,466 [main] INFO   adjustTimeMillis: 0
2023-07-20 18:10:11,466 [main] INFO   adjustAbsoluteMode: false
2023-07-20 18:10:11,526 [main] INFO  [JSON Manager]
2023-07-20 18:10:11,527 [main] INFO   realJsonParser: GsonJsonEngine
2023-07-20 18:10:11,531 [main] INFO   mapping: {CAMEL_TO_LOWER_SNAKE}
2023-07-20 18:10:11,531 [main] INFO  [Postbox]
2023-07-20 18:10:11,532 [main] INFO   postOffice: PostOffice@117d32e
2023-07-20 18:10:11,532 [main] INFO   postalParkingLot: SMailPostalParkingLot:{{category:{main}=motorbike:{session=javax.mail.Session@51d387d3}}}@2370ac7a
2023-07-20 18:10:11,532 [main] INFO   postalPersonnel: FessMailDeliveryDepartmentCreator$1:{SMailConventionReceptionist:{[]}@534180a2, batch:{[proofreader:{pmcomment}]}}@6cf0a747
2023-07-20 18:10:13,394 [main] INFO  Lasta Di boot successfully.
2023-07-20 18:10:13,395 [main] INFO    SmartDeploy Mode: Warm Deploy
2023-07-20 18:10:13,396 [main] INFO    Smart Package: org.codelibs.fess.app
2023-07-20 18:10:13,502 [main] INFO  Starting Crawler..
2023-07-20 18:10:13,754 [6XwsTokB6nLRQ4sdYkx_-1] INFO  [Crawling DocLib Folder] serverRelativeUrl:/sites/msteams_0dcc70/Shared_Documents/繝��Δ逕ｨ
2023-07-20 18:10:14,086 [6XwsTokB6nLRQ4sdYkx_-1] WARN  Error occured: Request failure. Cannot deserialize value of type `java.util.LinkedHashMap<java.lang.Object,java.lang.Object>` from Integer value (token `JsonToken.VALUE_NUMBER_INT`)
 at [Source: (String)"403 FORBIDDEN"; line: 1, column: 1]  [Retry:0]
2023-07-20 18:10:14,087 [6XwsTokB6nLRQ4sdYkx_-1] INFO  [Crawling DocLib Folder] serverRelativeUrl:/sites/msteams_0dcc70/Shared_Documents/繝��Δ逕ｨ
2023-07-20 18:10:14,112 [6XwsTokB6nLRQ4sdYkx_-1] WARN  Error occured: Request failure. Cannot deserialize value of type `java.util.LinkedHashMap<java.lang.Object,java.lang.Object>` from Integer value (token `JsonToken.VALUE_NUMBER_INT`)
 at [Source: (String)"403 FORBIDDEN"; line: 1, column: 1]  [Retry:1]
2023-07-20 18:10:14,112 [6XwsTokB6nLRQ4sdYkx_-1] INFO  [Crawling DocLib Folder] serverRelativeUrl:/sites/msteams_0dcc70/Shared_Documents/繝��Δ逕ｨ
2023-07-20 18:10:14,136 [6XwsTokB6nLRQ4sdYkx_-1] WARN  Error occured. Request failure. Cannot deserialize value of type `java.util.LinkedHashMap<java.lang.Object,java.lang.Object>` from Integer value (token `JsonToken.VALUE_NUMBER_INT`)
 at [Source: (String)"403 FORBIDDEN"; line: 1, column: 1]
org.codelibs.fess.ds.sharepoint.client.exception.SharePointClientException: Request failure. Cannot deserialize value of type `java.util.LinkedHashMap<java.lang.Object,java.lang.Object>` from Integer value (token `JsonToken.VALUE_NUMBER_INT`)
 at [Source: (String)"403 FORBIDDEN"; line: 1, column: 1]
	at org.codelibs.fess.ds.sharepoint.client.api.SharePointApi.doJsonRequest(SharePointApi.java:93) ~[fess-ds-sharepoint-14.9.0.jar:?]
	at org.codelibs.fess.ds.sharepoint.client.api.doclib.getfolder.GetFolder.execute(GetFolder.java:53) ~[fess-ds-sharepoint-14.9.0.jar:?]
	at org.codelibs.fess.ds.sharepoint.crawl.doclib.FolderCrawl.doCrawl(FolderCrawl.java:64) ~[fess-ds-sharepoint-14.9.0.jar:?]
	at org.codelibs.fess.ds.sharepoint.SharePointCrawler.doCrawl(SharePointCrawler.java:133) ~[fess-ds-sharepoint-14.9.0.jar:?]
	at org.codelibs.fess.ds.sharepoint.SharePointDataStore.storeData(SharePointDataStore.java:59) ~[fess-ds-sharepoint-14.9.0.jar:?]
	at org.codelibs.fess.ds.AbstractDataStore.store(AbstractDataStore.java:122) ~[classes/:?]
	at org.codelibs.fess.helper.DataIndexHelper$DataCrawlingThread.process(DataIndexHelper.java:218) ~[classes/:?]
	at org.codelibs.fess.helper.DataIndexHelper$DataCrawlingThread.run(DataIndexHelper.java:204) ~[classes/:?]
Caused by: com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize value of type `java.util.LinkedHashMap<java.lang.Object,java.lang.Object>` from Integer value (token `JsonToken.VALUE_NUMBER_INT`)
 at [Source: (String)"403 FORBIDDEN"; line: 1, column: 1]
	at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59) ~[jackson-databind-2.15.2.jar:2.15.2]
	at com.fasterxml.jackson.databind.DeserializationContext.reportInputMismatch(DeserializationContext.java:1752) ~[jackson-databind-2.15.2.jar:2.15.2]
	at com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1526) ~[jackson-databind-2.15.2.jar:2.15.2]
	at com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1473) ~[jackson-databind-2.15.2.jar:2.15.2]
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:460) ~[jackson-databind-2.15.2.jar:2.15.2]
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:32) ~[jackson-databind-2.15.2.jar:2.15.2]
	at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:323) ~[jackson-databind-2.15.2.jar:2.15.2]
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4825) ~[jackson-databind-2.15.2.jar:2.15.2]
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3772) ~[jackson-databind-2.15.2.jar:2.15.2]
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3740) ~[jackson-databind-2.15.2.jar:2.15.2]
	at org.codelibs.fess.ds.sharepoint.client.api.SharePointApi.doJsonRequest(SharePointApi.java:77) ~[fess-ds-sharepoint-14.9.0.jar:?]
	... 7 more
2023-07-20 18:10:14,159 [6XwsTokB6nLRQ4sdYkx_-1] INFO  Deleted 0 old docs.
2023-07-20 18:10:18,547 [DataStoreCrawler] INFO  [EXEC TIME] crawling time: 5005ms
2023-07-20 18:10:18,551 [main] INFO  Finished Crawler
2023-07-20 18:10:18,613 [main] INFO  [CRAWL INFO] DataCrawlExecTime=5005,DataCrawlEndTime=2023-07-20T18:10:18.551+0900,CrawlerEndTime=2023-07-20T18:10:18.551+0900,DataIndexExecTime=0,CrawlerStatus=true,CrawlerStartTime=2023-07-20T18:10:13.502+0900,DataIndexSize=0,CrawlerExecTime=5050,DataCrawlStartTime=2023-07-20T18:10:13.530+0900
2023-07-20 18:10:18,620 [main] INFO  Destroyed LaContainer.
4

JSONではないレスポンスが返ってきているようにも見えるので、ログレベルをデバッグに変えて、内容を確認してみると良いと思います。

5

ご回答ありがとうございます。
ログレベルをデバッグに変更して出力したところ、

2023-07-25 09:08:36,200 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 >> GET /sites/msteams_0dcc70/Shared%20Documents/Forms/AllItems.aspx/sites/msteams_0dcc70/_api/web/GetFolderByServerRelativeUrl('/sites/msteams_0dcc70/Shared_Documents/%E3%83%87%E3%83%A2%E7%94%A8') HTTP/1.1
2023-07-25 09:08:36,200 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 >> Accept: application/json
2023-07-25 09:08:36,200 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 >> Host: vicent001.sharepoint.com
2023-07-25 09:08:36,200 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 >> Connection: Keep-Alive
2023-07-25 09:08:36,200 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 >> User-Agent: Apache-HttpClient/4.5.14 (Java/17.0.2)
2023-07-25 09:08:36,200 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 >> Accept-Encoding: gzip,deflate
2023-07-25 09:08:36,200 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 >> "GET /sites/msteams_0dcc70/Shared%20Documents/Forms/AllItems.aspx/sites/msteams_0dcc70/_api/web/GetFolderByServerRelativeUrl('/sites/msteams_0dcc70/Shared_Documents/%E3%83%87%E3%83%A2%E7%94%A8') HTTP/1.1[\r][\n]"
2023-07-25 09:08:36,200 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 >> "Accept: application/json[\r][\n]"
2023-07-25 09:08:36,200 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 >> "Host: vicent001.sharepoint.com[\r][\n]"
2023-07-25 09:08:36,200 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 >> "Connection: Keep-Alive[\r][\n]"
2023-07-25 09:08:36,200 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 >> "User-Agent: Apache-HttpClient/4.5.14 (Java/17.0.2)[\r][\n]"
2023-07-25 09:08:36,200 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 >> "Accept-Encoding: gzip,deflate[\r][\n]"
2023-07-25 09:08:36,200 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 >> "[\r][\n]"
2023-07-25 09:08:36,232 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 << "HTTP/1.1 403 Forbidden[\r][\n]"
2023-07-25 09:08:36,232 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 << "Content-Length: 13[\r][\n]"
2023-07-25 09:08:36,232 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 << "Content-Type: text/plain; charset=utf-8[\r][\n]"
2023-07-25 09:08:36,232 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 << "P3P: CP="ALL IND DSP COR ADM CONo CUR CUSo IVAo IVDo PSA PSD TAI TELo OUR SAMo CNT COM INT NAV ONL PHY PRE PUR UNI"[\r][\n]"
2023-07-25 09:08:36,232 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 << "X-NetworkStatistics: 198,525568,0,9421,708860,62726,11542[\r][\n]"
2023-07-25 09:08:36,232 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 << "X-SharePointHealthScore: 3[\r][\n]"
2023-07-25 09:08:36,232 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 << "X-Forms_Based_Auth_Required: https://xxxxx/_forms/default.aspx?ReturnUrl=/_layouts/15/error.aspx&Source=%2fsites%2fmsteams_0dcc70%2fShared%2520Documents%2fForms%2fAllItems.aspx%2fsites%2fmsteams_0dcc70%2f_api%2fweb%2fGetFolderByServerRelativeUrl(%27%2fsites%2fmsteams_0dcc70%2fShared_Documents%2f%25E3%2583%2587%25E3%2583%25A2%25E7%2594%25A8%27)[\r][\n]"
2023-07-25 09:08:36,232 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 << "X-Forms_Based_Auth_Return_Url: https://xxxxx/_layouts/15/error.aspx[\r][\n]"
2023-07-25 09:08:36,232 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 << "X-MSDAVEXT_Error: 917656; Access+denied.+Before+opening+files+in+this+location%2c+you+must+first+browse+to+the+web+site+and+select+the+option+to+login+automatically.[\r][\n]"
2023-07-25 09:08:36,232 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 << "SPRequestGuid: d2c8c9a0-10de-2000-7cda-fcaf2744e31c[\r][\n]"
2023-07-25 09:08:36,232 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 << "request-id: d2c8c9a0-10de-2000-7cda-fcaf2744e31c[\r][\n]"
2023-07-25 09:08:36,232 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 << "MS-CV: oMnI0t4QACB82vyvJ0TjHA.0[\r][\n]"
2023-07-25 09:08:36,232 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 << "Strict-Transport-Security: max-age=31536000[\r][\n]"
2023-07-25 09:08:36,232 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 << "SPRequestDuration: 16[\r][\n]"
2023-07-25 09:08:36,232 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 << "SPIisLatency: 1[\r][\n]"
2023-07-25 09:08:36,247 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 << "X-IDCRL_AUTH_PARAMS_V1: IDCRL Type="BPOSIDCRL", EndPoint="/sites/msteams_0dcc70/_vti_bin/idcrl.svc/", RootDomain="sharepoint.com", Policy="MBI"[\r][\n]"
2023-07-25 09:08:36,247 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 << "X-Powered-By: ASP.NET[\r][\n]"
2023-07-25 09:08:36,247 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 << "MicrosoftSharePointTeamServices: 16.0.0.23912[\r][\n]"
2023-07-25 09:08:36,247 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 << "X-Content-Type-Options: nosniff[\r][\n]"
2023-07-25 09:08:36,247 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 << "X-MS-InvokeApp: 1; RequireReadOnly[\r][\n]"
2023-07-25 09:08:36,247 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 << "X-Cache: CONFIG_NOCACHE[\r][\n]"
2023-07-25 09:08:36,247 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 << "X-MSEdge-Ref: Ref A: 39B37EEE9E9B4EE4B11A60DF307F2875 Ref B: TYO01EDGE1708 Ref C: 2023-07-25T00:08:36Z[\r][\n]"
2023-07-25 09:08:36,247 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 << "Date: Tue, 25 Jul 2023 00:08:36 GMT[\r][\n]"
2023-07-25 09:08:36,247 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 << "[\r][\n]"
2023-07-25 09:08:36,247 [6XwsTokB6nLRQ4sdYkx_-1] DEBUG http-outgoing-0 << "403 FORBIDDEN"

というログが出力され、X-MSDAVEXT_Error: 917656に関して調査を行いShare Point内で先進認証を使用しないアプリの許可を行いましたがクロールが成功しませんでした。

こちらに関して何かご存知の事がありましたらお力添え頂けると幸いです。

6

横から失礼します。Auth Requiredなのでntlm認証が許可されていないのではないでしょうか。