Is there any way to set connection time to live to the web crawler? I’m using the version of 13.
In the middle of the web crawling, I got some IO Runtime Exception (NoHttpResponseException) from random large files. After some investigations, I came to a hypothesis that keepalive set on target sever might be longer than the TTL set on FESS. It seems connections in the pool left longer that the keepalive is used to obtain the data from the server, it causes NoHttpResponseException.