Crawling basic authentication sites

I am trying to crawl a basic authentication site. I configured “Web Authentication” option for this site but still the site is not getting crawled. As per crawl logs, the site is redirecting to login page and only indexing the login page. I tried configuring Request Header with Basic Authentication credentials, but still the behavior is same.
Seems crawler is not able to bypass login page using Request Header or Web Authentication. I tested this using CURL command and able to bypass the login page.

Can you please help debugging this issue further.

Basic authentication does not have a login page.
If a login page is displayed, it’s Form authentication.
Therefore I think the target site might support both Basic and Form authentication.

To change a log level for crawler, modify Default Crawler job at Scheduler page:

return container.getComponent("crawlJob").logLevel("debug").execute(executor);

and then check logs/fess-crawler.log.

Thanks. I am able to find the issue and now it is crawling fine.