Failed to access to https://www.zopyx.de

discuss · March 5, 2016, 9:18am

(from github.com/zopyx)
I have setup a crawler for

https://www.zopyx.com
https://www.zopyx.de

However the sites are not crawled…both sites are up, both sites have proper SSL certs…what is the issue here?

24195 2016-03-05 10:11:28,133 [Crawler-AVNB_jEZ0WG_R1xvnirN-1-1] INFO Failed to access to https://www.zopyx.de
24196 2016-03-05 10:11:28,151 [Crawler-AVNB_jEZ0WG_R1xvnirN-1-3] INFO Failed to access to https://www.zopyx.com

discuss · March 5, 2016, 10:07am

(from github.com/marevol)
I got the folloing debug log.

2016-03-05 19:04:01,740 [Crawler-20160305190349-1-2] DEBUG Failed to access to https://www.zopyx.com
org.codelibs.fess.crawler.exception.CrawlingAccessException: I/O exception(sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target): https://www.zopyx.com
        at org.codelibs.fess.crawler.client.http.HcHttpClient.processHttpMethod(HcHttpClient.java:753) ~[fess-crawler-1.0.6-SNAPSHOT.jar:na]
        at org.codelibs.fess.crawler.client.http.HcHttpClient.doHttpMethod(HcHttpClient.java:555) ~[fess-crawler-1.0.6-SNAPSHOT.jar:na]
        at org.codelibs.fess.crawler.client.http.HcHttpClient.doGet(HcHttpClient.java:514) ~[fess-crawler-1.0.6-SNAPSHOT.jar:na]
        at org.codelibs.fess.crawler.client.AbstractCrawlerClient.execute(AbstractCrawlerClient.java:66) ~[fess-crawler-1.0.6-SNAPSHOT.jar:na]
        at org.codelibs.fess.crawler.client.FaultTolerantClient.execute(FaultTolerantClient.java:67) ~[fess-crawler-1.0.6-SNAPSHOT.jar:na]
        at org.codelibs.fess.crawler.CrawlerThread.run(CrawlerThread.java:164) [fess-crawler-1.0.6-SNAPSHOT.jar:na]
        at java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]
Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) ~[na:1.8.0_72]
        at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1949) ~[na:1.8.0_72]
        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:302) ~[na:1.8.0_72]
        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296) ~[na:1.8.0_72]
        at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1509) ~[na:1.8.0_72]
        at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216) ~[na:1.8.0_72]
        at sun.security.ssl.Handshaker.processLoop(Handshaker.java:979) ~[na:1.8.0_72]
        at sun.security.ssl.Handshaker.process_record(Handshaker.java:914) ~[na:1.8.0_72]
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1062) ~[na:1.8.0_72]
        at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375) ~[na:1.8.0_72]
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403) ~[na:1.8.0_72]
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387) ~[na:1.8.0_72]
        at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:394) ~[httpclient-4.5.1.jar:4.5.1]
        at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:353) ~[httpclient-4.5.1.jar:4.5.1]
        at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:134) ~[httpclient-4.5.1.jar:4.5.1]
        at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353) ~[httpclient-4.5.1.jar:4.5.1]
        at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380) ~[httpclient-4.5.1.jar:4.5.1]
        at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) ~[httpclient-4.5.1.jar:4.5.1]
        at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184) ~[httpclient-4.5.1.jar:4.5.1]
        at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88) ~[httpclient-4.5.1.jar:4.5.1]
        at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.1.jar:4.5.1]
        at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184) ~[httpclient-4.5.1.jar:4.5.1]
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) ~[httpclient-4.5.1.jar:4.5.1]
        at org.codelibs.fess.crawler.client.http.HcHttpClient.executeHttpClient(HcHttpClient.java:772) ~[fess-crawler-1.0.6-SNAPSHOT.jar:na]
        at org.codelibs.fess.crawler.client.http.HcHttpClient.processHttpMethod(HcHttpClient.java:592) ~[fess-crawler-1.0.6-SNAPSHOT.jar:na]
        ... 6 common frames omitted
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:387) ~[na:1.8.0_72]
        at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:230) ~[na:1.8.0_72]
        at sun.security.validator.Validator.validate(Validator.java:260) ~[na:1.8.0_72]
        at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324) ~[na:1.8.0_72]
        at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229) ~[na:1.8.0_72]
        at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124) ~[na:1.8.0_72]
        at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1491) ~[na:1.8.0_72]
        ... 26 common frames omitted
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141) ~[na:1.8.0_72]
        at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126) ~[na:1.8.0_72]
        at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280) ~[na:1.8.0_72]
        at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:382) ~[na:1.8.0_72]
        ... 32 common frames omitted

discuss · March 5, 2016, 10:10am

(from zopyx (Andreas Jung) · GitHub)
My servers use certificates from Let’s Encrypt…my assumption is that the
SSL stack used in Fess can not validate such certs (all browsers can).

2016-03-05 11:07 GMT+01:00 Shinsuke Sugaya notifications@github.com:

I got the folloing debug log.

2016-03-05 19:04:01,740 [Crawler-20160305190349-1-2] DEBUG Failed to access to https://www.zopyx.com
org.codelibs.fess.crawler.exception.CrawlingAccessException: I/O exception(sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target): https://www.zopyx.com
at org.codelibs.fess.crawler.client.http.HcHttpClient.processHttpMethod(HcHttpClient.java:753) ~[fess-crawler-1.0.6-SNAPSHOT.jar:na]
at org.codelibs.fess.crawler.client.http.HcHttpClient.doHttpMethod(HcHttpClient.java:555) ~[fess-crawler-1.0.6-SNAPSHOT.jar:na]
at org.codelibs.fess.crawler.client.http.HcHttpClient.doGet(HcHttpClient.java:514) ~[fess-crawler-1.0.6-SNAPSHOT.jar:na]
at org.codelibs.fess.crawler.client.AbstractCrawlerClient.execute(AbstractCrawlerClient.java:66) ~[fess-crawler-1.0.6-SNAPSHOT.jar:na]
at org.codelibs.fess.crawler.client.FaultTolerantClient.execute(FaultTolerantClient.java:67) ~[fess-crawler-1.0.6-SNAPSHOT.jar:na]
at org.codelibs.fess.crawler.CrawlerThread.run(CrawlerThread.java:164) [fess-crawler-1.0.6-SNAPSHOT.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]
Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) ~[na:1.8.0_72]
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1949) ~[na:1.8.0_72]
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:302) ~[na:1.8.0_72]
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296) ~[na:1.8.0_72]
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1509) ~[na:1.8.0_72]
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216) ~[na:1.8.0_72]
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:979) ~[na:1.8.0_72]
at sun.security.ssl.Handshaker.process_record(Handshaker.java:914) ~[na:1.8.0_72]
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1062) ~[na:1.8.0_72]
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375) ~[na:1.8.0_72]
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403) ~[na:1.8.0_72]
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387) ~[na:1.8.0_72]
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:394) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:353) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:134) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184) ~[httpclient-4.5.1.jar:4.5.1]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) ~[httpclient-4.5.1.jar:4.5.1]
at org.codelibs.fess.crawler.client.http.HcHttpClient.executeHttpClient(HcHttpClient.java:772) ~[fess-crawler-1.0.6-SNAPSHOT.jar:na]
at org.codelibs.fess.crawler.client.http.HcHttpClient.processHttpMethod(HcHttpClient.java:592) ~[fess-crawler-1.0.6-SNAPSHOT.jar:na]
… 6 common frames omitted
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:387) ~[na:1.8.0_72]
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:230) ~[na:1.8.0_72]
at sun.security.validator.Validator.validate(Validator.java:260) ~[na:1.8.0_72]
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324) ~[na:1.8.0_72]
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229) ~[na:1.8.0_72]
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124) ~[na:1.8.0_72]
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1491) ~[na:1.8.0_72]
… 26 common frames omitted
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141) ~[na:1.8.0_72]
at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126) ~[na:1.8.0_72]
at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280) ~[na:1.8.0_72]
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:382) ~[na:1.8.0_72]
… 32 common frames omitted

—
Reply to this email directly or view it on GitHub
Failed to access to https://www.zopyx.de · Issue #412 · codelibs/fess · GitHub.

discuss · March 5, 2016, 10:23am

(from github.com/marevol)
This is JVM problem.
http://stackoverflow.com/questions/34110426/does-java-support-lets-encrypt-certificates

discuss · March 6, 2016, 3:35pm

(from github.com/zopyx)
This helped