解决: httpclient ssl 验证导致死锁问题
线上图片下载服务器平时运行正常,最近突然出现一种比较奇怪的现象,只接受请求,但却没有处理请求,最开始怀疑下载线程挂掉了,dump 项目线程后发现异常:
"pool-2-thread-1" prio=10 tid=0x00007f7fbc7c5000 nid=0x4e9 runnable [0x00007f800a05e000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:152) at java.net.SocketInputStream.read(SocketInputStream.java:122) at sun.security.ssl.InputRecord.readFully(InputRecord.java:442) at sun.security.ssl.InputRecord.readV3Record(InputRecord.java:554) at sun.security.ssl.InputRecord.read(InputRecord.java:509) at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927) - locked <0x00000006f886b898> (a java.lang.Object) at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312) - locked <0x00000006f886b948> (a java.lang.Object) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:275) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:254) at org.apache.http.impl.conn.HttpClientConnectionOperator.connect(HttpClientConnectionOperator.java:117) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:314) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:363) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:219) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:86) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:186) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)
所有下载线程都处于这种状态,项目中实际上时没有证书的,ssl验证肯定也不会过,java.net.SocketInputStream.socketRead0 基本上也不可能读取到数据,那就很可能是超时未设置或者设置之后未生效,后来发现,实际上是httpclient 4.3.6之前的一个bug,项目中用的是4.3.2,问题在httpclient/src/main/java/org/apache/http/conn/ssl/SSLConnectionSocketFactory.java:
参考链接:https://github.com/apache/httpcomponents-client/commit/d954cd287dfcdad8f153e61181e20d253175ca8c
因此将httpclient 版本更新至4.3.6之后就正常了。