生产环境,请求方调用我方地址,发生异常NoHttpResponseException,错误详情:
org.apache.http.NoHttpResponseException: 127.0.0.1:9000 failed to respondat org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157)at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)at com.http.CommonHttpClient.sendPost(CommonHttpClient.java:96)at com.http.CommonHttpClient.lambda$main$0(CommonHttpClient.java:121)at java.util.stream.ForEachOps$ForEachOp$OfInt.accept(ForEachOps.java:205)at java.util.Spliterators$IntArraySpliterator.forEachRemaining(Spliterators.java:1032)at java.util.Spliterator$OfInt.forEachRemaining(Spliterator.java:693)at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:291)at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)at java.util.concurrent.ForkJoinPool$WorkQueue.execLocalTasks(ForkJoinPool.java:1040)at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1058)at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
看一下报错的源代码,相关注释提示服务端关闭了连接导致的:
想要解决问题,首先需要复现一下,于是本地启动服务,设置tomcat保活时间1s,
server:tomcat:keep-alive-timeout: 1000
使用common-httpclient4.5.10连接池,模拟一下请求,这里设置:
connManager.setMaxTotal(5);
connManager.setDefaultMaxPerRoute(5);
因为连接就是一组 ip:port 抽象,所以在模拟代码里获取一下本地的连接端口,在main测试中模拟20个请求,发送请求后休眠一会,推进连接过期,服务端关闭连接:
BasicHttpContext httpContext = new BasicHttpContext();
final long l = System.currentTimeMillis();
try {response = httpClient.execute(httpPost, httpContext);local = JSONObject.parseObject(JSON.toJSONString(httpContext.getAttribute("http.connection"))).getString("localPort");HttpEntity httpEntity = response.getEntity();result = EntityUtils.toString(httpEntity, "utf-8");System.out.println("normal" + " : " + l + " : " + result + " : " + local);
} catch (Exception e) {System.out.println("error " + " : " + l + " : " + local);e.printStackTrace();
}
-----------------------------------------------------------------------------------
public static void main(String[] args) {try {IntStream.range(1, 20).parallel().sorted().forEach(e -> {try {sendPost("http://127.0.0.1:9000/receive", "{}");TimeUnit.MILLISECONDS.sleep(995L);} catch (Exception exception) {exception.printStackTrace();}});} catch (Exception e) {e.printStackTrace();}}
可以看到本地和服务器共有5个连接,51573,51574,51575,51576,51577,其中有两笔异常数据报错:NoHttpResponseException,其中75,76两个端口连接后续没有被继续使用,怀疑已经被关闭了看一下抓包数据:
这两个端口处确实已经四次挥手关闭连接了,看样子应该是连接已不可用,发起请求时返回的错误。如果连接不可用,连接池会清除掉不可用连接,重新创建新的连接。再看下httpclient默认的保活策略:
问题也复现了,相关原因也基本确定了,解决办法就是保证conn的可用性:
1.关闭keep-alive,设置header Connection close
2.客户端实现一下保活策略,时间小于服务端的keep-alive-timeout,连接提前过期释放;当然应该配置合理的连接池大小,不必过大
3.客户端发生此类错误时,应该启用重试机制