一次tomcat实战jvm线上问题定位排查和解决

问题分析：

问题来源是客户通过闲鱼找到了我这边进行问题的排查，给我发了一个报错日志，让我帮忙分析。

首先他的项目比较老，很多年以前开发的软件，但是具体做什么业务我不知道。下面来看日志的报错分析。

问题：tomcat自动关闭，频繁关闭。

jvm报错的一个错误信息：


Current thread (0x0000000079779800):  JavaThread "C2 CompilerThread2" daemon [_thread_blocked_trans, id=3012, stack(0x000000007aab0000,0x000000007abb0000)]siginfo: ExceptionCode=0x0, ExceptionInformation=0x0000000000000001 0x00000000009f0980 Registers:
RAX=0x00000000009f0000, RBX=0x0000000079779800, RCX=0x0000000000000980, RDX=0x0000000003380f50
RSP=0x000000007abaf5b0, RBP=0x0000000003380f50, RSI=0x0000000000000000, RDI=0x00000000792a11e0
R8 =0x0000000000000000, R9 =0x00000000796f1000, R10=0x0000000000000000, R11=0x0000000000000246
R12=0x0000000000000000, R13=0x000000007977bf50, R14=0x0000000000000000, R15=0x000000006ff98cc8
RIP=0x000000006f9a8688, EFLAGS=0x0000000000010202Top of Stack: (sp=0x000000007abaf5b0)
0x000000007abaf5b0:   0000000000000000 0000000003380f50
0x000000007abaf5c0:   0000000000000002 0000000000001388
0x000000007abaf5d0:   00000000792a11e0 000000006fbb8b04
0x000000007abaf5e0:   0000000079779800 0000000003380f50
0x000000007abaf5f0:   0000000000000000 000000007977bf50
0x000000007abaf600:   00000000792a1520 00000000792a1be0
0x000000007abaf610:   0000000003380f50 000000006fa47243
0x000000007abaf620:   000000007977bf50 0000000002dc19c0
0x000000007abaf630:   0000000079779800 00000000792a1520
0x000000007abaf640:   000000000ddab180 000000006fa47470
0x000000007abaf650:   0000000002dc19c0 0000000079779800
0x000000007abaf660:   0000000079779800 00000000ffffffff
0x000000007abaf670:   0000000000000000 000000006fc3ce4b
0x000000007abaf680:   0000000000000006 0000000002fc56f5
0x000000007abaf690:   0000000000000000 000000007940ff70
0x000000007abaf6a0:   0000000000000000 0000000079410bf0 Instructions: (pc=0x000000006f9a8688)
0x000000006f9a8668:   cc cc cc cc cc cc cc cc 48 83 ec 28 48 c1 e9 04
0x000000006f9a8678:   8b 05 a2 d1 80 00 48 23 c8 48 8b 05 90 d1 80 00
0x000000006f9a8688:   c7 04 08 01 00 00 00 eb 00 48 83 c4 28 c3 cc cc
0x000000006f9a8698:   cc cc cc cc cc cc cc cc 48 89 5c 24 08 57 48 83 Register to memory mapping:RAX=0x00000000009f0000 is an unknown value
RBX=0x0000000079779800 is a thread
RCX=0x0000000000000980 is an unknown value
RDX=0x0000000003380f50 is an unknown value
RSP=0x000000007abaf5b0 is pointing into the stack for thread: 0x0000000079779800
RBP=0x0000000003380f50 is an unknown value
RSI=0x0000000000000000 is an unknown value
RDI=0x00000000792a11e0 is an unknown value
R8 =0x0000000000000000 is an unknown value
R9 =0x00000000796f1000 is an unknown value
R10=0x0000000000000000 is an unknown value
R11=0x0000000000000246 is an unknown value
R12=0x0000000000000000 is an unknown value
R13=0x000000007977bf50 is an unknown value
R14=0x0000000000000000 is an unknown value
R15=0x000000006ff98cc8 is an unknown valueStack: [0x000000007aab0000,0x000000007abb0000],  sp=0x000000007abaf5b0,  free space=1021k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [jvm.dll+0x8688]
V  [jvm.dll+0x218b04]
V  [jvm.dll+0xa7243]
V  [jvm.dll+0xa7470]
V  [jvm.dll+0x2461df]
V  [jvm.dll+0x29eb9a]
C  [msvcr100.dll+0x21d9f]
C  [msvcr100.dll+0x21e3b]
C  [KERNEL32.DLL+0x17ac4]
C  [ntdll.dll+0x5a8c1]

错误分析

异常线程:
错误日志中的 JavaThread "C2 CompilerThread2" daemon 表示这是 Java 的 JIT 编译器线程崩溃了。C2 CompilerThread2 是 HotSpot JIT 编译器的线程，它负责编译 Java 字节码为机器码。如果 JIT 编译器遇到问题，可能会导致 JVM 崩溃。
指令和崩溃点: 在日志中，崩溃指令位于 jvm.dll 中，具体位置是 0x000000006f9a8688。这个指令处在 JVM 的本地代码部分，通常意味着有 JVM 内部的缺陷、驱动问题或者库文件冲突。
调用栈:
- msvcr100.dll 是 Microsoft Visual C++ Redistributable 的一个 DLL 文件，通常是由 C++ 编写的程序依赖的。如果你的程序使用了某些本地代码或与 C++ 库有交互（如 JNI 调用），这个 DLL 可能会受到影响。
- KERNEL32.DLL 和 ntdll.dll 是 Windows 操作系统的系统级 DLL，通常表示系统底层问题。
可能的原因:
- JIT 编译器缺陷：某些特定的字节码或代码模式可能触发了 JIT 编译器的 bug，导致内存损坏或无效访问。
- 操作系统或硬件问题：比如内存损坏、硬件冲突或者操作系统本身的问题（如 Windows 更新导致某些系统库出问题）。
- 依赖的本地库问题：如果你的程序依赖某些 C/C++ 库，或者与某些本地代码交互，可能是本地库出现了问题。

当然从上面来看不一定是可能的问题，上面只是做一个普通分析，通过看jvm回收的日志来看确实是正常的.

Other Threads:0x00000000783de000 VMThread [stack: 0x000000006f840000,0x000000006f940000] [id=928]0x0000000079878800 WatcherThread [stack: 0x000000007b8b0000,0x000000007b9b0000] [id=7408]VM state:synchronizing (normal execution)VM Mutex/Monitor currently owned by a thread:  ([mutex/lock_event])
[0x00000000033814d0] Safepoint_lock - owner thread: 0x00000000783de000
[0x00000000033807d0] Threads_lock - owner thread: 0x00000000783de000Heap:garbage-first heap   total 16777216K, used 397942K [0x0000000080000000, 0x0000000080804000, 0x0000000880000000)region size 8192K, 50 young (409600K), 7 survivors (57344K)Metaspace       used 32537K, capacity 33067K, committed 33280K, reserved 34816K

Java HotSpot(TM) 64-Bit Server VM (25.191-b12) for windows-amd64 JRE (1.8.0_191-b12), built on Oct  6 2018 09:29:03 by "java_re" with MS VC++ 10.0 (VS2010)
Memory: 4k page, physical 67100272k(64186852k free), swap 83877488k(80904696k free)
CommandLine flags: -XX:G1HeapRegionSize=2097152 -XX:InitialHeapSize=25769803776 -XX:MaxGCPauseMillis=200 -XX:MaxHeapSize=25769803776 -XX:+PrintGC -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:ReservedCodeCacheSize=268435456 -XX:ThreadStackSize=512 -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseG1GC -XX:-UseLargePagesIndividualAllocation 
2024-12-09T10:20:02.067+0800: 10.477: [GC pause (G1 Evacuation Pause) (young), 0.3448343 secs][Parallel Time: 151.8 ms, GC Workers: 23][GC Worker Start (ms): Min: 10477.8, Avg: 10478.0, Max: 10478.2, Diff: 0.4][Ext Root Scanning (ms): Min: 0.1, Avg: 0.6, Max: 3.0, Diff: 3.0, Sum: 12.7][Update RS (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.1][Processed Buffers: Min: 0, Avg: 0.0, Max: 0, Diff: 0, Sum: 0][Scan RS (ms): Min: 0.0, Avg: 0.1, Max: 0.3, Diff: 0.2, Sum: 3.4][Code Root Scanning (ms): Min: 0.0, Avg: 1.2, Max: 3.4, Diff: 3.4, Sum: 26.9][Object Copy (ms): Min: 113.4, Avg: 120.9, Max: 123.8, Diff: 10.4, Sum: 2781.8][Termination (ms): Min: 25.1, Avg: 28.1, Max: 34.9, Diff: 9.8, Sum: 646.3][Termination Attempts: Min: 4117, Avg: 4768.7, Max: 5237, Diff: 1120, Sum: 109679][GC Worker Other (ms): Min: 0.1, Avg: 0.4, Max: 1.1, Diff: 1.0, Sum: 8.1][GC Worker Total (ms): Min: 151.0, Avg: 151.3, Max: 151.5, Diff: 0.6, Sum: 3479.3][GC Worker End (ms): Min: 10629.1, Avg: 10629.2, Max: 10629.4, Diff: 0.2][Code Root Fixup: 0.2 ms][Code Root Purge: 0.2 ms][Clear CT: 0.8 ms][Other: 191.8 ms][Choose CSet: 0.0 ms][Ref Proc: 186.5 ms][Ref Enq: 0.8 ms][Redirty Cards: 0.7 ms][Humongous Register: 0.3 ms][Humongous Reclaim: 0.2 ms][Free CSet: 1.8 ms][Eden: 1228.0M(1228.0M)->0.0B(1074.0M) Survivors: 0.0B->154.0M Heap: 1229.0M(24.0G)->193.9M(24.0G)][Times: user=2.39 sys=1.13, real=0.34 secs] 
2024-12-09T10:20:03.607+0800: 12.028: [GC pause (Metadata GC Threshold) (young) (initial-mark), 0.0248216 secs][Parallel Time: 11.3 ms, GC Workers: 23][GC Worker Start (ms): Min: 12029.5, Avg: 12029.7, Max: 12029.9, Diff: 0.4][Ext Root Scanning (ms): Min: 1.4, Avg: 1.6, Max: 2.0, Diff: 0.7, Sum: 36.4][Update RS (ms): Min: 0.4, Avg: 0.6, Max: 0.7, Diff: 0.3, Sum: 13.9][Processed Buffers: Min: 1, Avg: 1.0, Max: 1, Diff: 0, Sum: 23][Scan RS (ms): Min: 0.1, Avg: 0.4, Max: 0.9, Diff: 0.8, Sum: 8.6][Code Root Scanning (ms): Min: 0.0, Avg: 0.2, Max: 3.4, Diff: 3.4, Sum: 4.7][Object Copy (ms): Min: 4.7, Avg: 7.8, Max: 8.1, Diff: 3.4, Sum: 180.2][Termination (ms): Min: 0.1, Avg: 0.1, Max: 0.2, Diff: 0.1, Sum: 3.2][Termination Attempts: Min: 12, Avg: 31.8, Max: 42, Diff: 30, Sum: 731][GC Worker Other (ms): Min: 0.0, Avg: 0.1, Max: 0.3, Diff: 0.2, Sum: 2.8][GC Worker Total (ms): Min: 10.6, Avg: 10.9, Max: 11.2, Diff: 0.6, Sum: 249.8][GC Worker End (ms): Min: 12040.4, Avg: 12040.5, Max: 12040.7, Diff: 0.2][Code Root Fixup: 0.1 ms][Code Root Purge: 0.1 ms][Clear CT: 0.7 ms][Other: 12.7 ms][Choose CSet: 0.0 ms][Ref Proc: 7.7 ms][Ref Enq: 0.1 ms][Redirty Cards: 0.6 ms][Humongous Register: 0.1 ms][Humongous Reclaim: 0.2 ms][Free CSet: 0.9 ms][Eden: 146.0M(1074.0M)->0.0B(1178.0M) Survivors: 154.0M->50.0M Heap: 338.9M(24.0G)->88.9M(24.0G)][Times: user=0.02 sys=0.00, real=0.02 secs] 
2024-12-09T10:20:03.631+0800: 12.053: [GC concurrent-root-region-scan-start]
2024-12-09T10:20:03.637+0800: 12.059: [GC concurrent-root-region-scan-end, 0.0060234 secs]
2024-12-09T10:20:03.637+0800: 12.059: [GC concurrent-mark-start]
2024-12-09T10:20:03.654+0800: 12.075: [GC concurrent-mark-end, 0.0157671 secs]
2024-12-09T10:20:03.654+0800: 12.075: [GC remark 2024-12-09T10:20:03.654+0800: 12.075: [Finalize Marking, 0.0010367 secs] 2024-12-09T10:20:03.655+0800: 12.076: [GC ref-proc, 0.0002724 secs] 2024-12-09T10:20:03.655+0800: 12.077: [Unloading, 0.0039069 secs], 0.0082044 secs][Times: user=0.00 sys=0.00, real=0.01 secs] 
2024-12-09T10:20:03.662+0800: 12.084: [GC cleanup 90M->51M(24G), 0.0110430 secs][Times: user=0.01 sys=0.34, real=0.01 secs] 
2024-12-09T10:20:03.673+0800: 12.095: [GC concurrent-cleanup-start]
2024-12-09T10:20:03.673+0800: 12.095: [GC concurrent-cleanup-end, 0.0000648 secs]
2024-12-09T10:20:07.916+0800: 16.363: [GC pause (Metadata GC Threshold) (young) (initial-mark), 0.0374830 secs][Parallel Time: 19.9 ms, GC Workers: 23][GC Worker Start (ms): Min: 16364.9, Avg: 16365.1, Max: 16365.3, Diff: 0.4][Ext Root Scanning (ms): Min: 4.0, Avg: 4.2, Max: 4.8, Diff: 0.7, Sum: 96.1][Update RS (ms): Min: 0.0, Avg: 0.0, Max: 0.1, Diff: 0.1, Sum: 0.3][Processed Buffers: Min: 0, Avg: 2.0, Max: 16, Diff: 16, Sum: 45][Scan RS (ms): Min: 0.0, Avg: 0.1, Max: 0.1, Diff: 0.1, Sum: 1.2][Code Root Scanning (ms): Min: 0.5, Avg: 1.3, Max: 2.8, Diff: 2.3, Sum: 30.8][Object Copy (ms): Min: 12.1, Avg: 13.6, Max: 14.4, Diff: 2.3, Sum: 312.8][Termination (ms): Min: 0.0, Avg: 0.1, Max: 0.1, Diff: 0.1, Sum: 2.2][Termination Attempts: Min: 1, Avg: 27.9, Max: 38, Diff: 37, Sum: 641][GC Worker Other (ms): Min: 0.0, Avg: 0.1, Max: 0.3, Diff: 0.3, Sum: 3.1][GC Worker Total (ms): Min: 19.2, Avg: 19.4, Max: 19.7, Diff: 0.5, Sum: 446.6][GC Worker End (ms): Min: 16384.4, Avg: 16384.5, Max: 16384.7, Diff: 0.2][Code Root Fixup: 0.4 ms][Code Root Purge: 0.1 ms][Clear CT: 0.6 ms][Other: 16.4 ms][Choose CSet: 0.0 ms][Ref Proc: 10.5 ms][Ref Enq: 0.1 ms][Redirty Cards: 0.6 ms][Humongous Register: 0.1 ms][Humongous Reclaim: 0.2 ms][Free CSet: 1.0 ms][Eden: 386.0M(1178.0M)->0.0B(1162.0M) Survivors: 50.0M->66.0M Heap: 436.0M(24.0G)->66.0M(24.0G)][Times: user=0.25 sys=0.00, real=0.04 secs] 
2024-12-09T10:20:07.953+0800: 16.401: [GC concurrent-root-region-scan-start]
2024-12-09T10:20:07.962+0800: 16.411: [GC concurrent-root-region-scan-end, 0.0096959 secs]
2024-12-09T10:20:07.962+0800: 16.411: [GC concurrent-mark-start]
2024-12-09T10:20:07.970+0800: 16.418: [GC concurrent-mark-end, 0.0074401 secs]
2024-12-09T10:20:07.970+0800: 16.418: [GC remark 2024-12-09T10:20:07.970+0800: 16.418: [Finalize Marking, 0.0009971 secs] 2024-12-09T10:20:07.971+0800: 16.420: [GC ref-proc, 0.0003945 secs] 2024-12-09T10:20:07.972+0800: 16.420: [Unloading, 0.0060639 secs], 0.0101454 secs][Times: user=0.00 sys=0.00, real=0.01 secs] 
2024-12-09T10:20:07.981+0800: 16.429: [GC cleanup 67M->67M(24G), 0.0105597 secs][Times: user=0.00 sys=0.36, real=0.01 secs] 
2024-12-09T10:20:18.563+0800: 27.052: [GC pause (Metadata GC Threshold) (young) (initial-mark), 0.0584057 secs][Parallel Time: 38.4 ms, GC Workers: 23][GC Worker Start (ms): Min: 27053.6, Avg: 27053.8, Max: 27053.9, Diff: 0.3][Ext Root Scanning (ms): Min: 2.7, Avg: 3.0, Max: 3.6, Diff: 0.9, Sum: 69.2][Update RS (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.1][Processed Buffers: Min: 0, Avg: 0.0, Max: 0, Diff: 0, Sum: 0][Scan RS (ms): Min: 0.0, Avg: 0.1, Max: 0.3, Diff: 0.3, Sum: 2.8][Code Root Scanning (ms): Min: 0.7, Avg: 1.8, Max: 7.0, Diff: 6.3, Sum: 41.4][Object Copy (ms): Min: 27.1, Avg: 32.1, Max: 33.2, Diff: 6.1, Sum: 737.6][Termination (ms): Min: 0.0, Avg: 0.6, Max: 0.8, Diff: 0.8, Sum: 14.2][Termination Attempts: Min: 1, Avg: 190.3, Max: 213, Diff: 212, Sum: 4378][GC Worker Other (ms): Min: 0.0, Avg: 0.1, Max: 0.3, Diff: 0.3, Sum: 3.4][GC Worker Total (ms): Min: 37.5, Avg: 37.8, Max: 38.0, Diff: 0.5, Sum: 868.8][GC Worker End (ms): Min: 27091.4, Avg: 27091.5, Max: 27091.7, Diff: 0.3][Code Root Fixup: 0.8 ms][Code Root Purge: 0.3 ms][Clear CT: 0.6 ms][Other: 18.3 ms][Choose CSet: 0.0 ms][Ref Proc: 11.4 ms][Ref Enq: 0.1 ms][Redirty Cards: 0.6 ms][Humongous Register: 0.1 ms][Humongous Reclaim: 0.1 ms][Free CSet: 2.3 ms][Eden: 1064.0M(1162.0M)->0.0B(1106.0M) Survivors: 66.0M->122.0M Heap: 1130.0M(24.0G)->122.0M(24.0G)][Times: user=0.84 sys=0.08, real=0.06 secs] 
2024-12-09T10:20:18.622+0800: 27.111: [GC concurrent-root-region-scan-start]
2024-12-09T10:20:18.644+0800: 27.134: [GC concurrent-root-region-scan-end, 0.0225791 secs]
2024-12-09T10:20:18.644+0800: 27.134: [GC concurrent-mark-start]
2024-12-09T10:20:18.649+0800: 27.138: [GC concurrent-mark-end, 0.0040234 secs]
2024-12-09T10:20:18.649+0800: 27.138: [GC remark 2024-12-09T10:20:18.649+0800: 27.138: [Finalize Marking, 0.0009443 secs] 2024-12-09T10:20:18.650+0800: 27.139: [GC ref-proc, 0.0002498 secs] 2024-12-09T10:20:18.650+0800: 27.139: [Unloading, 0.0083678 secs], 0.0124049 secs][Times: user=0.00 sys=0.00, real=0.01 secs] 
2024-12-09T10:20:18.662+0800: 27.151: [GC cleanup 125M->125M(24G), 0.0102792 secs][Times: user=0.01 sys=0.34, real=0.01 secs] 
2024-12-09T10:20:27.028+0800: 35.534: [GC pause (G1 Evacuation Pause) (young), 0.0844040 secs][Parallel Time: 61.7 ms, GC Workers: 23][GC Worker Start (ms): Min: 35534.0, Avg: 35534.2, Max: 35534.4, Diff: 0.3][Ext Root Scanning (ms): Min: 0.6, Avg: 1.2, Max: 9.7, Diff: 9.0, Sum: 28.5][Update RS (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.1][Processed Buffers: Min: 0, Avg: 0.0, Max: 0, Diff: 0, Sum: 0][Scan RS (ms): Min: 0.0, Avg: 0.1, Max: 0.3, Diff: 0.3, Sum: 3.1][Code Root Scanning (ms): Min: 0.0, Avg: 3.9, Max: 16.8, Diff: 16.8, Sum: 89.7][Object Copy (ms): Min: 42.3, Avg: 54.9, Max: 58.1, Diff: 15.8, Sum: 1263.8][Termination (ms): Min: 0.0, Avg: 0.9, Max: 1.2, Diff: 1.2, Sum: 21.8][Termination Attempts: Min: 1, Avg: 279.7, Max: 326, Diff: 325, Sum: 6433][GC Worker Other (ms): Min: 0.0, Avg: 0.1, Max: 0.3, Diff: 0.3, Sum: 3.3][GC Worker Total (ms): Min: 61.1, Avg: 61.3, Max: 61.6, Diff: 0.5, Sum: 1410.4][GC Worker End (ms): Min: 35595.4, Avg: 35595.5, Max: 35595.7, Diff: 0.3][Code Root Fixup: 1.3 ms][Code Root Purge: 0.3 ms][Clear CT: 0.7 ms][Other: 20.4 ms][Choose CSet: 0.0 ms][Ref Proc: 15.0 ms][Ref Enq: 0.1 ms][Redirty Cards: 0.6 ms][Humongous Register: 0.3 ms][Humongous Reclaim: 0.1 ms][Free CSet: 2.8 ms][Eden: 1106.0M(1106.0M)->0.0B(1074.0M) Survivors: 122.0M->154.0M Heap: 1228.0M(24.0G)->189.5M(24.0G)][Times: user=1.52 sys=0.00, real=0.08 secs] 
2024-12-09T10:20:58.788+0800: 67.315: [GC pause (G1 Evacuation Pause) (young), 0.0755674 secs][Parallel Time: 59.6 ms, GC Workers: 23][GC Worker Start (ms): Min: 67315.8, Avg: 67315.9, Max: 67316.1, Diff: 0.4][Ext Root Scanning (ms): Min: 0.5, Avg: 1.2, Max: 9.5, Diff: 9.0, Sum: 26.9][Update RS (ms): Min: 0.0, Avg: 0.7, Max: 1.3, Diff: 1.3, Sum: 16.1][Processed Buffers: Min: 0, Avg: 3.2, Max: 15, Diff: 15, Sum: 74][Scan RS (ms): Min: 0.0, Avg: 0.4, Max: 0.7, Diff: 0.6, Sum: 8.7][Code Root Scanning (ms): Min: 0.0, Avg: 1.0, Max: 1.8, Diff: 1.8, Sum: 23.4][Object Copy (ms): Min: 49.4, Avg: 55.1, Max: 56.2, Diff: 6.8, Sum: 1266.7][Termination (ms): Min: 0.0, Avg: 0.4, Max: 0.6, Diff: 0.5, Sum: 10.0][Termination Attempts: Min: 2, Avg: 133.0, Max: 162, Diff: 160, Sum: 3060][GC Worker Other (ms): Min: 0.0, Avg: 0.2, Max: 0.3, Diff: 0.3, Sum: 3.9][GC Worker Total (ms): Min: 58.6, Avg: 59.0, Max: 59.2, Diff: 0.6, Sum: 1355.9][GC Worker End (ms): Min: 67374.8, Avg: 67374.9, Max: 67375.1, Diff: 0.3][Code Root Fixup: 0.6 ms][Code Root Purge: 0.3 ms][Clear CT: 0.6 ms][Other: 14.5 ms][Choose CSet: 0.0 ms][Ref Proc: 9.7 ms][Ref Enq: 0.1 ms][Redirty Cards: 0.6 ms][Humongous Register: 0.3 ms][Humongous Reclaim: 0.1 ms][Free CSet: 2.1 ms][Eden: 1074.0M(1074.0M)->0.0B(1082.0M) Survivors: 154.0M->146.0M Heap: 1263.5M(24.0G)->295.7M(24.0G)][Times: user=1.39 sys=0.06, real=0.08 secs]

从jvm看不出问题那就看tomcat日志错误：

第一个报错：

十二月 07, 2024 3:44:43 下午 org.apache.catalina.core.StandardWrapperValve invoke
严重: 在路径为的上下文中，Servlet[springmvc]的Servlet.service（）引发了具有根本原因的异常Request processing failed; nested exception is org.springframework.orm.hibernate3.HibernateOptimisticLockingFailureException: Batch update returned unexpected row count from update [0]; actual row count: 0; expected: 1; nested exception is org.hibernate.StaleStateException: Batch update returned unexpected row count from update [0]; actual row count: 0; expected: 1
org.hibernate.StaleStateException: Batch update returned unexpected row count from update [0]; actual row count: 0; expected: 1at org.hibernate.jdbc.Expectations$BasicExpectation.checkBatched(Expectations.java:85)at org.hibernate.jdbc.Expectations$BasicExpectation.verifyOutcome(Expectations.java:70)at org.hibernate.jdbc.BatchingBatcher.checkRowCounts(BatchingBatcher.java:90)at org.hibernate.jdbc.BatchingBatcher.doExecuteBatch(BatchingBatcher.java:70)at org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:268)at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:268)at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:189)at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:321)at org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:51)at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1216)at org.hibernate.impl.SessionImpl.managedFlush(SessionImpl.java:383)at org.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:133)at org.hibernate.ejb.TransactionImpl.commit(TransactionImpl.java:76)at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:513)at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:755)at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:724)at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:475)at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:270)at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:94)at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:631)at com.hz.eshop.service.impl.CartServiceImpl$$EnhancerByCGLIB$$62d9bd09.getCurrent(<generated>)at com.hz.eshop.controller.shop.CartController.getCartItemCount(CartController.java:373)at com.hz.eshop.controller.shop.CartController$$FastClassByCGLIB$$7d8a5f04.invoke(<generated>)at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:698)at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:631)at com.hz.eshop.controller.shop.CartController$$EnhancerByCGLIB$$5b69eab2.getCartItemCount(<generated>)at sun.reflect.GeneratedMethodAccessor628.invoke(Unknown Source)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:498)at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:219)at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:132)at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104)at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:745)at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:686)at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80)at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:925)at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:856)at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:936)at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:838)at javax.servlet.http.HttpServlet.service(HttpServlet.java:647)at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:812)at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)at com.planetj.servlet.filter.compression.CompressingFilter.doFilter(CompressingFilter.java:270)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)at com.hz.eshop.web.filter.SiteStatusFilter.doFilterInternal(SiteStatusFilter.java:38)at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)at com.hz.eshop.web.filter.XxxFilter.doFilter(XxxFilter.java:19)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)at com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:168)at com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:233)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:604)at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:165)at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:104)at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:1025)at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:452)at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1201)at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:654)at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:319)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)at java.lang.Thread.run(Thread.java:748)

这个错看起来就是更新了事务处理出了问题，是批量更新操作，这是其中一个问题，但是不至于到tomcat直接挂掉。

第二个错误：


十二月 09, 2024 10:25:33 上午 org.apache.coyote.http11.AbstractHttp11Processor process
信息: 解析 HTTP 请求 header 错误注意：HTTP请求解析错误的进一步发生将记录在DEBUG级别。
java.lang.IllegalArgumentException: 在请求目标中找到无效字符。有效字符在RFC 7230和RFC 3986中定义at org.apache.coyote.http11.InternalNioInputBuffer.parseRequestLine(InternalNioInputBuffer.java:343)at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1108)at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:654)at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1782)at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1741)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)at java.lang.Thread.run(Thread.java:748)十二月 09, 2024 10:25:33 上午 org.apache.coyote.http11.AbstractHttp11Processor process
信息: 解析 HTTP 请求 header 错误注意：HTTP请求解析错误的进一步发生将记录在DEBUG级别。
java.lang.IllegalArgumentException: 在请求目标中找到无效字符。有效字符在RFC 7230和RFC 3986中定义at org.apache.coyote.http11.InternalNioInputBuffer.parseRequestLine(InternalNioInputBuffer.java:343)at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1108)at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:654)at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1782)at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1741)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)at java.lang.Thread.run(Thread.java:748)十二月 09, 2024 10:25:34 上午 org.apache.coyote.http11.AbstractHttp11Processor process
信息: 解析 HTTP 请求 header 错误注意：HTTP请求解析错误的进一步发生将记录在DEBUG级别。
java.lang.IllegalArgumentException: 在请求目标中找到无效字符。有效字符在RFC 7230和RFC 3986中定义at org.apache.coyote.http11.InternalNioInputBuffer.parseRequestLine(InternalNioInputBuffer.java:343)at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1108)at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:654)at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1782)at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1741)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)at java.lang.Thread.run(Thread.java:748)

这个这个错误一看就是有人在尝试通过非法字符篡改请求接口，尤其是当攻击者试图利用一些不规范的 HTTP 请求来攻击系统时，因为服务端的编码都是uft-8的字符，走客户端不可能出现不一样的编码，所以这很明显了是在url非法请求，这种直接去操作人家接口很明显了。

第三个错误：

十二月 11, 2024 8:48:16 上午 org.apache.catalina.core.StandardWrapperValve invoke
严重: 在路径为的上下文中，Servlet[springmvc]的Servlet.service（）引发了具有根本原因的异常Request processing failed; nested exception is java.lang.ArrayIndexOutOfBoundsException: 0
java.lang.ArrayIndexOutOfBoundsException: 0at com.hz.eshop.controller.shop.member.OrderController.OrderInvoice(OrderController.java:2546)at com.hz.eshop.controller.shop.member.OrderController$$FastClassByCGLIB$$3b69e21e.invoke(<generated>)at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:698)at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:631)at com.hz.eshop.controller.shop.member.OrderController$$EnhancerByCGLIB$$8bfa4a0d.OrderInvoice(<generated>)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:498)at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:219)at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:132)at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104)at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:745)at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:686)at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80)at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:925)at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:856)at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:936)at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:827)at javax.servlet.http.HttpServlet.service(HttpServlet.java:621)at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:812)at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)at com.planetj.servlet.filter.compression.CompressingFilter.doFilter(CompressingFilter.java:270)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)at com.hz.eshop.web.filter.SiteStatusFilter.doFilterInternal(SiteStatusFilter.java:38)at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)at com.hz.eshop.web.filter.XxxFilter.doFilter(XxxFilter.java:19)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)at com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:168)at com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:233)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:604)at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:165)at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:104)at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:1025)at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:452)at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1201)at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:654)at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1782)at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1741)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)at java.lang.Thread.run(Thread.java:748)

这个错误就更明显了一直参数越界了，说明人家在一直篡改参数请求，没有按照你的规范操作，导致数据对不上了直接报错了，这个错误你觉得不明显你说不是别人在请求，来看下面的问题。

第四个错误：

严重: 在路径为的上下文中，Servlet[springmvc]的Servlet.service（）引发了具有根本原因的异常Request processing failed; nested exception is net.sf.json.JSONException: JSONObject["openid"] not found.
net.sf.json.JSONException: JSONObject["openid"] not found.at net.sf.json.JSONObject.getString(JSONObject.java:2245)at com.hz.eshop.controller.shop.LoginController.wx_callback(LoginController.java:1080)at com.hz.eshop.controller.shop.LoginController$$FastClassByCGLIB$$bc3338bd.invoke(<generated>)at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:698)at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:631)at com.hz.eshop.controller.shop.LoginController$$EnhancerByCGLIB$$27d465a8.wx_callback(<generated>)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:498)at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:219)at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:132)at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104)at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:745)at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:686)at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80)at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:925)at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:856)at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:936)at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:827)at javax.servlet.http.HttpServlet.service(HttpServlet.java:621)at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:812)at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)at com.planetj.servlet.filter.compression.CompressingFilter.doFilter(CompressingFilter.java:270)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)at com.hz.eshop.web.filter.SiteStatusFilter.doFilterInternal(SiteStatusFilter.java:38)at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)at com.hz.eshop.web.filter.XxxFilter.doFilter(XxxFilter.java:19)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)at com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:168)at com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:233)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:604)at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:165)at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:104)at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:1025)at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:452)at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1201)at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:654)at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1782)at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1741)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)at java.lang.Thread.run(Thread.java:748)

这个错误足够明显了吧，你要再说不是别人在请求篡改说不过去了吧，我后台定义好的数据格式，怎么就会报json序列化异常呢？足够明显了吧。

第五个错误：

严重: 在路径为的上下文中，Servlet[springmvc]的Servlet.service（）引发了具有根本原因的异常Request processing failed; nested exception is java.lang.IllegalStateException: Invalid use of BasicClientConnManager: connection still allocated.
Make sure to release the connection before allocating another one.
java.lang.IllegalStateException: Invalid use of BasicClientConnManager: connection still allocated.
Make sure to release the connection before allocating another one.at org.apache.http.util.Asserts.check(Asserts.java:34)at org.apache.http.impl.conn.BasicClientConnectionManager.getConnection(BasicClientConnectionManager.java:160)at org.apache.http.impl.conn.BasicClientConnectionManager$1.getConnection(BasicClientConnectionManager.java:142)at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:422)at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863)at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)at com.hz.eshop.app.LoginController.wxAccountLogin(LoginController.java:529)at com.hz.eshop.app.LoginController$$FastClassByCGLIB$$f3d5abd6.invoke(<generated>)at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:698)at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:51)at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:631)at com.hz.eshop.app.LoginController$$EnhancerByCGLIB$$f7da36a1.wxAccountLogin(<generated>)at sun.reflect.GeneratedMethodAccessor1595.invoke(Unknown Source)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:498)at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:219)at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:132)at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104)at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:745)at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:686)at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80)at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:925)at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:856)at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:936)at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:838)at javax.servlet.http.HttpServlet.service(HttpServlet.java:647)at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:812)at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)at com.planetj.servlet.filter.compression.CompressingFilter.doFilter(CompressingFilter.java:270)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)at com.hz.eshop.web.filter.SiteStatusFilter.doFilterInternal(SiteStatusFilter.java:38)at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)at com.hz.eshop.web.filter.XxxFilter.doFilter(XxxFilter.java:19)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)at com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:168)at com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:233)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:604)at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:165)at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:104)at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:1025)at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:452)at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1201)at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:654)at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1782)at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1741)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)at java.lang.Thread.run(Thread.java:748)

这个错就直接看到了一直请求login登录，这个是业务上的一个微信登录，主要是由于 HTTP 连接的管理不当导致的，通常发生在一个连接被分配后，没有及时释放，导致后续的请求不能正常获取到连接。这种问题可能出现在并发请求、连接池管理不当等场景。

看到这里上面的问题我想你有答案了没错改代码：

接口加签名，接口做幂等，通过token令牌或者验签访问接口，保证接口不被重复请求，数据不会重复提交这是最靠谱的。

但是上面错误起码能给80%的把握定位是别人请求，但是具体的代码还是要通过arthas 进行调优查看的。