經(jīng)過(guò)上面一通手法操作，足以解決此類常規(guī)報(bào)錯(cuò)了，通常大多是原因各種循環(huán)遞歸、或數(shù)據(jù)庫(kù)慢查詢等。

Mat使用

在MAT中，會(huì)有兩種大小表示：

• Shallow Size：表示對(duì)象自身占用的內(nèi)存大小，不包括它引用的對(duì)象。
• Retained size：當(dāng)前對(duì)象內(nèi)存大小+當(dāng)前對(duì)象直接或間接引用的對(duì)象大小，全部的總和，簡(jiǎn)單理解，就是當(dāng)前對(duì)象被GC后，總共能釋放的內(nèi)存大小。

Histogram視圖

以Class Name為維度，分別展示各個(gè)類的對(duì)象數(shù)量。它默認(rèn)是以byte為單位的，

要顯示讓單位展示出來(lái)，點(diǎn)擊Window->Preferences選擇最后一項(xiàng)，點(diǎn)擊Apply and Close

再重新打開(kāi)Histogram視圖，就會(huì)生效了。

Leak Suspects

報(bào)表很直觀地展現(xiàn)了一個(gè)餅圖，圖中顏色深的部分表示可能存在內(nèi)存泄漏的嫌疑。

通過(guò)這個(gè)指標(biāo)可以快速定位內(nèi)存泄漏地方出現(xiàn)在哪個(gè)類方法里的哪行代碼。

本次問(wèn)題排查

1、 信息收集分析

因服務(wù)健康監(jiān)測(cè)無(wú)響應(yīng)，cpu及內(nèi)存情況正常，直接查看堆棧信息，看看線程都在干什么

jstack -l PID >> a.log

Jstack的輸出中，Java線程狀態(tài)主要是以下幾種：

• RUNNABLE 線程運(yùn)行中或I/O等待
• BLOCKED 線程在等待monitor鎖(synchronized關(guān)鍵字)
• TIMED_WAITING 線程在等待喚醒，但設(shè)置了時(shí)限
• WAITING 線程在無(wú)限等待喚醒

發(fā)現(xiàn)都是WAITING線程。

"http-nio-8888-exec-6666" #8833 daemon prio=5 os_prio=0 tid=0x00001f2f0016e100 nid=0x667d waiting on condition [0x00002f1de3c5200]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000007156a29c8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at com.alibaba.druid.pool.DruidDataSource.takeLast(DruidDataSource.java:1897)
at com.alibaba.druid.pool.DruidDataSource.getConnectionInternal(DruidDataSource.java:1458)
at com.alibaba.druid.pool.DruidDataSource.getConnectionDirect(DruidDataSource.java:1253)
at com.alibaba.druid.filter.FilterChainImpl.dataSource_connect(FilterChainImpl.java:4619)
at com.alibaba.druid.filter.stat.StatFilter.dataSource_getConnection(StatFilter.java:680)
at com.alibaba.druid.filter.FilterChainImpl.dataSource_connect(FilterChainImpl.java:4615)
at com.alibaba.druid.pool.DruidDataSource.getConnection(DruidDataSource.java:1231)
at com.alibaba.druid.pool.DruidDataSource.getConnection(DruidDataSource.java:1223)
at com.alibaba.druid.pool.DruidDataSource.getConnection(DruidDataSource.java:90)
at com.baomidou.dynamic.datasource.ds.ItemDataSource.getConnection(ItemDataSource.java:56)
at com.baomidou.dynamic.datasource.ds.AbstractRoutingDataSource.getConnection(AbstractRoutingDataSource.java:48)
at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111)
at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77)
at org.mybatis.spring.transaction.SpringManagedTransaction.openConnection(SpringManagedTransaction.java:82)
at org.mybatis.spring.transaction.SpringManagedTransaction.getConnection(SpringManagedTransaction.java:68)
at org.apache.ibatis.executor.BaseExecutor.getConnection(BaseExecutor.java:336)
at org.apache.ibatis.executor.SimpleExecutor.prepareStatement(SimpleExecutor.java:84)
at org.apache.ibatis.executor.SimpleExecutor.doQuery(SimpleExecutor.java:62)
at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:324)
at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:156)
at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:109)
at com.github.pagehelper.PageInterceptor.intercept(PageInterceptor.java:143)
at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:61)
at com.sun.proxy.$Proxy571.query(Unknown Source)

2、定位關(guān)鍵信息，追蹤源代碼

at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at com.alibaba.druid.pool.DruidDataSource.takeLast(DruidDataSource.java:1897)
DruidConnectionHolder takeLast() throws InterruptedException, SQLException {
try {
while (poolingCount == 0) {
emptySignal(); // send signal to CreateThread create connection

if (failFast && isFailContinuous()) {
throw new DataSourceNotAvailableException(createError);
}

notEmptyWaitThreadCount++;
if (notEmptyWaitThreadCount > notEmptyWaitThreadPeak) {
notEmptyWaitThreadPeak = notEmptyWaitThreadCount;
}
try {
// 數(shù)據(jù)庫(kù)的連接都沒(méi)有釋放且被占用，連接池中無(wú)可用連接，導(dǎo)致請(qǐng)求被阻塞
notEmpty.await(); // signal by recycle or creator
} finally {
notEmptyWaitThreadCount--;
}
notEmptyWaitCount++;

if (!enable) {
connectErrorCountUpdater.incrementAndGet(this);
throw new DataSourceDisableException();
}
}
} catch (InterruptedException ie) {
notEmpty.signal(); // propagate to non-interrupted thread
notEmptySignalCount++;
throw ie;
}

decrementPoolingCount();
DruidConnectionHolder last = connections[poolingCount];
connections[poolingCount] = null;

return last;
}

結(jié)合日志報(bào)錯(cuò)定位到問(wèn)題代碼。因報(bào)錯(cuò)可用連接沒(méi)有正常釋放，導(dǎo)致一直await卡死。

問(wèn)題代碼如下：

try {
SqlSession sqlSession = sqlSessionFactory.openSession(ExecutorType.BATCH);
TestMapper mapper = sqlSession.getMapper(TestMapper.class);
mapper.insetList(list);
sqlSession.flushStatements();
} catch (Exception e) {
e.printStackTrace();
}

問(wèn)題復(fù)現(xiàn)

按照以上信息在多活環(huán)境復(fù)現(xiàn)。因線程被打滿且都在等待導(dǎo)致監(jiān)控檢查無(wú)響應(yīng)。

tomcat線程被打滿：

圖片

tomcat默認(rèn)參數(shù)：

最大工作線程數(shù)，默認(rèn)200。

server.tomcat.max-threads=200

最大連接數(shù)默認(rèn)是10000

server.tomcat.max-connections=10000

等待隊(duì)列長(zhǎng)度，默認(rèn)100。

server.tomcat.accept-count=100

最小工作空閑線程數(shù)，默認(rèn)10。

server.tomcat.min-spare-threads=100

Druid連接池的默認(rèn)參數(shù)如下：

圖片

Druid連接池的配置參數(shù)如下：

解決

1、Druid連接池的配置超時(shí)參數(shù)

spring:
redis:
host: localhost
port: 6379
password:
datasource:
druid:
stat-view-servlet:
enabled: true
loginUsername: admin
loginPassword: 123456
dynamic:
druid:
initial-size: 5
min-idle: 5
maxActive: 20
maxWait: 60000
timeBetweenEvictionRunsMillis: 60000
minEvictableIdleTimeMillis: 300000
validationQuery: SELECT 1 FROM DUAL
testWhileIdle: true
testOnBorrow: false
testOnReturn: false
poolPreparedStatements: true
maxPoolPreparedStatementPerConnectionSize: 20
filters: stat,slf4j,wall
connectionProperties: druid.stat.mergeSql\=true;druid.stat.slowSqlMillis\=5000

2、異常及時(shí)關(guān)閉連接

sqlSession.close();

來(lái)源：blog.csdn.net/zhangcongyi420/article/details/131139599

End

標(biāo)簽：人工智能pythonpytorchchatgpt