Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stats: concurrent map read and map write #57958

Closed
lilinghai opened this issue Dec 4, 2024 · 0 comments · Fixed by #58106
Closed

stats: concurrent map read and map write #57958

lilinghai opened this issue Dec 4, 2024 · 0 comments · Fixed by #58106
Labels
affects-8.5 This bug affects the 8.5.x(LTS) versions. severity/major sig/planner SIG: Planner type/bug The issue is confirmed as a bug.

Comments

@lilinghai
Copy link
Contributor

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

million tables and dml

fatal error: concurrent map read and map write 

goroutine 1273832 [running]:
[github.com/pingcap/tidb/pkg/statistics.(*HistColl).ID2UniqueID(...)](http://github.com/pingcap/tidb/pkg/statistics.(*HistColl).ID2UniqueID(...))
	/workspace/source/tidb/pkg/statistics/table.go:925
[github.com/pingcap/tidb/pkg/planner/core.initStats](http://github.com/pingcap/tidb/pkg/planner/core.initStats)(0xc02b965348, {0x0, 0x0, 0x0})
	/workspace/source/tidb/pkg/planner/core/stats.go:527 +0x505
[github.com/pingcap/tidb/pkg/planner/core.deriveStats4DataSource](http://github.com/pingcap/tidb/pkg/planner/core.deriveStats4DataSource)({0x7159dc8?, 0xc02b965348}, {0x0?, 0x4dbdafe?, 0x20c0578?})
	/workspace/source/tidb/pkg/planner/core/stats.go:120 +0xbd
[github.com/pingcap/tidb/pkg/planner/core/operator/logicalop.(*DataSource).DeriveStats](http://github.com/pingcap/tidb/pkg/planner/core/operator/logicalop.(*DataSource).DeriveStats)(0x1?, {0xc380f0ecd8?, 0x4d7135d?, 0xc380f0ec98?}, 0x0?, {0x8?, 0x715deb0?, 0xc23dac4420?}, {0x0, 0x0, ...})
	/workspace/source/tidb/pkg/planner/core/operator/logicalop/logical_datasource.go:295 +0x34
[github.com/pingcap/tidb/pkg/planner/core/operator/logicalop.(*BaseLogicalPlan).RecursiveDeriveStats](http://github.com/pingcap/tidb/pkg/planner/core/operator/logicalop.(*BaseLogicalPlan).RecursiveDeriveStats)(0xc02b965368, {0x0, 0x0, 0x0})
	/workspace/source/tidb/pkg/planner/core/operator/logicalop/base_logical_plan.go:219 +0x2a4
[github.com/pingcap/tidb/pkg/planner/core/operator/logicalop.(*BaseLogicalPlan).RecursiveDeriveStats](http://github.com/pingcap/tidb/pkg/planner/core/operator/logicalop.(*BaseLogicalPlan).RecursiveDeriveStats)(0xc1709c2660, {0x0, 0x0, 0x0})
	/workspace/source/tidb/pkg/planner/core/operator/logicalop/base_logical_plan.go:212 +0x173
[github.com/pingcap/tidb/pkg/planner/core/operator/logicalop.(*BaseLogicalPlan).RecursiveDeriveStats](http://github.com/pingcap/tidb/pkg/planner/core/operator/logicalop.(*BaseLogicalPlan).RecursiveDeriveStats)(0xc02c0e03c0, {0x0, 0x0, 0x0})
	/workspace/source/tidb/pkg/planner/core/operator/logicalop/base_logical_plan.go:212 +0x173
[github.com/pingcap/tidb/pkg/planner/core.physicalOptimize](http://github.com/pingcap/tidb/pkg/planner/core.physicalOptimize)({0x715ade8, 0xc02c0e03c0}, 0xc44885d210)
	/workspace/source/tidb/pkg/planner/core/optimizer.go:1012 +0x116
[github.com/pingcap/tidb/pkg/planner/core.doOptimize](http://github.com/pingcap/tidb/pkg/planner/core.doOptimize)({0x70c77a8, 0xc484551a10}, {0x7116780, 0xc2603c99c0}, 0x7110a, {0x715ade8, 0xc02c0e03c0})
	/workspace/source/tidb/pkg/planner/core/optimizer.go:271 +0x128
[github.com/pingcap/tidb/pkg/planner/core.DoOptimize](http://github.com/pingcap/tidb/pkg/planner/core.DoOptimize)({0x70c77a8, 0xc484551a10}, {0x7116780, 0xc2603c99c0}, 0x7110a, {0x715ade8, 0xc02c0e03c0})
	/workspace/source/tidb/pkg/planner/core/optimizer.go:318 +0x125
[github.com/pingcap/tidb/pkg/planner.optimize](http://github.com/pingcap/tidb/pkg/planner.optimize)({0x70c77a8, 0xc484551a10}, {0x7116780, 0xc2603c99c0}, 0xc1cc9bef00, {0x711ca70, 0xc087471c20})
	/workspace/source/tidb/pkg/planner/optimize.go:529 +0x9d6
[github.com/pingcap/tidb/pkg/planner.Optimize](http://github.com/pingcap/tidb/pkg/planner.Optimize)({0x70c77a8, 0xc484551a10}, {0x7160020, 0xc3da1ebb08}, 0xc1cc9bedb0, {0x711ca70, 0xc087471c20})
	/workspace/source/tidb/pkg/planner/optimize.go:360 +0x196b
[github.com/pingcap/tidb/pkg/planner/core.generateNewPlan](http://github.com/pingcap/tidb/pkg/planner/core.generateNewPlan)({0x70c77a8, 0xc484551a10}, {0x7160020, 0xc3da1ebb08}, 0x0, {0x711ca70, 0xc087471c20}, 0xc295433500, {0xc3ee362a80, 0xde}, ...)
	/workspace/source/tidb/pkg/planner/core/plan_cache.go:338 +0x1cf
[github.com/pingcap/tidb/pkg/planner/core.GetPlanFromPlanCache](http://github.com/pingcap/tidb/pkg/planner/core.GetPlanFromPlanCache)({0x70c77a8, 0xc484551a10}, {0x7160020, 0xc3da1ebb08}, 0x0, {0x711ca70, 0xc087471c20}, 0xc295433500, {0xc1c4f0b000, 0x2, ...})
	/workspace/source/tidb/pkg/planner/core/plan_cache.go:239 +0x4ec
[github.com/pingcap/tidb/pkg/planner.OptimizeExecStmt](http://github.com/pingcap/tidb/pkg/planner.OptimizeExecStmt)({0x70c77a8, 0xc484551a10}, {0x7160020, 0xc3da1ebb08}, 0xc1cc9beb88, {0x711ca70, 0xc087471c20})
	/workspace/source/tidb/pkg/planner/optimize.go:552 +0x367
[github.com/pingcap/tidb/pkg/planner.Optimize](http://github.com/pingcap/tidb/pkg/planner.Optimize)({0x70c77a8, 0xc484551a10}, {0x7160020, 0xc3da1ebb08}, 0xc1cc9beb88, {0x711ca70, 0xc087471c20})
	/workspace/source/tidb/pkg/planner/optimize.go:170 +0x39e
[github.com/pingcap/tidb/pkg/executor.(*Compiler).Compile](http://github.com/pingcap/tidb/pkg/executor.(*Compiler).Compile)(0xc380f10fa8, {0x70c77a8?, 0xc484551a10?}, {0x70e3268, 0xc2c9da1a40})
	/workspace/source/tidb/pkg/executor/compiler.go:101 +0x41c
[github.com/pingcap/tidb/pkg/session.(*session).ExecuteStmt](http://github.com/pingcap/tidb/pkg/session.(*session).ExecuteStmt)(0xc3da1ebb08, {0x70c77a8?, 0xc4845518f0?}, {0x70e3268, 0xc2c9da1a40})
	/workspace/source/tidb/pkg/session/session.go:2099 +0x85a
[github.com/pingcap/tidb/pkg/server.(*TiDBContext).ExecuteStmt](http://github.com/pingcap/tidb/pkg/server.(*TiDBContext).ExecuteStmt)(0xc2601fb908, {0x70c77a8, 0xc4845518f0}, {0x70e3268, 0xc2c9da1a40})
	/workspace/source/tidb/pkg/server/driver_tidb.go:291 +0x97
[github.com/pingcap/tidb/pkg/server.(*clientConn).executePreparedStmtAndWriteResult](http://github.com/pingcap/tidb/pkg/server.(*clientConn).executePreparedStmtAndWriteResult)(0xc2604c6000, {0x70c77a8, 0xc4845518f0}, {0x71026d0, 0xc29641b780}, {0xc2095d37c0, 0x2, 0x2}, 0x0)
	/workspace/source/tidb/pkg/server/conn_stmt.go:306 +0x587
[github.com/pingcap/tidb/pkg/server.(*clientConn).executePlanCacheStmt](http://github.com/pingcap/tidb/pkg/server.(*clientConn).executePlanCacheStmt)(0xc2604c6000, {0x70c77a8, 0xc484551830}, {0x64dd540, 0xc29641b780}, {0xc2095d37c0, 0x2, 0x2}, 0x0)
	/workspace/source/tidb/pkg/server/conn_stmt.go:234 +0x1d0
[github.com/pingcap/tidb/pkg/server.(*clientConn).handleStmtExecute](http://github.com/pingcap/tidb/pkg/server.(*clientConn).handleStmtExecute)(0xc2604c6000, {0x70c77a8, 0xc484551830}, {0xc22dbffd61, 0x1b, 0x1b})
	/workspace/source/tidb/pkg/server/conn_stmt.go:225 +0xb29
[github.com/pingcap/tidb/pkg/server.(*clientConn).dispatch](http://github.com/pingcap/tidb/pkg/server.(*clientConn).dispatch)(0xc2604c6000, {0x70c77a8, 0xc0e549a900}, {0xc22dbffd60, 0x1c, 0x1c})
	/workspace/source/tidb/pkg/server/conn.go:1405 +0x12bb
[github.com/pingcap/tidb/pkg/server.(*clientConn).Run](http://github.com/pingcap/tidb/pkg/server.(*clientConn).Run)(0xc2604c6000, {0x70c77a8, 0xc0e549a900})
	/workspace/source/tidb/pkg/server/conn.go:1147 +0x528
[github.com/pingcap/tidb/pkg/server.(*Server).onConn](http://github.com/pingcap/tidb/pkg/server.(*Server).onConn)(0xc00dc91d00, 0xc2604c6000)
	/workspace/source/tidb/pkg/server/server.go:741 +0x81f
created by [github.com/pingcap/tidb/pkg/server.(*Server).startNetworkListener](http://github.com/pingcap/tidb/pkg/server.(*Server).startNetworkListener) in goroutine 17007
	/workspace/source/tidb/pkg/server/server.go:560 +0x674

2. What did you expect to see? (Required)

3. What did you see instead (Required)

4. What is your TiDB version? (Required)

Release Version: v8.5.0-alpha-279-g9812d85d0d
Edition: Community
Git Commit Hash: 9812d85
Git Branch: HEAD
UTC Build Time: 2024-12-03 17:04:33
GoVersion: go1.23.3
Race Enabled: false
Check Table Before Drop: false
Store: tikv

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects-8.5 This bug affects the 8.5.x(LTS) versions. severity/major sig/planner SIG: Planner type/bug The issue is confirmed as a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants