Skip to content

[fix](memtable) fix memtable memory limit waiting threads count error #59361

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
liaoxin01 merged 1 commit into from
Dec 26, 2025
Merged

[fix](memtable) fix memtable memory limit waiting threads count error #59361

liaoxin01 merged 1 commit into from
Dec 26, 2025

Conversation

@sollhui
Copy link
Contributor

@sollhui sollhui commented Dec 25, 2025 •
edited
Loading

What problem does this PR solve?

Introduced by #53070

If cancel_check && cancel_check() returns true, simply return without decrementing memtable_memory_limit_waiting _threads counter.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Dec 25, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@sollhui
Copy link
Contributor Author

sollhui commented Dec 25, 2025

run buildall

liaoxin01
liaoxin01 approved these changes Dec 25, 2025
View reviewed changes
Copy link
Contributor

@liaoxin01 liaoxin01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 25, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Dec 25, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Dec 25, 2025

PR approved by anyone and no changes requested.

@doris-robot
Copy link

doris-robot commented Dec 25, 2025

TPC-H: Total hot run time: 36008 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 49d7a15602b439db75aaf728321b1834c825c028, data reload: false

------ Round 1 ----------------------------------
q1	17606	4214	4063	4063
q2	2022	347	236	236
q3	10197	1326	726	726
q4	10232	876	319	319
q5	7528	2151	1866	1866
q6	185	165	136	136
q7	991	858	717	717
q8	9363	1500	1160	1160
q9	6958	5331	5347	5331
q10	6862	2420	1985	1985
q11	560	330	306	306
q12	665	734	590	590
q13	17807	3665	3051	3051
q14	296	298	274	274
q15	588	507	524	507
q16	707	691	642	642
q17	683	811	572	572
q18	7362	7580	7980	7580
q19	1334	991	654	654
q20	417	396	247	247
q21	4533	4225	4054	4054
q22	1152	1103	992	992
Total cold run time: 108048 ms
Total hot run time: 36008 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4558	4243	4208	4208
q2	344	426	325	325
q3	2487	2877	2518	2518
q4	1402	1882	1454	1454
q5	4573	4543	4481	4481
q6	209	166	131	131
q7	2042	1904	1907	1904
q8	2660	2543	2575	2543
q9	7553	7767	7436	7436
q10	2910	3142	2675	2675
q11	596	527	490	490
q12	633	711	560	560
q13	3253	3644	3038	3038
q14	294	275	265	265
q15	535	497	494	494
q16	633	676	619	619
q17	1139	1396	1379	1379
q18	7134	6997	7047	6997
q19	845	781	788	781
q20	1896	1937	1858	1858
q21	4667	4358	4190	4190
q22	1042	1031	1014	1014
Total cold run time: 51405 ms
Total hot run time: 49360 ms

@doris-robot
Copy link

doris-robot commented Dec 25, 2025

TPC-DS: Total hot run time: 178928 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 49d7a15602b439db75aaf728321b1834c825c028, data reload: false

query5	5043	592	429	429
query6	326	235	220	220
query7	4252	474	267	267
query8	308	249	242	242
query9	8757	2484	2477	2477
query10	552	385	330	330
query11	15427	15250	14860	14860
query12	185	119	118	118
query13	1264	500	388	388
query14	6276	3054	2771	2771
query14_1	2643	2602	2671	2602
query15	206	201	187	187
query16	928	476	462	462
query17	1134	746	618	618
query18	2554	448	382	382
query19	230	227	196	196
query20	122	116	114	114
query21	227	142	121	121
query22	4030	4063	3983	3983
query23	16712	16088	15908	15908
query23_1	16034	15997	16093	15997
query24	7347	1649	1250	1250
query24_1	1248	1232	1225	1225
query25	541	479	434	434
query26	1244	264	158	158
query27	2765	465	305	305
query28	4469	2115	2098	2098
query29	797	558	444	444
query30	323	241	214	214
query31	830	708	605	605
query32	82	71	69	69
query33	537	330	285	285
query34	899	916	533	533
query35	775	804	701	701
query36	857	916	843	843
query37	132	93	78	78
query38	2993	3026	3044	3026
query39	770	750	750	750
query39_1	713	709	687	687
query40	224	141	121	121
query41	69	81	64	64
query42	106	100	103	100
query43	417	444	411	411
query44	1307	749	743	743
query45	190	187	185	185
query46	872	984	598	598
query47	1697	1741	1668	1668
query48	308	321	244	244
query49	622	427	349	349
query50	662	289	224	224
query51	3850	3794	3945	3794
query52	108	108	101	101
query53	321	349	301	301
query54	281	252	248	248
query55	76	79	71	71
query56	298	305	314	305
query57	1166	1149	1084	1084
query58	269	262	247	247
query59	2449	2492	2445	2445
query60	308	313	289	289
query61	171	197	156	156
query62	713	747	643	643
query63	325	288	299	288
query64	4879	1298	1013	1013
query65	4008	3907	3933	3907
query66	1432	451	321	321
query67	14995	15096	14830	14830
query68	2756	1035	751	751
query69	441	350	309	309
query70	1029	1022	974	974
query71	322	310	267	267
query72	5602	4990	5030	4990
query73	480	538	304	304
query74	9031	8982	8693	8693
query75	3160	3198	2812	2812
query76	2888	1138	731	731
query77	359	406	306	306
query78	9439	9760	8864	8864
query79	1115	831	620	620
query80	1239	696	598	598
query81	552	275	238	238
query82	890	137	104	104
query83	424	253	248	248
query84	263	118	107	107
query85	941	518	472	472
query86	393	288	282	282
query87	3161	3253	3141	3141
query88	3224	2272	2275	2272
query89	456	425	401	401
query90	1951	156	153	153
query91	172	172	144	144
query92	69	69	62	62
query93	952	910	555	555
query94	532	297	265	265
query95	564	374	316	316
query96	595	467	213	213
query97	2324	2321	2203	2203
query98	213	200	203	200
query99	1325	1491	1261	1261
Total cold run time: 255011 ms
Total hot run time: 178928 ms

@doris-robot
Copy link

doris-robot commented Dec 25, 2025

ClickBench: Total hot run time: 27.6 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 49d7a15602b439db75aaf728321b1834c825c028, data reload: false

query1	0.05	0.05	0.05
query2	0.10	0.05	0.05
query3	0.26	0.09	0.09
query4	1.61	0.11	0.10
query5	0.28	0.26	0.26
query6	1.16	0.63	0.63
query7	0.03	0.03	0.03
query8	0.06	0.04	0.06
query9	0.56	0.51	0.49
query10	0.55	0.55	0.56
query11	0.17	0.11	0.12
query12	0.15	0.11	0.11
query13	0.63	0.59	0.61
query14	1.01	1.00	0.98
query15	0.81	0.79	0.82
query16	0.41	0.39	0.40
query17	1.05	1.01	1.03
query18	0.23	0.21	0.22
query19	1.93	1.82	1.87
query20	0.02	0.01	0.02
query21	15.44	0.29	0.13
query22	4.71	0.06	0.05
query23	16.02	0.28	0.10
query24	0.93	0.66	1.33
query25	0.11	0.13	0.06
query26	0.13	0.13	0.13
query27	0.06	0.07	0.09
query28	5.08	1.22	1.03
query29	12.60	4.04	3.19
query30	0.27	0.13	0.13
query31	2.82	0.64	0.39
query32	3.23	0.55	0.45
query33	2.94	3.01	3.05
query34	16.89	5.18	4.49
query35	4.63	4.54	4.60
query36	0.63	0.50	0.50
query37	0.11	0.06	0.06
query38	0.07	0.04	0.04
query39	0.05	0.03	0.03
query40	0.18	0.14	0.14
query41	0.09	0.03	0.03
query42	0.05	0.03	0.03
query43	0.05	0.04	0.03
Total cold run time: 98.16 s
Total hot run time: 27.6 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Dec 25, 2025

BE UT Coverage Report

Increment line coverage 0.00% (0/1) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 53.41% (18955/35487)
Line Coverage 39.28% (175812/447573)
Region Coverage 33.86% (136076/401891)
Branch Coverage 34.78% (58740/168897)

@hello-stephen
Copy link
Contributor

hello-stephen commented Dec 25, 2025

BE Regression && UT Coverage Report

Increment line coverage 0.00% (0/1) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 73.96% (25657/34692)
Line Coverage 61.36% (273933/446406)
Region Coverage 56.33% (228746/406094)
Branch Coverage 58.17% (98584/169477)

@hello-stephen
Copy link
Contributor

hello-stephen commented Dec 25, 2025

BE Regression && UT Coverage Report

Increment line coverage 0.00% (0/1) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 73.95% (25655/34692)
Line Coverage 61.36% (273908/446406)
Region Coverage 56.31% (228676/406094)
Branch Coverage 58.16% (98569/169477)

dataroaring
dataroaring approved these changes Dec 26, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@liaoxin01 liaoxin01 merged commit 680bbea into apache:master Dec 26, 2025
31 of 33 checks passed
github-actions bot pushed a commit that referenced this pull request Dec 26, 2025
@sollhui
[fix](memtable) fix memtable memory limit waiting threads count error (…
696d4ce 
…#59361)

### What problem does this PR solve?

Introduced by #53070

If `cancel_check && cancel_check()` returns true, simply return without
decrementing `memtable_memory_limit_waiting _threads` counter.
@github-actions github-actions bot mentioned this pull request Dec 26, 2025
Open
github-actions bot pushed a commit that referenced this pull request Dec 26, 2025
@sollhui
[fix](memtable) fix memtable memory limit waiting threads count error (…
da5f9ca 
…#59361)

### What problem does this PR solve?

Introduced by #53070

If `cancel_check && cancel_check()` returns true, simply return without
decrementing `memtable_memory_limit_waiting _threads` counter.
@github-actions github-actions bot mentioned this pull request Dec 26, 2025
Merged
yiguolei pushed a commit that referenced this pull request Dec 26, 2025
@github-actions @sollhui
branch-4.0: [fix](memtable) fix memtable memory limit waiting threads…
6b0f2fb 
… count error #59361 (#59393)

Cherry-picked from #59361

Co-authored-by: hui lai <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@liaoxin01 liaoxin01 liaoxin01 approved these changes

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.1.x dev/4.0.3-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@sollhui @Thearas @doris-robot @hello-stephen @liaoxin01 @dataroaring @yiguolei