Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase sleep time to avoid exemplar sampling rate limiting for openMetricsScrapeWithExemplars() #5908

Merged
merged 1 commit into from
Feb 10, 2025

Conversation

izeye
Copy link
Contributor

@izeye izeye commented Feb 8, 2025

This PR increases sleep time to avoid exemplar sampling rate limiting for the openMetricsScrapeWithExemplars() as it seems to be flaky, depending on scheduling.

Looking into a failure, its output is as follows:

java.lang.AssertionError: 	
Expecting actual:	
  "# TYPE my_counter counter	
# HELP my_counter  	
my_counter_total 1.0 # {span_id="1",trace_id="2"} 1.0 1738954792.084	
# TYPE summary_noHistogram summary	
# HELP summary_noHistogram  	
summary_noHistogram_count 3 # {span_id="17",trace_id="18"} 0.1 1738954792.129	
summary_noHistogram_sum 1.0E18	
# TYPE summary_noHistogram_max gauge	
# HELP summary_noHistogram_max  	
summary_noHistogram_max 1.0E18	
# TYPE summary_withHistogram histogram	
# HELP summary_withHistogram  	
summary_withHistogram_bucket{le="1.0"} 1 # {span_id="19",trace_id="20"} 0.15 1738954792.130	
summary_withHistogram_bucket{le="2.0"} 1	
summary_withHistogram_bucket{le="3.0"} 1	
summary_withHistogram_bucket{le="4.0"} 1	
summary_withHistogram_bucket{le="5.0"} 1	
summary_withHistogram_bucket{le="6.0"} 1	
summary_withHistogram_bucket{le="7.0"} 1	
summary_withHistogram_bucket{le="8.0"} 1	
summary_withHistogram_bucket{le="9.0"} 1	
summary_withHistogram_bucket{le="10.0"} 1	
summary_withHistogram_bucket{le="11.0"} 1	
summary_withHistogram_bucket{le="12.0"} 1	
summary_withHistogram_bucket{le="13.0"} 1	
summary_withHistogram_bucket{le="14.0"} 1	
summary_withHistogram_bucket{le="16.0"} 2 # {span_id="21",trace_id="22"} 15.0 1738954792.157	
summary_withHistogram_bucket{le="21.0"} 2	
summary_withHistogram_bucket{le="26.0"} 2	
summary_withHistogram_bucket{le="31.0"} 2	
summary_withHistogram_bucket{le="36.0"} 2	
summary_withHistogram_bucket{le="41.0"} 2	
summary_withHistogram_bucket{le="46.0"} 2	
summary_withHistogram_bucket{le="51.0"} 2	
summary_withHistogram_bucket{le="56.0"} 2	
summary_withHistogram_bucket{le="64.0"} 2	
summary_withHistogram_bucket{le="85.0"} 2	
summary_withHistogram_bucket{le="106.0"} 2	
summary_withHistogram_bucket{le="127.0"} 2	
summary_withHistogram_bucket{le="148.0"} 2	
summary_withHistogram_bucket{le="169.0"} 2	
summary_withHistogram_bucket{le="190.0"} 2	
summary_withHistogram_bucket{le="211.0"} 2	
summary_withHistogram_bucket{le="232.0"} 2	
summary_withHistogram_bucket{le="256.0"} 2	
summary_withHistogram_bucket{le="341.0"} 2	
summary_withHistogram_bucket{le="426.0"} 2	
summary_withHistogram_bucket{le="511.0"} 2	
summary_withHistogram_bucket{le="596.0"} 2	
summary_withHistogram_bucket{le="681.0"} 2	
summary_withHistogram_bucket{le="766.0"} 2	
summary_withHistogram_bucket{le="851.0"} 2	
summary_withHistogram_bucket{le="936.0"} 2	
summary_withHistogram_bucket{le="1024.0"} 2	
summary_withHistogram_bucket{le="1365.0"} 2	
summary_withHistogram_bucket{le="1706.0"} 2	
summary_withHistogram_bucket{le="2047.0"} 2	
summary_withHistogram_bucket{le="2388.0"} 2	
summary_withHistogram_bucket{le="2729.0"} 2	
summary_withHistogram_bucket{le="3070.0"} 2	
summary_withHistogram_bucket{le="3411.0"} 2	
summary_withHistogram_bucket{le="3752.0"} 2	
summary_withHistogram_bucket{le="4096.0"} 2	
summary_withHistogram_bucket{le="5461.0"} 2	
summary_withHistogram_bucket{le="6826.0"} 2	
summary_withHistogram_bucket{le="8191.0"} 2	
summary_withHistogram_bucket{le="9556.0"} 2	
summary_withHistogram_bucket{le="10921.0"} 2	
summary_withHistogram_bucket{le="12286.0"} 2	
summary_withHistogram_bucket{le="13651.0"} 2	
summary_withHistogram_bucket{le="15016.0"} 2	
summary_withHistogram_bucket{le="16384.0"} 2	
summary_withHistogram_bucket{le="21845.0"} 2	
summary_withHistogram_bucket{le="27306.0"} 2	
summary_withHistogram_bucket{le="32767.0"} 2	
summary_withHistogram_bucket{le="38228.0"} 2	
summary_withHistogram_bucket{le="43689.0"} 2	
summary_withHistogram_bucket{le="49150.0"} 2	
summary_withHistogram_bucket{le="54611.0"} 2	
summary_withHistogram_bucket{le="60072.0"} 2	
summary_withHistogram_bucket{le="65536.0"} 2	
summary_withHistogram_bucket{le="87381.0"} 2	
summary_withHistogram_bucket{le="109226.0"} 2	
summary_withHistogram_bucket{le="131071.0"} 2	
summary_withHistogram_bucket{le="152916.0"} 2	
summary_withHistogram_bucket{le="174761.0"} 2	
summary_withHistogram_bucket{le="196606.0"} 2	
summary_withHistogram_bucket{le="218451.0"} 2	
summary_withHistogram_bucket{le="240296.0"} 2	
summary_withHistogram_bucket{le="262144.0"} 2	
summary_withHistogram_bucket{le="349525.0"} 2	
summary_withHistogram_bucket{le="436906.0"} 2	
summary_withHistogram_bucket{le="524287.0"} 2	
summary_withHistogram_bucket{le="611668.0"} 2	
summary_withHistogram_bucket{le="699049.0"} 2	
summary_withHistogram_bucket{le="786430.0"} 2	
summary_withHistogram_bucket{le="873811.0"} 2	
summary_withHistogram_bucket{le="961192.0"} 2	
summary_withHistogram_bucket{le="1048576.0"} 2	
summary_withHistogram_bucket{le="1398101.0"} 2	
summary_withHistogram_bucket{le="1747626.0"} 2	
summary_withHistogram_bucket{le="2097151.0"} 2	
summary_withHistogram_bucket{le="2446676.0"} 2	
summary_withHistogram_bucket{le="2796201.0"} 2	
summary_withHistogram_bucket{le="3145726.0"} 2	
summary_withHistogram_bucket{le="3495251.0"} 2	
summary_withHistogram_bucket{le="3844776.0"} 2	
summary_withHistogram_bucket{le="4194304.0"} 2	
summary_withHistogram_bucket{le="5592405.0"} 2	
summary_withHistogram_bucket{le="6990506.0"} 2	
summary_withHistogram_bucket{le="8388607.0"} 2	
summary_withHistogram_bucket{le="9786708.0"} 2	
summary_withHistogram_bucket{le="1.1184809E7"} 2	
summary_withHistogram_bucket{le="1.258291E7"} 2	
summary_withHistogram_bucket{le="1.3981011E7"} 2	
summary_withHistogram_bucket{le="1.5379112E7"} 2	
summary_withHistogram_bucket{le="1.6777216E7"} 2	
summary_withHistogram_bucket{le="2.2369621E7"} 2	
summary_withHistogram_bucket{le="2.7962026E7"} 2	
summary_withHistogram_bucket{le="3.3554431E7"} 2	
summary_withHistogram_bucket{le="3.9146836E7"} 2	
summary_withHistogram_bucket{le="4.4739241E7"} 2	
summary_withHistogram_bucket{le="5.0331646E7"} 2	
summary_withHistogram_bucket{le="5.5924051E7"} 2	
summary_withHistogram_bucket{le="6.1516456E7"} 2	
summary_withHistogram_bucket{le="6.7108864E7"} 2	
summary_withHistogram_bucket{le="8.9478485E7"} 2	
summary_withHistogram_bucket{le="1.11848106E8"} 2	
summary_withHistogram_bucket{le="1.34217727E8"} 2	
summary_withHistogram_bucket{le="1.56587348E8"} 2	
summary_withHistogram_bucket{le="1.78956969E8"} 2	
summary_withHistogram_bucket{le="2.0132659E8"} 2	
summary_withHistogram_bucket{le="2.23696211E8"} 2	
summary_withHistogram_bucket{le="2.46065832E8"} 2	
summary_withHistogram_bucket{le="2.68435456E8"} 2	
summary_withHistogram_bucket{le="3.57913941E8"} 2	
summary_withHistogram_bucket{le="4.47392426E8"} 2	
summary_withHistogram_bucket{le="5.36870911E8"} 2	
summary_withHistogram_bucket{le="6.26349396E8"} 2	
summary_withHistogram_bucket{le="7.15827881E8"} 2	
summary_withHistogram_bucket{le="8.05306366E8"} 2	
summary_withHistogram_bucket{le="8.94784851E8"} 2	
summary_withHistogram_bucket{le="9.84263336E8"} 2	
summary_withHistogram_bucket{le="1.073741824E9"} 2	
summary_withHistogram_bucket{le="1.431655765E9"} 2	
summary_withHistogram_bucket{le="1.789569706E9"} 2	
summary_withHistogram_bucket{le="2.147483647E9"} 2	
summary_withHistogram_bucket{le="2.505397588E9"} 2	
summary_withHistogram_bucket{le="2.863311529E9"} 2	
summary_withHistogram_bucket{le="3.22122547E9"} 2	
summary_withHistogram_bucket{le="3.579139411E9"} 2	
summary_withHistogram_bucket{le="3.937053352E9"} 2	
summary_withHistogram_bucket{le="4.294967296E9"} 2	
summary_withHistogram_bucket{le="5.726623061E9"} 2	
summary_withHistogram_bucket{le="7.158278826E9"} 2	
summary_withHistogram_bucket{le="8.589934591E9"} 2	
summary_withHistogram_bucket{le="1.0021590356E10"} 2	
summary_withHistogram_bucket{le="1.1453246121E10"} 2	
summary_withHistogram_bucket{le="1.2884901886E10"} 2	
summary_withHistogram_bucket{le="1.4316557651E10"} 2	
summary_withHistogram_bucket{le="1.5748213416E10"} 2	
summary_withHistogram_bucket{le="1.7179869184E10"} 2	
summary_withHistogram_bucket{le="2.2906492245E10"} 2	
summary_withHistogram_bucket{le="2.8633115306E10"} 2	
summary_withHistogram_bucket{le="3.4359738367E10"} 2	
summary_withHistogram_bucket{le="4.0086361428E10"} 2	
summary_withHistogram_bucket{le="4.5812984489E10"} 2	
summary_withHistogram_bucket{le="5.153960755E10"} 2	
summary_withHistogram_bucket{le="5.7266230611E10"} 2	
summary_withHistogram_bucket{le="6.2992853672E10"} 2	
summary_withHistogram_bucket{le="6.8719476736E10"} 2	
summary_withHistogram_bucket{le="9.1625968981E10"} 2	
summary_withHistogram_bucket{le="1.14532461226E11"} 2	
summary_withHistogram_bucket{le="1.37438953471E11"} 2	
summary_withHistogram_bucket{le="1.60345445716E11"} 2	
summary_withHistogram_bucket{le="1.83251937961E11"} 2	
summary_withHistogram_bucket{le="2.06158430206E11"} 2	
summary_withHistogram_bucket{le="2.29064922451E11"} 2	
summary_withHistogram_bucket{le="2.51971414696E11"} 2	
summary_withHistogram_bucket{le="2.74877906944E11"} 2	
summary_withHistogram_bucket{le="3.66503875925E11"} 2	
summary_withHistogram_bucket{le="4.58129844906E11"} 2	
summary_withHistogram_bucket{le="5.49755813887E11"} 2	
summary_withHistogram_bucket{le="6.41381782868E11"} 2	
summary_withHistogram_bucket{le="7.33007751849E11"} 2	
summary_withHistogram_bucket{le="8.2463372083E11"} 2	
summary_withHistogram_bucket{le="9.16259689811E11"} 2	
summary_withHistogram_bucket{le="1.007885658792E12"} 2	
summary_withHistogram_bucket{le="1.099511627776E12"} 2	
summary_withHistogram_bucket{le="1.466015503701E12"} 2	
summary_withHistogram_bucket{le="1.832519379626E12"} 2	
summary_withHistogram_bucket{le="2.199023255551E12"} 2	
summary_withHistogram_bucket{le="2.565527131476E12"} 2	
summary_withHistogram_bucket{le="2.932031007401E12"} 2	
summary_withHistogram_bucket{le="3.298534883326E12"} 2	
summary_withHistogram_bucket{le="3.665038759251E12"} 2	
summary_withHistogram_bucket{le="4.031542635176E12"} 2	
summary_withHistogram_bucket{le="4.398046511104E12"} 2
summary_withHistogram_bucket{le="5.864062014805E12"} 2
summary_withHistogram_bucket{le="7.330077518506E12"} 2
summary_withHistogram_bucket{le="8.796093022207E12"} 2
summary_withHistogram_bucket{le="1.0262108525908E13"} 2
summary_withHistogram_bucket{le="1.1728124029609E13"} 2
summary_withHistogram_bucket{le="1.319413953331E13"} 2
summary_withHistogram_bucket{le="1.4660155037011E13"} 2
summary_withHistogram_bucket{le="1.6126170540712E13"} 2
summary_withHistogram_bucket{le="1.7592186044416E13"} 2
summary_withHistogram_bucket{le="2.3456248059221E13"} 2
summary_withHistogram_bucket{le="2.9320310074026E13"} 2
summary_withHistogram_bucket{le="3.5184372088831E13"} 2
summary_withHistogram_bucket{le="4.1048434103636E13"} 2
summary_withHistogram_bucket{le="4.6912496118441E13"} 2
summary_withHistogram_bucket{le="5.2776558133246E13"} 2
summary_withHistogram_bucket{le="5.8640620148051E13"} 2
summary_withHistogram_bucket{le="6.4504682162856E13"} 2
summary_withHistogram_bucket{le="7.0368744177664E13"} 2
summary_withHistogram_bucket{le="9.3824992236885E13"} 2
summary_withHistogram_bucket{le="1.17281240296106E14"} 2
summary_withHistogram_bucket{le="1.40737488355327E14"} 2
summary_withHistogram_bucket{le="1.64193736414548E14"} 2
summary_withHistogram_bucket{le="1.87649984473769E14"} 2
summary_withHistogram_bucket{le="2.1110623253299E14"} 2
summary_withHistogram_bucket{le="2.34562480592211E14"} 2
summary_withHistogram_bucket{le="2.58018728651432E14"} 2
summary_withHistogram_bucket{le="2.81474976710656E14"} 2
summary_withHistogram_bucket{le="3.75299968947541E14"} 2
summary_withHistogram_bucket{le="4.69124961184426E14"} 2
summary_withHistogram_bucket{le="5.62949953421311E14"} 2
summary_withHistogram_bucket{le="6.56774945658196E14"} 2
summary_withHistogram_bucket{le="7.50599937895081E14"} 2
summary_withHistogram_bucket{le="8.44424930131966E14"} 2
summary_withHistogram_bucket{le="9.38249922368851E14"} 2
summary_withHistogram_bucket{le="1.032074914605736E15"} 2
summary_withHistogram_bucket{le="1.125899906842624E15"} 2
summary_withHistogram_bucket{le="1.501199875790165E15"} 2
summary_withHistogram_bucket{le="1.876499844737706E15"} 2
summary_withHistogram_bucket{le="2.251799813685247E15"} 2
summary_withHistogram_bucket{le="2.627099782632788E15"} 2
summary_withHistogram_bucket{le="3.002399751580329E15"} 2
summary_withHistogram_bucket{le="3.37769972052787E15"} 2
summary_withHistogram_bucket{le="3.752999689475411E15"} 2
summary_withHistogram_bucket{le="4.128299658422952E15"} 2
summary_withHistogram_bucket{le="4.503599627370496E15"} 2
summary_withHistogram_bucket{le="6.004799503160661E15"} 2
summary_withHistogram_bucket{le="7.505999378950826E15"} 2
summary_withHistogram_bucket{le="9.007199254740991E15"} 2
summary_withHistogram_bucket{le="1.0508399130531156E16"} 2
summary_withHistogram_bucket{le="1.200959900632132E16"} 2
summary_withHistogram_bucket{le="1.3510798882111486E16"} 2
summary_withHistogram_bucket{le="1.5011998757901652E16"} 2
summary_withHistogram_bucket{le="1.6513198633691816E16"} 2
summary_withHistogram_bucket{le="1.8014398509481984E16"} 2
summary_withHistogram_bucket{le="2.4019198012642644E16"} 2
summary_withHistogram_bucket{le="3.0023997515803304E16"} 2
summary_withHistogram_bucket{le="3.6028797018963968E16"} 2
summary_withHistogram_bucket{le="4.2033596522124624E16"} 2
summary_withHistogram_bucket{le="4.8038396025285288E16"} 2
summary_withHistogram_bucket{le="5.4043195528445952E16"} 2
summary_withHistogram_bucket{le="6.0047995031606608E16"} 2
summary_withHistogram_bucket{le="6.6052794534767272E16"} 2
summary_withHistogram_bucket{le="7.2057594037927936E16"} 2
summary_withHistogram_bucket{le="9.6076792050570576E16"} 2
summary_withHistogram_bucket{le="1.20095990063213232E17"} 2
summary_withHistogram_bucket{le="1.44115188075855872E17"} 2
summary_withHistogram_bucket{le="1.68134386088498528E17"} 2
summary_withHistogram_bucket{le="1.92153584101141152E17"} 2
summary_withHistogram_bucket{le="2.16172782113783808E17"} 2
summary_withHistogram_bucket{le="2.40191980126426464E17"} 2
summary_withHistogram_bucket{le="2.64211178139069088E17"} 2
summary_withHistogram_bucket{le="2.8823037615171174E17"} 2
summary_withHistogram_bucket{le="3.843071682022823E17"} 2
summary_withHistogram_bucket{le="4.8038396025285293E17"} 2
summary_withHistogram_bucket{le="5.7646075230342349E17"} 2
summary_withHistogram_bucket{le="6.7253754435399411E17"} 2
summary_withHistogram_bucket{le="7.6861433640456461E17"} 2
summary_withHistogram_bucket{le="8.6469112845513523E17"} 2
summary_withHistogram_bucket{le="9.6076792050570586E17"} 2
summary_withHistogram_bucket{le="1.05684471255627635E18"} 2
summary_withHistogram_bucket{le="1.15292150460684698E18"} 2
summary_withHistogram_bucket{le="1.53722867280912922E18"} 2
summary_withHistogram_bucket{le="1.92153584101141171E18"} 2
summary_withHistogram_bucket{le="2.305843009213694E18"} 2
summary_withHistogram_bucket{le="2.6901501774159764E18"} 2
summary_withHistogram_bucket{le="3.0744573456182584E18"} 2
summary_withHistogram_bucket{le="3.4587645138205409E18"} 2
summary_withHistogram_bucket{le="3.8430716820228234E18"} 2
summary_withHistogram_bucket{le="4.2273788502251054E18"} 2
summary_withHistogram_bucket{le="+Inf"} 3
summary_withHistogram_count 3 # {span_id="21",trace_id="22"} 15.0 1738954792.157
summary_withHistogram_sum 5.0E18
# TYPE summary_withHistogram_max gauge
# HELP summary_withHistogram_max
summary_withHistogram_max 5.0E18
# TYPE summary_withSlos histogram
# HELP summary_withSlos
summary_withSlos_bucket{le="100.0"} 1 # {span_id="23",trace_id="24"} 10.0 1738954792.158
summary_withSlos_bucket{le="200.0"} 1
summary_withSlos_bucket{le="300.0"} 2 # {span_id="27",trace_id="28"} 250.0 1738954792.178
summary_withSlos_bucket{le="+Inf"} 3 # {span_id="25",trace_id="26"} 1000.0 1738954792.168
summary_withSlos_count 3 # {span_id="27",trace_id="28"} 250.0 1738954792.178
summary_withSlos_sum 1260.0
# TYPE summary_withSlos_max gauge
# HELP summary_withSlos_max
summary_withSlos_max 1000.0
# TYPE timer_noHistogram_seconds summary
# HELP timer_noHistogram_seconds
timer_noHistogram_seconds_count 3 # {span_id="3",trace_id="4"} 0.1 1738954792.087
timer_noHistogram_seconds_sum 0.45
# TYPE timer_noHistogram_seconds_max gauge
# HELP timer_noHistogram_seconds_max
timer_noHistogram_seconds_max 0.2
# TYPE timer_withHistogram_seconds histogram
# HELP timer_withHistogram_seconds
timer_withHistogram_seconds_bucket{le="0.001"} 0
timer_withHistogram_seconds_bucket{le="0.001048576"} 0
timer_withHistogram_seconds_bucket{le="0.001398101"} 0
timer_withHistogram_seconds_bucket{le="0.001747626"} 0
timer_withHistogram_seconds_bucket{le="0.002097151"} 0
timer_withHistogram_seconds_bucket{le="0.002446676"} 0
timer_withHistogram_seconds_bucket{le="0.002796201"} 0
timer_withHistogram_seconds_bucket{le="0.003145726"} 0
timer_withHistogram_seconds_bucket{le="0.003495251"} 0
timer_withHistogram_seconds_bucket{le="0.003844776"} 0
timer_withHistogram_seconds_bucket{le="0.004194304"} 0
timer_withHistogram_seconds_bucket{le="0.005592405"} 0
timer_withHistogram_seconds_bucket{le="0.006990506"} 0
timer_withHistogram_seconds_bucket{le="0.008388607"} 0
timer_withHistogram_seconds_bucket{le="0.009786708"} 0
timer_withHistogram_seconds_bucket{le="0.011184809"} 0
timer_withHistogram_seconds_bucket{le="0.01258291"} 0
timer_withHistogram_seconds_bucket{le="0.013981011"} 0
timer_withHistogram_seconds_bucket{le="0.015379112"} 1 # {span_id="5",trace_id="6"} 0.015 1738954792.088
timer_withHistogram_seconds_bucket{le="0.016777216"} 1
timer_withHistogram_seconds_bucket{le="0.022369621"} 1
timer_withHistogram_seconds_bucket{le="0.027962026"} 1
timer_withHistogram_seconds_bucket{le="0.033554431"} 1
timer_withHistogram_seconds_bucket{le="0.039146836"} 1
timer_withHistogram_seconds_bucket{le="0.044739241"} 1
timer_withHistogram_seconds_bucket{le="0.050331646"} 1
timer_withHistogram_seconds_bucket{le="0.055924051"} 1
timer_withHistogram_seconds_bucket{le="0.061516456"} 1
timer_withHistogram_seconds_bucket{le="0.067108864"} 1
timer_withHistogram_seconds_bucket{le="0.089478485"} 1
timer_withHistogram_seconds_bucket{le="0.111848106"} 1
timer_withHistogram_seconds_bucket{le="0.134217727"} 1
timer_withHistogram_seconds_bucket{le="0.156587348"} 2 # {span_id="7",trace_id="8"} 0.15 1738954792.098
timer_withHistogram_seconds_bucket{le="0.178956969"} 2
timer_withHistogram_seconds_bucket{le="0.20132659"} 2
timer_withHistogram_seconds_bucket{le="0.223696211"} 2
timer_withHistogram_seconds_bucket{le="0.246065832"} 2
timer_withHistogram_seconds_bucket{le="0.268435456"} 2
timer_withHistogram_seconds_bucket{le="0.357913941"} 2
timer_withHistogram_seconds_bucket{le="0.447392426"} 2
timer_withHistogram_seconds_bucket{le="0.536870911"} 2
timer_withHistogram_seconds_bucket{le="0.626349396"} 2
timer_withHistogram_seconds_bucket{le="0.715827881"} 2
timer_withHistogram_seconds_bucket{le="0.805306366"} 2
timer_withHistogram_seconds_bucket{le="0.894784851"} 2
timer_withHistogram_seconds_bucket{le="0.984263336"} 2
timer_withHistogram_seconds_bucket{le="1.073741824"} 2
timer_withHistogram_seconds_bucket{le="1.431655765"} 2
timer_withHistogram_seconds_bucket{le="1.789569706"} 2
timer_withHistogram_seconds_bucket{le="2.147483647"} 2
timer_withHistogram_seconds_bucket{le="2.505397588"} 2
timer_withHistogram_seconds_bucket{le="2.863311529"} 2
timer_withHistogram_seconds_bucket{le="3.22122547"} 2
timer_withHistogram_seconds_bucket{le="3.579139411"} 2
timer_withHistogram_seconds_bucket{le="3.937053352"} 2
timer_withHistogram_seconds_bucket{le="4.294967296"} 2
timer_withHistogram_seconds_bucket{le="5.726623061"} 2
timer_withHistogram_seconds_bucket{le="7.158278826"} 2
timer_withHistogram_seconds_bucket{le="8.589934591"} 2
timer_withHistogram_seconds_bucket{le="10.021590356"} 2
timer_withHistogram_seconds_bucket{le="11.453246121"} 2
timer_withHistogram_seconds_bucket{le="12.884901886"} 2
timer_withHistogram_seconds_bucket{le="14.316557651"} 2
timer_withHistogram_seconds_bucket{le="15.748213416"} 2
timer_withHistogram_seconds_bucket{le="17.179869184"} 2
timer_withHistogram_seconds_bucket{le="22.906492245"} 2
timer_withHistogram_seconds_bucket{le="28.633115306"} 2
timer_withHistogram_seconds_bucket{le="30.0"} 2
timer_withHistogram_seconds_bucket{le="+Inf"} 3 # {span_id="9",trace_id="10"} 60.0 1738954792.109
timer_withHistogram_seconds_count 3 # {span_id="9",trace_id="10"} 60.0 1738954792.109
timer_withHistogram_seconds_sum 60.165
# TYPE timer_withHistogram_seconds_max gauge
# HELP timer_withHistogram_seconds_max
timer_withHistogram_seconds_max 60.0
# TYPE timer_withSlos_seconds histogram
# HELP timer_withSlos_seconds
timer_withSlos_seconds_bucket{le="0.1"} 1 # {span_id="11",trace_id="12"} 0.015 1738954792.109
timer_withSlos_seconds_bucket{le="0.2"} 2 # {span_id="15",trace_id="16"} 0.15 1738954792.129
timer_withSlos_seconds_bucket{le="0.3"} 2
timer_withSlos_seconds_bucket{le="+Inf"} 3 # {span_id="13",trace_id="14"} 1.5 1738954792.119
timer_withSlos_seconds_count 3 # {span_id="15",trace_id="16"} 0.15 1738954792.129
timer_withSlos_seconds_sum 1.665
# TYPE timer_withSlos_seconds_max gauge
# HELP timer_withSlos_seconds_max
timer_withSlos_seconds_max 1.5
# EOF
"
to contain:
  "summary_withHistogram_bucket{le="16.0"} 2 # {span_id="23",trace_id="24"} 15.0 "

An exemplar sampling for the summary_withHistogram_bucket{le="+Inf"} 3 entry seems to have been missed.

I couldn't reproduce it locally, but if I reduce the sleep time to 1 ms, I could reproduce it.

This is an attempt to mitigate its flakiness by increasing chances for exemplar sampling to be scheduled in time.

See https://ge.micrometer.io/scans/tests?search.timeZoneId=Asia%2FSeoul&tests.container=io.micrometer.prometheusmetrics.PrometheusMeterRegistryTest&tests.sortField=FLAKY&tests.test=openMetricsScrapeWithExemplars()

…MetricsScrapeWithExemplars()

Signed-off-by: Johnny Lim <[email protected]>
@shakuzen shakuzen added the type: task A general task label Feb 10, 2025
@shakuzen shakuzen added this to the 1.13.11 milestone Feb 10, 2025
@shakuzen shakuzen merged commit f334566 into micrometer-metrics:1.13.x Feb 10, 2025
8 checks passed
@izeye izeye deleted the exemplar branch February 10, 2025 01:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: task A general task
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants