Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add randomized allocation sampling #100356

Closed
wants to merge 95 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
95 commits
Select commit Hold shift + click to select a range
3815aa8
This is an incomplete implementation of a new randomized allocation s…
noahfalk Feb 8, 2024
6a92fcb
Initial update
chrisnas Feb 21, 2024
cdbffc3
Update based on feedback
chrisnas Feb 22, 2024
924afd0
Rename fast_alloc_helper_limit_ptr to alloc_sampling
chrisnas Feb 23, 2024
e0ce093
Remove trailing whitespace in markdown
chrisnas Feb 23, 2024
396c05e
Add sampling threshold computation
chrisnas Feb 23, 2024
91b8b79
Take feedback into account and start to implement AllocationSampled e…
chrisnas Feb 27, 2024
65ff391
Fix typo
chrisnas Feb 27, 2024
b25adc6
Take review into account
chrisnas Feb 29, 2024
d939cdd
Emit the AllocationSampled event
chrisnas Mar 1, 2024
9b0f7da
Fix threshold computation error
chrisnas Mar 14, 2024
8875a4d
Deal with empty allocation context when fixing them
chrisnas Mar 16, 2024
f39a732
Handle the case of objects larger than an allocation context
chrisnas Mar 18, 2024
fbc0b6b
Take review into account
chrisnas Mar 21, 2024
32eb2fa
Take review into account for dynamically sampling
chrisnas Mar 22, 2024
518b4f7
Adding a first test that AllocationSampled is emitted when keyword/ve…
chrisnas Mar 26, 2024
b596f33
Check type name in AllocationSampled test
chrisnas Mar 27, 2024
984ed95
Compare perf impact of allocation sampling
chrisnas Mar 28, 2024
1c46a5d
Fix tests
chrisnas Apr 10, 2024
44e914b
Update tests and doc
chrisnas Apr 16, 2024
9eba36e
Update review
chrisnas Apr 19, 2024
b7614fd
Update based on review
chrisnas Apr 19, 2024
cbc5a12
Add tests and simple framework to measure statistical distribution of…
chrisnas Apr 19, 2024
826d8eb
Add x1/x2/x3 ratio check
chrisnas Apr 22, 2024
a56a5ee
Update array of double implementation for 32 bits
chrisnas Apr 22, 2024
48dbfa1
Updates
chrisnas May 7, 2024
7280c13
Fix markdown
chrisnas May 7, 2024
fd8a1b9
Add AllocationsRunEventSource to compute percentiles more easily
chrisnas May 8, 2024
a4aef1a
Allow percentiles computation
chrisnas May 9, 2024
08ec139
Add ratio based array allocations
chrisnas May 10, 2024
0bbbfa9
take review into account
chrisnas May 13, 2024
cf29e9b
Refactor the code and update upscaling method
chrisnas May 14, 2024
26e8fe0
Add runs for allocations
chrisnas May 14, 2024
52d4071
Handle padding for objects alignment
chrisnas May 16, 2024
db506a3
take GC padding into account
chrisnas May 16, 2024
8522866
Update allocation sampling distribution results
chrisnas May 16, 2024
e1f8ece
Update benchmarks with events
chrisnas May 16, 2024
32f6d31
Beginning of NativeAOT support
chrisnas May 18, 2024
9e31a6e
Fix trailing spaces
chrisnas May 21, 2024
f9f5224
Updates based on PR comments
chrisnas May 21, 2024
97a3bbd
Proposed changes
noahfalk May 21, 2024
734b773
Fix README
noahfalk May 21, 2024
a6120ac
- fix Alloc helper to get the address of the allocated object for All…
chrisnas May 22, 2024
3d0bcd1
Update the design doc
noahfalk May 23, 2024
549e7b0
Merge branch 'main' into chrisnas/alloc_sampling
chrisnas May 23, 2024
147447a
Fix compilation errors for Linux and related to merged changes in Thr…
chrisnas May 24, 2024
2b41997
Fix compilation issue
chrisnas May 24, 2024
4618b37
Restore Thread randomizer initialization
chrisnas May 24, 2024
918fe0c
Remove the randomizer from Thread to have just a singleton
chrisnas May 27, 2024
e82162d
Add a lazily allocated CLRRandom in Thread class
chrisnas May 28, 2024
09eae74
Start a no op implementation of randomized sampling for NativeAOT
chrisnas May 28, 2024
6badfb2
Fix typos in markdown
chrisnas May 29, 2024
d0e46a4
Fix linux compilation errors
chrisnas May 29, 2024
ef5b819
Make alloc context randomizer per thread lazily allocated
chrisnas May 29, 2024
ff77f7a
Fix missing rename in assembly code to use combined_limit instead of …
chrisnas May 29, 2024
77cb736
Fix build errors dues to explicit net8.0 usage in tests
chrisnas May 29, 2024
c40b1a2
Filter out AllocationTick when AllocationSampled is enabled
chrisnas May 30, 2024
a62ea75
Try a fix for build errors related to .csproj that are not really tests
chrisnas May 30, 2024
07ab88b
Avoid touching the IGCToCLREventSink interface because AllocationSampled
chrisnas May 30, 2024
d04b655
Update AllocationTick filtering implementation
chrisnas May 31, 2024
d565045
Fix double[] implementation
chrisnas May 31, 2024
b7bd18f
Fix test due to added field in the event payload
chrisnas May 31, 2024
7cbcb43
Implement emitting the AllocationSampled event in NativeAOT
chrisnas May 31, 2024
86f24c6
First no op implementation for NativeAOT with updated assembly code
chrisnas May 31, 2024
51d9935
Fix typos
chrisnas Jun 3, 2024
a9ef0e5
Fix DEBUG compilation error
chrisnas Jun 3, 2024
dd7e00d
Fix linux/x86 compilation errors
chrisnas Jun 3, 2024
a86a98b
Fix build error
chrisnas Jun 3, 2024
512f011
Try to fix "static" related compilation errors
chrisnas Jun 3, 2024
532f095
Fix test in 32 bit
chrisnas Jun 4, 2024
4fc9b8d
Avoid running AllocationSampled test with Mono + Fix compilation issue
chrisnas Jun 8, 2024
9174212
Start to implement the randomizer with existing 32 bit random
chrisnas Jun 8, 2024
541f59f
Fix compilation error
chrisnas Jun 12, 2024
40f96f3
Allow eventing functions to be called from thread.cpp
chrisnas Jun 14, 2024
29c16b5
Merge branch 'main' into chrisnas/alloc_sampling
chrisnas Jun 17, 2024
20c6d51
Fix assert in AOT
chrisnas Jun 17, 2024
c5ccad8
Fix link issue
chrisnas Jun 24, 2024
39afdf2
Fix NativeAOT build
chrisnas Jun 25, 2024
b2db19e
Merge branch 'main' into chrisnas/alloc_sampling
chrisnas Jun 26, 2024
e4ef681
Fix missing references to gc_alloc_context
chrisnas Jun 26, 2024
abfe6cb
Fix compilation issues
chrisnas Jun 27, 2024
997a293
Update x86 implementation
chrisnas Jun 28, 2024
041e397
Fix
chrisnas Jun 28, 2024
e312bf6
Fixes
chrisnas Jul 1, 2024
ec16fa0
Merge remote-tracking branch 'origin' into chrisnas/alloc_sampling
chrisnas Jul 5, 2024
fd382d3
Fix rebase
chrisnas Jul 5, 2024
a2e30cc
Fix linux gcc build issues
chrisnas Jul 5, 2024
83611b8
Add missing alloc context fixup and randomizer code
chrisnas Jul 9, 2024
c908fe4
Merge remote-tracking branch 'origin' into chrisnas/alloc_sampling
chrisnas Jul 9, 2024
dc02b7c
Fix issue when TLS for alloc context is not initialized
chrisnas Jul 10, 2024
85625cd
Merge remote-tracking branch 'origin' into chrisnas/alloc_sampling
chrisnas Jul 10, 2024
911d5d7
Change randomizer to use sxoshiro128++ for better statistical distrib…
chrisnas Jul 11, 2024
91d58dc
Fix possible crash in GCInterface_GetTotalAllocatedBytesPrecise
chrisnas Jul 11, 2024
11177d9
Fix possible desync between gc_alloc_context and combined_limit
chrisnas Jul 12, 2024
ce40d3d
Merge remote-tracking branch 'origin' into chrisnas/alloc_sampling
chrisnas Jul 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Take review into account
chrisnas committed Mar 21, 2024

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
commit fbc0b6b35f2de8f090fc85cfc49298be5738efe2
6 changes: 1 addition & 5 deletions src/coreclr/vm/ClrEtwAll.man
Original file line number Diff line number Diff line change
@@ -467,8 +467,6 @@
value="40" eventGUID="{CC82530E-A21C-4EB0-A68A-FA4F7E66498F}"
message="$(string.RuntimePublisher.AllocationSamplingTaskMessage)">
<opcodes>
<!-- TODO: I don't know which value should be picked for this opcode-->
<opcode name="AllocationSampled" message="$(string.RuntimePublisher.AllocationSampledOpcodeMessage)" symbol="CLR_ALLOCATIONSAMPLED_OPCODE" value="11"> </opcode>
</opcodes>
</task>
<!--Next available ID is 41-->
@@ -3175,7 +3173,6 @@
</template>

<template tid="AllocationSampled">
<!--TODO: is it a problem to reuse the GC allocation kind?-->
<data name="AllocationKind" inType="win:UInt32" map="GCAllocationKindMap" />
<data name="ClrInstanceID" inType="win:UInt16" />
<data name="TypeID" inType="win:Pointer" />
@@ -4292,7 +4289,7 @@

<!-- Allocation sampling events -->
<event value="303" version="0" level="win:Informational" template="AllocationSampled"
keywords="AllocationSamplingKeyword" opcode="AllocationSampled"
keywords="AllocationSamplingKeyword"
task="AllocationSampling"
symbol="AllocationSampled" message="$(string.RuntimePublisher.AllocationSampledEventMessage)"/>

@@ -9331,7 +9328,6 @@
<string id="RuntimePublisher.ExecutionCheckpointOpcodeMessage" value="ExecutionCheckpoint" />

<string id="RuntimePublisher.ProfilerOpcodeMessage" value="ProfilerMessage" />
<string id="RuntimePublisher.AllocationSampledOpcodeMessage" value="AllocationSampled" />

<string id="RundownPublisher.GCSettingsOpcodeMessage" value="GCSettingsRundown" />

24 changes: 11 additions & 13 deletions src/coreclr/vm/gcenv.ee.cpp
Original file line number Diff line number Diff line change
@@ -451,23 +451,21 @@ void InvokeGCAllocCallback(ee_alloc_context* pEEAllocContext, enum_alloc_context

// The allocation context might be modified by the callback, so we need to save
// the remaining sampling budget and restore it after the callback if needed.
size_t currentSamplingBudget = (pEEAllocContext->alloc_sampling == nullptr) ? -1 : (size_t)(pEEAllocContext->alloc_sampling - pAllocContext->alloc_ptr);
size_t currentSize = pAllocContext->alloc_limit - pAllocContext->alloc_ptr;
size_t currentSamplingBudget = (size_t)(pEEAllocContext->alloc_sampling - pAllocContext->alloc_ptr);
size_t currentSize = (size_t)(pAllocContext->alloc_limit - pAllocContext->alloc_ptr);

fn(pAllocContext, param);

if (currentSamplingBudget != -1)
// If the GC changed the size of the allocation context, we need to recompute the sampling limit
// This includes the case where the AC was initially zero-sized/uninitialized.
if (currentSize != (size_t)(pAllocContext->alloc_limit - pAllocContext->alloc_ptr))
{
// if the GC changed the size of the allocation context, we need to recompute the sampling limit
if (currentSize != (size_t)(pAllocContext->alloc_limit - pAllocContext->alloc_ptr))
{
pEEAllocContext->ComputeSamplingLimit(GetThread()->GetRandom());
}
else
{
// restore the remaining sampling budget when the GC simply moved the allocation context in memory
pEEAllocContext->alloc_sampling = pAllocContext->alloc_ptr + currentSamplingBudget;
}
pEEAllocContext->ComputeSamplingLimit(GetThread()->GetRandom());
}
else
{
// Restore the remaining sampling budget as the size is the same.
pEEAllocContext->alloc_sampling = pAllocContext->alloc_ptr + currentSamplingBudget;
}
}

8 changes: 4 additions & 4 deletions src/coreclr/vm/gcheaputilities.h
Original file line number Diff line number Diff line change
@@ -45,20 +45,20 @@ typedef struct _ee_alloc_context
// If sampling is on then we'd do some pseudo-random number generation to decide what is
// the next sampled byte in the gc_alloc_context, if any.

// compute the next sampling limit based on an exponential distribution
size_t threshold = ComputeExponetialRandom(pRandomizer);
// compute the next sampling limit based on a geometric distribution
size_t threshold = ComputeExponentialRandom(pRandomizer);

// if the threshold is larger than the allocation context, no sampling will occur
alloc_sampling = Min(gc_alloc_context.alloc_ptr + threshold, gc_alloc_context.alloc_limit);
}

static inline bool IsSampled(CLRRandom* pRandomizer, size_t range)
{
size_t threshold = ComputeExponetialRandom(pRandomizer);
size_t threshold = ComputeExponentialRandom(pRandomizer);
return (threshold < range);
}
private:
static inline size_t ComputeExponetialRandom(CLRRandom* pRandomizer)
static inline size_t ComputeExponentialRandom(CLRRandom* pRandomizer)
{
// compute a thres based on an exponential distribution
double probability = pRandomizer->NextDouble();
2 changes: 0 additions & 2 deletions src/coreclr/vm/gchelpers.cpp
Original file line number Diff line number Diff line change
@@ -248,8 +248,6 @@ inline Object* Alloc(ee_alloc_context* pEEAllocContext, size_t size, GC_ALLOC_FL
// only SOH allocations require sampling threshold to be recomputed
if ((flags & GC_ALLOC_USER_OLD_HEAP) == 0)
{
// the sampling threshold was crossed, so compute a new one (even if some
// free space is left in the allocation context)
pEEAllocContext->ComputeSamplingLimit(pCurrentThread->GetRandom());
}