Exposes minimal GC stats via gc_usage

[mihails-strasuns]

https://issues.dlang.org/show_bug.cgi?id=16019

This is one very minor utility that we have in Sociomantic GC API that makes it different from current upstream one - and which is used quite extensively. One of most important use cases of having these stats is to be able write a test that memory consumption is O(1) for a function that can't be @nogc.

Ping @MartinNowak

[dlang-bot]

Fix	Bugzilla	Description
✓	16019	Implement a way to check GC usage stats from application

[schveiguy]

Whitespace issue.

[schveiguy]

1. Why does the more general gc_stats function not suffice?
2. I'm not sure freeblocks is of the same unit as usedsize. Your function gives the impression that free is in bytes, but GCStats says freeblocks is in "blocks", whatever that means.

[mihails-strasuns]

Why does the more general gc_stats function not suffice?

It is currently not exposed being current GC implementation detail. As far as I understand, reason for that was that full gc_stats structs contains metrics that possibly have no meaning for different GC implementation and the exposed API must be generally applicable. @MartinNowak should know more.

I'm not sure freeblocks is of the same unit as usedsize

I wasn't sure either but code looks like it : https://github.com/dlang/druntime/blob/master/src/gc/gc.d#L1262-L1264

[mihails-strasuns]

I wasn't sure either but code looks like it

Oops, it must be freelistsizeof course, not freeblocks!

[schveiguy]

It is currently not exposed being current GC implementation detail.

What I think may be a better option than creating a redundant, less powerful, but more general and stable function, is to document which of the GCStats members are finalized. So for example, free bytes and used bytes is likely a stat we require ALL GC implementations to provide, so they must fill in those pieces. If we want to change the other stats, or add more (or make them customizable, etc.), then that can happen without affecting code that is written to use GCStats.freebytes and GCStats.usedbytes. You just need an alias from freelistbytes to more general freebytes.

The gc_stats function itself can remain undocumented for now (and sociomantic just knows that it's there). This can't be any worse than your current situation.

[mihails-strasuns]

So what API would you envision for actual public function? One additional problem with exposing GCStats is that right now relation between core.memory and GC is only using extern(C) linkage, with no actual imports. struct doesn't fit that scheme well - probably it is time to move API parts of gc.proxy in separate module and make it imported by both.

[MartinNowak]

Let's return a struct instead of using out parameters.
That implies moving GCStats to core.memory.

[MartinNowak]

In fact it's already there, so please just forward to gc_stats here, and change the GCStats members.

[mihails-strasuns]

In fact it's already there

? There is no GCStats in core.memory, it is only defined in gc stats module. This is why I am not sure about how to proceed here.

How about introducing new core.gcapi module that can be imported by both GC implementations and core.memory? This whole business of manually maintaining extern(C) declarations in sync feels like legacy.

[mihails-strasuns]

Ping @MartinNowak

[rainers]

How about introducing new core.gcapi module that can be imported by both GC implementations and core.memory?

That's more or less what gc.gcinterface does in #1581.

This whole business of manually maintaining extern(C) declarations in sync feels like legacy.

I agree. If the extern(C) interface is to stay (it is barely useful to begin with), it should be opt-in provided by a GC proxy. Having the calls go through an extern(C) interface also inhibits some inlining for the common case.

[MartinNowak]

I'd opt for

struct GCStats
{
    size_t heapsize;        /// total size of GC heap
    size_t freesize;        /// free bytes on the GC heap (might only get updated after a collection)
}

atm.

[PetarKirov]

@Dicebot you may want to have a look at PR #1581 as there will be a merge conflict in of the two PRs, depending on which PR is merged first.

[mihails-strasuns]

Thanks, will keep it in mind

[rainers]

So what API would you envision for actual public function?

A common technique is to have a field with flags for valid info fields in the info struct, for example https://technet.microsoft.com/en-us/library/bb146197(v=vs.110).aspx
The call could also just request the fields it is interested in, so you don't have to pay for extensive heap analysis if you just want to get the overall used memory.

One additional problem with exposing GCStats is that right now relation between core.memory

Importing core from the GC should be no problem, it already does that for other modules.

[mihails-strasuns]

A common technique is to have a field with flags for valid info fields ...

Sounds like a good approach indeed.

Importing core from the GC should be no problem, it already does that for other modules.

It looks semantically wrong though, considering it is core.memory that calls GC functions, not other way around.

[PetarKirov]

It looks semantically wrong though

I thought so too, but it turns out it is by design. See:
#1581 (comment)

[mihails-strasuns]

@ZombineDev that doesn't contradict my proposal of gcapi module - it would contain the very same èxtern(C) declarations but imported by both gc proxy and core modules.

[MartinNowak]

A common technique is to have a field with flags for valid info fields ...

Sounds like a good approach indeed.

Let's not overcomplicate things, we can go with a minimal API first and extend it later, e.g. by adding a name argument and a separate float value for that field.

so you don't have to pay for extensive heap analysis if you just want to get the overall used memory

We won't add any expensive stat functions. The values might be outdated and/or estimates.

The path forward here is the following.

• change the fields of GCStats (this has never been public API)
• move gc.stats.GCStats to core.memory
• add core.memory.GC.stats to forward to gc.proxy.gc_stats
• reopen against master (new features don't go into stable)

[mihails-strasuns]

Replaced by #1610