Memory hogging happening in ReflectionBasedDestruturer

[NizamLZ]

Issue Description

Server memory is suddenly getting filled from 11% to 99% within ~3 mins. Came to this conclusion based on the memory dumps.

Unexpected EF Core entities found in memory.

Version Serilog.Exceptions NuGet package?

5.3.1

[krajek]

Hi @NizamLZ , thank you for the report and sorry for the problem.

Could you tell us more about your use case?

Do you have any idea how you got millions of objects in such a short time? Destructurer works only when there are exceptions, do you have that many exceptions? Could you show me what kind of exceptions are you throwing?

The <String, Object> dictionary that is mentioned is a local variable so currently I assume it is not a culprit of a memory leak.

[NizamLZ]

Hi @krajek,

I agree, its not a leak.

The stack trace showed that there was an exception(logger.log invocation), not millions, only one. The control went into Destructurer and did not come out.
I believe Destructurer reached the DB Context->Db Set or directly DB Set and started reading it, making the EF Core hit the db and get the entires.

One thing to note is that all of the entities in memory belong to same entity type but their values are different which makes me believe that the records are read from the database rather than being the clone of the same object.

I can share the dot memory workspace file (~2.5GB), It needs a system with more than 32GB of ram, to which I no longer have access and cannot look deeper into the call stack etc. by myself.

[krajek]

Got it. What is the type of an exception?

A) In general, if you have a problem with the particular type of exceptions you can implement custom destructurer. In your case, you would just ignore properties that reach DbContext and avoid the problem altogether.

B) #117 Lately there was an accepted PR that seems to be closely related to your problem. I didn't follow closely, but that may be helpful.

C) It could be useful to use some SQL Profiler to investigate if the assumptions that DbContext started reading the whole table is correct.

[RehanSaeed]

Hey @NizamLZ, hope all is well.

If this is EF Core, then there is a new package to deal with it's exceptions.

[NizamLZ]

@krajek, They are sql timeout exceptions. Thanks for options A & C, will try to do one of those.
I knew B and hence updated the package to 5.2.1 very recently. This started after that itself. Never happened with old version, hence wanted to let you guys know.

Hey @RehanSaeed, all good here. Hope you are doing good as well.
Which package are you referring to?

[RehanSaeed]

There are two packages on the home page. One for SqlException and one for EF Core exceptions.

[joelweiss]

@NizamLZ

I knew B and hence updated the package to 5.2.1 very recently.

The latest package is not enough, did you add the DbUpdateExceptionDestructurer during setup?

[RehanSaeed]

Closing. Feel free to reopen if the other packages I mentioned are not enough.