Proof-of-concept for storing results in memory more efficiently

[thessem]

No description provided.

[IamXander]

I thought we agreed the path forward on this one was just to warn the user when they go over on memory? Are you sure GZIPing in memory will

1. Save space
2. Be easy to debug
3. Not effect the performance test

[thessem]

I thought we agreed the path forward on this one was just to warn the user when they go over on memory? Are you sure GZIPing in memory will

1. Save space

2. Be easy to debug

3. Not effect the performance test

This is something I did on my own time out of interest just to see if I could. On another personal project I found a 4x space saving with a technique like this, with very little performance impact (CPUs are very good at compression these days). On point (2), it can't be any harder than using a linked list with millions of entries (which is causing a lot of stress on the allocator and is giving us a fair bit of pointer overhead).

I have no proof of (1) or (3), and would like to go and find it on my own time. As for DEVPROD-2165, this was our agreed approach and the Jira integration shouldn't have progressed it. I will change the branch name so it doesn't cause noise over there.