Document problems #448
Document problems #448
Conversation
PROBLEMS.md
Outdated
|
|
||
| For [god objects](https://en.wikipedia.org/wiki/God_object) (like the Shop model at Shopify), it’s typical to have many associations. It’s for developers appealing to embed many of those associations into the god model’s IDC blob with `cache_has_many :images, embed: true`. As a side effect, over time the god model’s IDC blob may grow over the top. It gets even worse when some of the embedded records get larger, and the total IDC blob gets into as much as a megabyte in size. This puts pressure on both the application and memcached, because the blob needs to be serialized and unserialized each time and transferred over the network. | ||
|
|
||
| ## Overfetching |
There was a problem hiding this comment.
Is this really a separate issue? It seems like another issue that comes from embedded associations.
Another possible issue that can come from embedded associations is that cache invalidations for a model can be amplified by the number of models that embed them. This can further aggravate the large cache blob problem by embedding an association that gets invalidated more frequently.
PROBLEMS.md
Outdated
|
|
||
| If that key associated with a record becomes busy and hot, that memcached instance will become saturated on the network and the number of bytes that it can send out. In cloud environments, it’s a limit that is easy to hit. | ||
|
|
||
| ## Thundering herd |
There was a problem hiding this comment.
This one seems like it is more clearly just an issue that needs fixing. We even have a pull request with a possible fix (#373). So why document it here instead of as a github issue? You can always reference issues from the documentation
Co-Authored-By: Hormoz Kheradmand <[email protected]>
|
Thanks for feedback, I've addressed all points. I also renamed the file to CAVEATS.md (thanks to Hormoz for helping me to pick a better word) |
There's some context around "what's bad with IDC" at Shopify. I wanted to take a step at documenting some of those to make it easier to pass to other people on the team. I preferred to have it as a markdown file within this public repo for better discoverability, compared to having it as a private doc at Shopify that might get lost.
I also tried not to get into what would be solutions for those problems because there's too many possibilities :)