Skip to content
This repository has been archived by the owner on Aug 30, 2019. It is now read-only.

[trace] Moving quantization before truncation of spans #324

Merged
merged 4 commits into from
Oct 25, 2017

Conversation

ryplo
Copy link

@ryplo ryplo commented Oct 18, 2017

This is to ensure that strings (ex. query strings) stored in certain fields, such as resources are still parseable and are quantized properly.
Tested in staging and prod

@palazzem palazzem requested a review from ufoot October 19, 2017 12:26
@palazzem palazzem added the core label Oct 19, 2017
@palazzem
Copy link

@ufoot I think it's safe to ship that huge improvement together with the distributed sampling. If you don't have strong opinion, I'm going to add that one to our 5.18.2 milestone.

@ufoot ufoot added this to the 5.18.2 milestone Oct 20, 2017
Copy link

@ufoot ufoot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few remarks about testing.

@@ -178,6 +178,7 @@ func (a *Agent) Process(t model.Trace) {

for i := range t {
t[i] = quantizer.Quantize(t[i])
t[i].Truncate()
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm generally speaking in for this, the only thing I'm worried about is Quantize eating lots of CPU when the resource is very long. We have a protection against this, the watchdog is going to drop traces at pre-sampling stage if this happens. I think it's fine to ship this as is, but let's keep an eye on performance, not saying we should not do it, just, pay attention to side effects.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point! Will keep that in mind

@@ -0,0 +1,66 @@
package model

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we lack a test here, something that would test that say UPDATE blogpost SET comment='<...something possibly huge, at least longer than 5000 char...>' WHERE blogpost_id=42; and see how that ends. We should end up with the right quantization, which shows the WHERE clause. My understanding is that the purpose of this PR is to treat that case, because before, truncation would have removed the WHERE clause before anything else. I might have missed that test, or an equivalent, if that's the case, just point me to it.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed with Christian - decided to add a test mimicking agent behaviour to test joined effort of quantizer and truncator.

@ryplo ryplo force-pushed the rachel/truncate-after-quantize branch from 95cc2f0 to 54ab7a0 Compare October 20, 2017 16:51
@ryplo ryplo force-pushed the rachel/truncate-after-quantize branch from 54ab7a0 to 356c3fa Compare October 23, 2017 14:51
@ryplo ryplo merged commit 2ba2b5a into master Oct 25, 2017
@ryplo ryplo deleted the rachel/truncate-after-quantize branch October 25, 2017 20:43
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants