Skip to content
This repository has been archived by the owner on Aug 30, 2019. It is now read-only.

Replace Span with a protobuf model #339

Merged
merged 1 commit into from
Dec 6, 2017
Merged

Conversation

LotharSee
Copy link

@LotharSee LotharSee commented Dec 4, 2017

The core model.Span is now coming from a protobuf model. That's to prepare the Agent to new API endpoints using protobuf. By using internally the same model, it will keep the code simple, without the need to cast to a special struct for the serialization.

Use gogoproto extensions to allow to keep the same JSON/msgpack fields/tags as before.

Comes with some cleanups, required to restructure the code:

  • Introduce WeightedSpan to carry the "cached" topLevel and weight attributes (which are stored in the Metrics map, but accessing it from the Concentrator while the Sampler modifies it can cause crashes). It makes clear that this extended structured is only used in the Concentrator.
  • Remove some useless types or methods.
  • Replace some signatures to be by reference instead of copy.

Results of impacted benchmarks:


# before
BenchmarkHandleSpan-4              	                  	  300000	      5702 ns/op	    2976 B/op	      21 allocs/op
BenchmarkHandleSpanSublayers-4              	         	  100000	     16152 ns/op	    5456 B/op	      56 allocs/op
BenchmarkComputeSublayers-4               	           	  100000	     12539 ns/op	    3282 B/op	      37 allocs/op

BenchmarkAgentTraceProcessing-4                         	   10000	    173126 ns/op	   50889 B/op	     702 allocs/op
BenchmarkAgentTraceProcessingWithFiltering-4            	   10000	    169601 ns/op	   45926 B/op	     629 allocs/op
BenchmarkAgentTraceProcessingWithWorstCaseFiltering-4   	   10000	    265090 ns/op	   50981 B/op	     700 allocs/op
BenchmarkHandleSpanRandom-4                             	   10000	    155103 ns/op	   29036 B/op	     377 allocs/op


# after
BenchmarkHandleSpan-4                        	        	  200000	      6494 ns/op	    3168 B/op	      30 allocs/op
BenchmarkHandleSpanSublayers-4                    	   	  100000	     15966 ns/op	    5456 B/op	      56 allocs/op
BenchmarkComputeSublayers-4                       	   	  100000	     13909 ns/op	    3282 B/op	      37 allocs/op

BenchmarkAgentTraceProcessing-4                         	   10000	    146068 ns/op	   50014 B/op	     716 allocs/op
BenchmarkAgentTraceProcessingWithFiltering-4            	   10000	    145527 ns/op	   44779 B/op	     640 allocs/op
BenchmarkAgentTraceProcessingWithWorstCaseFiltering-4   	    5000	    248793 ns/op	   50630 B/op	     720 allocs/op
BenchmarkHandleSpanRandom-4                             	   20000	     77989 ns/op	   28261 B/op	     392 allocs/op

Basically, no big significant changes. Some extra allocations because of the new WeightedTrace struct, but faster code, mostly because of references and removal of an unecessary ComputeTopLevel. Most are actually related to the test implementation itself (which highlight that we could afford better/more isolated benchmarks).

The core model.Span is now coming from protobuf.
Use gogoproto extensions to allow to keep the same
JSON/msgpack fields/tags as before.

To make it work, some other cleanups:
- Introduce WeightedSpan to carry the "cached"
topLevel and weight attributes. It makes clear
that was only used in the Concentrator.
- Remove some useless types or methods.
@LotharSee LotharSee requested review from AlexJF and ufoot December 4, 2017 18:08
@LotharSee LotharSee changed the title Replace Span with a protobuf model, clean code Replace Span with a protobuf model Dec 4, 2017
Copy link

@ufoot ufoot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. I'm wondering about the naming, but since I have no clear replacement, this can wait until we actually release this.

package model

// WeightedSpan extends Span to contain weights required by the Concentrator.
type WeightedSpan struct {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wondering about the name here, it's certainly Weighted but it also has TopLevel. And I suspect it could have more tomorrow, as in, any information that could be inferred from trace content, but has been processed once for all & cached. Sadly I do not have a nice replacement here, ResolvedSpan is not good too I think, because we use the Resolve term for other things. It just itches me that Weighted is not really about weight but "weight + other things".

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EnhancedSpan perhaps?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I didn't find anything better. In a way the topLevel can be seen as a boolean weight on if the stat should be computed or not ; but it might be a bit far fetch.

I'll keep it as it until we actually put something else in it.

@@ -217,7 +218,10 @@ func (a *Agent) Process(t model.Trace) {
rate *= a.Receiver.preSampler.Rate()
sampler.SetTraceAppliedSampleRate(root, rate)

// Need to do this computation before entering the concentrator
// as they access the Metrics map, which is not thread safe.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 much appreciated comment.

@LotharSee LotharSee merged commit 84f5220 into master Dec 6, 2017
@dtilghman dtilghman deleted the benjamin/protobuf-span branch December 7, 2017 05:14
@palazzem palazzem added this to the 5.21.1 milestone Jan 27, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants