Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added net/http-tracer #84

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions tracer/contrib/net/httptrace/example_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
package httptrace_test

import (
"fmt"
"net/http"

"github.com/DataDog/dd-trace-go/tracer"
"github.com/DataDog/dd-trace-go/tracer/contrib/net/httptrace"
)

func handler(w http.ResponseWriter, r *http.Request) {
span := tracer.SpanFromContextDefault(r.Context())
fmt.Printf("tracing service:%s resource:%s", span.Service, span.Resource)
w.Write([]byte("hello world"))
}

func Example() {
mux := http.NewServeMux()
mux.Handle("/users", handler)
mux.Handle("/anything", handler)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

handler can't be used as a http.Handler, use http.HandlerFunc(handler) it will create an http.Handler struct

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

httpTracer := httptrace.NewHttpTracer("fake-service", tracer.DefaultTracer)

http.ListenAndServe(":8080", httpTracer.Handler(mux))
}
122 changes: 122 additions & 0 deletions tracer/contrib/net/httptrace/httptrace.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
package httptrace

import (
"net/http"
"strconv"

"github.com/DataDog/dd-trace-go/tracer"
"github.com/DataDog/dd-trace-go/tracer/ext"
)

// HttpTracer is used to trace requests in a net/http server.
type HttpTracer struct {
tracer *tracer.Tracer
service string
}

// NewHttpTracer creates a HttpTracer for the given service and tracer.
func NewHttpTracer(service string, t *tracer.Tracer) *HttpTracer {
t.SetServiceInfo(service, "net/http", ext.AppTypeWeb)
return &HttpTracer{
tracer: t,
service: service,
}
}

// Handler will return a Handler that will wrap tracing around the
// given handler.
func (h *HttpTracer) Handler(handler http.Handler) http.Handler {
return h.TraceHandlerFunc(http.HandlerFunc(func(writer http.ResponseWriter, req *http.Request) {
handler.ServeHTTP(writer, req)
}))
}

// TraceHandlerFunc will return a HandlerFunc that will wrap tracing around the
// given handler func.
func (h *HttpTracer) TraceHandlerFunc(handler http.HandlerFunc) http.HandlerFunc {

return func(writer http.ResponseWriter, req *http.Request) {

// bail out if tracing isn't enabled.
if !h.tracer.Enabled() {
handler(writer, req)
return
}

// trace the request
tracedRequest, span := h.trace(req)
defer span.Finish()

// trace the response
tracedWriter := newTracedResponseWriter(span, writer)

// run the request
handler(tracedWriter, tracedRequest)
}
}

// span will create a span for the given request.
func (h *HttpTracer) trace(req *http.Request) (*http.Request, *tracer.Span) {
resource := req.Method + " " + req.URL.Path

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will create a unique resource for each new path, which can become problematic as soon as you have path parameters (eg. "/article/:id") in some routes, as is the case for most APIs nowadays.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you are correct, but is there a way to do that with the default go handler? I'm not sure Go's net/http package natively supports pattern matching the way httprouter and gorilla/mux do.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK, net/http doesn't implement pattern matching and named path parameters as most 3rd party go routers do. I therefore think that with the current go ecosystem, we need one integration per HTTP router (cf. my current PR to integrate github.com/julienschmidt/httprouter, or any of the other router integrations), your implementation would still work for the default http mux, but not as a generic go http integration.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense. This is not as much a replacement/catchall then, and instead just something that works with the base net/http functions. I wonder if we should use this from the other two to build the span & have a resource builder func that can build the pattern for the endpoint.

\braindump

Copy link
Member

@raphaelgavache raphaelgavache Jul 21, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello, thank you for that PR FreekingDean. I actually found a nice pattern matching function in net/url. You can try this on the playground:

package main

import (
	"fmt"
	"net/url"
)

func main() {
	u, _ := url.Parse("/books?author=:author&subject=")
	fmt.Println(u.Path)
        // /books
	u, _ = url.Parse("/search?q=net%2Furl&rlz=1C5CHFA_enUS730US731&oq=net%2Furl&aqs=chrome..69i57j69i58j69i61l3j69i59.1391j0j7&sourceid=chrome&ie=UTF-8#q=golang+parse+url+path")
	fmt.Println(u.Path)
        // /search
}

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @furmmon! That is awesome. I think the main issue would be resource pattern matching when a parameter is in the path: www.api.com/apps/:app_id/get_resource_info In this case the methods I provide would not be sufficient to pull out the relevant path and instead create a new resource for each call to that endpoint:

/apps/1/get_resource_info
/apps/2/get_resource_info
/apps/3/get_resource_info

In this case I think it may be best to have path be absolute in this base case (IE the way I described) but also allow individual libraries to supply the path. (For use with gomux & httprouter)

Copy link
Member

@raphaelgavache raphaelgavache Jul 21, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

edit - comment outdated


span := h.tracer.NewRootSpan("http.request", h.service, resource)
span.Type = ext.HTTPType
span.SetMeta(ext.HTTPMethod, req.Method)
span.SetMeta(ext.HTTPURL, req.URL.Path)

// patch the span onto the request context.
treq := SetRequestSpan(req, span)
return treq, span
}

// tracedResponseWriter is a small wrapper around an http response writer that will
// intercept and store the status of a request.
type tracedResponseWriter struct {
span *tracer.Span
w http.ResponseWriter
status int
}

func newTracedResponseWriter(span *tracer.Span, w http.ResponseWriter) *tracedResponseWriter {
return &tracedResponseWriter{
span: span,
w: w,
}
}

func (t *tracedResponseWriter) Header() http.Header {
return t.w.Header()
}

func (t *tracedResponseWriter) Write(b []byte) (int, error) {
if t.status == 0 {
t.WriteHeader(http.StatusOK)
}
return t.w.Write(b)
}

func (t *tracedResponseWriter) WriteHeader(status int) {
t.w.WriteHeader(status)
t.status = status
t.span.SetMeta(ext.HTTPCode, strconv.Itoa(status))
if status >= 500 && status < 600 {
t.span.Error = 1
}
}

// SetRequestSpan sets the span on the request's context.
func SetRequestSpan(r *http.Request, span *tracer.Span) *http.Request {
if r == nil || span == nil {
return r
}

ctx := tracer.ContextWithSpan(r.Context(), span)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This way of setting the span into the context is deprecated, we changed to
ctx := span.Context(r.Context())

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

return r.WithContext(ctx)
}

// GetRequestSpan will return the span associated with the given request. It
// will return nil/false if it doesn't exist.
func GetRequestSpan(r *http.Request) (*tracer.Span, bool) {
span, ok := tracer.SpanFromContext(r.Context())
return span, ok
}
168 changes: 168 additions & 0 deletions tracer/contrib/net/httptrace/httptrace_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
package httptrace

import (
"net/http"
"net/http/httptest"
"testing"

"github.com/DataDog/dd-trace-go/tracer"
"github.com/stretchr/testify/assert"
)

func TestHttpTracerDisabled(t *testing.T) {
assert := assert.New(t)

testTracer, testTransport, httpTracer := getTestTracer("disabled-service")
handler := httpTracer.TraceHandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_, err := w.Write([]byte("disabled!"))
assert.Nil(err)
// Ensure we have no tracing context.
span, ok := tracer.SpanFromContext(r.Context())
assert.Nil(span)
assert.False(ok)
})
testTracer.SetEnabled(false) // the key line in this test.

// make the request
req := httptest.NewRequest("GET", "/disabled", nil)
writer := httptest.NewRecorder()
handler.ServeHTTP(writer, req)
assert.Equal(writer.Code, 200)
assert.Equal(writer.Body.String(), "disabled!")

// assert nothing was traced.
testTracer.ForceFlush()
traces := testTransport.Traces()
assert.Len(traces, 0)
}

func TestHttpTracer200(t *testing.T) {
assert := assert.New(t)

// setup
tracer, transport, router := setup(t)

// Send and verify a 200 request
url := "/200"
req := httptest.NewRequest("GET", url, nil)
writer := httptest.NewRecorder()
router.ServeHTTP(writer, req)
assert.Equal(writer.Code, 200)
assert.Equal(writer.Body.String(), "200!")

// ensure properly traced
tracer.ForceFlush()
traces := transport.Traces()
assert.Len(traces, 1)
spans := traces[0]
assert.Len(spans, 1)

s := spans[0]
assert.Equal(s.Name, "http.request")
assert.Equal(s.Service, "my-service")
assert.Equal(s.Resource, "GET "+url)
assert.Equal(s.GetMeta("http.status_code"), "200")
assert.Equal(s.GetMeta("http.method"), "GET")
assert.Equal(s.GetMeta("http.url"), url)
assert.Equal(s.Error, int32(0))
}

func TestHttpTracer500(t *testing.T) {
assert := assert.New(t)

// setup
tracer, transport, router := setup(t)

// SEnd and verify a 200 request
url := "/500"
req := httptest.NewRequest("GET", url, nil)
writer := httptest.NewRecorder()
router.ServeHTTP(writer, req)
assert.Equal(writer.Code, 500)
assert.Equal(writer.Body.String(), "500!\n")

// ensure properly traced
tracer.ForceFlush()
traces := transport.Traces()
assert.Len(traces, 1)
spans := traces[0]
assert.Len(spans, 1)

s := spans[0]
assert.Equal(s.Name, "http.request")
assert.Equal(s.Service, "my-service")
assert.Equal(s.Resource, "GET "+url)
assert.Equal(s.GetMeta("http.status_code"), "500")
assert.Equal(s.GetMeta("http.method"), "GET")
assert.Equal(s.GetMeta("http.url"), url)
assert.Equal(s.Error, int32(1))
}

// test handlers

func handler200(t *testing.T) http.HandlerFunc {
assert := assert.New(t)
return func(w http.ResponseWriter, r *http.Request) {
_, err := w.Write([]byte("200!"))
assert.Nil(err)
span := tracer.SpanFromContextDefault(r.Context())
assert.Equal(span.Service, "my-service")
assert.Equal(span.Duration, int64(0))
}
}

func handler500(t *testing.T) http.HandlerFunc {
assert := assert.New(t)
return func(w http.ResponseWriter, r *http.Request) {
http.Error(w, "500!", http.StatusInternalServerError)
span := tracer.SpanFromContextDefault(r.Context())
assert.Equal(span.Service, "my-service")
assert.Equal(span.Duration, int64(0))
}
}

func setup(t *testing.T) (*tracer.Tracer, *dummyTransport, *http.ServeMux) {
tracer, transport, ht := getTestTracer("my-service")

mux := http.NewServeMux()

h200 := handler200(t)
h500 := handler500(t)

mux.Handle("/200", ht.Handler(h200))
mux.Handle("/500", ht.Handler(h500))

return tracer, transport, mux
}

// getTestTracer returns a Tracer with a DummyTransport
func getTestTracer(service string) (*tracer.Tracer, *dummyTransport, *HttpTracer) {
transport := &dummyTransport{}
tracer := tracer.NewTracerTransport(transport)
muxTracer := NewHttpTracer(service, tracer)
return tracer, transport, muxTracer
}

// dummyTransport is a transport that just buffers spans and encoding
type dummyTransport struct {
traces [][]*tracer.Span
services map[string]tracer.Service
}

func (t *dummyTransport) SendTraces(traces [][]*tracer.Span) (*http.Response, error) {
t.traces = append(t.traces, traces...)
return nil, nil
}

func (t *dummyTransport) SendServices(services map[string]tracer.Service) (*http.Response, error) {
t.services = services
return nil, nil
}

func (t *dummyTransport) Traces() [][]*tracer.Span {
traces := t.traces
t.traces = nil
return traces
}

func (t *dummyTransport) SetHeader(key, value string) {}