Add python sentry (and sentry & django) integrations

[macobo]

This creates a PostHogIntegration to use inside sentry_sdk to capture events in PostHog, and a PostHogDistinctIdMiddleware which allows creating links to PostHog in Sentry.

Usage examples in README.md and sentry_django_example repo.

For using inside PostHog/posthog, it would look like this(settings.py):

+POSTHOG_DJANGO = {
+    "distinct_id": lambda request: request.user and request.user.distinct_id
+}
+
 if not TEST:
     if os.environ.get("SENTRY_DSN"):
         # https://docs.sentry.io/platforms/python/
         sentry_sdk.init(
             dsn=os.environ["SENTRY_DSN"],
-            integrations=[DjangoIntegration(), CeleryIntegration(), RedisIntegration()],
+            integrations=[DjangoIntegration(), CeleryIntegration(), RedisIntegration(), PostHogIntegration()],
             request_bodies="always",
+            before_send=update_posthog
         )

 if get_bool_from_env("DISABLE_SECURE_SSL_REDIRECT", False):
@@ -210,6 +216,7 @@ MIDDLEWARE = [
     "django.contrib.messages.middleware.MessageMiddleware",
     "django.middleware.clickjacking.XFrameOptionsMiddleware",
     "whitenoise.middleware.WhiteNoiseMiddleware",
+    "posthoganalytics.sentry.django.PosthogDistinctIdMiddleware"
 ]

Testing this using unit tests or something similar is hard. Open to suggestions, I've created a manual end-to-end test which integrates with a Sentry + PostHog installation.

[macobo]

# settings.py
middlewares += ['ourSentryMiddleWare']

POSTHOG_DJANGO = {
  distinct_id: lambda request: request.user.id
}

[neilkakkar]

An overall question about this ticket: Why isn't this a part of core PostHog? Since it's a middleware that depends on

1. Running only PostHog on Django with Sentry
2. The specific middleware added to the list

and can't be used anywhere else.

Maybe https://github.com/PostHog/posthog/blob/master/posthog/middleware.py is a better place to add this?

[mariusandra]

I think the original idea between "adding sentry to X" has been that if you are building your own python app, and use both posthog and sentry, the two libraries could coordinate. What this means is that you would have a link to the user in posthog in sentry, and on the other side PostHog would also capture the error in a $exception event.

I think that's how it works with the posthog-js library, but double check to be sure. I think it was @timgl and @macobo (and or @yakkomajuri ?) who built the original JS sentry integration.

[neilkakkar]

Ah, that makes some sense.

So, the idea is to integrate posthog-python and sentry together, such that posthog.capture is included inside Sentry via a PostHogSentryIntegration, which has nothing to do with the main Django PostHog app. (while at the same time, adding the posthog_distinct_id to Sentry as well)

I think that's what posthog-js is doing: https://github.com/PostHog/posthog-js/blob/6bf9b1e0f59849cae9a1e00e56092a06842a8142/src/posthog-core.js#L1391-L1415

Edit: .. Diving deeper into posthog-js, things seem a bit weirder: posthog-js generates its own distinct_ids (for things like autocapture?) while posthog-python is stateless - it depends on user input to tell it what distinct_ids to work with. We could work around this by creating an Integration that takes in a callable (which must be user defined), that returns the current distinct_id.

Am I missing something obvious?

[macobo]

We could work around this by creating an Integration that takes in a callable (which must be user defined), that returns the current distinct_id.

There's no good way to figure that out from the global context, hence the django middleware (you can get the user from request)

[neilkakkar]

I guess the part I don't understand is: there's no django middleware on posthog-python. We're dealing directly with the API. So, how does this work?

So, let's say someone's building HogFlixApp, and using both, Sentry and PostHog. So, in their Flask app, they

import posthog
import sentry_sdk

sentry_sdk.init(integrations=[posthog.sentry.PostHogSentryIntegration(get_distinct_id?)]

which somehow sends things to both Sentry and PostHog.

[macobo]

To get request-specific information in sentry from flask, it also installs a middleware: https://github.com/getsentry/sentry-python/blob/master/sentry_sdk/integrations/flask.py

You hit the nail on the head before:

1. You need user information to provide any value with the integration
2. There's no real way to get that user information with "vanilla" python without user hooking posthog-python into $framework_of_choice AND sentry

If you're still confused happy to pass on context on monday or let marius do it. :)

[neilkakkar]

.. Okay, that clears things up a lot! So, it's for someone else building HogFlixApp only using Django with Sentry, and not just plain python / some other framework.

And after that expansion, compressing information back into the name of the ticket makes sense again 😂.

The point of confusion was the main repo: it's not our main PostHog repo, but randomDjangoApp.

[neilkakkar]

@macobo requesting review (can't add you the usual way since you're the PR creator)

[neilkakkar]

Friendly ping, @yakkomajuri xD

[yakkomajuri]

One nit otherwise looks good to me. Would be good to get some tests though although I appreciate it might be a tiny bit annoying to set up

[yakkomajuri]

nit: can we substitute all instances of "organisation" for the american spelling "organization"? Just to be consistent since we refer to it as the latter everywhere in the product

[neilkakkar]

re: tests - apologies if it wasn't clear: there is a working end to end test (hence the sample Django app setup), it just isn't automated, since it involves setting up Sentry DSN + PostHog keys on the fly. (and replication instructions in README)

I can't unit-test this works, since the Integration is managed by Sentry, and the middleware is managed by Django, neither of which is in our control.

But, if you have any ideas on how to go about it - all ears!