feat: Write person ID and properties to redis

[tiina303]

Problem

For #9179

Storing person info in redis:

1. teamId, distinctId -> personId
2. teamId, personId -> createdAt
3. teamId, personId -> properties

Changes

👉 Stay up-to-date with PostHog coding conventions for a smoother review.

How did you test this code?

Ran locally with PERSON_INFO_TO_REDIS_TEAMS="1,2,3,4,5" ./bin/start
Sent an event with posthog python >>> posthog.capture('test-id3', 'test-event', properties={ '$set': { 'userProperty': 16 } })
Verified that info was in Redis as expected:

$ redis-cli
>  keys person*
1) "person_props:1:5"
2) "person_create_at:1:5"
3) "person_id:1:test-id3"
127.0.0.1:6379> keys person*
1) "person_props:1:5"
2) "person_create_at:1:5"
3) "person_id:1:test-id3"
127.0.0.1:6379> get person_props:1:5
"\"{\\\"$initial_geoip_city_name\\\":\\\"Sydney\\\",\\\"$initial_geoip_country_name\\\":\\\"Australia\\\",\\\"$initial_geoip_country_code\\\":\\\"AU\\\",\\\"$initial_geoip_continent_name\\\":\\\"Oceania\\\",\\\"$initial_geoip_continent_code\\\":\\\"OC\\\",\\\"$initial_geoip_postal_code\\\":\\\"1148\\\",\\\"$initial_geoip_latitude\\\":-33.8715,\\\"$initial_geoip_longitude\\\":151.2006,\\\"$initial_geoip_time_zone\\\":\\\"Australia/Sydney\\\",\\\"$initial_geoip_subdivision_1_code\\\":\\\"NSW\\\",\\\"$initial_geoip_subdivision_1_name\\\":\\\"New South Wales\\\",\\\"userProperty\\\":16,\\\"$geoip_city_name\\\":\\\"Sydney\\\",\\\"$geoip_country_name\\\":\\\"Australia\\\",\\\"$geoip_country_code\\\":\\\"AU\\\",\\\"$geoip_continent_name\\\":\\\"Oceania\\\",\\\"$geoip_continent_code\\\":\\\"OC\\\",\\\"$geoip_postal_code\\\":\\\"1148\\\",\\\"$geoip_latitude\\\":-33.8715,\\\"$geoip_longitude\\\":151.2006,\\\"$geoip_time_zone\\\":\\\"Australia/Sydney\\\",\\\"$geoip_subdivision_1_code\\\":\\\"NSW\\\",\\\"$geoip_subdivision_1_name\\\":\\\"New South Wales\\\"}\""
127.0.0.1:6379> get person_create_at:1:5
"\"2022-04-12T13:24:00.825Z\""
127.0.0.1:6379> get person_id:1:test-id3
"5"

Changed the userProperty with py >>> posthog.capture('test-id3', 'test-event', properties={ '$set': { 'userProperty': 334 } })
In redis:

127.0.0.1:6379> get person_props:1:5
"\"{\\\"userProperty\\\":334,\\\"$geoip_latitude\\\":-33.8715,\\\"$geoip_city_name\\\":\\\"Sydney\\\",\\\"$geoip_longitude\\\":151.2006,\\\"$geoip_time_zone\\\":\\\"Australia/Sydney\\\",\\\"$geoip_postal_code\\\":\\\"1148\\\",\\\"$geoip_country_code\\\":\\\"AU\\\",\\\"$geoip_country_name\\\":\\\"Australia\\\",\\\"$geoip_continent_code\\\":\\\"OC\\\",\\\"$geoip_continent_name\\\":\\\"Oceania\\\",\\\"$initial_geoip_latitude\\\":-33.8715,\\\"$initial_geoip_city_name\\\":\\\"Sydney\\\",\\\"$initial_geoip_longitude\\\":151.2006,\\\"$initial_geoip_time_zone\\\":\\\"Australia/Sydney\\\",\\\"$geoip_subdivision_1_code\\\":\\\"NSW\\\",\\\"$geoip_subdivision_1_name\\\":\\\"New South Wales\\\",\\\"$initial_geoip_postal_code\\\":\\\"1148\\\",\\\"$initial_geoip_country_code\\\":\\\"AU\\\",\\\"$initial_geoip_country_name\\\":\\\"Australia\\\",\\\"$initial_geoip_continent_code\\\":\\\"OC\\\",\\\"$initial_geoip_continent_name\\\":\\\"Oceania\\\",\\\"$initial_geoip_subdivision_1_code\\\":\\\"NSW\\\",\\\"$initial_geoip_subdivision_1_name\\\":\\\"New South Wales\\\"}\""

Added second user and merged

>>> posthog.capture('test-id4', 'test-event', properties={ '$set': { 'userProperty': 600 } })
>>> posthog.alias('test-id3', 'test-id4')

redis. Note that we didn't expire the old props and created_at, but that personId isn't used anymore.

> keys person*
1) "person_props:1:5"
2) "person_create_at:1:5"
3) "person_id:1:test-id4"
4) "person_props:1:6"
5) "person_id:1:test-id3"
6) "person_create_at:1:6"
> get person_id:1:test-id4
"5"
127.0.0.1:6379> get person_id:1:test-id3
"5"
9> get person_props:1:5
"\"{\\\"userProperty\\\":334,\\\"$geoip_latitude\\\":-33.8715,\\\"$geoip_city_name\\\":\\\"Sydney\\\",\\\"$geoip_longitude\\\":151.2006,\\\"$geoip_time_zone\\\":\\\"Australia/Sydney\\\",\\\"$geoip_postal_code\\\":\\\"1148\\\",\\\"$geoip_country_code\\\":\\\"AU\\\",\\\"$geoip_country_name\\\":\\\"Australia\\\",\\\"$geoip_continent_code\\\":\\\"OC\\\",\\\"$geoip_continent_name\\\":\\\"Oceania\\\",\\\"$initial_geoip_latitude\\\":-33.8715,\\\"$initial_geoip_city_name\\\":\\\"Sydney\\\",\\\"$initial_geoip_longitude\\\":151.2006,\\\"$initial_geoip_time_zone\\\":\\\"Australia/Sydney\\\",\\\"$geoip_subdivision_1_code\\\":\\\"NSW\\\",\\\"$geoip_subdivision_1_name\\\":\\\"New South Wales\\\",\\\"$initial_geoip_postal_code\\\":\\\"1148\\\",\\\"$initial_geoip_country_code\\\":\\\"AU\\\",\\\"$initial_geoip_country_name\\\":\\\"Australia\\\",\\\"$initial_geoip_continent_code\\\":\\\"OC\\\",\\\"$initial_geoip_continent_name\\\":\\\"Oceania\\\",\\\"$initial_geoip_subdivision_1_code\\\":\\\"NSW\\\",\\\"$initial_geoip_subdivision_1_name\\\":\\\"New South Wales\\\"}\""

[yakkomajuri]

Thanks for submitting the WIP PR, appreciate the visibility into the work

[yakkomajuri]

Looking good. One comment that the buffer depends on this work, and it will use the person created_at field to determine what events should be sent there.

Could you (either in this PR or the next) add a cache for created_at (either as its own thing or bundled in the props by caching a larger subset of "person" instead of just props at the top level).

[yakkomajuri]

looks good - a few nits.

also, can we add a metric for each of the cache functions? redisSet already has metrics but I'd love to see a breakdown across each of the operations more clearly

[tiina303]

also, can we add a metric for each of the cache functions? redisSet already has metrics but I'd love to see a breakdown across each of the operations more clearly

They are all just calling redisSet directly are you predicting it to be slower for setting properties or what would you use it for? That said this could be good for the fetching part, which might end up going to DB and updating the cache. - we'd want to know the cache misses there etc.

[yakkomajuri]

would be best to have this to the same default as the config but sure

[tiina303]

set it like that on purpose to expire fast if not set.