Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: upgrade python to 3.11 🐍 #22932

Merged
merged 48 commits into from
Jun 21, 2024
Merged
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
595927e
3.11
aspicer Jun 12, 2024
41cf645
push
aspicer Jun 12, 2024
3b21fbc
3.11
aspicer Jun 12, 2024
74a8e30
update
aspicer Jun 12, 2024
2ae8c84
dockerfile changes
aspicer Jun 14, 2024
c645dd1
merge
aspicer Jun 14, 2024
a73d843
Merge remote-tracking branch 'origin/master' into aspicer/3.11
aspicer Jun 14, 2024
e49560e
remove uid
aspicer Jun 14, 2024
9526708
lxml upgrade
aspicer Jun 14, 2024
acfe178
Merge branch 'aspicer/3.11' of github.com:PostHog/posthog into aspice…
aspicer Jun 14, 2024
5b0a900
requirements.txt
aspicer Jun 14, 2024
b855075
yarl
aspicer Jun 14, 2024
70e8a9b
upgrades
aspicer Jun 14, 2024
7bab5b3
push
aspicer Jun 14, 2024
24c2606
requirements
aspicer Jun 14, 2024
c1d6851
upgrades
aspicer Jun 14, 2024
c012945
djangorestframework upgrade
aspicer Jun 14, 2024
b52345a
StrEnum
aspicer Jun 14, 2024
42c0b8d
Update query snapshots
github-actions[bot] Jun 14, 2024
9fd2eba
strenum
aspicer Jun 14, 2024
fd43a08
Merge remote-tracking branch 'origin/master' into aspicer/3.11
aspicer Jun 14, 2024
d59101a
Merge remote-tracking branch 'origin/aspicer/3.11' into aspicer/3.11
aspicer Jun 14, 2024
c310fb5
bump build version
aspicer Jun 14, 2024
5f1b5df
dockerfile
aspicer Jun 14, 2024
205fb53
remove migration check
aspicer Jun 14, 2024
958f45b
remove migration checking
aspicer Jun 14, 2024
5ae720a
serialier patch
aspicer Jun 14, 2024
6bc98d0
remove serializer in places it doesn't belong
aspicer Jun 14, 2024
dd4d7ed
Update query snapshots
github-actions[bot] Jun 14, 2024
6a28f70
Merge branch 'master' into aspicer/3.11
aspicer Jun 14, 2024
e9b4964
merge
aspicer Jun 18, 2024
3850eb4
merge
aspicer Jun 18, 2024
79424b4
error tracking order
aspicer Jun 18, 2024
e654576
hm
aspicer Jun 18, 2024
4786e20
see why it isn't starting
aspicer Jun 18, 2024
fb938e5
hm
aspicer Jun 18, 2024
5fe14ea
upgdate unit json tpl
aspicer Jun 18, 2024
b814c51
put back
aspicer Jun 18, 2024
fe3a2d5
update requirements
aspicer Jun 18, 2024
a0cc810
fix requirements
aspicer Jun 18, 2024
caafbd7
Merge branch 'master' into aspicer/3.11
aspicer Jun 18, 2024
35ab02f
Merge remote-tracking branch 'origin/master' into aspicer/3.11
aspicer Jun 18, 2024
d674f63
module sdk
aspicer Jun 18, 2024
6c6b5db
true
aspicer Jun 18, 2024
0649f1b
Merge remote-tracking branch 'origin/master' into aspicer/3.11
aspicer Jun 19, 2024
e3ba4a9
Merge remote-tracking branch 'origin/master' into aspicer/3.11
aspicer Jun 21, 2024
7187455
Merge remote-tracking branch 'origin/master' into aspicer/3.11
aspicer Jun 21, 2024
aaad5e7
Merge branch 'master' into aspicer/3.11
aspicer Jun 21, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .devcontainer/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
# experience as rich as possible. Perhaps later down the line it might be worth
# rolling our own
#
FROM mcr.microsoft.com/vscode/devcontainers/python:3.10-bullseye
FROM mcr.microsoft.com/vscode/devcontainers/python:3.11-bullseye

# Make sure all exit codes on pipes cause failures
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
Expand All @@ -32,7 +32,7 @@
netcat brotli curl \
&& rm -rf /var/lib/apt/lists/*

RUN curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg | sudo dd of=/usr/share/keyrings/githubcli-archive-keyring.gpg \

Check failure on line 35 in .devcontainer/Dockerfile

View workflow job for this annotation

GitHub Actions / Lint changed Dockerfiles

Do not use sudo as it leads to unpredictable behavior. Use a tool like gosu to enforce root
&& sudo chmod go+r /usr/share/keyrings/githubcli-archive-keyring.gpg \
&& echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" | sudo tee /etc/apt/sources.list.d/github-cli.list > /dev/null \
&& sudo apt update \
Expand Down
2 changes: 1 addition & 1 deletion .github/actions/run-backend-tests/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ name: Run Django tests
inputs:
python-version:
required: true
description: Python version, e.g. 3.10.10
description: Python version, e.g. 3.11.9
clickhouse-server-image:
required: true
description: ClickHouse server image tag, e.g. clickhouse/clickhouse-server:latest
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/benchmark.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: 3.10.10
python-version: 3.11.9
cache: 'pip'
cache-dependency-path: '**/requirements*.txt'
token: ${{ secrets.POSTHOG_BOT_GITHUB_TOKEN }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci-backend-update-test-timing.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ jobs:
concurrency: 1
group: 1
token: ${{ secrets.POSTHOG_BOT_GITHUB_TOKEN }}
python-version: '3.10.10'
python-version: '3.11.9'
clickhouse-server-image: 'clickhouse/clickhouse-server:23.12.5.81-alpine'
segment: 'FOSS'
person-on-events: false
Expand Down
10 changes: 5 additions & 5 deletions .github/workflows/ci-backend.yml
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: 3.10.10
python-version: 3.11.9
cache: 'pip'
cache-dependency-path: '**/requirements*.txt'
token: ${{ secrets.POSTHOG_BOT_GITHUB_TOKEN }}
Expand Down Expand Up @@ -163,7 +163,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: 3.10.10
python-version: 3.11.9
cache: 'pip'
cache-dependency-path: '**/requirements*.txt'
token: ${{ secrets.POSTHOG_BOT_GITHUB_TOKEN }}
Expand Down Expand Up @@ -231,7 +231,7 @@ jobs:
strategy:
fail-fast: false
matrix:
python-version: ['3.10.10']
python-version: ['3.11.9']
clickhouse-server-image: ['clickhouse/clickhouse-server:23.12.5.81-alpine']
segment: ['Core']
person-on-events: [false, true]
Expand All @@ -242,7 +242,7 @@ jobs:
- segment: 'Temporal'
person-on-events: false
clickhouse-server-image: 'clickhouse/clickhouse-server:23.12.5.81-alpine'
python-version: '3.10.10'
python-version: '3.11.9'
concurrency: 1
group: 1

Expand Down Expand Up @@ -330,7 +330,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: 3.10.10
python-version: 3.11.9
cache: 'pip'
cache-dependency-path: '**/requirements*.txt'
token: ${{ secrets.POSTHOG_BOT_GITHUB_TOKEN }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci-hog.yml
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ jobs:
if: needs.changes.outputs.hog == 'true'
uses: actions/setup-python@v5
with:
python-version: 3.10.10
python-version: 3.11.9
cache: 'pip'
cache-dependency-path: '**/requirements*.txt'
token: ${{ secrets.POSTHOG_BOT_GITHUB_TOKEN }}
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/ci-plugin-server.yml
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ jobs:
if: needs.changes.outputs.plugin-server == 'true'
uses: actions/setup-python@v5
with:
python-version: 3.10.10
python-version: 3.11.9
cache: 'pip'
cache-dependency-path: '**/requirements*.txt'
token: ${{ secrets.POSTHOG_BOT_GITHUB_TOKEN }}
Expand Down Expand Up @@ -207,7 +207,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: 3.10.10
python-version: 3.11.9
cache: 'pip'
cache-dependency-path: '**/requirements*.txt'
token: ${{ secrets.POSTHOG_BOT_GITHUB_TOKEN }}
Expand Down
3 changes: 2 additions & 1 deletion .run/Celery Beat.run.xml
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,9 @@
<env name="REPLAY_EMBEDDINGS_ALLOWED_TEAM" value="1,2,3" />
</envs>
<option name="SDK_HOME" value="" />
<option name="SDK_NAME" value="Python 3.11 (posthog2)" />
<option name="WORKING_DIRECTORY" value="$PROJECT_DIR$/env/bin" />
<option name="IS_MODULE_SDK" value="true" />
<option name="IS_MODULE_SDK" value="false" />
<option name="ADD_CONTENT_ROOTS" value="true" />
<option name="ADD_SOURCE_ROOTS" value="true" />
<EXTENSION ID="PythonCoverageRunConfigurationExtension" runner="coverage.py" />
Expand Down
3 changes: 2 additions & 1 deletion .run/Celery Threads.run.xml
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,9 @@
<env name="REPLAY_EMBEDDINGS_ALLOWED_TEAM" value="1,2,3" />
</envs>
<option name="SDK_HOME" value="$PROJECT_DIR$/env/bin/python" />
<option name="SDK_NAME" value="Python 3.11 (posthog2)" />
<option name="WORKING_DIRECTORY" value="$PROJECT_DIR$/env/bin" />
<option name="IS_MODULE_SDK" value="true" />
<option name="IS_MODULE_SDK" value="false" />
<option name="ADD_CONTENT_ROOTS" value="true" />
<option name="ADD_SOURCE_ROOTS" value="true" />
<EXTENSION ID="PythonCoverageRunConfigurationExtension" runner="coverage.py" />
Expand Down
8 changes: 7 additions & 1 deletion bin/build-schema-python.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ set -e

# Generate schema.py from schema.json
datamodel-codegen \
--class-name='SchemaRoot' --collapse-root-models --target-python-version 3.10 --disable-timestamp \
--class-name='SchemaRoot' --collapse-root-models --target-python-version 3.11 --disable-timestamp \
--use-one-literal-as-default --use-default --use-default-kwarg --use-subclass-enum \
--input frontend/src/queries/schema.json --input-file-type jsonschema \
--output posthog/schema.py --output-model-type pydantic_v2.BaseModel \
Expand All @@ -29,3 +29,9 @@ if [[ "$OSTYPE" == "darwin"* ]]; then
else
sed -i -e 's/Optional\[PropertyOperator\] = \("[A-Za-z_]*"\)/Optional[PropertyOperator] = PropertyOperator(\1)/g' posthog/schema.py
fi

# Replace class Foo(str, Enum) with class Foo(StrEnum) for proper handling in format strings in python 3.11
# Remove this when https://github.com/koxudaxi/datamodel-code-generator/issues/1313 is resolved

sed -i -e 's/str, Enum/StrEnum/g' posthog/schema.py
sed -i 's/from enum import Enum/from enum import Enum, StrEnum/g' posthog/schema.py
2 changes: 1 addition & 1 deletion ee/api/test/__snapshots__/test_time_to_see_data.ambr
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
"first_name": "",
"last_name": "",
"email": "",
"is_email_verified": false
"is_email_verified": null
}
},
"children": [
Expand Down
2 changes: 1 addition & 1 deletion mypy.ini
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[mypy]
python_version = 3.10
python_version = 3.11
plugins =
mypy_django_plugin.main,
mypy_drf_plugin.main,
Expand Down
26 changes: 26 additions & 0 deletions posthog/api/routing.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,32 @@
class DefaultRouterPlusPlus(ExtendedDefaultRouter):
"""DefaultRouter with optional trailing slash and drf-extensions nesting."""

# This is an override because of changes in djangorestframework 3.15, which is required for python 3.11
# changes taken from and explained here: https://github.com/nautobot/nautobot/pull/5546/files#diff-81850a2ccad5814aab4f477d447f85cc0a82e9c10fd88fd72327cda51a750471R30
def _register(self, prefix, viewset, basename=None):
"""
Override DRF's BaseRouter.register() to bypass an unnecessary restriction added in version 3.15.0.
(Reference: https://github.com/encode/django-rest-framework/pull/8438)
"""
if basename is None:
basename = self.get_default_basename(viewset)

# DRF:
# if self.is_already_registered(basename):
# msg = (f'Router with basename "{basename}" is already registered. '
# f'Please provide a unique basename for viewset "{viewset}"')
# raise ImproperlyConfigured(msg)
#
# We bypass this because we have at least one use case (/api/extras/jobs/) where we are *intentionally*
# registering two viewsets with the same basename, but have carefully defined them so as not to conflict.

# resuming standard DRF code...
self.registry.append((prefix, viewset, basename))

# invalidate the urls cache
if hasattr(self, "_urls"):
del self._urls

def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.trailing_slash = r"/?"
Expand Down
2 changes: 1 addition & 1 deletion posthog/batch_exports/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -199,7 +199,7 @@ def interval_time_delta(self) -> timedelta:
raise ValueError(f"Invalid interval: '{self.interval}'")


class BatchExportLogEntryLevel(str, enum.Enum):
class BatchExportLogEntryLevel(enum.StrEnum):
"""Enumeration of batch export log levels."""

DEBUG = "DEBUG"
Expand Down
4 changes: 2 additions & 2 deletions posthog/clickhouse/table_engines.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
import uuid
from enum import Enum
from enum import StrEnum
from typing import Optional

from django.conf import settings


class ReplicationScheme(str, Enum):
class ReplicationScheme(StrEnum):
NOT_SHARDED = "NOT_SHARDED"
SHARDED = "SHARDED"
REPLICATED = "REPLICATED"
Expand Down
26 changes: 13 additions & 13 deletions posthog/constants.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from enum import Enum
from enum import StrEnum
from typing import Literal

from semantic_version import Version
Expand All @@ -9,7 +9,7 @@

# N.B. Keep this in sync with frontend enum (types.ts)
# AND ensure it is added to the Billing Service
class AvailableFeature(str, Enum):
class AvailableFeature(StrEnum):
ZAPIER = "zapier"
ORGANIZATIONS_PROJECTS = "organizations_projects"
PROJECT_BASED_PERMISSIONING = "project_based_permissioning"
Expand Down Expand Up @@ -215,19 +215,19 @@ class AvailableFeature(str, Enum):
BREAKDOWN_TYPES = Literal["event", "person", "cohort", "group", "session", "hogql"]


class FunnelOrderType(str, Enum):
class FunnelOrderType(StrEnum):
STRICT = "strict"
UNORDERED = "unordered"
ORDERED = "ordered"


class FunnelVizType(str, Enum):
class FunnelVizType(StrEnum):
TRENDS = "trends"
TIME_TO_CONVERT = "time_to_convert"
STEPS = "steps"


class FunnelCorrelationType(str, Enum):
class FunnelCorrelationType(StrEnum):
EVENTS = "events"
PROPERTIES = "properties"
EVENT_WITH_PROPERTIES = "event_with_properties"
Expand All @@ -240,7 +240,7 @@ class FunnelCorrelationType(str, Enum):
PERSON_UUID_FILTER = "person_uuid"


class AnalyticsDBMS(str, Enum):
class AnalyticsDBMS(StrEnum):
POSTGRES = "postgres"
CLICKHOUSE = "clickhouse"

Expand All @@ -251,34 +251,34 @@ class AnalyticsDBMS(str, Enum):
MONTHLY_ACTIVE = "monthly_active"


class RetentionQueryType(str, Enum):
class RetentionQueryType(StrEnum):
RETURNING = "returning"
TARGET = "target"
TARGET_FIRST_TIME = "target_first_time"


class ExperimentSignificanceCode(str, Enum):
class ExperimentSignificanceCode(StrEnum):
SIGNIFICANT = "significant"
NOT_ENOUGH_EXPOSURE = "not_enough_exposure"
LOW_WIN_PROBABILITY = "low_win_probability"
HIGH_LOSS = "high_loss"
HIGH_P_VALUE = "high_p_value"


class ExperimentNoResultsErrorKeys(str, Enum):
class ExperimentNoResultsErrorKeys(StrEnum):
NO_EVENTS = "no-events"
NO_FLAG_INFO = "no-flag-info"
NO_CONTROL_VARIANT = "no-control-variant"
NO_TEST_VARIANT = "no-test-variant"
NO_RESULTS = "no-results"


class PropertyOperatorType(str, Enum):
class PropertyOperatorType(StrEnum):
AND = "AND"
OR = "OR"


class BreakdownAttributionType(str, Enum):
class BreakdownAttributionType(StrEnum):
FIRST_TOUCH = "first_touch"
# FIRST_TOUCH attribution means the breakdown value is the first property value found within all funnel steps
LAST_TOUCH = "last_touch"
Expand All @@ -294,7 +294,7 @@ class BreakdownAttributionType(str, Enum):
GROUP_TYPES_LIMIT = 5


class EventDefinitionType(str, Enum):
class EventDefinitionType(StrEnum):
# Mimics EventDefinitionType in frontend/src/types.ts
ALL = "all"
ACTION_EVENT = "action_event"
Expand All @@ -303,7 +303,7 @@ class EventDefinitionType(str, Enum):
EVENT_CUSTOM = "event_custom"


class FlagRequestType(str, Enum):
class FlagRequestType(StrEnum):
DECIDE = "decide"
LOCAL_EVALUATION = "local-evaluation"

Expand Down
4 changes: 2 additions & 2 deletions posthog/decorators.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from enum import Enum
from enum import StrEnum
from functools import wraps
from typing import Any, TypeVar, Union, cast
from collections.abc import Callable
Expand All @@ -17,7 +17,7 @@
from .utils import generate_cache_key, get_safe_cache


class CacheType(str, Enum):
class CacheType(StrEnum):
TRENDS = "Trends"
FUNNEL = "Funnel"
RETENTION = "Retention"
Expand Down
5 changes: 2 additions & 3 deletions posthog/demo/matrix/randomization.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,11 @@
from enum import Enum
from enum import StrEnum

import mimesis
import mimesis.random

WeightedPool = tuple[list[str], list[int]]


class Industry(str, Enum):
class Industry(StrEnum):
TECHNOLOGY = "technology"
FINANCE = "finance"
MEDIA = "media"
Expand Down
4 changes: 2 additions & 2 deletions posthog/demo/products/hedgebox/models.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import datetime as dt
import math
from dataclasses import dataclass, field
from enum import Enum, auto
from enum import auto, StrEnum
from typing import (
TYPE_CHECKING,
Any,
Expand Down Expand Up @@ -66,7 +66,7 @@ class HedgeboxSessionIntent(SimSessionIntent):
DOWNGRADE_PLAN = auto()


class HedgeboxPlan(str, Enum):
class HedgeboxPlan(StrEnum):
PERSONAL_FREE = "personal/free"
PERSONAL_PRO = "personal/pro"
BUSINESS_STANDARD = "business/standard"
Expand Down
Loading
Loading