Token Limit Exceeded: Please try again with a shorted query upgrading from 23.11.2

[JohnCFerguson]

Self-Hosted Version

23.11.2

CPU Architecture

x86_64

Docker Version

27.2.0, build 3ab4256

Docker Compose Version

2.20.3

Machine Specification

• My system meets the minimum system requirements of Sentry

Steps to Reproduce

1. Checkout latest release of Sentry as per the releases and upgrading docs
2. sudo ./install.sh

Expected Result

Sentry upgrades and is able to start

Actual Result

`▶ Ensuring proper PostgreSQL version ...

▶ Setting up / migrating database ...
Container sentry-self-hosted-memcached-1 Creating
Container sentry-self-hosted-symbolicator-1 Creating
Container sentry-self-hosted-postgres-1 Creating
Container sentry-self-hosted-smtp-1 Creating
Container sentry-self-hosted-redis-1 Running
Container sentry-self-hosted-clickhouse-1 Running
Container sentry-self-hosted-zookeeper-1 Running
Container sentry-self-hosted-kafka-1 Running
Container sentry-self-hosted-snuba-api-1 Creating
Container sentry-self-hosted-snuba-subscription-consumer-transactions-1 Creating
Container sentry-self-hosted-snuba-transactions-consumer-1 Creating
Container sentry-self-hosted-snuba-outcomes-consumer-1 Creating
Container sentry-self-hosted-snuba-consumer-1 Creating
Container sentry-self-hosted-vroom-1 Creating
Container sentry-self-hosted-snuba-replacer-1 Creating
Container sentry-self-hosted-snuba-subscription-consumer-events-1 Creating
Container sentry-self-hosted-snuba-api-1 Created
Container sentry-self-hosted-snuba-outcomes-consumer-1 Created
Container sentry-self-hosted-smtp-1 Created
Container sentry-self-hosted-memcached-1 Created
Container sentry-self-hosted-snuba-replacer-1 Created
Container sentry-self-hosted-vroom-1 Created
Container sentry-self-hosted-snuba-consumer-1 Created
Container sentry-self-hosted-snuba-transactions-consumer-1 Created
Container sentry-self-hosted-snuba-subscription-consumer-events-1 Created
Container sentry-self-hosted-postgres-1 Created
Container sentry-self-hosted-snuba-subscription-consumer-transactions-1 Created
Container sentry-self-hosted-symbolicator-1 Created
Container sentry-self-hosted-smtp-1 Starting
Container sentry-self-hosted-memcached-1 Starting
Container sentry-self-hosted-symbolicator-1 Starting
Container sentry-self-hosted-zookeeper-1 Waiting
Container sentry-self-hosted-postgres-1 Starting
Container sentry-self-hosted-zookeeper-1 Healthy
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-redis-1 Waiting
Container sentry-self-hosted-clickhouse-1 Waiting
Container sentry-self-hosted-redis-1 Waiting
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-redis-1 Waiting
Container sentry-self-hosted-clickhouse-1 Waiting
Container sentry-self-hosted-clickhouse-1 Waiting
Container sentry-self-hosted-clickhouse-1 Waiting
Container sentry-self-hosted-clickhouse-1 Waiting
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-redis-1 Waiting
Container sentry-self-hosted-clickhouse-1 Waiting
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-redis-1 Waiting
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-redis-1 Waiting
Container sentry-self-hosted-kafka-1 Waiting
Container sentry-self-hosted-clickhouse-1 Waiting
Container sentry-self-hosted-redis-1 Waiting
Container sentry-self-hosted-clickhouse-1 Healthy
Container sentry-self-hosted-redis-1 Healthy
Container sentry-self-hosted-redis-1 Healthy
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-redis-1 Healthy
Container sentry-self-hosted-redis-1 Healthy
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-redis-1 Healthy
Container sentry-self-hosted-redis-1 Healthy
Container sentry-self-hosted-clickhouse-1 Healthy
Container sentry-self-hosted-snuba-consumer-1 Starting
Container sentry-self-hosted-clickhouse-1 Healthy
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-snuba-transactions-consumer-1 Starting
Container sentry-self-hosted-snuba-subscription-consumer-events-1 Starting
Container sentry-self-hosted-clickhouse-1 Healthy
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-snuba-subscription-consumer-transactions-1 Starting
Container sentry-self-hosted-clickhouse-1 Healthy
Container sentry-self-hosted-snuba-replacer-1 Starting
Container sentry-self-hosted-redis-1 Healthy
Container sentry-self-hosted-kafka-1 Healthy
Container sentry-self-hosted-vroom-1 Starting
Container sentry-self-hosted-clickhouse-1 Healthy
Container sentry-self-hosted-snuba-api-1 Starting
Container sentry-self-hosted-clickhouse-1 Healthy
Container sentry-self-hosted-snuba-outcomes-consumer-1 Starting
Container sentry-self-hosted-memcached-1 Started
Container sentry-self-hosted-postgres-1 Started
Container sentry-self-hosted-smtp-1 Started
Container sentry-self-hosted-symbolicator-1 Started
Container sentry-self-hosted-snuba-subscription-consumer-transactions-1 Started
Container sentry-self-hosted-snuba-consumer-1 Started
Container sentry-self-hosted-snuba-subscription-consumer-events-1 Started
Container sentry-self-hosted-snuba-replacer-1 Started
Container sentry-self-hosted-snuba-outcomes-consumer-1 Started
Container sentry-self-hosted-snuba-api-1 Started
Container sentry-self-hosted-vroom-1 Started
Container sentry-self-hosted-snuba-transactions-consumer-1 Started
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
Running migrations for default
Operations to perform:
Apply all migrations: auth, contenttypes, feedback, hybridcloud, nodestore, replays, sentry, sessions, sites, social_auth
Running migrations:
No migrations to apply.
Creating missing DSNs
Correcting Group.num_comments counter
14:01:18 [INFO] sentry.outboxes: Executing outbox replication backfill
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/sentry/db/postgres/decorators.py", line 91, in inner
return func(self, sql, *args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/sentry/db/postgres/base.py", line 86, in execute
return self.cursor.execute(sql, clean_bad_params(params))
psycopg2.errors.UndefinedColumn: column sentry_team.actor_id does not exist
LINE 1: ...g", "sentry_team"."name", "sentry_team"."status", "sentry_te...
^

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python3.9/site-packages/sentry/db/postgres/decorators.py", line 77, in inner
raise_the_exception(self.db, e)
File "/usr/local/lib/python3.9/site-packages/sentry/db/postgres/decorators.py", line 75, in inner
return func(self, *args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/sentry/db/postgres/decorators.py", line 18, in inner
return func(self, *args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/sentry/db/postgres/decorators.py", line 95, in inner
raise exc_info0.with_traceback(exc_info[2])
File "/usr/local/lib/python3.9/site-packages/sentry/db/postgres/decorators.py", line 91, in inner
return func(self, sql, *args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/sentry/db/postgres/base.py", line 86, in execute
return self.cursor.execute(sql, clean_bad_params(params))
psycopg2.errors.UndefinedColumn: UndefinedColumn('column sentry_team.actor_id does not exist\nLINE 1: ...g", "sentry_team"."name", "sentry_team"."status", "sentry_te...\n ^\n')
SQL: SELECT "sentry_team"."id", "sentry_team"."organization_id", "sentry_team"."slug", "sentry_team"."name", "sentry_team"."status", "sentry_team"."actor_id", "sentry_team"."idp_provisioned", "sentry_team"."date_added", "sentry_team"."org_role" FROM "sentry_team" WHERE ("sentry_team"."id" >= %s AND "sentry_team"."id" <= %s)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/bin/sentry", line 8, in
sys.exit(main())
File "/usr/local/lib/python3.9/site-packages/sentry/runner/init.py", line 195, in main
func(**kwargs)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/sentry/runner/decorators.py", line 29, in inner
return ctx.invoke(f, *args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/sentry/runner/commands/upgrade.py", line 117, in upgrade
_upgrade(
File "/usr/local/lib/python3.9/site-packages/sentry/runner/commands/upgrade.py", line 71, in _upgrade
post_upgrade.send(sender=SiloMode.get_current_mode(), interactive=interactive)
File "/usr/local/lib/python3.9/site-packages/django/dispatch/dispatcher.py", line 180, in send
return [
File "/usr/local/lib/python3.9/site-packages/django/dispatch/dispatcher.py", line 181, in
(receiver, receiver(signal=self, sender=sender, **named))
File "/usr/local/lib/python3.9/site-packages/sentry/db/models/outboxes.py", line 458, in run_outbox_replications_for_self_hosted
while backfill_outboxes_for(
File "/usr/local/lib/python3.9/site-packages/sentry/tasks/backfill_outboxes.py", line 177, in backfill_outboxes_for
batch = process_outbox_backfill_batch(
File "/usr/local/lib/python3.9/site-packages/sentry/tasks/backfill_outboxes.py", line 131, in process_outbox_backfill_batch
for inst in model.objects.filter(id__gte=processing_state.low, id__lte=processing_state.up):
File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 280, in iter
self._fetch_all()
File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 1324, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 51, in iter
results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
File "/usr/local/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 1175, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python3.9/site-packages/sentry_sdk/integrations/django/init.py", line 641, in execute
result = real_execute(self, sql, params)
File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 66, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python3.9/site-packages/django/db/utils.py", line 90, in exit
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python3.9/site-packages/sentry/db/postgres/decorators.py", line 77, in inner
raise_the_exception(self.db, e)
File "/usr/local/lib/python3.9/site-packages/sentry/db/postgres/decorators.py", line 75, in inner
return func(self, *args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/sentry/db/postgres/decorators.py", line 18, in inner
return func(self, *args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/sentry/db/postgres/decorators.py", line 95, in inner
raise exc_info0.with_traceback(exc_info[2])
File "/usr/local/lib/python3.9/site-packages/sentry/db/postgres/decorators.py", line 91, in inner
return func(self, sql, *args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/sentry/db/postgres/base.py", line 86, in execute
return self.cursor.execute(sql, clean_bad_params(params))
django.db.utils.ProgrammingError: UndefinedColumn('column sentry_team.actor_id does not exist\nLINE 1: ...g", "sentry_team"."name", "sentry_team"."status", "sentry_te...\n ^\n')
SQL: SELECT "sentry_team"."id", "sentry_team"."organization_id", "sentry_team"."slug", "sentry_team"."name", "sentry_team"."status", "sentry_team"."actor_id", "sentry_team"."idp_provisioned", "sentry_team"."date_added", "sentry_team"."org_role" FROM "sentry_team" WHERE ("sentry_team"."id" >= %s AND "sentry_team"."id" <= %s)
Error in install/set-up-and-migrate-database.sh:12.
'$dcr web upgrade' exited with status 1
-> ./install.sh:main:34
--> install/set-up-and-migrate-database.sh:source:12

Looks like you've already sent this error to us, we're on it :)`

Event ID

No response