Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Invalid PostgreSQL generated on query with distinct, annotation and order_by #1878

Open
markmiscavage opened this issue Feb 6, 2025 · 0 comments · May be fixed by #1886
Open

Invalid PostgreSQL generated on query with distinct, annotation and order_by #1878

markmiscavage opened this issue Feb 6, 2025 · 0 comments · May be fixed by #1886
Labels
bug Something isn't working

Comments

@markmiscavage
Copy link

markmiscavage commented Feb 6, 2025

Describe the bug
When using asyncpg if I generate a query that has a distinct clause, an annotation with case expression, and order by the annotation it generates the following error tortoise.exceptions.OperationalError: for SELECT DISTINCT, ORDER BY expressions must appear in select list.

To Reproduce

"""
This example demonstrates most basic operations with single model
"""

from enum import Enum

from tortoise import Tortoise, fields, run_async
from tortoise.contrib.test import init_memory_sqlite
from tortoise.expressions import Case, Q, When
from tortoise.models import Model


class Grade(str, Enum):
    APLUS = "A+"
    A = "A"
    BPLUS = "B+"
    B = "B"
    C = "C"

    @property
    def sortable_str(self):
        """Map letter grades so they sort correctly.

        (A+, A, B+) become (A_0, A_1, B_0).
        """
        letter = self[0]

        if len(self) == 1:
            suffix = "1"
        elif len(self) == 2 and self[1] == "+":
            suffix = "0"
        else:
            raise NotImplementedError

        return f"{letter}_{suffix}"


class Event(Model):
    id = fields.IntField(primary_key=True)
    name = fields.TextField()
    datetime = fields.DatetimeField(null=True)
    grade = fields.CharEnumField(enum_type=Grade)

    class Meta:
        table = "event"

    def __str__(self):
        return self.name


async def run():
    await Tortoise.init(
        {
            "connections": {
                "default": {
                    "engine": "tortoise.backends.asyncpg",
                    "credentials": {
                        "host": "localhost",
                        "port": "5432",
                        "user": "tortoise",
                         "password": "qwerty123",
                        "database": "test",
                    },
                }
            },
            "apps": {"models": {"models": ["__main__"], "default_connection": "default"}},
        },
        _create_db=True,
    )
    await Tortoise.generate_schemas()

    event = await Event.create(name="Test", grade=Grade.A)

    events = Event.all().distinct()
    whens = [When(Q(grade=x.value), then=x.sortable_str) for x in Grade]
    whens.append(When(Q(grade__isnull=True), then="Z"))
    events = events.annotate(grade_orderable=Case(*whens))
    events = events.order_by('grade_orderable', '-datetime')
    print(events.sql(params_inline=True))
    print(await events)
    await Tortoise._drop_databases()


if __name__ == "__main__":
    run_async(run())`

Expected behavior
If I replace the case statement in the ORDER BY with the named case statement in the SELECT, Postgres seems happy

  SELECT DISTINCT
  "datetime",
  "grade",
  "name",
  "id",
  CASE
    WHEN "grade" = 'A+' THEN 'A_0'
    WHEN "grade" = 'A' THEN 'A_1'
    WHEN "grade" = 'B+' THEN 'B_0'
    WHEN "grade" = 'B' THEN 'B_1'
    WHEN "grade" = 'C' THEN 'C_1'
    WHEN "grade" IS NULL THEN 'Z'
    ELSE NULL
  END "grade_orderable"
FROM
  "event"
ORDER BY
  "grade_orderable" ASC,
  "datetime" DESC

Additional context
When creating the minimal example for this, I discovered in-memory sqllite works correctly with the SQL generated leading me to believe this is solely a PostgreSQL problem. I'm not sure if this ultimately is a PyPika problem but I figured I would post here first since I'm able to generate the simple example posted above and I'm not sure how to generate a good PyPika example to post there.

@henadzit henadzit added the bug Something isn't working label Feb 17, 2025
@henadzit henadzit linked a pull request Feb 17, 2025 that will close this issue
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants