You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When using the to_sql function in PyAthena with if_exists="replace", SQL statements like DROP TABLE fail if the table name or schema contains special characters (e.g., hyphens -). This happens because Athena requires such identifiers to be enclosed in backticks (`identifier`) to avoid syntax errors.
Steps to Reproduce
Create a schema or table name containing special characters, such as my-schema.my-table.
Use the to_sql function with the if_exists="replace" parameter.
Observe that the generated DROP TABLE or DDL query fails with a syntax error.
Suggested Solution
Modify the to_sql implementation to properly escape identifiers wherever they appear in SQL statements. Specifically:
DROP TABLE Statements:
Update to:
DROPTABLE`{schema}`.`{name}`
CREATE TABLE (DDL) Statements:
Ensure that schema and table names are enclosed in backticks in the generated DDL.
Information Schema Queries:
Escape schema and table names:
Update the Codebase:
Apply the escape_identifier function wherever schema or table names are referenced in SQL.
Why This Matters
Properly escaping identifiers ensures compatibility with Athena's SQL syntax rules, avoids syntax errors, and makes PyAthena more robust when dealing with schemas or tables containing special characters. This change would also make the library easier to use without requiring additional handling from users.
The text was updated successfully, but these errors were encountered:
Issue Summary
When using the
to_sql
function in PyAthena withif_exists="replace"
, SQL statements likeDROP TABLE
fail if the table name or schema contains special characters (e.g., hyphens-
). This happens because Athena requires such identifiers to be enclosed in backticks (`identifier
`) to avoid syntax errors.Steps to Reproduce
my-schema.my-table
.to_sql
function with theif_exists="replace"
parameter.DROP TABLE
or DDL query fails with a syntax error.Suggested Solution
Modify the
to_sql
implementation to properly escape identifiers wherever they appear in SQL statements. Specifically:DROP TABLE Statements:
Update to:
CREATE TABLE (DDL) Statements:
Ensure that schema and table names are enclosed in backticks in the generated DDL.
Information Schema Queries:
Escape schema and table names:
Add a Utility for Escaping:
Create a helper function for escaping identifiers:
Update the Codebase:
Apply the
escape_identifier
function wherever schema or table names are referenced in SQL.Why This Matters
Properly escaping identifiers ensures compatibility with Athena's SQL syntax rules, avoids syntax errors, and makes PyAthena more robust when dealing with schemas or tables containing special characters. This change would also make the library easier to use without requiring additional handling from users.
The text was updated successfully, but these errors were encountered: