Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added ADR document describing why the notion of dialects was introduced in the common sql provider #45456

Merged
merged 27 commits into from
Jan 24, 2025

Conversation

dabla
Copy link
Contributor

@dabla dabla commented Jan 7, 2025

Added ADR document describing why the notion of dialects was introduced in the common sql provider.


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

…ects was introduced in the common sql provider
@dabla
Copy link
Contributor Author

dabla commented Jan 7, 2025

@potiuk @jscheffl As promised a document describing the dialects in common sql provider. If any remarks or improvements are necessary please let me know.

@eladkal
Copy link
Contributor

eladkal commented Jan 7, 2025

NIce! I think we are also missing user facing docs about what dialects are and how to use them (ADR is for Airflow developers to document our design choices but it's not really a user facing doc)

@dabla
Copy link
Contributor Author

dabla commented Jan 7, 2025

NIce! I think we are also missing user facing docs about what dialects are and how to use them (ADR is for Airflow developers to document our design choices but it's not really a user facing doc)

Where should I put those user facing docs? Is that under docs/apache-airflow-providers-common-sql? Should I create a new dialects.rst document?

@eladkal
Copy link
Contributor

eladkal commented Jan 7, 2025

Where should I put those user facing docs? Is that under docs/apache-airflow-providers-common-sql? Should I create a new dialects.rst document?

Yes. You dont have to create new .rst
Depends on the scope of what you write. If its 1-2 pargraphs it might fit in the current rst we have.
However you may want to mention in the providers we have dedicated dialect how to use it and link to the doc in common.sql so there may be need for doc changea in several providers

@eladkal
Copy link
Contributor

eladkal commented Jan 10, 2025

@dabla do you intend to add the docs in this PR?

@dabla
Copy link
Contributor Author

dabla commented Jan 10, 2025

@dabla do you intend to add the docs in this PR?

I just committed a new dialects.rst and adapted the index.rst this morning, what do you think about it?

Copy link
Contributor

@jscheffl jscheffl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, "late to the party" now digging through my backlog... wanted to review earlier.

I really like this and the description.

For the docs build problem I see two options: (1) [not sure if this really works] outside the TOC create a small RST just with the heading and pointing to the other doctree - or (2) if you want to have exactly the same doc in each provider then use a symlink in GIT such that the RST is maintained in one place and each doctree has the same. (As long as we not move the providers the symlink is working in Sphinx)

Can you add some example the the dialects.rst or is this already contained in the implementation PR?

@dabla
Copy link
Contributor Author

dabla commented Jan 15, 2025

Sorry, "late to the party" now digging through my backlog... wanted to review earlier.

I really like this and the description.

For the docs build problem I see two options: (1) [not sure if this really works] outside the TOC create a small RST just with the heading and pointing to the other doctree - or (2) if you want to have exactly the same doc in each provider then use a symlink in GIT such that the RST is maintained in one place and each doctree has the same. (As long as we not move the providers the symlink is working in Sphinx)

Can you add some example the the dialects.rst or is this already contained in the implementation PR?

Will look into it. I also opened another PR related to this which fixes an issue with reserved words and special characters using the dialects.

@dabla
Copy link
Contributor Author

dabla commented Jan 15, 2025

Sorry, "late to the party" now digging through my backlog... wanted to review earlier.

I really like this and the description.

For the docs build problem I see two options: (1) [not sure if this really works] outside the TOC create a small RST just with the heading and pointing to the other doctree - or (2) if you want to have exactly the same doc in each provider then use a symlink in GIT such that the RST is maintained in one place and each doctree has the same. (As long as we not move the providers the symlink is working in Sphinx)

Can you add some example the the dialects.rst or is this already contained in the implementation PR?

Not that found of symlinks, dunno how git will handle those. Aren't http references an option? I saw it's also used within rst.

For example (won't exist atm ofc):

https://airflow.apache.org/docs/apache-airflow-providers-common-sql/stable/dialects.html

@dabla dabla requested a review from jscheffl January 15, 2025 19:14
@dabla dabla requested review from eladkal and potiuk January 20, 2025 09:33
@dabla
Copy link
Contributor Author

dabla commented Jan 24, 2025

@eladkal I've duplicated the dialects document in each provider, we can sort it out later as you mentioned

@eladkal eladkal merged commit ccbd536 into apache:main Jan 24, 2025
45 checks passed
@utkarsharma2 utkarsharma2 added the changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..) label Jan 27, 2025
gpathak128 pushed a commit to gpathak128/airflow that referenced this pull request Jan 29, 2025
apache#45456)

* docs: Added a markdown ADR document describing why the notion of dialects was introduced in the common sql provider

* docs: Added reference to the dialects in the Airflow common sql provider documentation

* docs: Reformatted the ADR

* refactor: Added dialects reference in mssql and postgres provider

* refactor: Duplicated dialects.rst to mssql and postgres provider until we sort out how to redirect to the common sql provider one

---------

Co-authored-by: David Blain <[email protected]>
got686-yandex pushed a commit to got686-yandex/airflow that referenced this pull request Jan 30, 2025
apache#45456)

* docs: Added a markdown ADR document describing why the notion of dialects was introduced in the common sql provider

* docs: Added reference to the dialects in the Airflow common sql provider documentation

* docs: Reformatted the ADR

* refactor: Added dialects reference in mssql and postgres provider

* refactor: Duplicated dialects.rst to mssql and postgres provider until we sort out how to redirect to the common sql provider one

---------

Co-authored-by: David Blain <[email protected]>
niklasr22 pushed a commit to niklasr22/airflow that referenced this pull request Feb 8, 2025
apache#45456)

* docs: Added a markdown ADR document describing why the notion of dialects was introduced in the common sql provider

* docs: Added reference to the dialects in the Airflow common sql provider documentation

* docs: Reformatted the ADR

* refactor: Added dialects reference in mssql and postgres provider

* refactor: Duplicated dialects.rst to mssql and postgres provider until we sort out how to redirect to the common sql provider one

---------

Co-authored-by: David Blain <[email protected]>
ambika-garg pushed a commit to ambika-garg/airflow that referenced this pull request Feb 17, 2025
apache#45456)

* docs: Added a markdown ADR document describing why the notion of dialects was introduced in the common sql provider

* docs: Added reference to the dialects in the Airflow common sql provider documentation

* docs: Reformatted the ADR

* refactor: Added dialects reference in mssql and postgres provider

* refactor: Duplicated dialects.rst to mssql and postgres provider until we sort out how to redirect to the common sql provider one

---------

Co-authored-by: David Blain <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:providers changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..) provider:common-sql
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants