Merge branch 'develop' into b/dx-300/azure-blob-warning

* develop: [BUGFIX] Copy previous versions after checking out the the current commit (#7142) [DOCS] Remove sitemap.xml (#7141) [MAINTENANCE] mypy `v1.0.0` (#7138) [DOCS]: fix capitalization of Slack (#7136) [MAINTENANCE] format notebooks with `black` (#7054) [DOCS] Use named snippets part 1 (#7131)
great-expectations · Feb 14, 2023 · cc69fce · cc69fce
2 parents 725ff36 + 55c3b2a
commit cc69fce
Show file tree

Hide file tree

Showing 55 changed files with 2,586 additions and 3,118 deletions.
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -15,7 +15,7 @@ repos:
   - repo: https://github.com/psf/black
     rev: 22.3.0
     hooks:
-      - id: black
+      - id: black-jupyter
   - repo: https://github.com/charliermarsh/ruff-pre-commit
     rev: 'v0.0.241'
     hooks:

diff --git a/contrib/cli/requirements.txt b/contrib/cli/requirements.txt
@@ -1,7 +1,7 @@
-black==22.3.0        # Linting / code style
+black[jupyter]==22.3.0 # Linting / code style
 Click>=7.1.2         # CLI tooling
 cookiecutter==1.7.3  # Project templating
-mypy==0.991          # Type checker
+mypy==1.0.0          # Type checker
 pydantic>=1.0,<2.0    # Needed for mypy plugin
 pytest>=5.3.5        # Test framework
 ruff==0.0.241        # Linting / code style

diff --git a/docs/README.md b/docs/README.md
@@ -60,9 +60,9 @@ The following are a few details about other files Docusaurus uses that you may w
 - `../babel.config.js`: Babel config file used when building
 - `../package.json`: dependencies and scripts
 - `../yarn.lock`: dependency lock file that ensures reproducibility
-- `sitemap.xml`: After any changes to sidebars.js are made, use https://www.xml-sitemaps.com/ with the temporary netlify url and then download the resulting sitemap.xml to update the existing sitemap.xml file. Make sure to change the temporary netlify url to the real docs url https://docs.greatexpectations.io/docs/ in the sitemap.xml file.
+
+sitemap.xml is built and uploaded by a netlify plugin during the documentation build process. 
 
 ## Documentation changes checklist
 
-1. If you have made changes to sidebar.js, regenerate and update sitemap.xml per the above instructions
-2. For any pages you have moved or removed, update _redirects to point from the old to the new content location
+1. For any pages you have moved or removed, update _redirects to point from the old to the new content location
diff --git a/docs/build_docs b/docs/build_docs
@@ -26,16 +26,16 @@ echo "Installing dev dependencies"
 echo "Building sphinx API docs."
 (cd docs/sphinx_api_docs_source; invoke docs)
 
-echo "Copying previous versions"
-curl "https://superconductive-public.s3.us-east-2.amazonaws.com/oss_docs_versions.zip" -o "oss_docs_versions.zip"
-unzip -o oss_docs_versions.zip -d .
-
 if [ "$PULL_REQUEST" == "false" ]
 then
   echo "Not in a pull request. Checking back out current commit ${CURRENT_COMMIT} to build the rest of the docs."
   git checkout "$CURRENT_COMMIT"
   git pull
 fi
 
+echo "Copying previous versions"
+curl "https://superconductive-public.s3.us-east-2.amazonaws.com/oss_docs_versions.zip" -o "oss_docs_versions.zip"
+unzip -o oss_docs_versions.zip -d .
+
 echo "Building docusaurus docs."
 yarn build
diff --git a/docs/deployment_patterns/how_to_use_great_expectations_in_aws_glue.md b/docs/deployment_patterns/how_to_use_great_expectations_in_aws_glue.md
@@ -10,7 +10,7 @@ We will cover case with RuntimeDataConnector and use S3 as metadata store.
 ### 0. Pre-requirements
 
 - Configure great_expectations.yaml and upload to your S3 bucket or generate it dynamically from code
-```yaml file=../../tests/integration/docusaurus/deployment_patterns/aws_glue_deployment_patterns_great_expectations.yaml#L1-L70
+```yaml name="tests/integration/docusaurus/deployment_patterns/aws_glue_deployment_patterns_great_expectations.yaml"
 ```
 
 
@@ -20,24 +20,24 @@ You need to add to your AWS Glue Spark Job Parameters to install great expectati
   — additional-python-modules great_expectations
 ```
 Then import necessary libs:
-```python file=../../tests/integration/docusaurus/deployment_patterns/aws_glue_deployment_patterns.py#L1-L13
+```python name="tests/integration/docusaurus/deployment_patterns/aws_glue_deployment_patterns.py imports"
 ```
 
 ### 2. Set up Great Expectations
 Here we initialize a Spark and Glue, and read great_expectations.yaml
-```python file=../../tests/integration/docusaurus/deployment_patterns/aws_glue_deployment_patterns.py#L15-L22
+```python name="tests/integration/docusaurus/deployment_patterns/aws_glue_deployment_patterns.py set up gx"
 ```
 
 ### 3. Connect to your data
-```python file=../../tests/integration/docusaurus/deployment_patterns/aws_glue_deployment_patterns.py#L24-L43
+```python name="tests/integration/docusaurus/deployment_patterns/aws_glue_deployment_patterns.py connect to data"
 ```
 
 ### 4. Create Expectations
-```python file=../../tests/integration/docusaurus/deployment_patterns/aws_glue_deployment_patterns.py#L45-L62
+```python name="tests/integration/docusaurus/deployment_patterns/aws_glue_deployment_patterns.py create expectations"
 ```
 
 ### 5. Validate your data
-```python file=../../tests/integration/docusaurus/deployment_patterns/aws_glue_deployment_patterns.py#L64-L78
+```python name="tests/integration/docusaurus/deployment_patterns/aws_glue_deployment_patterns.py validate your data"
 ```
 
 ### 6. Congratulations!