Skip to content

Commit

Permalink
Notebook/Jupyter: Add example using JupySQL
Browse files Browse the repository at this point in the history
  • Loading branch information
amotl committed Jan 22, 2025
1 parent bd0571f commit 8c31d35
Show file tree
Hide file tree
Showing 10 changed files with 2,926 additions and 0 deletions.
94 changes: 94 additions & 0 deletions .github/workflows/notebook-jupyter.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
name: pandas

on:
pull_request:
branches: ~
paths:
- '.github/workflows/notebook-jupyter.yml'
- 'notebook/jupyter/**'
- '/requirements.txt'
push:
branches: [ main ]
paths:
- '.github/workflows/notebook-jupyter.yml'
- 'notebook/jupyter/**'
- '/requirements.txt'

# Allow job to be triggered manually.
workflow_dispatch:

# Run job each night after CrateDB nightly has been published.
schedule:
- cron: '0 3 * * *'

# Cancel in-progress jobs when pushing to the same branch.
concurrency:
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.ref }}

jobs:
test:
name: "
Python: ${{ matrix.python-version }}
CrateDB: ${{ matrix.cratedb-version }}
on ${{ matrix.os }}"
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ 'ubuntu-latest' ]
python-version: [ '3.8', '3.13' ]
cratedb-version: [ 'nightly' ]

env:
OS_TYPE: ${{ matrix.os }}
PYTHON_VERSION: ${{ matrix.python-version }}
UV_SYSTEM_PYTHON: true

services:
cratedb:
image: crate/crate:${{ matrix.cratedb-version }}
ports:
- 4200:4200
- 5432:5432
env:
CRATE_HEAP_SIZE: 4g

steps:

- name: Acquire sources
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
architecture: x64
cache: 'pip'
cache-dependency-path: |
notebook/jupyter/pyproject.toml
- name: Set up uv
uses: astral-sh/setup-uv@v5
with:
cache-dependency-glob: |
notebook/jupyter/pyproject.toml
cache-suffix: ${{ matrix.python-version }}
enable-cache: true
version: "latest"

- name: Install project
run: |
uv pip install notebook/jupyter[all]
- name: Validate notebook/jupyter
run: |
cd notebook/jupyter
pytest
- name: Build docs for notebook/jupyter
run: |
cd notebook/jupyter
poe check
poe docs-html
poe docs-linkcheck
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
.idea
.venv*
__pycache__
_build
.coverage*
.env
.DS_Store
Expand All @@ -14,3 +15,4 @@ logs.log
*.tmp
*.swp
*.bak
*.egg-info
8 changes: 8 additions & 0 deletions notebook/jupyter/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# CrateDB Jupyter Examples

A few examples using CrateDB from Jupyter Notebooks.
If you are here already, we recommend to check out the [JupySQL] examples
at least.


[JupySQL]: https://jupysql.ploomber.io/
31 changes: 31 additions & 0 deletions notebook/jupyter/conftest.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
import os
from pathlib import Path

import pytest
import sqlalchemy as sa
from pueblo.testing.notebook import generate_tests


def pytest_generate_tests(metafunc):
"""
Generate pytest test case per Jupyter Notebook.
"""
here = Path(__file__).parent
generate_tests(metafunc, path=here)


@pytest.fixture(autouse=True)
def reset_database_tables():
"""
Before running a test case, reset relevant tables in database.
"""

connection_string = os.environ.get("CRATEDB_CONNECTION_STRING")

engine = sa.create_engine(connection_string, echo=os.environ.get("DEBUG"))
connection = engine.connect()

reset_tables = []

for table in reset_tables:
connection.execute(sa.text(f"DROP TABLE IF EXISTS {table};"))
6 changes: 6 additions & 0 deletions notebook/jupyter/docs/conf.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
project = "CrateDB Jupyter Examples"
copyright = "2023-2025, The CrateDB Developers"
author = "The CrateDB Developers"

html_theme = "furo"
extensions = ["myst_nb"]
14 changes: 14 additions & 0 deletions notebook/jupyter/docs/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# CrateDB Jupyter Examples

This is a little Sphinx documentation stub that includes Jupyter Notebooks,
by using [MyST-NB].

```{toctree}
:maxdepth: 2
:glob:
*
```


[MyST-NB]: https://myst-nb.readthedocs.io/
1 change: 1 addition & 0 deletions notebook/jupyter/docs/jupysql.ipynb
Loading

0 comments on commit 8c31d35

Please sign in to comment.