Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Set up basic API for data curation #1

Merged
merged 14 commits into from
Jun 10, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .flake8
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
[flake8]
max-line-length = 120
ignore = E402, E265, F403, W503, W504, E731
exclude = .github, .git, venv*, docs, build
per-file-ignores = **/__init__.py:F401
12 changes: 12 additions & 0 deletions .github/FUNDING.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# These are supported funding model platforms

github: # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2]
patreon: # Replace with a single Patreon username
open_collective: pyronear
ko_fi: # Replace with a single Ko-fi username
tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
liberapay: # Replace with a single Liberapay username
issuehunt: # Replace with a single IssueHunt username
otechie: # Replace with a single Otechie username
custom: # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2']
57 changes: 57 additions & 0 deletions .github/validate_headers.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
from datetime import datetime
from pathlib import Path

shebang = ["#!usr/bin/python\n"]
blank_line = "\n"

# Possible years
starting_year = 2022
current_year = datetime.now().year

year_options = [f"{current_year}"] + [f"{year}-{current_year}" for year in range(starting_year, current_year)]
copyright_notices = [[f"# Copyright (C) {year_str}, Pyronear.\n"] for year_str in year_options]
license_notice = [
"# This program is licensed under the Apache License version 2.\n",
"# See LICENSE or go to <https://www.apache.org/licenses/LICENSE-2.0.txt> for full license details.\n",
]

# Define all header options
HEADERS = [
shebang + [blank_line] + copyright_notice + [blank_line] + license_notice for copyright_notice in copyright_notices
] + [copyright_notice + [blank_line] + license_notice for copyright_notice in copyright_notices]


IGNORED_FILES = ["version.py", "__init__.py"]
FOLDERS = ["src/app"]


def main():

invalid_files = []

# For every python file in the repository
for folder in FOLDERS:
for source_path in Path(__file__).parent.parent.joinpath(folder).rglob("**/*.py"):
if source_path.name not in IGNORED_FILES:
# Parse header
header_length = max(len(option) for option in HEADERS)
current_header = []
with open(source_path) as f:
for idx, line in enumerate(f):
current_header.append(line)
if idx == header_length - 1:
break
# Validate it
if not any(
"".join(current_header[: min(len(option), len(current_header))]) == "".join(option)
for option in HEADERS
):
invalid_files.append(source_path)

if len(invalid_files) > 0:
invalid_str = "\n- " + "\n- ".join(map(str, invalid_files))
raise AssertionError(f"Invalid header in the following files:{invalid_str}")


if __name__ == "__main__":
main()
34 changes: 34 additions & 0 deletions .github/workflows/builds.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
name: builds

on:
push:
branches: main
pull_request:
branches: main

jobs:
install:
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest]
python: [3.7, 3.8]
steps:
- uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v1
with:
python-version: ${{ matrix.python }}
architecture: x64
- name: Cache python modules
uses: actions/cache@v2
with:
path: ~/.cache/pip
key: ${{ runner.os }}-deps-${{ matrix.python }}-${{ hashFiles('src/app/requirements.txt') }}-${{ hashFiles('**/*.py') }}
restore-keys: |
${{ runner.os }}-deps-${{ matrix.python }}-${{ hashFiles('src/app/requirements.txt') }}-
- name: Install project
run: |
python -m pip install --upgrade pip
pip install -r src/app/requirements.txt
75 changes: 75 additions & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
name: api

on:
push:
branches: main
pull_request:
branches: main

jobs:
docker-ready:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest]
steps:
- uses: actions/checkout@v2
- name: Build & run docker
env:
QARNOT_TOKEN: ${{ secrets.QARNOT_TOKEN }}
BUCKET_NAME: ${{ secrets.BUCKET_NAME }}
BUCKET_MEDIA_FOLDER: ${{ secrets.BUCKET_MEDIA_FOLDER }}
BUCKET_ANNOT_FOLDER: ${{ secrets.BUCKET_ANNOT_FOLDER }}
run: docker-compose up -d --build
- name: Docker sanity check
run: sleep 10 && nc -vz localhost 8080
- name: Ping server
run: curl http://localhost:8080/docs

pytest:
runs-on: ubuntu-latest
needs: docker-ready
steps:
- uses: actions/checkout@v2
- name: Build & run docker
env:
QARNOT_TOKEN: ${{ secrets.QARNOT_TOKEN }}
BUCKET_NAME: ${{ secrets.BUCKET_NAME }}
BUCKET_MEDIA_FOLDER: ${{ secrets.BUCKET_MEDIA_FOLDER }}
BUCKET_ANNOT_FOLDER: ${{ secrets.BUCKET_ANNOT_FOLDER }}
run: docker-compose up -d --build
- name: Install dependencies in docker
run: |
docker-compose exec -T pyrostorage python -m pip install --upgrade pip
docker-compose exec -T pyrostorage pip install -r requirements-dev.txt
- name: Run docker test
run: |
docker-compose exec -T pyrostorage coverage --version
docker-compose exec -T pyrostorage coverage run -m pytest tests/
docker-compose exec -T pyrostorage coverage xml
docker cp pyro-storage_pyrostorage_1:/app/coverage.xml .

- name: Upload coverage to Codecov
uses: codecov/codecov-action@v1
with:
file: ./coverage.xml
flags: unittests
fail_ci_if_error: true

headers:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest]
python: [3.7]
steps:
- uses: actions/checkout@v2
with:
persist-credentials: false
- name: Set up Python
uses: actions/setup-python@v1
with:
python-version: ${{ matrix.python }}
architecture: x64
- name: Run unittests
run: python .github/validate_headers.py
75 changes: 75 additions & 0 deletions .github/workflows/style.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
name: style

on:
push:
branches: main
pull_request:
branches: main

jobs:
flake8:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest]
python: [3.7]
steps:
- uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v1
with:
python-version: ${{ matrix.python }}
architecture: x64
- name: Run flake8
run: |
pip install flake8
flake8 --version
flake8 ./

isort:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest]
python: [3.7]
steps:
- uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v1
with:
python-version: ${{ matrix.python }}
architecture: x64
- name: Run isort
run: |
pip install isort
isort --version
isort .
if [ -n "$(git status --porcelain --untracked-files=no)" ]; then exit 1; else echo "All clear"; fi

mypy:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest]
python: [3.7]
steps:
- uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v1
with:
python-version: ${{ matrix.python }}
architecture: x64
- name: Cache python modules
uses: actions/cache@v2
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pkg-deps-${{ matrix.python }}-${{ hashFiles('src/app/requirements.txt') }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r src/app/requirements.txt --upgrade
pip install mypy
- name: Run mypy
run: |
mypy --version
mypy
128 changes: 128 additions & 0 deletions CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
# Contributor Covenant Code of Conduct

## Our Pledge

We as members, contributors, and leaders pledge to make participation in our
community a harassment-free experience for everyone, regardless of age, body
size, visible or invisible disability, ethnicity, sex characteristics, gender
identity and expression, level of experience, education, socio-economic status,
nationality, personal appearance, race, religion, or sexual identity
and orientation.

We pledge to act and interact in ways that contribute to an open, welcoming,
diverse, inclusive, and healthy community.

## Our Standards

Examples of behavior that contributes to a positive environment for our
community include:

* Demonstrating empathy and kindness toward other people
* Being respectful of differing opinions, viewpoints, and experiences
* Giving and gracefully accepting constructive feedback
* Accepting responsibility and apologizing to those affected by our mistakes,
and learning from the experience
* Focusing on what is best not just for us as individuals, but for the
overall community

Examples of unacceptable behavior include:

* The use of sexualized language or imagery, and sexual attention or
advances of any kind
* Trolling, insulting or derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or email
address, without their explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting

## Enforcement Responsibilities

Community leaders are responsible for clarifying and enforcing our standards of
acceptable behavior and will take appropriate and fair corrective action in
response to any behavior that they deem inappropriate, threatening, offensive,
or harmful.

Community leaders have the right and responsibility to remove, edit, or reject
comments, commits, code, wiki edits, issues, and other contributions that are
not aligned to this Code of Conduct, and will communicate reasons for moderation
decisions when appropriate.

## Scope

This Code of Conduct applies within all community spaces, and also applies when
an individual is officially representing the community in public spaces.
Examples of representing our community include using an official e-mail address,
posting via an official social media account, or acting as an appointed
representative at an online or offline event.

## Enforcement

Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the community leaders responsible for enforcement at
[email protected].
All complaints will be reviewed and investigated promptly and fairly.

All community leaders are obligated to respect the privacy and security of the
reporter of any incident.

## Enforcement Guidelines

Community leaders will follow these Community Impact Guidelines in determining
the consequences for any action they deem in violation of this Code of Conduct:

### 1. Correction

**Community Impact**: Use of inappropriate language or other behavior deemed
unprofessional or unwelcome in the community.

**Consequence**: A private, written warning from community leaders, providing
clarity around the nature of the violation and an explanation of why the
behavior was inappropriate. A public apology may be requested.

### 2. Warning

**Community Impact**: A violation through a single incident or series
of actions.

**Consequence**: A warning with consequences for continued behavior. No
interaction with the people involved, including unsolicited interaction with
those enforcing the Code of Conduct, for a specified period of time. This
includes avoiding interactions in community spaces as well as external channels
like social media. Violating these terms may lead to a temporary or
permanent ban.

### 3. Temporary Ban

**Community Impact**: A serious violation of community standards, including
sustained inappropriate behavior.

**Consequence**: A temporary ban from any sort of interaction or public
communication with the community for a specified period of time. No public or
private interaction with the people involved, including unsolicited interaction
with those enforcing the Code of Conduct, is allowed during this period.
Violating these terms may lead to a permanent ban.

### 4. Permanent Ban

**Community Impact**: Demonstrating a pattern of violation of community
standards, including sustained inappropriate behavior, harassment of an
individual, or aggression toward or disparagement of classes of individuals.

**Consequence**: A permanent ban from any sort of public interaction within
the community.

## Attribution

This Code of Conduct is adapted from the [Contributor Covenant][homepage],
version 2.0, available at
https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.

Community Impact Guidelines were inspired by [Mozilla's code of conduct
enforcement ladder](https://github.com/mozilla/diversity).

[homepage]: https://www.contributor-covenant.org

For answers to common questions about this code of conduct, see the FAQ at
https://www.contributor-covenant.org/faq. Translations are available at
https://www.contributor-covenant.org/translations.
Loading