Skip to content

Commit

Permalink
fix: cleanup
Browse files Browse the repository at this point in the history
  • Loading branch information
lwasser committed Dec 20, 2024
1 parent 66891a0 commit 07c81e4
Show file tree
Hide file tree
Showing 4 changed files with 97 additions and 32 deletions.
47 changes: 33 additions & 14 deletions ci-data/ci.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,37 @@
# What is continuous integration?
(ci-cd)=
# Continuous Integration and Continuous Deployment (CI/CD) For Python Packages

When you’re ready to publish your code online, you can setup Continuous Integration (CI). CI is a platform that allows you to specify and run jobs or workflows that you define.
When you develop, work on, and contribute to software, there is more to consider than
just writing code. Having tests and checks ensures that your code
runs reliably and follows a consistent format is also important. You can use
**Continuous Integration (CI)** and **Continuous
Deployment (CD)** to run tests and checks on your code every time someone suggests a change online
in a platform like GitHub or GitLab.

- **Continuous Integration (CI):** Automates the process of running tests,
code checks, and other workflows each time code is updated.
- **Continuous Deployment (CD):** Extends CI by allowing you to automate publishing your package to PyPI, publishing your documentation, and more.

CI and CD streamline software development by automating repetitive
tasks and ensuring code quality and consistency. Having CI setup also makes it easier for new contributors
to contribute to your code base without setting up all your test suites and
other local checks.

## What is continuous integration?

When you’re ready to publish your code online, you can set up Continuous Integration (CI). CI is a platform that allows you to specify and run jobs or workflows you define.
These workflows include:

- Running your test suite
- Running code checkers / linters / spellcheck
- Building your documentation

CI allows you to automate running workflows across a suite of environments including:
CI allows you to automate running workflows across a suite of environments, including:

- environments containing different Python versions and
- different operating systems (Mac, Linux, Windows).

### What is Continuous Deployment (CD)?
## What is Continuous Deployment (CD)?

Continuous deployment (CD) extends the CI process by automating the deployment of code changes to production or staging environments. In the case of your open source tool, CD can be used to:

Expand All @@ -21,26 +40,26 @@ Continuous deployment (CD) extends the CI process by automating the deployment o

It is also used once your conda-forge recipe is set up to keep your package up to date on conda-forge.

### Why Use CI
### Why use CI

CI can be configured to run a workflow on every commit pushed to GitHub and every pull request opened. This ensures that any changes made to your package are tested across environments before they are merged into the main branch of your code.
CI can be configured to run a workflow on every commit pushed to GitHub and every pull request opened. This ensures that any changes made to your package are tested across environments before merging into the main branch of your code.

These checks are particularly useful if someone new is contributing to your code. Every change a contributor makes will be tested when it’s pushed to your code repository.
These checks are particularly useful if someone new is contributing to your code. Every contributor's change will be tested when pushed to your code repository.

Together, CI and CD streamline the process of building, testing, and deploying code. They aim to improve the efficiency, quality, and reliability of software development and publication.
Together, CI and CD streamline the process of building, testing, and deploying code. They aim to improve software development and publication efficiency, quality, and reliability.

```{note}
All pyOpenSci packages must use some form of continuous integration. Even if you are not planning to go through peer review, we strongly recommend that you use continuous integration too!
All pyOpenSci packages must use some form of continuous integration. Even if you are not planning to go through peer review, we strongly recommend that you use continuous integration, too!
```

In the case of GitHub actions (which we will focus on here), CI workflows are running on online servers that support GitHub.

## CI / CD Platforms
## CI / CD platforms

There are numerous platforms available for CI/CD. Here, we will focus on GitHub Actions (GHA) which is built into GitHub. GitHub is the most commonly used platform to store scientific open source software.
There are numerous platforms available for CI/CD. Here, we will focus on GitHub Actions (GHA), built into GitHub. GitHub is the most commonly used platform to store scientific open-source software.

:::{note}
If you are using [GitLab](https://about.gitlab.com/) CI/CD many of the principles described here will apply, however the workflow files may look different.
If you use [GitLab](https://about.gitlab.com/) CI/CD, many of the principles described here will apply. However, the workflow files may look different.
:::

### If you aren't sure, use GitHub Actions
Expand All @@ -60,6 +79,6 @@ templates that you can easily use and adapt to your own needs.

## Embrace automation

By embracing CI/CD, you can ensure that your code runs as you expect it to across the diverse landscapes of user environments. Further you can
automate certain checks (and in some cases code fixes) including linting and code style. You can even automate spell checking your documentation
By embracing CI/CD, you can ensure that your code runs as you expect it to across the diverse landscapes of user environments. Further, you can
automate certain checks (and, in some cases, code fixes), including linting and code style. You can even automate spell-checking your documentation
and docstrings!
1 change: 0 additions & 1 deletion ci-data/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,5 +7,4 @@


What is CI? <ci.md>
Data <data.md>
:::
18 changes: 6 additions & 12 deletions index.md
Original file line number Diff line number Diff line change
Expand Up @@ -286,22 +286,23 @@ to see here clarified in this guide, [we invite you to open an issue on GitHub.]

If you have questions about our peer review process or packaging in general, you are welcome to use our [pyOpenSci Discourse forum](https://pyopensci.discourse.group/).

This is a living guide that is updated as tools and best practices evolve in the Python packaging ecosystem. We will be adding new content over the next year.
This living Python packaging guide is updated as tools and best practices evolve in the Python packaging ecosystem. We will be adding new content over the next year.


```{toctree}
:::{toctree}
:hidden:
:caption: Tutorials

Tutorials <tutorials/intro>
```
:::

```{toctree}
:::{toctree}
:hidden:
:caption: Packaging

Packaging <package-structure-code/intro>

```
:::

:::{toctree}
:hidden:
Expand All @@ -311,13 +312,6 @@ Documentation <documentation/index>

:::

:::{toctree}
:hidden:
:caption: Build & Publish

Packaging <package-structure-code/intro>

:::

:::{toctree}
:hidden:
Expand Down
63 changes: 58 additions & 5 deletions tests/code-cov.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,66 @@
# Code coverage for your Python package test suite

Code coverage is the amount of your package's codebase that is run as a part of running your project's tests. A good rule of thumb is to ensure that every line of your code is run at least once during testing. However, note that good code coverage does not guarantee that your package is well-tested. For example, you may run all of your lines of code, but not account for many edge-cases that users may have. Ultimately, you should think carefully about the way your package will be used, and decide whether your tests adequately cover all of that usage.
Code coverage measures how much of your package's code runs during testing.
Achieving high coverage can help ensure the reliability of your codebase, but
it’s not a guarantee of quality. Below, we outline key considerations for
using code coverage effectively.

Some common services for analyzing code coverage are [codecov.io](https://codecov.io/) and [coveralls.io](https://coveralls.io/). These projects are free for open source tools, and will provide dashboards that tell you how much of your codebase is covered during your tests. We recommend setting up an account (on either CodeCov or Coveralls), and using it to keep track of your code coverage.
## Why aim for high code coverage?

```{figure} ../images/code-cov-stravalib.png
A good practice is to ensure that every line of your code runs at least once
during your test suite. This helps you:

- Identify untested parts of your codebase.
- Catch bugs that might otherwise go unnoticed.
- Build confidence in your software's stability.

## Limitations of code coverage

While high code coverage is valuable, it has its limits:

- **Difficult-to-test code:** Some parts of your code might be challenging to
test, either due to complexity or limited resources.
- **Missed edge cases:** Running all lines of code doesn’t guarantee that edge
cases are handled correctly.

Ultimately, you should focus on how your package will be used and ensure your
tests cover those scenarios adequately.

## Tools for analyzing Python package code coverage

Some common services for analyzing code coverage are [codecov.io](https://codecov.io/) and [coveralls.io](https://coveralls.io/). These projects are free for open source tools and will provide dashboards that tell you how much of your codebase is covered during your tests. We recommend setting up an account (on either CodeCov or Coveralls) and using it to keep track of your code coverage.

:::{figure} ../images/code-cov-stravalib.png
:height: 450px
:alt: Screenshot of the code cov service - showing test coverage for the stravalib package. in this image you can see a list of package modules and the associated number of lines and % lines covered by tests. at the top of the image you can see what branch is being evaluated and the path to the repository being shown.
:alt: Screenshot of the code cov service - showing test coverage for the stravalib package. This image shows a list of package modules and the associated number of lines and % lines covered by tests. At the top of the image, you can see what branch is being evaluated and the path to the repository.

The CodeCov platform is a useful tool if you wish to track code coverage visually. Using it, you can not only get the same summary information that you can get with the **pytest-cov** extension. You can also see what lines are covered by your tests and what lines are not covered. Code coverage is useful for evaluating unit tests and/or how much of your package code is "covered". It, however, will not evaluate things like integration tests and end-to-end workflows.

:::

The CodeCov platform is a useful tool if you wish to visually track code coverage. Using it you can not only get the same summary information that you can get with **pytest-cov** extension. You can also get a visual representation of what lines are covered by your tests and what lines are not covered. Code coverage is mostly useful for evaluating unit tests and/or how much of your package code is "covered". It however will not evaluate things like integration tests and end-to-end workflows.


:::{admonition} Typing & MyPy coverage
You can also create and upload typing reports to CodeCov.
:::

## Exporting Local Coverage Reports

In addition to using services like CodeCov or Coveralls, you can generate local coverage reports directly using the **coverage.py** tool. This can be especially useful if you want to create reports in Markdown or HTML format for offline use or documentation.

To generate a coverage report in **Markdown** format, run:

```bash
$ python -m coverage report --format=markdown
```
This command will produce a Markdown-formatted coverage summary that you can easily include in project documentation or share with your team.

To generate an HTML report that provides a detailed, interactive view of which lines are covered, use:

```bash
python -m coverage html
```

The generated HTML report will be saved in a directory named htmlcov by default. Open the index.html file in your browser to explore your coverage results.

These local reports are an excellent way to quickly review coverage without setting up an external service.

0 comments on commit 07c81e4

Please sign in to comment.