Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dev #1

Merged
merged 66 commits into from
Nov 30, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
c4307be
Add readme
RobertRosca Nov 16, 2020
6e663cd
Add initial repository files
RobertRosca Nov 16, 2020
475b154
Copy ipykernel kernelspec
RobertRosca Nov 16, 2020
09e6d98
Add modifications
RobertRosca Nov 16, 2020
c4d8f36
Use setup.py instead
RobertRosca Nov 16, 2020
ba2947e
Update gitignore
RobertRosca Nov 16, 2020
fe77fa6
Add clarifying comments
RobertRosca Nov 16, 2020
c931055
Adjust how mod argument is used
RobertRosca Nov 16, 2020
0f0ca1c
Update argparse descriptions
RobertRosca Nov 16, 2020
cbd9459
Add vip ipykernel launcher
RobertRosca Nov 16, 2020
7fd9306
Add attribution
RobertRosca Nov 16, 2020
9fcd806
Sort imports
RobertRosca Nov 16, 2020
add6ef2
Add resources logos
RobertRosca Nov 16, 2020
9b6511a
Rename install app
RobertRosca Nov 16, 2020
4891d63
Add tests
RobertRosca Nov 16, 2020
0638c2d
Add note about logging
RobertRosca Nov 16, 2020
c53591b
Update dependencies
RobertRosca Nov 16, 2020
ddbdf50
Update readme
RobertRosca Nov 16, 2020
91de134
Add pytest-cov to test requirements
RobertRosca Nov 16, 2020
a74b69f
Add tests
RobertRosca Nov 16, 2020
48a7c96
Add publish to pypi
RobertRosca Nov 16, 2020
2da9231
Add codecov
RobertRosca Nov 16, 2020
49343d6
Expand todo
RobertRosca Nov 16, 2020
9a3266d
Sort imports
RobertRosca Nov 16, 2020
9c3319f
Fix pytest-cov version
RobertRosca Nov 16, 2020
0d9ece8
Add resources dir to package data
RobertRosca Nov 16, 2020
4325419
Bump version
RobertRosca Nov 16, 2020
6fa531e
Remove old logging attempt
RobertRosca Nov 16, 2020
bb0afa0
Resolve path to system python3
RobertRosca Nov 16, 2020
7fd00ae
Run test with shutil.which instaed of sys.executable
RobertRosca Nov 16, 2020
47358fa
Set version to alpha, use get version in setup.py
RobertRosca Nov 16, 2020
4154802
Expand readme
RobertRosca Nov 17, 2020
3fbed51
Fix typo
RobertRosca Nov 17, 2020
a56fd2f
Clarify venv priority order
RobertRosca Nov 18, 2020
7b39838
Clarify tests with comments
RobertRosca Nov 18, 2020
a896bcb
Add notebook tests
RobertRosca Nov 18, 2020
ad8f288
Depend on ipykernel, patch it
RobertRosca Nov 18, 2020
07e5764
Adjust tests for new patching
RobertRosca Nov 18, 2020
afdaa94
Move launcher entry point
RobertRosca Nov 18, 2020
61867b5
Use the launcher module correctly
RobertRosca Nov 18, 2020
434964c
Move venv search out of launcher
RobertRosca Nov 18, 2020
b33ab9d
Rename tests
RobertRosca Nov 18, 2020
868f7b5
Update gitignore
RobertRosca Nov 18, 2020
5989355
Adjust formatting
RobertRosca Nov 18, 2020
2bdd09e
Expand python test versions
RobertRosca Nov 18, 2020
732913b
Minor fixes and cleanup
RobertRosca Nov 18, 2020
eeecb10
Update badge formatting
RobertRosca Nov 19, 2020
47bb351
Space out VENV_NAMES
RobertRosca Nov 20, 2020
115a75b
Update tests
RobertRosca Nov 20, 2020
27eb005
Bump version
RobertRosca Nov 20, 2020
f9a944f
Standardise python module calls
RobertRosca Nov 30, 2020
d658053
No need for glob, use `is_file` to check for python3
RobertRosca Nov 30, 2020
22bcc89
Copy argv before modifying it
RobertRosca Nov 30, 2020
d3b2e66
Replace `subprocess.run` call with `os.execv`
RobertRosca Nov 30, 2020
f2fb54b
Use `monkeypatch.chdir` instead of `os.chdir`
RobertRosca Nov 30, 2020
5ff0abb
Fix nested calls to pytest
RobertRosca Nov 30, 2020
059a546
Use call to venv module instead of subprocess
RobertRosca Nov 30, 2020
606b963
Sort imports
RobertRosca Nov 30, 2020
33a310c
Add .vscode to gitignore
RobertRosca Nov 30, 2020
b5c8480
Standardise python calls for builds
RobertRosca Nov 30, 2020
5fe1651
Move import to after sys.path manipulation
RobertRosca Nov 30, 2020
c4768e1
Update readme
RobertRosca Nov 30, 2020
4e0f2a9
Bump version to 1.0.0
RobertRosca Nov 30, 2020
b1e3a16
Use minimum versions for install requires
RobertRosca Nov 30, 2020
e92a954
Change lifecycle to active
RobertRosca Nov 30, 2020
f7ef410
Add dependabot integration and constraints
RobertRosca Nov 30, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
version: 2
updates:
- package-ecosystem: "pip"
directory: "/.github/dependabot/"
schedule:
interval: "daily"
20 changes: 20 additions & 0 deletions .github/dependabot/constraints.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
backcall==0.2.0
decorator==4.4.2
ipykernel==5.3.4
ipython==7.19.0
ipython-genutils==0.2.0
jedi==0.17.2
jupyter-client==6.1.7
jupyter-core==4.7.0
parso==0.7.1
pexpect==4.8.0
pickleshare==0.7.5
prompt-toolkit==3.0.8
ptyprocess==0.6.0
Pygments==2.7.2
python-dateutil==2.8.1
pyzmq==20.0.0
six==1.15.0
tornado==6.1
traitlets==5.0.5
wcwidth==0.2.5
34 changes: 34 additions & 0 deletions .github/workflows/python-publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# This workflows will upload a Python Package using Twine when a release is created
# For more information see: https://help.github.com/en/actions/language-and-framework-guides/using-python-with-github-actions#publishing-to-package-registries

name: Upload Python Package

on:
release:
types: [created]

jobs:
deploy:

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2

- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: '3.8'

- name: Install dependencies
run: |
python3 -m pip install --upgrade pip
python3 -m pip install setuptools wheel twine

- name: Build and publish
env:
TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
run: |
python3 -m setup.py sdist bdist_wheel
python3 -m twine upload dist/*
48 changes: 48 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# This workflow will install Python dependencies, run tests and lint with a variety of Python versions
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions

name: Tests

on:
push:
branches: [ main ]
pull_request:
branches: [ main ]

jobs:
tests:

runs-on: ubuntu-latest
strategy:
matrix:
python-version: [ 3.6, 3.7, 3.8, 3.9 ]

steps:
- uses: actions/checkout@v2

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}

- uses: actions/cache@v1
with:
path: ~/.cache/pip
key: ${{ runner.os }}-${{ matrix.python-version }}-pip-${{ hashFiles('**/setup.py') }}-${{ hashFiles('.github/dependabot/constraints.txt') }}

- name: Install dependencies
run: |
python3 -m pip install --upgrade pip

if [ ! ${{ matrix.python-version }} == "3.6" ]; then
# Our constraints can't be applied to python 3.6
python3 -m pip install ".[test]" --constraint .github/dependabot/constraints.txt
else
python3 -m pip install ".[test]"
fi

- name: Test with pytest
run: |
python3 -m pytest -s --verbose --cov=vip_ipykernel

- uses: codecov/codecov-action@v1
9 changes: 9 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,11 @@
.venv
.venv/

*.egg-info
**/__pycache__/**

**/.ipynb_checkpoints

.coverage

.vscode
132 changes: 132 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
[![Lifecycle](https://www.repostatus.org/badges/latest/active.svg)](https://www.repostatus.org/#active)
[![Release](https://img.shields.io/github/release/robertrosca/vip-ipykernel.svg)](https://github.com/robertrosca/vip-ipykernel/releases/latest)

[![Tests](https://github.com/RobertRosca/vip-ipykernel/workflows/Tests/badge.svg)](https://github.com/RobertRosca/vip-ipykernel/actions?query=workflow%3ATests)
[![Codecov](https://codecov.io/gh/RobertRosca/vip-ipykernel/branch/main/graph/badge.svg)](https://codecov.io/gh/RobertRosca/vip-ipykernel)

# ViP IPykernel

Venv in Parent IPykernel

- [ViP IPykernel](#vip-ipykernel)
- [Overview](#overview)
- [How it Works](#how-it-works)
- [Caveats and Gotchas](#caveats-and-gotchas)
- [VSCode Jupyter Notebook Integration](#vscode-jupyter-notebook-integration)
- [Venv Names](#venv-names)
- [Acknowledgements](#acknowledgements)
- [Todo](#todo)

## Overview

Do you use `venv`'s for all of your environments? Do you run Jupyter out of a
system/user installed location or via JupyterHub? Are you bored of making a
kernel for every single venv? Then this is the package for you!

vip-ipykernel overwrites the default `python3` kernel and replaces it with one
which will traverse directories upwards until it finds a `.venv` directory, if
it finds one then it will start the kernel with python out of that directory, if
it does not find a venv then it will carry on with the default python3.

NOTE: Your venv **must have ipykernel installed in it**, as this 'kernel' just
searches for and launches ipykernel out of the local venv. If ipykernel is not
available inside the venv then it will fail to start.

This only needs to be installed once, you can do this with `pip install
vip-ipykernel --user` to install it into your local user environment.

Once the package is installed, run `python3 -m vip_ipykernel.kernelspec --user`
to install the kernel, now when you run a notebook with the default `python3`
kernel it will instead use the venv in a parent directory.

If you want to revert the changes, run `python3 -m ipykernel install --user`,
this will re-install the default `python3` kernel.

Alternatively, if you don't want to overwrite the default kernel, then you can
pass a name (`python3 -m vip_ipykernel.kernelspec --user --name venv-kernel`) to
so that the kernel appears separately in the list of kernels and the default
behaviour is not modified.

## How it Works

The standard python3 kernel is:

```
{
"argv": [
"/usr/bin/python3",
"-m",
"ipykernel_launcher",
"-f",
"{connection_file}"
],
"display_name": "Python 3",
"language": "python"
}
```

This just says "Run using `python3` to run `ipykernel_launcher` with an argument
`-f {connection_file}`". When you install the vip ipykernel this is replace by:

```
{
"argv": [
"/usr/bin/python3",
"-m",
"vip_ipykernel_launcher",
"-m",
"ipykernel_launcher",
"-f",
"{connection_file}"
],
"display_name": "Python 3",
"language": "python"
}
```

Which will instead run the `vip_ipykernel.vip_ipykernel_launcher` module,
passing it the arguments `-m ipykernel_launcher -f {connection_file}`. The
module runs a function `venv_search` which looks in the current directory, and
upwards to any parent directories, until it finds a `.venv` or `venv` directory
containing `bin/python3`.

If it finds a venv with python3 in it, it passes the arguments `-m
ipykernel_launcher -f {connection_file}` to that python executable, which starts
and connects the kernel from that venv to your current session, in the same way
that a kernel installed for that specific venv would.

If it does not find a venv, then it will default to the system python executable
and behave like the standard `python3` kernel.

## Caveats and Gotchas

### VSCode Jupyter Notebook Integration

VSCode manages kernels for its notebooks with its own system, so it will not use
the vip-ipykernel.

### Venv Names

Currently only venv's named `.venv` or `venv` are searched for, if your venv has
a different name it won't be found, and if you have multiple venv's available
then the first one (sorted alphanumerically, so `.venv` takes priority over
`venv`) will be used.

## Acknowledgements

The kernel implementation and tests are largely copy-and-paste'd directly from
the [ipykernel project](https://github.com/ipython/ipykernel) with some minor
modifications made to search for a venv and launch python out of it if possible.

## Todo

- [ ] Expand tests to different versions of ipykernel/jupyter_core
- [ ] Look at ways to show kernel errors
- [ ] Support for other environments:
- [ ] Poetry-created venvs (`poetry env info --path`)
- [ ] Pipenv-created venvs
- [ ] Pyenv-created venvs
- [ ] Conda-created environments
- [ ] User-configured venvs
- [ ] Reading from vscode configuration?
- [ ] etc...
45 changes: 45 additions & 0 deletions setup.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
from setuptools import setup
import codecs
import os.path


def read(rel_path):
here = os.path.abspath(os.path.dirname(__file__))
with codecs.open(os.path.join(here, rel_path), 'r') as fp:
return fp.read()


def get_version(rel_path):
for line in read(rel_path).splitlines():
if line.startswith('__version__'):
delim = '"' if '"' in line else "'"
return line.split(delim)[1]
else:
raise RuntimeError("Unable to find version string.")


setup(
name='vip-ipykernel',
url='https://github.com/RobertRosca/vip-ipykernel',
long_description=read("README.md"),
long_description_content_type='text/markdown',
version=get_version("./src/vip_ipykernel/__init__.py"),
python_requires='==3.*,>=3.6.0',
author='Robert Rosca',
author_email='[email protected]',
py_modules=['vip_ipykernel_launcher'],
packages=['vip_ipykernel'],
package_dir={"": "src"},
install_requires=[
'jupyter-client>=4.2',
'ipykernel>=4.4',
],
extras_require={
'test': [
'pytest==5.*,>=5.2.0',
'pytest-cov==2.*,>=2.10.1',
'nose==1.*,>=1.3.7',
'nbval==0.9.*,>=0.9.6',
],
},
)
30 changes: 30 additions & 0 deletions src/vip_ipykernel/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
__version__ = '1.0.0'

import sys
from pathlib import Path

VENV_NAMES = [
'.venv',
'venv',
]

ANCHOR = Path(Path.cwd().anchor)


def venv_search(prefix: Path = Path('.')) -> Path:
prefix = prefix.absolute()

found_venvs = []

for venv in VENV_NAMES:
path = prefix / venv / 'bin' / 'python3'
if path.is_file():
found_venvs.append(path.absolute())

if any(found_venvs):
# If there are multiple venvs just return the first one
return found_venvs[0].absolute()
elif prefix == ANCHOR:
return Path(sys.executable).resolve()
else:
return venv_search(prefix=prefix.parent)
62 changes: 62 additions & 0 deletions src/vip_ipykernel/kernelspec.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
import sys

import ipykernel.kernelspec
from ipykernel.kernelspec import (
KERNEL_NAME,
RESOURCES,
InstallIPythonKernelSpecApp,
get_kernel_dict,
install,
make_ipkernel_cmd,
)


def make_vip_ipkernel_cmd(
mod='ipykernel_launcher', executable=None, extra_arguments=None, **kw
):
"""Build Popen command list for launching an ViP-IPython kernel.

Parameters
----------
mod : str, optional (default 'ipykernel_launcher')
A string of an IPython module whose __main__ starts an IPython kernel

executable : str, optional (default sys.executable)
The Python executable to use for the kernel process.

extra_arguments : list, optional
A list of extra arguments to pass when executing the launch code.

Returns
-------

A Popen command list
"""

# Copyright (c) IPython Development Team.
# Distributed under the terms of the Modified BSD License.

if executable is None:
executable = sys.executable
extra_arguments = extra_arguments or []
# When installing the ViP IPykernel, the first `-m` module call points to
# our `vip_ipykernel_launcher`, and the second module call points to the
# desired ipykernel launcher module
arguments = [
executable,
'-m',
'vip_ipykernel_launcher',
'-m',
mod,
'-f',
'{connection_file}',
]
arguments.extend(extra_arguments)

return arguments


ipykernel.kernelspec.make_ipkernel_cmd = make_vip_ipkernel_cmd

if __name__ == '__main__':
InstallIPythonKernelSpecApp.launch_instance()
Loading