Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release v3.0.4 #806

Merged
merged 122 commits into from
Nov 28, 2023
Merged
Changes from 1 commit
Commits
Show all changes
122 commits
Select commit Hold shift + click to select a range
ee3e4fe
Reenable develop
evhub Jul 29, 2023
200dede
Remove --history-file
evhub Jul 29, 2023
24bcd15
Add --incremental
evhub Jul 31, 2023
27a15c7
Fix --incremental
evhub Aug 1, 2023
f7e4fbc
Fix incremental test
evhub Aug 1, 2023
1c4a14f
Slightly improve incremental mode
evhub Aug 2, 2023
aa88407
Further fix --incremental
evhub Aug 3, 2023
779f9dc
Attempt to fix tests
evhub Aug 3, 2023
22bcb28
Fix tests, incremental log message
evhub Aug 3, 2023
7d95dc1
Fix unused import errors
evhub Aug 6, 2023
6473ac8
Add missing import test
evhub Aug 6, 2023
e402c58
Make where use temp vars
evhub Sep 2, 2023
479c6aa
Fix where statements
evhub Sep 2, 2023
00ef984
Add numpy install extra
evhub Sep 2, 2023
f4e3ebd
Add attritemgetter partials
evhub Sep 23, 2023
af257bc
Fix docs, tests
evhub Sep 23, 2023
f55e552
Fix jupyter console
evhub Sep 23, 2023
0c1cada
Bump IPython
evhub Sep 27, 2023
440334b
Backport ExceptionGroup
evhub Oct 10, 2023
b1a5f43
Improve reqs, tests
evhub Oct 10, 2023
87a5325
Improve reqs, tests
evhub Oct 10, 2023
021ad34
Fix reqs
evhub Oct 11, 2023
e819bed
Bump reqs
evhub Oct 11, 2023
595487a
Fix py37
evhub Oct 11, 2023
1967d14
Fix py<=38
evhub Oct 12, 2023
40ee558
Fix py37, docs
evhub Oct 13, 2023
f0b4a60
Fix mypy errors
evhub Oct 13, 2023
ef54243
Fix lots of tests
evhub Oct 15, 2023
5b51c4c
Update pre-commit
evhub Oct 15, 2023
c2165d0
Fix more tests
evhub Oct 15, 2023
b8665d6
Increase stack
evhub Oct 17, 2023
29a8345
3.12 prep
evhub Oct 22, 2023
f34759c
Fix typo in pure-Python example for scan()
Starwort Oct 22, 2023
26dbdec
Fix typo in pure-Python example for scan() (#790)
evhub Oct 22, 2023
66e5254
Rename process/thread maps
evhub Oct 28, 2023
586dd5e
Add mapreduce, improve collectby, process/thread maps
evhub Oct 28, 2023
2e03ba0
recursive_iterator to recursive_generator
evhub Oct 28, 2023
d034af3
Fix readthedocs
evhub Oct 28, 2023
746bf58
Improve header
evhub Oct 29, 2023
06e1586
Add process/thread versions of collectby/mapreduce
evhub Oct 30, 2023
4e11b6d
Remove docstring
evhub Oct 30, 2023
57c0386
Improve docs
evhub Oct 30, 2023
f24588e
Minor doc cleanup
evhub Oct 31, 2023
5baa6fc
Improve mapreduce
evhub Oct 31, 2023
959ce34
Improve mapreduce/collectby
evhub Oct 31, 2023
0c7a5af
Add Expected.handle
evhub Oct 31, 2023
c13fa4c
Add Expected.expect_error
evhub Nov 2, 2023
6c9c113
Fix typing
evhub Nov 3, 2023
061e67f
Fix no wrap test
evhub Nov 3, 2023
77daeb9
Add pyspy
evhub Nov 3, 2023
fb04ee6
Improve exception formatting
evhub Nov 4, 2023
61bd787
Improve profiling
evhub Nov 4, 2023
f1759b1
Add async_map
evhub Nov 4, 2023
a99a76d
Fix tests
evhub Nov 4, 2023
c3b0273
Improve partials
evhub Nov 4, 2023
fd5a17f
Reduce jobs
evhub Nov 4, 2023
e093c9b
Improve tests perf
evhub Nov 4, 2023
f5eb7fd
Disable jobs on pypy
evhub Nov 5, 2023
827c06c
Improve header
evhub Nov 5, 2023
d1b1cde
Add find_packages, improve override
evhub Nov 5, 2023
484965f
Fix syntax
evhub Nov 5, 2023
cedb148
Prevent unwanted multiprocessing
evhub Nov 5, 2023
c83609f
Clean up docs
evhub Nov 5, 2023
352b942
Fix test errors
evhub Nov 5, 2023
27b8c7a
Fix more tests
evhub Nov 6, 2023
143fa61
Improve pyparsing usage
evhub Nov 6, 2023
0e6f193
Add more profiling
evhub Nov 6, 2023
b001630
Clean up profiling
evhub Nov 6, 2023
78542b8
Improve --profile
evhub Nov 7, 2023
f496a4d
Further fix profiling
evhub Nov 7, 2023
42281fe
More profiling improvements
evhub Nov 8, 2023
e5bfd25
Add adaptive parsing support
evhub Nov 8, 2023
dd0ba2a
More adaptive improvements
evhub Nov 9, 2023
1cfbe06
Use fast grammar methods
evhub Nov 10, 2023
607c3b7
Fix fast parse methods
evhub Nov 10, 2023
9d2acaa
Actually enable fast parse methods
evhub Nov 10, 2023
b8d1581
Improve exceptions
evhub Nov 11, 2023
ec1c1dd
Lots of optimizations
evhub Nov 12, 2023
bb83c9c
Improve --incremental
evhub Nov 12, 2023
8ce71c8
Further improve --incremental
evhub Nov 12, 2023
0b2db7e
Always use cache
evhub Nov 12, 2023
5753c2b
More performance tuning
evhub Nov 13, 2023
8679c8c
Robustify os operations
evhub Nov 13, 2023
807eb9f
Improve any_of
evhub Nov 13, 2023
4961812
Improve incremental
evhub Nov 14, 2023
f9e3b4a
Use newest cpyparsing
evhub Nov 14, 2023
e1b9f80
Improve grammar
evhub Nov 14, 2023
a079bf0
Improve incremental
evhub Nov 15, 2023
11320e8
Update docs
evhub Nov 15, 2023
592bc78
Fix incremental parsing
evhub Nov 17, 2023
cd7d9d8
Improve incremental parsing
evhub Nov 17, 2023
71c2659
Fix dependencies
evhub Nov 17, 2023
a14a829
Fix tests
evhub Nov 17, 2023
459bf34
More robustification
evhub Nov 17, 2023
0ab1785
Fix tests
evhub Nov 20, 2023
3c28785
Fix py2 tests
evhub Nov 20, 2023
0110d61
Improve command running
evhub Nov 20, 2023
ea531c2
Improve call_output
evhub Nov 22, 2023
164fb56
Enable adaptive
evhub Nov 23, 2023
57eddf7
Fix errors
evhub Nov 23, 2023
a3d33ec
Fix more issues
evhub Nov 24, 2023
f797edf
Prepare for release
evhub Nov 24, 2023
417ce32
Fix syntax for py2
evhub Nov 24, 2023
ac16733
Update to require new cPyparsing
evhub Nov 24, 2023
68cdcff
Clean up docs
evhub Nov 24, 2023
58e2aba
Fix stdin reading
evhub Nov 24, 2023
1983871
More fixes
evhub Nov 25, 2023
6f17751
Add test
evhub Nov 25, 2023
db22bcd
Fix py2
evhub Nov 25, 2023
895a0f1
Fix fancy call_output
evhub Nov 25, 2023
3651477
Further fix py2
evhub Nov 25, 2023
7a1169f
Disable fancy call_output
evhub Nov 25, 2023
4c50275
Fix py2 pickling
evhub Nov 26, 2023
84cc54f
Fix extras test
evhub Nov 26, 2023
aa6c431
Fix parsing numbers
evhub Nov 26, 2023
1721293
Further fix number parsing
evhub Nov 26, 2023
055b1e5
Fix failing tests
evhub Nov 26, 2023
9b0ee55
Fix extras tests
evhub Nov 27, 2023
c14014a
Prepare for v3.0.4
evhub Nov 27, 2023
019348d
Fix test cache management
evhub Nov 27, 2023
dfd40bd
Fix typos
evhub Nov 27, 2023
d2191ee
Fix test
evhub Nov 28, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add find_packages, improve override
Resolves   #798, #800.
  • Loading branch information
evhub committed Nov 5, 2023

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
commit d1b1cde1dd6736d206514e0dc1ddb0e29d6c5df2
55 changes: 42 additions & 13 deletions DOCS.md
Original file line number Diff line number Diff line change
@@ -3106,7 +3106,7 @@ def fib(n):

**override**(_func_)

Coconut provides the `@override` decorator to allow declaring a method definition in a subclass as an override of some parent class method. When `@override` is used on a method, if a method of the same name does not exist on some parent class, the class definition will raise a `RuntimeError`.
Coconut provides the `@override` decorator to allow declaring a method definition in a subclass as an override of some parent class method. When `@override` is used on a method, if a method of the same name does not exist on some parent class, the class definition will raise a `RuntimeError`. `@override` works with other decorators such as `@classmethod` and `@staticmethod`, but only if `@override` is the outer-most decorator.

Additionally, `override` will present to type checkers as [`typing_extensions.override`](https://pypi.org/project/typing-extensions/).

@@ -4672,6 +4672,12 @@ Executes the given _args_ as if they were fed to `coconut` on the command-line,

Has the same effect of setting the command-line flags on the given _state_ object as `setup` (with the global `state` object used when _state_ is `False`).

#### `cmd_sys`

**coconut.api.cmd_sys**(_args_=`None`, *, _argv_=`None`, _interact_=`False`, _default\_target_=`"sys"`, _state_=`False`)

Same as `coconut.api.cmd` but _default\_target_ is `"sys"` rather than `None` (universal).

#### `coconut_exec`

**coconut.api.coconut_exec**(_expression_, _globals_=`None`, _locals_=`None`, _state_=`False`, _keep\_internal\_state_=`None`)
@@ -4684,18 +4690,6 @@ Version of [`exec`](https://docs.python.org/3/library/functions.html#exec) which

Version of [`eval`](https://docs.python.org/3/library/functions.html#eval) which can evaluate Coconut code.

#### `version`

**coconut.api.version**(**[**_which_**]**)

Retrieves a string containing information about the Coconut version. The optional argument _which_ is the type of version information desired. Possible values of _which_ are:

- `"num"`: the numerical version (the default)
- `"name"`: the version codename
- `"spec"`: the numerical version with the codename attached
- `"tag"`: the version tag used in GitHub and documentation URLs
- `"-v"`: the full string printed by `coconut -v`

#### `auto_compilation`

**coconut.api.auto_compilation**(_on_=`True`, _args_=`None`, _use\_cache\_dir_=`None`)
@@ -4712,6 +4706,41 @@ If _use\_cache\_dir_ is passed, it will turn on or off the usage of a `__coconut

Switches the [`breakpoint` built-in](https://www.python.org/dev/peps/pep-0553/) which Coconut makes universally available to use [`coconut.embed`](#coconut-embed) instead of [`pdb.set_trace`](https://docs.python.org/3/library/pdb.html#pdb.set_trace) (or undoes that switch if `on=False`). This function is called automatically when `coconut.api` is imported.

#### `find_and_compile_packages`

**coconut.api.find_and_compile_packages**(_where_=`"."`, _exclude_=`()`, _include_=`("*",)`)

Behaves similarly to [`setuptools.find_packages`](https://setuptools.pypa.io/en/latest/userguide/quickstart.html#package-discovery) except that it finds Coconut packages rather than Python packages, and compiles any Coconut packages that it finds in-place.

Note that if you want to use `find_and_compile_packages` in your `setup.py`, you'll need to include `coconut` as a [build-time dependency in your `pyproject.toml`](https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/#build-time-dependencies).

##### Example

```coconut_python
# if you put this in your setup.py, your Coconut package will be compiled in-place whenever it is installed

from setuptools import setup
from coconut.api import find_and_compile_packages

setup(
name=...,
version=...,
packages=find_and_compile_packages(),
)
```

#### `version`

**coconut.api.version**(**[**_which_**]**)

Retrieves a string containing information about the Coconut version. The optional argument _which_ is the type of version information desired. Possible values of _which_ are:

- `"num"`: the numerical version (the default)
- `"name"`: the version codename
- `"spec"`: the numerical version with the codename attached
- `"tag"`: the version tag used in GitHub and documentation URLs
- `"-v"`: the full string printed by `coconut -v`

#### `CoconutException`

If an error is encountered in a api function, a `CoconutException` instance may be raised. `coconut.api.CoconutException` is provided to allow catching such errors.
47 changes: 43 additions & 4 deletions coconut/api.py
Original file line number Diff line number Diff line change
@@ -23,15 +23,17 @@
import os.path
import codecs
from functools import partial
from setuptools import PackageFinder
try:
from encodings import utf_8
except ImportError:
utf_8 = None

from coconut.root import _coconut_exec
from coconut.util import override
from coconut.integrations import embed
from coconut.exceptions import CoconutException
from coconut.command import Command
from coconut.command.command import Command
from coconut.command.cli import cli_version
from coconut.command.util import proc_run_args
from coconut.compiler import Compiler
@@ -42,7 +44,6 @@
coconut_kernel_kwargs,
default_use_cache_dir,
coconut_cache_dir,
coconut_run_kwargs,
)

# -----------------------------------------------------------------------------------------------------------------------
@@ -68,9 +69,16 @@ def get_state(state=None):
def cmd(cmd_args, **kwargs):
"""Process command-line arguments."""
state = kwargs.pop("state", False)
cmd_func = kwargs.pop("_cmd_func", "cmd")
if isinstance(cmd_args, (str, bytes)):
cmd_args = cmd_args.split()
return get_state(state).cmd(cmd_args, **kwargs)
return getattr(get_state(state), cmd_func)(cmd_args, **kwargs)


def cmd_sys(*args, **kwargs):
"""Same as api.cmd() but defaults to --target sys."""
kwargs["_cmd_func"] = "cmd_sys"
return cmd(*args, **kwargs)


VERSIONS = {
@@ -214,7 +222,7 @@ def cmd(self, *args):
"""Run the Coconut compiler with the given args."""
if self.command is None:
self.command = Command()
return self.command.cmd(list(args) + self.args, interact=False, **coconut_run_kwargs)
return self.command.cmd_sys(list(args) + self.args, interact=False)

def compile(self, path, package):
"""Compile a path to a file or package."""
@@ -315,6 +323,7 @@ def compile_coconut(cls, source):
cls.coconut_compiler = Compiler(**coconut_kernel_kwargs)
return cls.coconut_compiler.parse_sys(source)

@override
@classmethod
def decode(cls, input_bytes, errors="strict"):
"""Decode and compile the given Coconut source bytes."""
@@ -347,3 +356,33 @@ def get_coconut_encoding(encoding="coconut"):


codecs.register(get_coconut_encoding)


# -----------------------------------------------------------------------------------------------------------------------
# SETUPTOOLS:
# -----------------------------------------------------------------------------------------------------------------------

class CoconutPackageFinder(PackageFinder, object):

_coconut_command = None

@classmethod
def _coconut_compile(cls, path):
"""Run the Coconut compiler with the given args."""
if cls._coconut_command is None:
cls._coconut_command = Command()
return cls._coconut_command.cmd_sys([path], interact=False)

@override
@classmethod
def _looks_like_package(cls, path, _package_name):
is_coconut_package = any(
os.path.isfile(os.path.join(path, "__init__" + ext))
for ext in code_exts
)
if is_coconut_package:
cls._coconut_compile(path)
return is_coconut_package


find_and_compile_packages = CoconutPackageFinder.find
7 changes: 7 additions & 0 deletions coconut/api.pyi
Original file line number Diff line number Diff line change
@@ -21,6 +21,8 @@ from typing import (
Text,
)

from setuptools import find_packages as _find_packages

from coconut.command.command import Command

class CoconutException(Exception):
@@ -50,6 +52,8 @@ def cmd(
"""Process command-line arguments."""
...

cmd_sys = cmd


VERSIONS: Dict[Text, Text] = ...

@@ -150,3 +154,6 @@ def auto_compilation(
def get_coconut_encoding(encoding: Text = ...) -> Any:
"""Get a CodecInfo for the given Coconut encoding."""
...


find_and_compile_packages = _find_packages
10 changes: 8 additions & 2 deletions coconut/command/command.py
Original file line number Diff line number Diff line change
@@ -72,7 +72,7 @@
create_package_retries,
default_use_cache_dir,
coconut_cache_dir,
coconut_run_kwargs,
coconut_sys_kwargs,
interpreter_uses_incremental,
disable_incremental_for_len,
)
@@ -165,10 +165,16 @@ def start(self, run=False):
dest = os.path.join(os.path.dirname(source), coconut_cache_dir)
else:
dest = os.path.join(source, coconut_cache_dir)
self.cmd(args, argv=argv, use_dest=dest, **coconut_run_kwargs)
self.cmd_sys(args, argv=argv, use_dest=dest)
else:
self.cmd()

def cmd_sys(self, *args, **in_kwargs):
"""Same as .cmd(), but uses defaults from coconut_sys_kwargs."""
out_kwargs = coconut_sys_kwargs.copy()
out_kwargs.update(in_kwargs)
return self.cmd(*args, **out_kwargs)

# new external parameters should be updated in api.pyi and DOCS
def cmd(self, args=None, argv=None, interact=True, default_target=None, use_dest=None):
"""Process command-line arguments."""
5 changes: 4 additions & 1 deletion coconut/command/command.pyi
Original file line number Diff line number Diff line change
@@ -15,7 +15,10 @@ Description: MyPy stub file for command.py.
# MAIN:
# -----------------------------------------------------------------------------------------------------------------------

from typing import Callable


class Command:
"""Coconut command-line interface."""
...
cmd: Callable
cmd_sys: Callable
6 changes: 6 additions & 0 deletions coconut/compiler/templates/header.py_template
Original file line number Diff line number Diff line change
@@ -1649,6 +1649,12 @@ class override(_coconut_baseclass):
def __init__(self, func):
self.func = func
def __get__(self, obj, objtype=None):
self_func_get = _coconut.getattr(self.func, "__get__", None)
if self_func_get is not None:
if objtype is None:
return self_func_get(obj)
else:
return self_func_get(obj, objtype)
if obj is None:
return self.func
{return_method_of_self_func}
4 changes: 3 additions & 1 deletion coconut/constants.py
Original file line number Diff line number Diff line change
@@ -669,7 +669,7 @@ def get_path_env_var(env_var, default):
# always use atomic --xxx=yyy rather than --xxx yyy
# and don't include --run, --quiet, or --target as they're added separately
coconut_base_run_args = ("--keep-lines",)
coconut_run_kwargs = dict(default_target="sys") # passed to Command.cmd
coconut_sys_kwargs = dict(default_target="sys") # passed to Command.cmd

default_mypy_args = (
"--pretty",
@@ -902,6 +902,7 @@ def get_path_env_var(env_var, default):
("async_generator", "py35"),
("exceptiongroup", "py37;py<311"),
("anyio", "py36"),
"setuptools",
),
"cpython": (
"cPyparsing",
@@ -1043,6 +1044,7 @@ def get_path_env_var(env_var, default):
# don't upgrade this; it breaks on Python 3.4
("pygments", "py<39"): (2, 3),
# don't upgrade these; they break on Python 2
"setuptools": (44,),
("jupyter-client", "py<35"): (5, 3),
("pywinpty", "py<3;windows"): (0, 5),
("jupyter-console", "py<35"): (5, 2),
3 changes: 1 addition & 2 deletions coconut/integrations.py
Original file line number Diff line number Diff line change
@@ -23,7 +23,6 @@

from coconut.constants import (
coconut_kernel_kwargs,
coconut_run_kwargs,
enabled_xonsh_modes,
interpreter_uses_incremental,
)
@@ -77,7 +76,7 @@ def magic(line, cell=None):
# first line in block is cmd, rest is code
line = line.strip()
if line:
api.cmd(line, state=magic_state, **coconut_run_kwargs)
api.cmd_sys(line, state=magic_state)
code = cell
compiled = api.parse(code, state=magic_state)
except CoconutException:
2 changes: 1 addition & 1 deletion coconut/root.py
Original file line number Diff line number Diff line change
@@ -26,7 +26,7 @@
VERSION = "3.0.3"
VERSION_NAME = None
# False for release, int >= 1 for develop
DEVELOP = 19
DEVELOP = 20
ALPHA = False # for pre releases rather than post releases

assert DEVELOP is False or DEVELOP >= 1, "DEVELOP must be False or an int >= 1"
28 changes: 28 additions & 0 deletions coconut/tests/main_test.py
Original file line number Diff line number Diff line change
@@ -105,6 +105,8 @@
additional_dest = os.path.join(base, "dest", "additional_dest")

src_cache_dir = os.path.join(src, coconut_cache_dir)
cocotest_dir = os.path.join(src, "cocotest")
agnostic_dir = os.path.join(cocotest_dir, "agnostic")

runnable_coco = os.path.join(src, "runnable.coco")
runnable_py = os.path.join(src, "runnable.py")
@@ -472,6 +474,26 @@ def using_coconut(fresh_logger=True, fresh_api=False):
logger.copy_from(saved_logger)


def remove_pys_in(dirpath):
removed_pys = 0
for fname in os.listdir(dirpath):
if fname.endswith(".py"):
rm_path(os.path.join(dirpath, fname))
removed_pys += 1
return removed_pys


@contextmanager
def using_pys_in(dirpath):
"""Remove *.py in dirpath at start and finish."""
remove_pys_in(dirpath)
try:
yield
finally:
removed_pys = remove_pys_in(dirpath)
assert removed_pys > 0, os.listdir(dirpath)


@contextmanager
def using_sys_path(path, prepend=False):
"""Adds a path to sys.path."""
@@ -797,6 +819,12 @@ def test_import_hook(self):
reload(runnable)
assert runnable.success == "<success>"

def test_find_packages(self):
with using_pys_in(agnostic_dir):
with using_coconut():
from coconut.api import find_and_compile_packages
assert find_and_compile_packages(cocotest_dir) == ["agnostic"]

def test_runnable(self):
run_runnable()

2 changes: 2 additions & 0 deletions coconut/tests/src/cocotest/agnostic/suite.coco
Original file line number Diff line number Diff line change
@@ -1066,6 +1066,8 @@ forward 2""") == 900
assert safe_raise_exc(IOError).handle(IOError, const 10).unwrap() == 10 == safe_raise_exc(IOError).expect_error(IOError).result_or(10)
assert pickle_round_trip(ident$(1))() == 1 == pickle_round_trip(ident$(x=?))(1)
assert x_or_y(x=1) == (1, 1) == x_or_y(y=1)
assert DerivedWithMeths().cls_meth()
assert DerivedWithMeths().static_meth()

with process_map.multiple_sequential_calls(): # type: ignore
assert process_map(tuple <.. (|>)$(to_sort), qsorts) |> list == [to_sort |> sorted |> tuple] * len(qsorts)
14 changes: 14 additions & 0 deletions coconut/tests/src/cocotest/agnostic/util.coco
Original file line number Diff line number Diff line change
@@ -881,6 +881,20 @@ class inh_inh_A(inh_A):
@override
def true(self) = False

class BaseWithMeths:
@classmethod
def cls_meth(cls) = False
@staticmethod
def static_meth() = False

class DerivedWithMeths(BaseWithMeths):
@override
@classmethod
def cls_meth(cls) = True
@override
@staticmethod
def static_meth() = True

class MyExc(Exception):
def __init__(self, m):
super().__init__(m)
5 changes: 5 additions & 0 deletions coconut/util.py
Original file line number Diff line number Diff line change
@@ -107,6 +107,11 @@ def __init__(self, func):
self.func = func

def __get__(self, obj, objtype=None):
if hasattr(self.func, "__get__"):
if objtype is None:
return self.func.__get__(obj)
else:
return self.func.__get__(obj, objtype)
if obj is None:
return self.func
if PY2: