Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

transform and models #52

Merged
merged 168 commits into from
Dec 12, 2022
Merged
Show file tree
Hide file tree
Changes from 112 commits
Commits
Show all changes
168 commits
Select commit Hold shift + click to select a range
cb4ddf6
add io test
giovp Aug 17, 2022
0981821
io for transforms, fixed bugs when testing on the merfish dataset
LucaMarconato Aug 17, 2022
707f047
merged with main
LucaMarconato Aug 17, 2022
b7cb647
bugfix
LucaMarconato Aug 17, 2022
561d252
fixed tests
LucaMarconato Aug 17, 2022
b10419c
Merge branch 'tests/io' into feature/transform
LucaMarconato Aug 17, 2022
31525d0
add io test (#21)
giovp Aug 18, 2022
140198f
workaround against bug with backed dask arrays
LucaMarconato Aug 22, 2022
5764882
Merge branch 'feature/transform' of https://github.com/scverse/spatia…
LucaMarconato Aug 22, 2022
7ece121
woring on a bug with io; fixed repr for points
LucaMarconato Sep 1, 2022
ce23cb7
quick code changes for to_zarr bug report
LucaMarconato Sep 5, 2022
67c1d7a
patched tests after discussion on the to_zarr bug
LucaMarconato Sep 5, 2022
6221fa2
more io tests, fixed related bugs
LucaMarconato Sep 28, 2022
f36ed27
first io support for polygons
LucaMarconato Sep 29, 2022
087600d
fix precommits
giovp Sep 30, 2022
14c4cd0
fix flake8
giovp Sep 30, 2022
98cd743
wip implementation of the ngff transform
LucaMarconato Sep 30, 2022
f8517d7
Merge branch 'feature/transform' of https://github.com/scverse/spatia…
LucaMarconato Sep 30, 2022
ba85569
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 30, 2022
3db58b7
wip transformations, tests pass but io not compatible
LucaMarconato Oct 2, 2022
b694423
Merge branch 'feature/transform' of https://github.com/scverse/spatia…
LucaMarconato Oct 2, 2022
db61de8
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 2, 2022
3a83b21
wip write for new transforms specs
LucaMarconato Oct 3, 2022
569abdd
Merge branch 'feature/transform' of https://github.com/scverse/spatia…
LucaMarconato Oct 3, 2022
018f290
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 3, 2022
833bb5c
fstring in format.py
LucaMarconato Oct 3, 2022
ccfe5e9
Merge branch 'feature/transform' of https://github.com/scverse/spatia…
LucaMarconato Oct 3, 2022
8460500
fix in format.py
LucaMarconato Oct 3, 2022
f00f7fd
io working for the new transform specs (no coord spaces yet)
LucaMarconato Oct 3, 2022
983b638
fix in affine
LucaMarconato Oct 3, 2022
26450a7
init
giovp Oct 12, 2022
e8b5268
init
giovp Oct 12, 2022
641e654
merge
giovp Oct 12, 2022
12c2b0b
add 3D and 2D
giovp Oct 12, 2022
45d3f27
update pandera pandas dataframe
giovp Oct 13, 2022
72de782
add polygons
giovp Oct 13, 2022
6b63e9a
Merge branch 'main' into feature/transform
giovp Oct 22, 2022
cf3eeb3
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 22, 2022
d67779d
Merge branch 'feature/transform' into feature/schema
giovp Oct 22, 2022
8293dfb
remove comment
giovp Oct 22, 2022
cbe4607
io for multiscale image
giovp Oct 23, 2022
38646cf
start unpacking pre-commit
giovp Oct 23, 2022
6de8b14
wip transformations
LucaMarconato Oct 27, 2022
8a102c8
workflow yml
LucaMarconato Oct 27, 2022
9eee828
improve models
giovp Oct 27, 2022
9a6f502
fix pylance
giovp Oct 27, 2022
14ae46f
update pre-commit
giovp Oct 27, 2022
ea5b2b0
fix dispatch
giovp Oct 28, 2022
962b7b5
renaming
LucaMarconato Nov 2, 2022
a4a5261
Merge branch 'main' into feature/transform
LucaMarconato Nov 2, 2022
1a261cb
Merge branch 'feature/transform' of https://github.com/scverse/spatia…
LucaMarconato Nov 2, 2022
10411a0
Merge branch 'main' into feature/transform
LucaMarconato Nov 2, 2022
6b22a3e
wip refactoring transformations
LucaMarconato Nov 2, 2022
17eb545
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 2, 2022
9d220c2
merge
giovp Nov 2, 2022
39bbfe4
fixes validation
giovp Nov 2, 2022
c342faf
fix validation
giovp Nov 2, 2022
63ca2b7
refactored transformations, started adding more solid tests
LucaMarconato Nov 2, 2022
30dba39
Merge branch 'feature/transform' of https://github.com/scverse/spatia…
LucaMarconato Nov 2, 2022
db07d35
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 2, 2022
9ea586d
passing tests for identity, map_axis, translation, scale, affine, rot…
LucaMarconato Nov 7, 2022
3ddb939
fix
LucaMarconato Nov 7, 2022
9243b09
add tests for multiscale
giovp Nov 8, 2022
b2c69cb
merge
giovp Nov 8, 2022
9bac0bc
update tests with labels
giovp Nov 8, 2022
8c40133
before changing the validation logic in Sequence
LucaMarconato Nov 8, 2022
d613c09
tests passing for sequence, bydimension
LucaMarconato Nov 8, 2022
057f88a
fixed precommit
LucaMarconato Nov 8, 2022
c602783
fixed tests
LucaMarconato Nov 8, 2022
46f70d0
add schema for polygons
giovp Nov 12, 2022
6d8f665
Merge branch 'feature/transform' into feature/schema
giovp Nov 12, 2022
0e58bf4
add constants
giovp Nov 12, 2022
5715dae
add transform to validate
giovp Nov 12, 2022
0ebdd3f
remove coordinate transforms and add writer for polygons
giovp Nov 12, 2022
5b9fc42
add roundtrip test for polygons
giovp Nov 12, 2022
b229550
update installation
giovp Nov 13, 2022
480d9b7
general fixes
giovp Nov 13, 2022
581a715
add shapes
giovp Nov 13, 2022
669ba7a
add io for shapes
giovp Nov 13, 2022
e2678b2
add tests shapes
giovp Nov 13, 2022
e9c9564
logging
giovp Nov 14, 2022
23fdf4c
add validate table to spatialdata
giovp Nov 14, 2022
46f81fa
Merge branch 'main' into feature/schema
giovp Nov 14, 2022
ff6185d
add write
giovp Nov 14, 2022
cc9a533
fix validate
giovp Nov 14, 2022
34ede2c
add tests for tables
giovp Nov 15, 2022
07dc775
fix
LucaMarconato Nov 19, 2022
7bbd1e0
refactor splitting validate and parse
giovp Nov 20, 2022
1f60aad
add points and fix io metadata
giovp Nov 20, 2022
0d24153
Merge branch 'feature/schema' into feature/transform_and_schema
LucaMarconato Nov 25, 2022
f8012ee
Merge branch 'feature/schema' into feature/transform_and_schema
LucaMarconato Nov 25, 2022
8e8053b
update
giovp Nov 25, 2022
38578c2
fixing the merge
LucaMarconato Nov 25, 2022
a15fc0a
update
giovp Nov 25, 2022
7a5ffa2
fix multipolygon
giovp Nov 25, 2022
17e7f1c
Merge branch 'feature/transform' into feature/schema
giovp Nov 25, 2022
25cb52c
Merge branch 'feature/schema' into feature/transform_and_schema
giovp Nov 29, 2022
4a134fe
cleanup and start adding tests
giovp Nov 29, 2022
e8ddb3a
add tests
giovp Nov 29, 2022
745a429
update tests and requirements
giovp Nov 29, 2022
2bb48ca
update req
giovp Nov 29, 2022
b4bd4c6
add pygeos
giovp Nov 29, 2022
2628922
fix requirements
giovp Nov 29, 2022
1657169
fix shapely installation
kevinyamauchi Nov 29, 2022
ae95cac
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 29, 2022
b6edf52
fix commas
kevinyamauchi Nov 29, 2022
eb674d1
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 29, 2022
d497799
add rich dependency
kevinyamauchi Nov 29, 2022
e8a9484
reviewed giovp pr
LucaMarconato Nov 29, 2022
c95c69c
Merge branch 'feature/transform_and_schema' of https://github.com/scv…
LucaMarconato Nov 30, 2022
0d3e338
general bugfix. In details:
LucaMarconato Nov 30, 2022
f21065a
bug fixes, mainly with polygons. Detailed description: better io with
LucaMarconato Nov 30, 2022
9a81caa
minor fixes. In details: added get_transform, set_transform helper
LucaMarconato Dec 1, 2022
fa658df
assigning default coordinate system upon parsing; improved parser
LucaMarconato Dec 1, 2022
f1b17a1
make codecov work
giovp Dec 1, 2022
1ec2125
try again to run codecov
giovp Dec 1, 2022
484a79b
revert
giovp Dec 2, 2022
416474e
update job
giovp Dec 2, 2022
ca2791e
add token
giovp Dec 2, 2022
0713f91
update codecov.yml
giovp Dec 2, 2022
a551a81
try fix
giovp Dec 2, 2022
ecafff7
specify xml format
giovp Dec 2, 2022
e502f42
fix polygons and add version
giovp Dec 4, 2022
178d24c
refactor write shapes and points and add versioning
giovp Dec 4, 2022
1790ba5
re add validate_coordinate_transform removed by mistake
giovp Dec 4, 2022
c77c93b
read and write cleanup
giovp Dec 4, 2022
d6beadd
cleanup and tests
giovp Dec 4, 2022
5dee787
add format for tables
giovp Dec 4, 2022
4b94ae2
fixes in performance for points and anndata (but will be replaced by …
LucaMarconato Dec 6, 2022
181a8c9
fixed tests and mypy
LucaMarconato Dec 6, 2022
3a1ef8f
added tqdm dependency to fix tests
LucaMarconato Dec 6, 2022
1ee2f4d
removed functions: (from|to)_json, get_coordinate_transformation_from…
LucaMarconato Dec 7, 2022
f32a254
removed legacy write() from _spatialdata.py
LucaMarconato Dec 7, 2022
7cac887
Update spatialdata/_core/transformations.py
LucaMarconato Dec 7, 2022
480fcca
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Dec 7, 2022
6ecae16
fix
LucaMarconato Dec 7, 2022
07bdc51
Merge branch 'feature/transform_and_schema' of https://github.com/scv…
LucaMarconato Dec 7, 2022
2a5fcfc
isaac refactoring of transformations; mypy to fix
LucaMarconato Dec 7, 2022
191d1fd
cleaup parser for raster
giovp Dec 7, 2022
af1df61
init models
giovp Dec 7, 2022
f0957ed
fixed mypy (removed Mapping, using Dict instead)
LucaMarconato Dec 8, 2022
4f4ddc0
Merge branch 'feature/transform_and_schema' of https://github.com/scv…
LucaMarconato Dec 8, 2022
0d7fc26
Apply suggestions from code review
LucaMarconato Dec 8, 2022
dc670f0
implemented review comments
LucaMarconato Dec 8, 2022
27af6d6
Merge branch 'feature/transform_and_schema' of https://github.com/scv…
LucaMarconato Dec 8, 2022
422f56d
Merge branch 'main' into feature/transform_and_schema
giovp Dec 8, 2022
ae1b868
add pre-commit and fix docstrings
giovp Dec 8, 2022
c6b12b2
fix
LucaMarconato Dec 8, 2022
1695b02
Merge branch 'feature/transform_and_schema' of https://github.com/scv…
LucaMarconato Dec 8, 2022
f2d4955
fix autogeneration
giovp Dec 8, 2022
7932faf
using geopandas for points io
LucaMarconato Dec 8, 2022
afc987d
Merge branch 'feature/transform_and_schema' of https://github.com/scv…
LucaMarconato Dec 8, 2022
ea06156
fix models
giovp Dec 8, 2022
c81008a
add tqdm
giovp Dec 9, 2022
a236e77
fix in test_models
LucaMarconato Dec 9, 2022
b5e73f3
fix polygon model
giovp Dec 10, 2022
5ed1a21
test models points
giovp Dec 10, 2022
e6671a2
add test shapes
giovp Dec 10, 2022
f026964
test table model
giovp Dec 10, 2022
fa55f59
fix table model
giovp Dec 10, 2022
533fb7a
try codecov one more time
giovp Dec 10, 2022
0ef1aa5
install codecov
giovp Dec 10, 2022
44edabd
Merge branch 'main' into feature/transform_and_schema
giovp Dec 12, 2022
efda324
update docs
giovp Dec 12, 2022
1bf287d
add register
giovp Dec 12, 2022
d5e7a60
Merge branch 'fix/docs' into feature/transform_and_schema
giovp Dec 12, 2022
32dd480
suppress warning
giovp Dec 12, 2022
933ea31
fix docs
giovp Dec 12, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,6 @@ __pycache__/
# data
spatialdata-sandbox
*.zarr/

# temp files
temp/
12 changes: 6 additions & 6 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ repos:
hooks:
- id: black
- repo: https://github.com/pre-commit/mirrors-prettier
rev: v2.7.1
rev: v3.0.0-alpha.4
hooks:
- id: prettier
- repo: https://github.com/asottile/blacken-docs
Expand All @@ -23,7 +23,7 @@ repos:
hooks:
- id: isort
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v0.982
rev: v0.991
hooks:
- id: mypy
additional_dependencies: [numpy==1.22.0, types-requests]
Expand All @@ -40,7 +40,7 @@ repos:
- flake8-bugbear
- flake8-blind-except
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.3.0
rev: v4.4.0
hooks:
- id: detect-private-key
- id: check-ast
Expand All @@ -50,7 +50,7 @@ repos:
- id: trailing-whitespace
- id: check-case-conflict
- repo: https://github.com/PyCQA/autoflake
rev: v1.7.1
rev: v2.0.0
hooks:
- id: autoflake
args:
Expand All @@ -59,7 +59,7 @@ repos:
- --remove-unused-variable
- --ignore-init-module-imports
- repo: https://github.com/PyCQA/flake8
rev: 5.0.4
rev: 6.0.0
hooks:
- id: flake8
additional_dependencies:
Expand All @@ -70,7 +70,7 @@ repos:
- flake8-bugbear
- flake8-blind-except
- repo: https://github.com/asottile/pyupgrade
rev: v3.0.0
rev: v3.2.2
hooks:
- id: pyupgrade
args: [--py3-plus, --py38-plus, --keep-runtime-typing]
14 changes: 14 additions & 0 deletions docs/design_doc.md
Original file line number Diff line number Diff line change
Expand Up @@ -280,6 +280,20 @@ Transforms map Elements between coordinate systems. Each `Transform` object must

Here is a short list of examples of the elements used to represent some spatial omics datasets. Real world will be available as notebooks [in this repository](https://github.com/scverse/spatialdata-notebooks), furthermore, some draft [implementations are available here](https://github.com/giovp/spatialdata-sandbox).

API

```python
import spatialdata as sd
from spatialdata import SpatialData

sdata = SpatialData(...)
points = sd.transform(sdata.points["image1"], tgt="tgt_space")
sdata = sd.transform(sdata, tgt="tgt_space")
```

The transfromation object should not have a method to apply itself to an element.
`SpatialData` can have a `transform` method, that can be applied to either a `SpatialData` object or an element.

#### Layout of a SpatialData object

The layout of some common datasets.
Expand Down
15 changes: 14 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,20 @@ version = "0.0.1.dev0"
requires-python = ">=3.8"
license = {file = "LICENSE"}
readme = "README.md"
dependencies = ["anndata","numpy","xarray","zarr","ome_zarr"]
dependencies = [
"anndata",
"numpy",
"xarray",
"zarr",
"ome_zarr",
"spatial_image",
"multiscale_spatial_image",
"xarray-schema",
"geopandas",
"pandera",
"shapely==2.0rc1",
"rich",
]

[project.optional-dependencies]
dev = [
Expand Down
48 changes: 46 additions & 2 deletions spatialdata/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,49 @@

__version__ = version("spatialdata")

from spatialdata._core.spatialdata import SpatialData
from spatialdata._core.transform import Transform
__all__ = [
"SpatialData",
"Identity",
# "MapIndex",
"MapAxis",
"Translation",
"Scale",
"Affine",
"Rotation",
"Sequence",
# "Displacements",
# "Coordinates",
# "VectorField",
# "InverseOf",
# "Bijection",
# "ByDimension",
"Image3DModel",
"Image2DModel",
"Labels2DModel",
"Labels3DModel",
"PointsModel",
"PolygonsModel",
"ShapesModel",
"TableModel",
]

from spatialdata._core._spatialdata import SpatialData
from spatialdata._core.models import (
Image2DModel,
Image3DModel,
Labels2DModel,
Labels3DModel,
PointsModel,
PolygonsModel,
ShapesModel,
TableModel,
)
from spatialdata._core.transformations import ( # Bijection,; ByDimension,; Coordinates,; Displacements,; InverseOf,; MapIndex,; VectorField,
Affine,
Identity,
MapAxis,
Rotation,
Scale,
Sequence,
Translation,
)
15 changes: 15 additions & 0 deletions spatialdata/_constants/_constants.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
from enum import unique

from spatialdata._constants._enum import ModeEnum


@unique
class RasterType(ModeEnum):
IMAGE = "Image"
LABEL = "Label"


@unique
class ShapeType(ModeEnum):
CIRCLE = "Circle"
SQUARE = "Square"
2 changes: 1 addition & 1 deletion spatialdata/_core/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
from spatialdata._core.spatialdata import SpatialData
from spatialdata._core._spatialdata import SpatialData
Loading