Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speedup import time with lazy imports #987

Merged
merged 6 commits into from
Dec 5, 2024
Merged

Speedup import time with lazy imports #987

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
f94cd67
Initial commit
mataton Nov 27, 2024
1355492
Implement simple solution
mataton Dec 3, 2024
c6a72e0
Remove comments and spaces
mataton Dec 3, 2024
2a297f5
Fix linting
mataton Dec 3, 2024
1a57b11
Merge branch 'master' into lazy
mataton Dec 3, 2024
b903c7e
Update Changelog
mataton Dec 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions ChangeLog.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ BIOM-Format ChangeLog
biom-2.1.16-dev
---------------

Performance improvements:

* Decreased execution time of `import biom` by half with lazy imports. See PR[#987](https://github.com/biocore/biom-format/pull/987)

Maintenance:

* Python 3.7 and 3.8 removed from CI as they are [end-of-life](https://devguide.python.org/versions/). Python 3.13 added to CI. See PR[#986](https://github.com/biocore/biom-format/pull/986).
Expand Down
8 changes: 5 additions & 3 deletions biom/table.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -172,8 +172,6 @@
# -----------------------------------------------------------------------------

import numpy as np
import scipy.stats
import h5py
from copy import deepcopy
from datetime import datetime
from json import dumps as _json_dumps, JSONEncoder
Expand All @@ -184,7 +182,6 @@
from numpy import ndarray, asarray, zeros, newaxis
from scipy.sparse import (coo_matrix, csc_matrix, csr_matrix, isspmatrix,
vstack, hstack, dok_matrix)
import pandas as pd
import re
from biom.exception import (TableException, UnknownAxisError, UnknownIDError,
DisjointIDError)
Expand Down Expand Up @@ -3234,6 +3231,8 @@ def rankdata(self, axis='sample', inplace=True, method='average'):
o4 1.0 4.0 1.0

"""
import scipy.stats

def f(val, id_, _):
return scipy.stats.rankdata(val, method=method)
return self.transform(f, axis=axis, inplace=inplace)
Expand Down Expand Up @@ -4108,6 +4107,7 @@ def from_hdf5(cls, h5grp, ids=None, axis='sample', parse_fs=None,
>>> t = Table.from_hdf5(f, ids=["GG_OTU_1"],
... axis='observation') # doctest: +SKIP
"""
import h5py
if not isinstance(h5grp, (h5py.Group, h5py.File)):
raise ValueError("h5grp does not appear to be an HDF5 file or "
"group")
Expand Down Expand Up @@ -4341,6 +4341,7 @@ def to_dataframe(self, dense=False):
index = self.ids(axis='observation')
columns = self.ids()

import pandas as pd
if dense:
mat = self.matrix_data.toarray()
constructor = pd.DataFrame
Expand Down Expand Up @@ -4442,6 +4443,7 @@ def metadata_to_dataframe(self, axis):
O1 Bacteria Firmicutes
O2 Bacteria Bacteroidetes
"""
import pandas as pd
md = self.metadata(axis=axis)
if md is None:
raise KeyError("%s does not have metadata" % axis)
Expand Down
Loading