Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MissingIDFieldWarning in reads #1306

Open
jorenham opened this issue Jan 13, 2025 · 5 comments
Open

MissingIDFieldWarning in reads #1306

jorenham opened this issue Jan 13, 2025 · 5 comments
Assignees
@mwouts

Comments

@jorenham
Copy link

When I use mkdocs-jupyter to build the Lmo docs, I see many MissingIDFieldWarning's:

INFO    -  Cleaning site directory
INFO    -  Building documentation to directory: /home/joren/Workspace/lmo/site
INFO    -  MissingIDFieldWarning: Cell is missing an id field, this will become a hard error in future nbformat versions. You may want to use `normalize()` on your notebooks before validations (available since nbformat 5.1.4). Previous versions of nbformat are fixing this issue transparently, and will stop doing so in the future.
             File "/home/joren/Workspace/lmo/.venv/lib/python3.13/site-packages/nbformat/validator.py", line 351, in _normalize
               warnings.warn(
             File "/home/joren/Workspace/lmo/.venv/lib/python3.13/site-packages/nbformat/__init__.py", line 96, in
               validate(nb)
INFO    -  MissingIDFieldWarning: Cell is missing an id field, this will become a hard error in future nbformat versions. You may want to use `normalize()` on your notebooks before validations (available since nbformat 5.1.4). Previous versions of nbformat are fixing this issue transparently, and will stop doing so in the future.
             File "/home/joren/Workspace/lmo/.venv/lib/python3.13/site-packages/nbformat/validator.py", line 351, in _normalize
               warnings.warn(
             File "/home/joren/Workspace/lmo/.venv/lib/python3.13/site-packages/nbformat/__init__.py", line 96, in
               validate(nb)
INFO    -  MissingIDFieldWarning: Cell is missing an id field, this will become a hard error in future nbformat versions. You may want to use `normalize()` on your notebooks before validations (available since nbformat 5.1.4). Previous versions of nbformat are fixing this issue transparently, and will stop doing so in the future.
             File "/home/joren/Workspace/lmo/.venv/lib/python3.13/site-packages/nbformat/validator.py", line 351, in _normalize
               warnings.warn(
             File "/home/joren/Workspace/lmo/.venv/lib/python3.13/site-packages/nbformat/__init__.py", line 96, in
               validate(nb)
INFO    -  MissingIDFieldWarning: Cell is missing an id field, this will become a hard error in future nbformat versions. You may want to use `normalize()` on your notebooks before validations (available since nbformat 5.1.4). Previous versions of nbformat are fixing this issue transparently, and will stop doing so in the future.
             File "/home/joren/Workspace/lmo/.venv/lib/python3.13/site-packages/nbformat/validator.py", line 351, in _normalize
               warnings.warn(
             File "/home/joren/Workspace/lmo/.venv/lib/python3.13/site-packages/nbformat/__init__.py", line 96, in
               validate(nb)
INFO    -  MissingIDFieldWarning: Cell is missing an id field, this will become a hard error in future nbformat versions. You may want to use `normalize()` on your notebooks before validations (available since nbformat 5.1.4). Previous versions of nbformat are fixing this issue transparently, and will stop doing so in the future.
             File "/home/joren/Workspace/lmo/.venv/lib/python3.13/site-packages/nbformat/validator.py", line 351, in _normalize
               warnings.warn(
             File "/home/joren/Workspace/lmo/.venv/lib/python3.13/site-packages/nbformat/__init__.py", line 96, in
               validate(nb)
INFO    -  MissingIDFieldWarning: Cell is missing an id field, this will become a hard error in future nbformat versions. You may want to use `normalize()` on your notebooks before validations (available since nbformat 5.1.4). Previous versions of nbformat are fixing this issue transparently, and will stop doing so in the future.
             File "/home/joren/Workspace/lmo/.venv/lib/python3.13/site-packages/nbformat/validator.py", line 351, in _normalize
               warnings.warn(
             File "/home/joren/Workspace/lmo/.venv/lib/python3.13/site-packages/nbformat/__init__.py", line 96, in
               validate(nb)
INFO    -  MissingIDFieldWarning: Cell is missing an id field, this will become a hard error in future nbformat versions. You may want to use `normalize()` on your notebooks before validations (available since nbformat 5.1.4). Previous versions of nbformat are fixing this issue transparently, and will stop doing so in the future.
             File "/home/joren/Workspace/lmo/.venv/lib/python3.13/site-packages/nbformat/validator.py", line 351, in _normalize
               warnings.warn(
             File "/home/joren/Workspace/lmo/.venv/lib/python3.13/site-packages/nbformat/__init__.py", line 96, in
               validate(nb)
INFO    -  MissingIDFieldWarning: Cell is missing an id field, this will become a hard error in future nbformat versions. You may want to use `normalize()` on your notebooks before validations (available since nbformat 5.1.4). Previous versions of nbformat are fixing this issue transparently, and will stop doing so in the future.
             File "/home/joren/Workspace/lmo/.venv/lib/python3.13/site-packages/nbformat/validator.py", line 351, in _normalize
               warnings.warn(
             File "/home/joren/Workspace/lmo/.venv/lib/python3.13/site-packages/nbformat/__init__.py", line 96, in
               validate(nb)
INFO    -  MissingIDFieldWarning: Cell is missing an id field, this will become a hard error in future nbformat versions. You may want to use `normalize()` on your notebooks before validations (available since nbformat 5.1.4). Previous versions of nbformat are fixing this issue transparently, and will stop doing so in the future.
             File "/home/joren/Workspace/lmo/.venv/lib/python3.13/site-packages/nbformat/validator.py", line 351, in _normalize
               warnings.warn(
             File "/home/joren/Workspace/lmo/.venv/lib/python3.13/site-packages/nbformat/__init__.py", line 96, in
               validate(nb)
INFO    -  MissingIDFieldWarning: Cell is missing an id field, this will become a hard error in future nbformat versions. You may want to use `normalize()` on your notebooks before validations (available since nbformat 5.1.4). Previous versions of nbformat are fixing this issue transparently, and will stop doing so in the future.
             File "/home/joren/Workspace/lmo/.venv/lib/python3.13/site-packages/nbformat/validator.py", line 351, in _normalize
               warnings.warn(
             File "/home/joren/Workspace/lmo/.venv/lib/python3.13/site-packages/nbformat/__init__.py", line 96, in
               validate(nb)
INFO    -  MissingIDFieldWarning: Cell is missing an id field, this will become a hard error in future nbformat versions. You may want to use `normalize()` on your notebooks before validations (available since nbformat 5.1.4). Previous versions of nbformat are fixing this issue transparently, and will stop doing so in the future.
             File "/home/joren/Workspace/lmo/.venv/lib/python3.13/site-packages/nbformat/validator.py", line 351, in _normalize
               warnings.warn(
             File "/home/joren/Workspace/lmo/.venv/lib/python3.13/site-packages/nbformat/__init__.py", line 96, in
               validate(nb)
INFO    -  MissingIDFieldWarning: Cell is missing an id field, this will become a hard error in future nbformat versions. You may want to use `normalize()` on your notebooks before validations (available since nbformat 5.1.4). Previous versions of nbformat are fixing this issue transparently, and will stop doing so in the future.
             File "/home/joren/Workspace/lmo/.venv/lib/python3.13/site-packages/nbformat/validator.py", line 351, in _normalize
               warnings.warn(
             File "/home/joren/Workspace/lmo/.venv/lib/python3.13/site-packages/nbformat/__init__.py", line 96, in
               validate(nb)
INFO    -  Converting notebook (execute=False): /home/joren/Workspace/lmo/docs/examples/lmm.ipynb
WARNING:traitlets:Alternative text is missing on 1 image(s).
INFO    -  Converting notebook (execute=False): /home/joren/Workspace/lmo/docs/examples/visual_intro.ipynb
WARNING:traitlets:Alternative text is missing on 6 image(s).
INFO    -  Documentation built in 10.68 seconds

to repro:

git clone [email protected]:jorenham/Lmo.git
cd Lmo
uv run mkdocs build

A text search in the .venv showed that the relevant nbformat.reads call can only originate from juypytext. Specifically, at

jupytext/src/jupytext/jupytext.py

Line 368 in 8409313

nb = nbformat.reads(text, as_version, **kwargs)
and

jupytext/src/jupytext/formats.py

Line 394 in 8409313

nbformat.reads(text, as_version=4)
.

I'm not familiar with the jupytext or nbformat codebases, but as the warning message suggests, I'm guessing that this could be solved by calling nbformat.validator.normalize before nbformat.reads 🤷🏻
@mwouts
Copy link
Owner

mwouts commented Jan 13, 2025

Thank you @jorenham for reporting this, I will look into this, especially as it sounds that this will become a hard error at some point!

@mwouts mwouts self-assigned this Jan 13, 2025
@mwouts
Copy link
Owner

mwouts commented Jan 13, 2025

At this stage I have not yet been able to reproduce the warning. Could you confirm which version of nbformat you are using?

I think that it would be useful if Jupytext could tell about which file is being read, I could try to add that - indeed at the two locations you point out that information is not available as nbformat.reads directly gets the file content.

Are you able to reproduce the problem with a simpler snippet in your env, e.g. something like this?

from pathlib import Path
from nbformat import reads

with open('docs/examples/visual_intro.ipynb') as fp:
    nb = reads(fp.read(), as_version=4)

@jorenham
Copy link
Author

At this stage I have not yet been able to reproduce the warning.

Did you try these repro steps?

git clone [email protected]:jorenham/Lmo.git
cd Lmo
uv run mkdocs build

Could you confirm which version of nbformat you are using?

$ uv tree --package jupytext
Resolved 147 packages in 1ms
jupytext v1.16.6
├── markdown-it-py v3.0.0
│   └── mdurl v0.1.2
├── mdit-py-plugins v0.4.2
│   └── markdown-it-py v3.0.0 (*)
├── nbformat v5.10.4
│   ├── fastjsonschema v2.21.1
│   ├── jsonschema v4.23.0
│   │   ├── attrs v24.3.0
│   │   ├── jsonschema-specifications v2024.10.1
│   │   │   └── referencing v0.35.1
│   │   │       ├── attrs v24.3.0
│   │   │       └── rpds-py v0.22.3
│   │   ├── referencing v0.35.1 (*)
│   │   └── rpds-py v0.22.3
│   ├── jupyter-core v5.7.2
│   │   ├── platformdirs v4.3.6
│   │   └── traitlets v5.14.3
│   └── traitlets v5.14.3
├── packaging v24.2
└── pyyaml v6.0.2
(*) Package tree already displayed

(ran while still within the Lmo directory)

so it's the latests v5.10.4

Are you able to reproduce the problem with a simpler snippet in your env, e.g. something like this?

from pathlib import Path
from nbformat import reads

with open('docs/examples/visual_intro.ipynb') as fp:
    nb = reads(fp.read(), as_version=4)

I also can't repo it this way, so perhaps mkdocs-jupyter does some additional pre-processing or something?

ping @danielfrg

@mwouts
Copy link
Owner

mwouts commented Jan 15, 2025

Yes I did try the uv command and I got this:

Logs of $uv run mkdocs build 
uv run mkdocs build
INFO    -  DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
             File "/home/marc/GitHub/Lmo/.venv/lib/python3.12/site-packages/pybtex/plugin/__init__.py", line 26, in <module>
               import pkg_resources
             File "/home/marc/GitHub/Lmo/.venv/lib/python3.12/site-packages/pkg_resources/__init__.py", line 98, in <module>
               warnings.warn(
INFO    -  DeprecationWarning: Jupyter is migrating its paths to use standard platformdirs
           given by the platformdirs library.  To remove this warning and
           see the appropriate new directories, set the environment variable
           `JUPYTER_PLATFORM_DIRS=1` and then run `jupyter --paths`.
           The use of platformdirs will be the default in `jupyter_core` v6
             File "/home/marc/GitHub/Lmo/.venv/lib/python3.12/site-packages/jupyter_core/utils/__init__.py", line 90, in deprecation
               warnings.warn(message, DeprecationWarning, stacklevel=stacklevel + 1)
             File "/home/marc/GitHub/Lmo/.venv/lib/python3.12/site-packages/jupyter_client/connect.py", line 22, in
               from jupyter_core.paths import jupyter_data_dir, jupyter_runtime_dir, secure_write
INFO    -  Loading data from bib files: ['docs/bib/refs.bib']
INFO    -  DeprecationWarning: open_text is deprecated. Use files() instead. Refer to
           https://importlib-resources.readthedocs.io/en/latest/using.html#migrating-from-legacy for migration advice.
             File "/home/marc/GitHub/Lmo/.venv/lib/python3.12/site-packages/latexcodec/codec.py", line 102, in load_unicode_latex_table
               with pkg_resources.open_text('latexcodec', 'table.txt') as datafile:
             File "/usr/lib/python3.12/importlib/resources/_legacy.py", line 18, in wrapper
               warnings.warn(
INFO    -  Cleaning site directory
INFO    -  Building documentation to directory: /home/marc/GitHub/Lmo/site
INFO    -  Converting notebook (execute=False): /home/marc/GitHub/Lmo/docs/examples/lmm.ipynb
WARNING:traitlets:Alternative text is missing on 1 image(s).
INFO    -  Converting notebook (execute=False): /home/marc/GitHub/Lmo/docs/examples/visual_intro.ipynb
WARNING:traitlets:Alternative text is missing on 6 image(s).
INFO    -  Documentation built in 6.82 seconds

This is my package tree:

$ uv tree --package jupytext
Resolved 147 packages in 1ms
jupytext v1.16.6
├── markdown-it-py v3.0.0
│   └── mdurl v0.1.2
├── mdit-py-plugins v0.4.2
│   └── markdown-it-py v3.0.0 (*)
├── nbformat v5.10.4
│   ├── fastjsonschema v2.21.1
│   ├── jsonschema v4.23.0
│   │   ├── attrs v24.3.0
│   │   ├── jsonschema-specifications v2024.10.1
│   │   │   └── referencing v0.35.1
│   │   │       ├── attrs v24.3.0
│   │   │       └── rpds-py v0.22.3
│   │   ├── referencing v0.35.1 (*)
│   │   └── rpds-py v0.22.3
│   ├── jupyter-core v5.7.2
│   │   ├── platformdirs v4.3.6
│   │   └── traitlets v5.14.3
│   └── traitlets v5.14.3
├── packaging v24.2
└── pyyaml v6.0.2
(*) Package tree already displayed

I was thinking that maybe we could turn the warning into an error with e.g. warnings.filterwarnings("error") to better see the call stack?

@jorenham
Copy link
Author

It's so strange that you're not seeing it. Maybe try again with uv run --python 3.13 ...? Because when I run with --python 3.12, I still see the warnigns, but without the stacktrace.

Judging by the the INFO line prefixes, there's some awkward logging stuff going on. So there's also a chance that this is interfering with the output in some way.

I was thinking that maybe we could turn the warning into an error with e.g. warnings.filterwarnings("error") to better see the call stack?

That's a good idea!
I tried running it with PYTHONWARNINGS="always,error::FutureWarning" in an .env file and ran uv run --python 3.13 --env-file .env mkdocs build -c -s. But now (only) the MissingIDFieldWarning warnings (info) logs disappear... So apparently "Volkswagen testing" is being used 😅.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mwouts mwouts

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jorenham @mwouts