Data viewer cannot handle pathlib.WindowsPath objectes in a pandas DataFrame

[ghost]

Applies To

• Notebooks (.ipynb files)
• Interactive Window and/or Cell Scripts (.py files with #%% markers)

What happened?

I start with a DataFrame with an int and str column, which I combine into another str column that represents a relative path:

df = pd.DataFrame(data=dict(a=[1, 2, 3], b='output'))
df['fname'] = "./" + df.b + "/" + df.a.astype(str) + ".json"
print(df.to_markdown())

	a	b	fname
0	1	output	./output/1.json
1	2	output	./output/2.json
2	3	output	./output/3.json

When I open this in the Data Viewer (right click variable in debug panel -> View Value in Data Viewer) this shows perfectly fine:

Now I create a new column where I convert the fname to an actual Path object

from pathlib import Path
df['path'] = df.fname.apply(Path)
print(df.to_markdown())

	a	b	fname	path
0	1	output	./output/1.json	output\1.json
1	2	output	./output/2.json	output\2.json
2	3	output	./output/3.json	output\3.json

Expected outcome: this DataFrame can be viewed in the Data Viewer similarly to when there was no path column.

Actual outcome: The Data Viewer panel opens, shows the column header but no rows and keeps loading:

A notification appears that says:

Unexpected end of JSON input. View Jupyter log for further details

The log is attached.

If I convert the relative path to an absolute path, this does not change the outcome:

df['path'] = df.path.apply(lambda x: x.resolve())
print(df.to_markdown())

	a	b	fname	path
0	1	output	./output/1.json	L:\redacted\output\1.json
1	2	output	./output/2.json	L:\redacted\output\2.json
2	3	output	./output/3.json	L:\redacted\output\3.json

VS Code Version

Version: 1.77.0 (system setup) Commit: 7f329fe6c66b0f86ae1574c2911b681ad5a45d63 Date: 2023-03-29T10:02:16.981Z Electron: 19.1.11 Chromium: 102.0.5005.196 Node.js: 16.14.2 V8: 10.2.154.26-electron.0 OS: Windows_NT x64 10.0.14393 Sandboxed: No

Jupyter Extension Version

v2023.3.1201040234

Jupyter logs

info 19:02:44.968: Found debugAdapterPython on Debug Configuration to use
info 19:02:45.068: Process Execution: > l:\<redacted>\.conda\envs\myenv\python.exe -c "import pandas;print(pandas.__version__)"
> l:\<redacted>\.conda\envs\myenv\python.exe -c "import pandas;print(pandas.__version__)"
info 19:02:46.195: Loading webview. View is notset
info 19:02:46.196: Loading web view...
info 19:02:46.196: Webview panel created.
info 19:02:46.362: Web view react rendered
error 19:02:46.489: [SyntaxError: Unexpected end of JSON input
	at JSON.parse (<anonymous>)
	at hw.getDataFrameRows (c:\Users\<redacted>\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\out\extension.node.js:24:238210)
	at async fE.getRows (c:\Users\<redacted>\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\out\extension.node.js:24:520193)
	at async c:\Users\<redacted>\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\out\extension.node.js:24:508264
	at async Zk.wrapRequest (c:\Users\<redacted>\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\out\extension.node.js:24:508541)]
warn 19:02:46.489: DataScience Error [SyntaxError: Unexpected end of JSON input
	at JSON.parse (<anonymous>)
	at hw.getDataFrameRows (c:\Users\<redacted>\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\out\extension.node.js:24:238210)
	at async fE.getRows (c:\Users\<redacted>\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\out\extension.node.js:24:520193)
	at async c:\Users\<redacted>\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\out\extension.node.js:24:508264
	at async Zk.wrapRequest (c:\Users\<redacted>\.vscode\extensions\ms-toolsai.jupyter-2023.3.1201040234\out\extension.node.js:24:508541)]

Coding Language and Runtime Version

Python 3.9.4

Language Extension Version (if applicable)

Python Extension v2023.6.1

Anaconda Version (if applicable)

conda 22.9.0

Running Jupyter locally or remotely?

Local

[amunger]

closing as the jupyter data viewer was deprecated #15486