Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use shutil for cross-volume moves #96

Merged
merged 1 commit into from
Apr 23, 2024

Conversation

piercefreeman
Copy link
Owner

@piercefreeman piercefreeman commented Apr 23, 2024

We leverage the local "/tmp" directory extensively in building so we can avoid polluting the local project filesystem until we are sure that we have a successful build result. Upon completion, we want to move files from the temporary directory to the local project. Mostly this is okay since we operate within the same device - but there are some situations (docker, remote CI builds, booting from removable storage) where the OS's default /tmp path is on a different volume than the project itself. Our previous logic would crash in these cases with a Invalid cross-device link error.

This PR fixes this bug by moving to shutil.move for moving temporary files. It will default to a regular replace while within the same volume and fallback to a copy/delete in the case of differing volumes.

11.66   File "/root/.cache/pypoetry/virtualenvs/myproject-VA82Wl8V-py3.11/lib/python3.11/site-packages/mountaineer/cli.py", line 422, in handle_build
11.66     js_compiler.build()
11.66   File "/root/.cache/pypoetry/virtualenvs/myproject-VA82Wl8V-py3.11/lib/python3.11/site-packages/mountaineer/client_builder/builder.py", line 68, in build
11.66     asyncio.run(self.async_build())
11.66   File "/usr/lib/python3.11/asyncio/runners.py", line 190, in run
11.66     return runner.run(main)
11.66            ^^^^^^^^^^^^^^^^
11.66   File "/usr/lib/python3.11/asyncio/runners.py", line 118, in run
11.66     return self._loop.run_until_complete(task)
11.66            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
11.66   File "/usr/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
11.66     return future.result()
11.66            ^^^^^^^^^^^^^^^
11.66   File "/root/.cache/pypoetry/virtualenvs/myproject-VA82Wl8V-py3.11/lib/python3.11/site-packages/mountaineer/client_builder/builder.py", line 94, in async_build
11.66     await self.build_javascript_chunks()
11.66   File "/root/.cache/pypoetry/virtualenvs/myproject-VA82Wl8V-py3.11/lib/python3.11/site-packages/mountaineer/client_builder/builder.py", line 477, in build_javascript_chunks
11.66     self.move_build_artifacts_into_project()
11.66   File "/root/.cache/pypoetry/virtualenvs/myproject-VA82Wl8V-py3.11/lib/python3.11/site-packages/mountaineer/client_builder/builder.py", line 495, in move_build_artifacts_into_project
11.66     self.view_root.get_managed_static_dir(tmp_build=True).rename(tmp_static_dir)
11.66   File "/usr/lib/python3.11/pathlib.py", line 1176, in rename
11.66     os.rename(self, target)
11.66 OSError: [Errno 18] Invalid cross-device link: '/usr/src/app/myproject/views/_static/tmp' -> '/tmp/tmpad4vf3eo/static'

@piercefreeman piercefreeman force-pushed the bug/fix-cross-device-building branch from b72bc68 to 889490a Compare April 23, 2024 14:50
@piercefreeman piercefreeman merged commit 5620f7d into main Apr 23, 2024
11 checks passed
@piercefreeman piercefreeman deleted the bug/fix-cross-device-building branch April 23, 2024 14:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant