Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

could other_files be not dereferenced? #390

Closed
yarikoptic opened this issue Jan 18, 2023 · 2 comments
Closed

could other_files be not dereferenced? #390

yarikoptic opened this issue Jan 18, 2023 · 2 comments
Labels
R-wontfix Resolution: This will not be fixed. The report might be wrong or out of scope

Comments

@yarikoptic
Copy link

yarikoptic commented Jan 18, 2023

❯ reprozip trace -d /tmp/reprozip-trace-`git describe` /bin/ls -l
...
Configuration file written in /tmp/reprozip-trace-1.1-274-gcac47797/config.yml
Edit that file then run the packer -- use 'reprozip pack -h' for help

❯ grep /bin/ls /tmp/reprozip-trace-1.1-274-gcac47797/*
/tmp/reprozip-trace-1.1-274-gcac47797/config.yml:  - /bin/ls
/tmp/reprozip-trace-1.1-274-gcac47797/config.yml:  binary: /bin/ls
/tmp/reprozip-trace-1.1-274-gcac47797/config.yml:  - "/usr/bin/ls" # 143.80 KB
grep: /tmp/reprozip-trace-1.1-274-gcac47797/trace.sqlite3: binary file matches

so - running /bin/ls has /usr/bin/ls listed in other_files of the config.yml, although that one nohow explicitly re-executed by /bin/ls

❯ strace -f -o /tmp/ls-strace.log /bin/ls -l > /dev/null
❯ grep bin/ls /tmp/ls-strace.log
1952259 execve("/bin/ls", ["/bin/ls", "-l"], 0x7fffe83bf550 /* 101 vars */) = 0
❯ md5sum {,/usr}/bin/ls
26f446f5c92841a0ba71dd041011baa7  /bin/ls
26f446f5c92841a0ba71dd041011baa7  /usr/bin/ls

and likely simply because we have now on debian systems

❯ ls -l /bin
lrwxrwxrwx 1 root root 7 Nov  5  2019 /bin -> usr/bin/

which would be all nice and dandy (for my desires of tracing in reproman), if dpkg could locate /usr/bin/ls one but it can't:

❯ dpkg -S /bin/ls /usr/bin/ls
coreutils: /bin/ls
dpkg-query: no path found matching pattern /usr/bin/ls

so I wonder if tracing could avoid dereferencing paths or it is unavoidable?

yarikoptic added a commit to yarikoptic/ReproNim that referenced this issue Jan 18, 2023
quick summary of the problem: on debian there is bin -> usr/bin symlink,
but then /bin/ls is the one known to the package. reprozip resolves path
to /usr/bin/ls and we end up without package information.

More info on VIDA-NYU/reprozip#390
and reproman specific issue ReproNim#591
@remram44
Copy link
Member

reprozip writes a list of files, with their canonical names. There is no /bin/ls on disk, there is only a symlink /bin and a single file /usr/bin/ls.

If you want to know which files were executed, the trace has the correct information (in the executed_files table). The /bin symlink will also be automatically included in the other_files since it has to exist for /bin/ls to be reached.

I am not sure I understand your use-case. The point of config.yml is to list the files to be packed, and /bin/ls is not an existing file. Could you perhaps say a bit more about what you want to do?

@remram44
Copy link
Member

I would add that maybe if you want information that is not intended for creating an RPZ, the config.yml might be the wrong file to read. Reading trace.sqlite3 is probably more appropriate, you can even adapt the get_files() function to create the list you want from it. My version of this function does other things you might not want, such as applying filters that will silently remove some files (like .pyc files) that we don't want to pack.

@remram44 remram44 added the R-wontfix Resolution: This will not be fixed. The report might be wrong or out of scope label Jan 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
R-wontfix Resolution: This will not be fixed. The report might be wrong or out of scope
Projects
None yet
Development

No branches or pull requests

2 participants