If there is any .so (e.g., _C.so) file in a package, Pynguin stops generating tests and reports FileNotFoundError: [Errno 2] No such file or directory: '.../_C.py'

[mohrez86]

I am trying to generate tests for the project torchvision.

After compiling torchvision, a file named _C.so is generated inside the package torchvision. Running Pynguin on the compiled project results in the following error:
FileNotFoundError: [Errno 2] No such file or directory: '.../_C.py'.

I found a workaround for now (I just need to add an empty file named _C.py next to the _C.so file).

I looked into Pynguin's source code. I think the bug is inside the following function:
https://github.com/se2p/pynguin/blob/main/pynguin/analyses/seeding/constantseeding.py#L139, which is:

• Module: pynguin/analyses/seeding/constantseeding.py
• Function: _find_modules
• Line: 139

The problem is that the function iter_modules returns .so files as well as .py files.
If I am right, this function needs to check if the modules returned by iter_modules are definitely .py files and skipped otherwise.

This bug (if it is a bug) can be reproduced by simply putting an empty .so file in a package and trying to generate tests for the modules inside that package, using Pynguin.

PS: awesome tool!

[stephanlukasczyk]

Hi @mohrez86 ,

Thank you for your interest in Pynguin and your warm words. And also thank you for reporting this issue.

I agree with your observation and the bug location. We have, honestly, never tried Pynguin on a project that uses modules written in C, i.e., compile down to some *.so file. Thus, we have not yet found the issue ourselves.

I'll play around with some native-code projects to maybe find some more problems and provide a fix release in the next couple of days.

Edit: Could you maybe provide the command line arguments you are using?

Best,
Stephan

[mohrez86]

Hi @stephanlukasczyk,

Thank you for the prompt reply.

Edit: Could you maybe provide the command line arguments you are using?

Sure. I used Pynguin running the following command:

pynguin \
  --project-path "/home/moe/vision" \
  --output-path "/home/moe/vision/test_gen" \
  --module-name torchvision.datasets.samplers.clip_sampler \
  -v \
  --budget 60

Regards,
Moe

[stephanlukasczyk]

Looking into this, I have yet another solution for you:

The issue is raised by the static constant seeding, which takes constants (i.e., numbers and strings) from the source code and uses them as seeded values for the input generation (this can be beneficial for test generation, as it was shown, e.g., by Fraser and Arcuri in their 2012 paper “The Seed is Strong: Seeding Strategies in Search-Based Software Testing”).

You can easily switch this of by adding --constant-seeding False to your command line.

Anyway, this is a bug that will be fixed in the near future.