Allow setting open build env vars in .readthedocs.yml config file

[webknjaz]

Details

• Read the Docs project URL: N/A
• Build URL (if applicable): N/A
• Read the Docs username (if applicable): N/A

Expected Result

I want to be able to set public env vars via .readthedocs.yml.

Actual Result

I can only do this in the web UI.

Use-case

I have an in-tree Sphinx extension and import paths are set up via tox. Since RTD doesn't allow specifying a custom build tool and has a hardcoded command, I had to seek another way of making it importable. I've chosen to set a PYTHONPATH variable to address this. But I'd prefer an explicit way of doing it.

Bonus thoughts

Setting "special" env vars may conflict with RTD's special values. It may be a good idea to enable first-class support for adding custom import location as a YAML sequence in the config and then properly merging it with the existing PYTHONPATH.

[flying-sheep]

Other use case: installing packages from custom public indices via PIP_FIND_LINKS="https://wheels.example.com" where e.g.

$ curl $PIP_FIND_LINKS
<a href="my_special_use_pkg-0.4-py3-none-any.whl">my-special-use-pkg</a>

[agjohnson]

Though we have this functionality in our UI, this seems like a great feature for our config file. This is handy for non-secret environment variables, as noted, but also for helping increase the odds of a build being reproducible without extra project configuration.

I raised issues with the scope of environment variables defined in our configuration through build.jobs. @humitos would you agree this feature makes sense without being at odds with the plans for build.jobs?

[humitos]

@agjohnson

I raised issues with the scope of environment variables defined in our configuration through build.jobs. @humitos would you agree this feature makes sense without being at odds with the plans for build.jobs?

Yes. I think this feature makes sense and can work well together with build.jobs. All the environment variables will be loaded after parsing the config file and will be passed to all commands (default ones ran by Read the Docs and custom ones defined by users)

[agjohnson]

Great! This seems like a great feature to add, I'm putting this on our 2022Q2 roadmap 👍

[agjohnson]

I lied, we didn't get to this in 2022. I'm going to keep this on the backlog for now.

[ericholscher]

Also just a note, I think you can probably hack this with something like:

build
  jobs:
    pre_build: echo "export FOO=bar" >> ~/.bash_profile

No guarantee it continues to work though 🙃

[humitos]

@ericholscher I tested your idea, but it seems it doesn't work as expected 😞 . I quickly tested this at https://readthedocs.org/projects/test-builds/builds/18714592/

I suppose it's because each command is executed with /bin/sh and it not loads any of the bash initial files:

readthedocs.org/readthedocs/doc_builder/environments.py

Lines 374 to 379 in fba1428

	return (
	"/bin/sh -c '{prefix}{cmd}'".format(
	prefix=prefix,
	cmd=command,
	)
	)

[agjohnson]

sh only evaluates .profile and friends if it's a login shell (sh -l/sh --login), so this wouldn't work without some more creative solutions

[humitos]

@agjohnson would it be safe to pass --login? If the answer is true, that this is all we need, I'd 👍🏼 on moving forward with this. At least, it will give users a workaround for this use case.

[agjohnson]

That is a really good question! I don't know for certain here, it would be worth a test.

Executing in a login shell would open the door to using direnv, which I might suggest over a solution like ☝️

It's also possible to author a local .profile in the repo and copy that into the /home/docs path, which is probably okay too.

[humitos]

This is issue is kind of related to #10462. We should consider that case when working on this idea.

[pllim]

FWIW, I was just looking for this feature. I was trying to set NUMPY_EXPERIMENTAL_DTYPE_API for some weird package I am trying to build for (spacetelescope/spherical_geometry#241).

[humitos]

@pllim you can define the environment variables for the project in the meanwhile. Read more at https://docs.readthedocs.io/en/stable/environment-variables.html