Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplifying adding runenvs in virtual environment #5580

Closed
xq114 opened this issue Sep 6, 2024 · 9 comments
Closed

Simplifying adding runenvs in virtual environment #5580

xq114 opened this issue Sep 6, 2024 · 9 comments

Comments

@xq114
Copy link
Contributor

xq114 commented Sep 6, 2024

Is your feature request related to a problem? Please describe.

目前在虚拟环境中加环境变量是比较麻烦的:必须先定义一个package,再在on_load里加环境变量,还要加上on_fetch让xmake不报错:

package("__phony")
on_load(function (package)
    package:addenv("PYTHONPATH", path.join(os.scriptdir(), "bin"))
end)
on_fetch(function (package, opt) return {} end)
package_end()
add_requires("__phony")

这种写法繁琐、非常容易出错。然而加环境变量的需求在多语言项目中普遍存在,例如C++/Python混合项目里,编译出的.pyd要在python里测试,就要把python module加入环境变量PYTHONPATH才行

Describe the solution you'd like

目前定义一个add_runenvs()函数就行:

function add_runenvs(envs)
package("__phony")
on_load(function (package)
    package:addenv("TESTFILE", io.readfile("testfile"))
    for name, value in pairs(envs) do
        package:addenv(name, value)
    end
end)
on_fetch(function (package, opt) return {} end)
package_end()
add_requires("__phony")
end

测试:

add_runenvs({TESTENV = path.join(os.scriptdir(), "bin"), TESTENV2 = "", TESTENV3 = winos.version()})

可以正确生成结果。要加入xmake内置函数的话,加一些处理应该就行:

function add_runenvs(envs)
envs = envs or {}
local envnames = table.orderkeys(envs)
if #envnames == 0 then return end -- check input
package("__" .. envnames[1] .. #envnames) -- deduplicate name
on_load(function (package)
    for name, value in pairs(envs) do
        package:addenv(name, value)
    end
end)
on_fetch(function (package, opt) return {} end)
package_end()
add_requires("__" .. envnames[1] .. #envnames)
end

这样多次调用add_runenvs也没有问题了。剩下两个问题:

  1. 这种方式的add_runenvs只能定义description scope可用的值。例如需要读取文件,定义 TESTFILE = io.readfile("testfile") ,就只能手写繁琐的package定义了。
  2. 这种方式定义的package尽管是fetchonly的空package,仍然会在.xmake/packages产生多余的空文件夹。有没有可能对fetchonly的package不要生成package hash文件夹?

Describe alternatives you've considered

function add_metapackage(load)
package("__phony")
on_load(load)
on_fetch(function (package, opt) return {} end)
package_end()
add_requires("__phony")
end

仍保留package定义的部分给用户,但传入一个on_load函数:

add_metapackage(function (package)
    package:addenv("TESTENV", io.readfile("testfile"))
end)

这样可以解决上面的问题1,只是需要多写一点代码。并且这种形式不仅限于加环境变量,也可以用来做别的事情。(这里去重的话只能用load函数的地址了;我不太清楚怎么做)

Additional context

No response

@waruqi
Copy link
Member

waruqi commented Sep 7, 2024

试下这个 patch, #5582

脚本,table 描述域设置都可以

includes("@builtin/xrepo")

xrepo_addenvs(function (package)
    package:addenv("FOO", "FOO")
end)

xrepo_addenvs({BAR = "BAR"})

@waruqi waruqi added this to the v2.9.5 milestone Sep 7, 2024
@waruqi
Copy link
Member

waruqi commented Sep 7, 2024

另外加了一个可以快速设置单个 envs 的

xrepo_addenv("ZOO", "ZOO")

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


In addition, one can quickly set up a single envs

xrepo_addenv("ZOO", "ZOO")

waruqi added a commit that referenced this issue Sep 8, 2024
@waruqi waruqi closed this as completed Sep 8, 2024
@xq114
Copy link
Contributor Author

xq114 commented Sep 8, 2024

这种方式定义的package尽管是fetchonly的空package,仍然会在.xmake/packages产生多余的空文件夹。有没有可能对fetchonly的package不要生成package hash文件夹?

这块好像还没解决

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


Even though the package defined in this way is a fetchonly empty package, it will still generate extra empty folders in .xmake/packages. Is it possible not to generate a package hash folder for fetchonly packages?

This seems to have not been resolved yet

@waruqi
Copy link
Member

waruqi commented Sep 8, 2024

这种方式定义的package尽管是fetchonly的空package,仍然会在.xmake/packages产生多余的空文件夹。有没有可能对fetchonly的package不要生成package hash文件夹?

这块好像还没解决

再试试

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


Even though the package defined in this way is a fetchonly empty package, it will still generate extra empty folders in .xmake/packages. Is it possible not to generate a package hash folder for fetchonly packages?

This seems to have not been resolved yet

try again

@xq114 xq114 mentioned this issue Sep 10, 2024
10 tasks
@xq114
Copy link
Contributor Author

xq114 commented Oct 8, 2024

这里实现还是有问题,当加的变量比较复杂(比如路径)

xrepo_addenv("CMAKE_PREFIX_PATH", "C:\\Users\\xq114\\Downloads\\libtorch")

会报错

error: @programdir\core\main.lua:329: @programdir\modules\async\runjobs.lua:325: @programdir\core\package\package.lua:718: cannot create filelock for package(__xrepo_addenv_CMAKE_PREFIX_PATHC:\Users\xq114\Downloads\libtorch)!

去重的时候简单拼接还是不行,会有非法字符,包括xrepo_addenvs使用的table也不行,含有":"字符。理想方式是使用hash,但description scope用不了hash.md5

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


There are still problems with the implementation here, when the variables added are more complex (such as paths)

xrepo_addenv("CMAKE_PREFIX_PATH", "C:\\Users\\xq114\\Downloads\\libtorch")

Will report an error

error: @programdir\core\main.lua:329: @programdir\modules\async\runjobs.lua:325: @programdir\core\package\package.lua:718: cannot create filelock for package(__xrepo_addenv_CMAKE_PREFIX_PATHC:\Users\ xq114\Downloads\libtorch)!

When removing duplicates, simple splicing still doesn't work. There will be illegal characters, including the table used by xrepo_addenvs, which contains the ":" character. The ideal way is to use hash, but description scope cannot use hash.md5

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants