You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Given pixi doesn't have an RFC process for feature discussion, I figured an issue here probably is the best place to discuss this feature idea. This feature is already in development as magic create in the magic CLI to solve problems with AI development use cases downstream where we are flushing out design issues, but we eventually want to share this feature upstream once it's a bit more flushed out and working.
pixi create
This is a proposal for a new feature called pixi create. This command enhances the project scaffolding capabilities of pixi, drawing inspiration from the npm init/npm create command by allowing for customizable "generator" packages.
Motivation
While pixi init provides a basic structure for new projects, there is a need for a more comprehensive solution to generate diverse project types, potentially spanning multiple programming languages. The idea is that feature can help setup full-fledged project scaffolds complete with best practices of developing with different frameworks. Some setups are difficult to get right out of the box for different frameworks and application types. The pixi create command aims to fulfill this requirement by leveraging a system of generator packages.
For example, a generator for a web framework could generate a Dockerfile to assist with deployment but also with local development that makes use of pixi-docker. A generator for a python library scafold could show the best patterns for using “pixi build” to show how to a build backend and publish to both Conda and PYPI. A generator for pytorch could show best practices for cross-platform building using different versions of pytorch (CPU, GPU, etc). A generator can also create .devcontainer configs around the project to assist with development.
Generators can be interactive, allowing for easy user customization of scaffold projects to pick perhaps different optional dependencies or integrations, database engines, tool support, etc. Generators can also be built to be non-interactive where you want to automate project generation with tooling.
Functionality
The pixi create command operates as follows:
Generator Packages:
Generator packages will be conda packages, identifiable by the "create-" prefix.
These packages will be responsible for the actual generation of project scaffolding.
They will be hosted on conda channels and installed using the same backend as pixi global except without directly exposed binaries in the path.
Differences from pixi init
pixi init is designed for lightweight project initialization that works even on existing projects.
pixi create is intended for more complex project generation, supporting a wider range of project types and languages. It’s generally not intended for use on an existing code base.
pixi init provides a basic structure where pixi create offers greater flexibility through the use of specialized generator packages.
Command Execution:
When pixi create <generator> <target_directory> is executed:
pixi will download and install the specified <generator> package into a new global environment.
A command within the <generator> package will be invoked, passing all arguments from pixi create to it.
The generator package will then handle the creation of the project structure within the <target_directory>.
To assist with the development and debug of new generators, an env + task can be passed in an existing project on disk to invoke instead of using one installed from a package into a global env.
pixi will fetch and install the "create-python" package.
The "create-python" command (or its designated entry point) will be executed by passing through any additional args to the generator.
The "create-python" package will generate the Python project scaffolding in the "./my_python_project" directory.
Internal Implementation
Generator Package Structure:
Generator packages will adhere to a standardized structure to ensure compatibility with pixi create. Generator packages can be built using any language or framework as long as they can shipped as conda packages themselves. A simple generator could be written in Python and generate files using simple jinja templates, perhaps leaning on copier.
The generator MUST specify a script within the conda package that matches the package's name, so a generator named "create-python-project" should have a bin/create-python-project script within the package.
A generator package should contain the code and templates to generate the project structure but can fetch templates, assets, and scaffold files from the web.
Generator package environments:
Since generators are installed implicitly, they need an environment other than the project environment to execute. Before executing, generators will be installed in the temp env folder in a similar fashion to pixi global (except when an explicit project and task are passed in).
Special Considerations/TBD
Generators may want to download dependencies and initialize different envs in the new project directory while generating rather than defer to the first run of the project, perhaps to invoke tools from dependent packages on the newly generated project folder. We may have to consider re-entertrancy back into pixi in that case.
The exact arguments for pixi create are still in flux.
The text was updated successfully, but these errors were encountered:
Problem description
Given pixi doesn't have an RFC process for feature discussion, I figured an issue here probably is the best place to discuss this feature idea. This feature is already in development as
magic create
in the magic CLI to solve problems with AI development use cases downstream where we are flushing out design issues, but we eventually want to share this feature upstream once it's a bit more flushed out and working.pixi create
This is a proposal for a new feature called
pixi create
. This command enhances the project scaffolding capabilities of pixi, drawing inspiration from thenpm init
/npm create
command by allowing for customizable "generator" packages.Motivation
While
pixi init
provides a basic structure for new projects, there is a need for a more comprehensive solution to generate diverse project types, potentially spanning multiple programming languages. The idea is that feature can help setup full-fledged project scaffolds complete with best practices of developing with different frameworks. Some setups are difficult to get right out of the box for different frameworks and application types. Thepixi create
command aims to fulfill this requirement by leveraging a system of generator packages.For example, a generator for a web framework could generate a Dockerfile to assist with deployment but also with local development that makes use of
pixi-docker
. A generator for a python library scafold could show the best patterns for using “pixi build” to show how to a build backend and publish to both Conda and PYPI. A generator for pytorch could show best practices for cross-platform building using different versions of pytorch (CPU, GPU, etc). A generator can also create .devcontainer configs around the project to assist with development.Generators can be interactive, allowing for easy user customization of scaffold projects to pick perhaps different optional dependencies or integrations, database engines, tool support, etc. Generators can also be built to be non-interactive where you want to automate project generation with tooling.
Functionality
The
pixi create
command operates as follows:Generator Packages:
Generator packages will be conda packages, identifiable by the "create-" prefix.
These packages will be responsible for the actual generation of project scaffolding.
They will be hosted on conda channels and installed using the same backend as
pixi global
except without directly exposed binaries in the path.Differences from
pixi init
pixi init
is designed for lightweight project initialization that works even on existing projects.pixi create
is intended for more complex project generation, supporting a wider range of project types and languages. It’s generally not intended for use on an existing code base.pixi init
provides a basic structure wherepixi create
offers greater flexibility through the use of specialized generator packages.Command Execution:
When
pixi create <generator> <target_directory>
is executed:pixi will download and install the specified
<generator>
package into a new global environment.A command within the
<generator>
package will be invoked, passing all arguments frompixi create
to it.The generator package will then handle the creation of the project structure within the
<target_directory>
.To assist with the development and debug of new generators, an env + task can be passed in an existing project on disk to invoke instead of using one installed from a package into a global env.
Example
pixi create python ./my_python_project -- --some-arg=hello
In this example:
pixi will fetch and install the "create-python" package.
The "create-python" command (or its designated entry point) will be executed by passing through any additional args to the generator.
The "create-python" package will generate the Python project scaffolding in the "./my_python_project" directory.
Internal Implementation
Generator Package Structure:
Generator packages will adhere to a standardized structure to ensure compatibility with
pixi create
. Generator packages can be built using any language or framework as long as they can shipped as conda packages themselves. A simple generator could be written in Python and generate files using simple jinja templates, perhaps leaning on copier.The generator MUST specify a script within the conda package that matches the package's name, so a generator named "create-python-project" should have a
bin/create-python-project
script within the package.A generator package should contain the code and templates to generate the project structure but can fetch templates, assets, and scaffold files from the web.
Generator package environments:
Since generators are installed implicitly, they need an environment other than the project environment to execute. Before executing, generators will be installed in the temp env folder in a similar fashion to
pixi global
(except when an explicit project and task are passed in).Special Considerations/TBD
Generators may want to download dependencies and initialize different envs in the new project directory while generating rather than defer to the first run of the project, perhaps to invoke tools from dependent packages on the newly generated project folder. We may have to consider re-entertrancy back into pixi in that case.
The exact arguments for
pixi create
are still in flux.The text was updated successfully, but these errors were encountered: