Skip to content

Commit

Permalink
[develop2] reference methods (#2953)
Browse files Browse the repository at this point in the history 
* [develop2] reference methods

* Update reference/conanfile/methods/build.rst

Co-authored-by: SSE4 <[email protected]>

* Update reference/conanfile/methods/compatibility.rst

Co-authored-by: SSE4 <[email protected]>

* Update reference/conanfile/methods/compatibility.rst

Co-authored-by: SSE4 <[email protected]>

* Update reference/conanfile/methods/export_sources.rst

Co-authored-by: SSE4 <[email protected]>

* review

* wip

* wip

* wip

* Add grammar review

* wip

* wip

* Update configure.rst

* Update reference/conanfile/methods/source.rst

Co-authored-by: Rubén Rincón Blanco <[email protected]>

* Update reference/conanfile/methods/source.rst

Co-authored-by: Rubén Rincón Blanco <[email protected]>

* Update reference/conanfile/methods/source.rst

Co-authored-by: Rubén Rincón Blanco <[email protected]>

* Update reference/conanfile/methods/source.rst

Co-authored-by: Rubén Rincón Blanco <[email protected]>

* Update reference/conanfile/methods/system_requirements.rst

Co-authored-by: Rubén Rincón Blanco <[email protected]>

* Update reference/conanfile/methods/source.rst

Co-authored-by: Rubén Rincón Blanco <[email protected]>

* Update reference/conanfile/methods/system_requirements.rst

Co-authored-by: Rubén Rincón Blanco <[email protected]>

* Update reference/conanfile/methods/validate.rst

Co-authored-by: Rubén Rincón Blanco <[email protected]>

* Update reference/conanfile/methods/validate.rst

Co-authored-by: Rubén Rincón Blanco <[email protected]>

* Update reference/conanfile/methods/system_requirements.rst

Co-authored-by: Rubén Rincón Blanco <[email protected]>

* fix command

* wip

* added references to tutorial

* cross referencing

* fix ref

---------

Co-authored-by: SSE4 <[email protected]>
Co-authored-by: Rubén Rincón Blanco <[email protected]>
Co-authored-by: Rubén Rincón Blanco <[email protected]>
Co-authored-by: czoido <[email protected]>
  • Loading branch information
@memsharded @SSE4
@AbrilRBS @czoido
5 people authored Feb 9, 2023
1 parent 3f537ec commit 900bc27
Show file tree
Hide file tree
Showing 32 changed files with 1,640 additions and 162 deletions.
201 changes: 51 additions & 150 deletions reference/conanfile/methods.rst
View file
Original file line number Diff line number Diff line change
@@ -1,155 +1,56 @@
.. spelling::

ing
ver

.. _conan_conanfile_methods:
.. _reference_conanfile_methods:

Methods
=======


requirements()
--------------

Requirement traits
^^^^^^^^^^^^^^^^^^

Traits are properties of a requires clause. They determine how various parts of a
dependency are treated and propagated by Conan. Values for traits are usually computed by
Conan based on dependency's :ref:`reference_conanfile_attributes_package_type`, but can
also be specified manually.

A good introduction to traits is provided in the `Advanced Dependencies Model in Conan 2.0
<https://youtu.be/kKGglzm5ous>`_ presentation.

In the example below ``headers`` and ``libs`` are traits.

.. code-block:: python
self.requires("math/1.0", headers=True, libs=True)
headers
~~~~~~~

Indicates that there are headers that are going to be ``#included`` from this package at
compile time. The dependency will be in the host context.

libs
~~~~

The dependency contains some library or artifact that will be used at link time of the
consumer. This trait will typically be ``True`` for direct shared and static libraries,
but could be false for indirect static libraries that are consumed via a shared library.
The dependency will be in the host context.

build
~~~~~

This dependency is a build tool, an application or executable, like cmake, that is used
exclusively at build time. It is not linked/embedded into binaries, and will be in the
build context.

run
~~~

This dependency contains some executables, either apps or shared libraries that need to be
available to execute (typically in the path, or other system env-vars). This trait can be
``True`` for ``build=False``, in that case, the package will contain some executables that
can run in the host system when installing it, typically like an end-user application.
This trait can be ``True`` for ``build=True``, the package will contain executables that
will run in the build context, typically while being used to build other packages.

visible
~~~~~~~

This ``require`` will be propagated downstream, even if it doesn't propagate ``headers``,
``libs`` or ``run`` traits. Requirements that propagate downstream can cause version
conflicts. This is typically ``True``, because in most cases, having 2 different versions of
the same library in the same dependency graph is at least complicated, if not directly
violating ODR or causing linking errors. It can be set to ``False`` in advanced scenarios,
when we want to use different versions of the same package during the build.

transitive_headers
~~~~~~~~~~~~~~~~~~

If ``True`` the headers of the dependency will be visible downstream.

transitive_libs
~~~~~~~~~~~~~~~

If ``True`` the libraries to link with of the dependency will be visible downstream.

test
~~~~

This requirement is a test library or framework, like Catch2 or gtest. It is mostly a
library that needs to be included and linked, but that will not be propagated downstream.

package_id_mode
~~~~~~~~~~~~~~~

If the recipe wants to specify how the dependency version affects the current package
``package_id``, can be directly specified here.

While it could be also done in the ``package_id()`` method, it seems simpler to be able to
specify it in the ``requires`` while avoiding some ambiguities.

.. code-block:: python
# We set the package_id_mode so it is part of the package_id
self.tool_requires("tool/1.1.1", package_id_mode="minor_mode")
Which would be equivalent to:

.. code-block:: python
def package_id(self):
self.info.requires["tool"].minor_mode()
force
~~~~~

This ``requires`` will force its version in the dependency graph upstream, overriding
other existing versions even of transitive dependencies, and also solving potential
existing conflicts.

override
~~~~~~~~

The same as the ``force`` trait, but not adding a ``direct`` dependency. If there is no
transitive dependency to override, this ``require`` will be discarded. This trait only
exists at the time of defining a ``requires``, but it will not exist as an actual
``requires`` once the graph is fully evaluated

direct
~~~~~~

If the dependency is a direct one, that is, it has explicitly been declared by the current
recipe, or if it is a transitive one.


validate_build()
----------------

The ``validate_build()`` method is used to verify if a configuration is valid for building a package. It is different
from the ``validate()`` method that checks if the binary package is "impossible" or invalid for a given configuration.

The ``validate_build()`` method has to use always the ``self.settings`` and ``self.options``:

.. code-block:: python
from conan import ConanFile
from conan.errors import ConanInvalidConfiguration
class myConan(ConanFile):
name = "foo"
version = "1.0"
settings = "os", "arch", "compiler"
def package_id(self):
# For this package, it doesn't matter the compiler used for the binary package
del self.info.settings.compiler
def validate_build(self):
# But we know this cannot be build with "gcc"
if self.settings.compiler == "gcc":
raise ConanInvalidConfiguration("This doesn't build in GCC")
.. toctree::
:maxdepth: 1
:hidden:

methods/build
methods/build_id
methods/build_requirements
methods/compatibility
methods/configure
methods/config_options
methods/export
methods/export_sources
methods/generate
methods/init
methods/layout
methods/package
methods/package_id
methods/package_info
methods/requirements
methods/set_name
methods/set_version
methods/source
methods/system_requirements
methods/test
methods/validate
methods/validate_build


- :doc:`build() <methods/build>`: Contains the build instructions to build a package from source
- :doc:`build_id() <methods/build_id>`: Allows reusing the same build to create different package binaries
- :doc:`build_requirements() <methods/build_requirements>`: Defines ``tool_requires`` and ``test_requires``
- :doc:`compatibility() <methods/compatibility>`: Defines binary compatibility at the recipe level
- :doc:`configure() <methods/configure>`: Allows configuring settings and options while computing dependencies
- :doc:`config_options() <methods/config_options>`: Configure options while computing dependency graph
- :doc:`export() <methods/export>`: Copies files that are part of the recipe
- :doc:`export_sources() <methods/export_sources>`: Copies files that are part of the recipe sources
- :doc:`generate() <methods/generate>`: Generates the files that are necessary for building the package
- :doc:`init() <methods/init>`: Special initialization of recipe when extending from ``python_requires``
- :doc:`layout() <methods/layout>`: Defines the relative project layout, source folders, build folders, etc.
- :doc:`package() <methods/package>`: Copies files from build folder to the package folder.
- :doc:`package_id() <methods/package_id>`: Defines special logic for computing the binary ``package_id`` identifier
- :doc:`package_info() <methods/package_info>`: Provide information for consumers of this package about libraries, folders, etc.
- :doc:`requirements() <methods/requirements>`: Define the dependencies of the package
- :doc:`set_name() <methods/set_name>`: Dynamically define the name of a package
- :doc:`set_version() <methods/set_version>`: Dynamically define the version of a package.
- :doc:`source() <methods/source>`: Define the dependencies of the package
- :doc:`system_requirements() <methods/system_requirements>`: Call system package managers like Apt to install system packages
- :doc:`test() <methods/test>`: Run some simple package test (exclusive of ``test_package``)
- :doc:`validate() <methods/validate>`: Define if the current package is invalid (cannot work) with the current configuration.
- :doc:`validate_build() <methods/validate_build>`: Define if the current package cannot be created with the current configuration.
52 changes: 52 additions & 0 deletions reference/conanfile/methods/build.rst
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
.. _reference_conanfile_methods_build:

build()
=======

The ``build()`` method is used to define the build from source of the package. In practice this means calling some build system, which could be done explicitly or using any of the build helpers provided by Conan:


.. code-block:: python
from conan.tools.cmake import CMake
class Pkg(ConanFile):
def build(self):
# Either using some of the Conan built-in helpers
cmake = CMake(self)
cmake.configure() # equivalent to self.run("cmake . <other args>")
cmake.build() # equivalent to self.run("cmake --build . <other args>")
cmake.test() # equivalent to self.run("cmake --target=RUN_TESTS")
# Or it could run your own build system or scripts
self.run("mybuildsystem . --configure")
self.run("mybuildsystem . --build")
For more information about the existing built-in build system integrations, visit :ref:`conan_tools`.

The ``build()`` method should be as simple as possible, just wrapping the command line invocations
that a developer would do in the simplest possible way. The ``generate()`` method is the one responsible
for preparing the build, creating toolchain files, CMake presets, or any other files which are necessary
so developers could easily call the build system by hand. This allows for much better integrations with IDEs and
improves the developer experience. The result is that in practice the ``build()`` method should be relatively simple.

The ``build()`` method is the right place to build and run unit tests, before packaging, and raising errors if those tests fail, interrupting the process, and not even packaging the final binaries.
The built-in helpers will skip the unit tests if the ``tools.build:skip_test`` configuration is defined. For custom integrations, it is expected that the method checks this ``conf`` value in order to skip building and running tests, which can be useful for some CI scenarios.

The ``build()`` method runs once per unique configuration, so if there are some source operations like applying patches that are done conditionally to different configurations, they could be also applied in the
``build()`` method, before the actual build. It is important to note that in this case the ``no_copy_source`` attribute cannot be set to ``True``.



.. note::

**Best practices**

- The ``build()`` method should be as simple as possible, the heavy lifting of preparing the build should happen in the ``generate()`` method in order to achieve a good developer experience that can easily build locally with just ``conan install .``, plus directly calling the build system or opening their IDE.


.. seealso::

Follow the :ref:`tutorial about building packages<tutorial_creating_build>` for more information about building from sources.
58 changes: 58 additions & 0 deletions reference/conanfile/methods/build_id.rst
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
.. _reference_conanfile_methods_build_id:

build_id()
==========

The ``build_id()`` method allows to re-use the same build to create different binary packages in the cache,
potentially saving build time as it can avoid some unnecessary re-builds. It is therefore an optimization method.

In the general case, there is one build folder for each binary package, with the exact same ``package_id`` of the package. However this behavior
can be changed, there are a couple of scenarios that this might be useful:

- The package build scripts generate several different configurations at once (like both debug and release artifacts) in the same run, without the possibility of building each configuration separately.
- The package build scripts generate one binary configuration, but different artifacts that can be packaged separately. For example if there are some test executables, you might want to create two packages: one just containing the library for general usage, and another one also containing the tests (for compliance, later reproducibility, debugging, etc).

In the first case, we could for example write:

.. code-block:: python
settings = "os", "compiler", "arch", "build_type"
def build_id(self):
self.info_build.settings.build_type = "Any"
This recipe will generate a final different package with a different ``package_id`` for debug and release configurations. But as the ``build_id()`` will generate the
same ``build_id`` for any ``build_type``, then just one folder and one ``build()`` will be done, building both debug and release artifacts,
and then the ``package()`` method will be called for each configuration, and it should package the artifacts conditionally to the ``self.settings.build_type`` value. Different builds will still be
executed if using different compilers or architectures.

Other information like custom package options can also be changed:

.. code-block:: python
def build_id(self):
self.info_build.options.myoption = 'MyValue' # any value possible
self.info_build.options.fullsource = 'Always'
If the ``build_id()`` method does not modify the ``info_build`` data, and it still produces a different id than
the ``package_id``, then the standard behavior will be applied. Consider the following:

.. code-block:: python
settings = "os", "compiler", "arch", "build_type"
def build_id(self):
if self.settings.os == "Windows":
self.info_build.settings.build_type = "Any"
This will only produce a different ``build_id`` if the package is for Windows, thus running ``build()`` just
once for all ``build_type`` values. The behavior
in any other OS will be the standard one, as if the ``build_id()`` method was not defined, running
one different ``build()`` for each ``build_type``.


.. note::

**Best practices**

Conan strongly recommends to use one package binary with its own ``package_id`` for each different configuration. The goal of the ``build_id()`` method is to deal with legacy build scripts that cannot easily be changed to do the build of one configuration each time.
58 changes: 58 additions & 0 deletions reference/conanfile/methods/build_requirements.rst
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
.. _reference_conanfile_methods_build_requirements:

build_requirements()
====================

The ``build_requirements()`` method is functionally equivalent to the ``requirements()`` one, it is executed just after it. It is not strictly necessary, in theory everything that is inside this method, could be done in the end of the ``requirements()`` one. Still, ``build_requirements()`` is good for having a dedicated place to define ``tool_requires`` and ``test_requires``:

.. code-block:: python
def build_requirements(self):
self.tool_requires("cmake/3.23.5")
self.test_requires("gtest/1.13.0")
For simple cases the attribute syntax can be enough, like ``tool_requires = "cmake/3.23.5"`` and ``test_requires = "gtest/1.13.0"``. The method form can be necessary for conditional or parameterized requirements.

The ``tool_requires`` and ``test_requires`` methods are just a specialized instance of ``requires`` with some predefined trait values. See the :ref:`requires() reference<reference_conanfile_methods_requirements>` for more information about traits.

tool_requires
-------------

The ``tool_requires`` is equivalent to ``requires()`` with the following traits:

- ``build=True``. This dependency is in the "build" context, being necessary at build time, but not at application runtime, and will receive the "build" profile and configuration.
- ``visible=False``. The dependency to a tool requirement is not propagated downstream. For example, one package can call ``tool_requires("cmake/3.23.5")``, but that doesn't mean that the consumer packages also use ``cmake``, they could even use a different build system, or a different version, without causing conflicts.
- ``run=True``. This dependency has some executables or runtime that needs to be ran at build time.
- ``headers=False`` A tool requirement does not have headers.
- ``libs=False``: A tool requirement does not have libraries to be linked by the consumer (if it had libraries they would be in the "build" context and could be incompatible with the "host" context of the consumer package).

test_requires
-------------

The ``test_requires`` is equivalent to ``requires()`` with the following traits:

- ``test=True``. This dependency is a "test" dependency, existing in the "host" context, but not aiming to be part of the final product.
- ``visible=False``. The dependency to a test requirement is not propagated downstream. For example, one package can call ``self.test_requires("gtest/1.13.0")``, but that doesn't mean that the consumer packages also use ``gtest``, they could even use a different test framework, or the same ``gtest`` with a different version, without causing conflicts.


It is possible to further modify individual traits of ``tool_requires()`` and ``test_requires()`` if necessary, for example:

.. code-block:: python
def build_requirements(self):
self.tool_requires("cmake/3.23.5", options={"shared": False})
.. note::

**Best practices**

- ``tool_requires`` are exclusively for build time **tools**, not for libraries that would be included and linked into the consumer package. For libraries with some special characteristics, use a ``requires()`` with custom trait values.
- The ``self.test_requires()`` and ``self.tool_requires()`` methods should exclusively be used in the ``build_requirements()`` method, with the only possible exception being the ``requirements()`` method. Using them in any other method is forbidden. To access information about dependencies when necessary in some methods, the :ref:`self.dependencies<conan_conanfile_model_dependencies>` attribute should be used.


.. seealso::

- Follow the :ref:`tutorial about consuming Conan packages as tools<consuming_packages_tool_requires>`.
- Read the :ref:`tutorial about creating tool_requires packages<tutorial_other_tool_requires_packages>`.
Loading

0 comments on commit 900bc27

Please sign in to comment.