From 78220a44bb19a0f8e556d81b6851888ab47b2b81 Mon Sep 17 00:00:00 2001 From: dmerejkowsky Date: Sat, 16 Dec 2023 15:36:51 +0000 Subject: [PATCH] =?UTF-8?q?Deploying=20to=20gh-pages=20from=20@=20your-too?= =?UTF-8?q?ls/tsrc@4330356696708f35bd7404acf8165b31ff7d8ee1=20=F0=9F=9A=80?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ref/cli/index.html | 4 ++-- ref/sync/index.html | 7 ++++--- search/search_index.json | 2 +- sitemap.xml.gz | Bin 127 -> 127 bytes 4 files changed, 7 insertions(+), 6 deletions(-) diff --git a/ref/cli/index.html b/ref/cli/index.html index 5e0eb722..7f1d2f98 100644 --- a/ref/cli/index.html +++ b/ref/cli/index.html @@ -988,10 +988,10 @@

Usage&
  • Shows repositories not on the expected branch
    • -
    tsrc sync [--correct-branch/-c]
    +
    tsrc sync [--no-correct-branch]
    Updates all the repositories and shows a summary at the end. If any of the repositories is not on the configured branch, but it is clean -and the --correct-branch/-c flag is set, then the branch is changed to +and the --no-correct-branch flag is NOT set, then the branch is changed to the configured one and then the repository is updated. Otherwise that repository will not be not updated.
    tsrc version
    diff --git a/ref/sync/index.html b/ref/sync/index.html index af30f447..ac7d9cb1 100644 --- a/ref/sync/index.html +++ b/ref/sync/index.html @@ -789,9 +789,10 @@

    Sync algorithmtsrc is a command-line tool that helps you manage groups of git repositories.

    It works by listing the repositories in a file called manifest.yml that looks like this:

    repos:\n  - dest: foo\n    url: git@example.com:foo.git\n\n  - dest: bar\n    url: git@example.com:bar.git\n

    You can then use:

    • tsrc init <manifest url> to create a workspace containing the foo and bar repository

    • tsrc sync to synchronize all repos in the workspace.

    • ... and many more commands. Run tsrc help to list them, or read the command line reference

    "},{"location":"#tutorial","title":"Tutorial","text":"

    Interested in using tsrc in your own organization?

    Proceed to the getting started tutorial!

    "},{"location":"#guides","title":"Guides","text":"

    Once you've learn how to setup tsrc for your organization, feel free to read the following guides - tsrc supports a variety of use cases beyond just listing git repositories to be cloned or synchronized and are described here:

    • Editing the manifest safely
    • Editing workspace configuration
    • Using groups
    • Using several remotes
    • Using fixed git references
    • Performing file system operations
    • Running a command for each repo in the workspace
    • Using tsrc with continuous integration
    "},{"location":"#reference","title":"Reference","text":"
    • Command line interface
    • Sync algorithm
    • Manifest configuration
    • Workspace configuration
    "},{"location":"#contributing","title":"Contributing","text":"
    • Using the issue tracker
    • Suggesting changes
    • Code Manifesto
    "},{"location":"#useful_links","title":"Useful links","text":"
    • FAQ
    • Changelog
    "},{"location":"changelog/","title":"Changelog","text":""},{"location":"changelog/#271_2022-05-30","title":"2.7.1 (2022-05-30)","text":"
    • In addition to preserving the order repos are listed in the manifest, tsrc now makes sure repos included via groups are processed before the other repos. See #356 for details. Thanks to @raabf for the bug report and code review!
    "},{"location":"changelog/#270_2022-05-14","title":"2.7.0 (2022-05-14)","text":"
    • Show which git commands are run by default.
    • tsrc init: fix order of operations - clone the local manifest before writing the workspace configuration. Fixes #344, where users could not run init a second time if the previous call failed. Bug report by @cgestes.
    • tsrc init: do not assume the default branch of the manifest is master. Note that master is still hard-coded in a few places. See #347 for details.
    • Breaking: When using -j 1, do not sort repositories by lexical order of destination, but preserve the order in which they were specified in the manifest. Suggested by @raabf.
    "},{"location":"changelog/#260_2022-03-27","title":"2.6.0 (2022-03-27)","text":"
    • Allow to use python -m tsrc in addition to just tsrc
    • Documentation updates
    • Bump mypy
    "},{"location":"changelog/#250_2022-12-21","title":"2.5.0 (2022-12-21)","text":""},{"location":"changelog/#highlights","title":"Highlights","text":"
    • Bump minimum supported version to Python 3.7.
    • Introduce ignore_submodules repository option - Patch by Thomas Hiscock.
    • When -j is not used, try getting the default jobs value from the TSRC_PARALLEL_JOBS environment variable. Patch by Marcin Jaworski.
    "},{"location":"changelog/#other","title":"Other","text":"
    • Bump mkdocs from 1.2.2 to 1.2.3
    • Bump pygit2
    • Bump linters (flake8, mypy, black)
    • Remove dependency on attr
    "},{"location":"changelog/#241_2021-11-05","title":"2.4.1 (2021-11-05)","text":"
    • Ad more guides in the documentation
    • Display absolute paths when performing file system operations
    • tsrc sync now uses parallel jobs by default. Use -j1 to force sequential processing. Patch by @gdubicki
    "},{"location":"changelog/#240_2021-08-22","title":"2.4.0 (2021-08-22)","text":""},{"location":"changelog/#highlights_1","title":"Highlights","text":"

    • All of tsrc commands can now be run in parallel. Try for instance tsrc sync -j auto.

    • tsrc foreach now sets a bunch of environment variables. This allows developers to add new behaviors to tsrc without having to change its source code. See the relevant guide for more information.

    • Augment documentation with more use cases and examples (still a work in progress)

    "},{"location":"changelog/#breaking_changes","title":"Breaking changes","text":"

    • Remove tsrc version - Use tsrc --version instead.

    • The 'parallel' feature caused the output of some commands like foreach or log to change slightly. Hopefully tsrc output is now more consistent.

    "},{"location":"changelog/#bug_fixes","title":"Bug fixes","text":"

    • Fix crash when running tsrc without any arguments

    • Fix crash when trying to clone repositories in some rare corner cases (like the destination existing but not being a directory)

    "},{"location":"changelog/#internal_changes","title":"Internal changes","text":"
    • Make all tsrc imports consistent
    • Fix error when calling repr on tsrc Errors.
    "},{"location":"changelog/#231_2021-06-28","title":"2.3.1 (2021-06-28)","text":""},{"location":"changelog/#bug_fixes_and_small_improvements","title":"Bug fixes and small improvements","text":"
    • Fix #268: tsrc apply-manifest now performs file system operation
    • Always display workspace path at the beginning of any action
    • Skip \"performing file system operations\" message if there is no work to be done
    "},{"location":"changelog/#internal_changes_1","title":"Internal changes","text":"
    • Bump linters and formatters (black, mypy, isort ...)
    • Use copier to simplify maintenance of tools configuration
    "},{"location":"changelog/#230_2021-05-31","title":"2.3.0 (2021-05-31)","text":""},{"location":"changelog/#repo_selection","title":"Repo selection","text":"
    • Added -r (regex) and -i (inverse regex) params for filtering repos. Patch by @xzr
    "},{"location":"changelog/#add_support_for_submodules","title":"Add support for submodules","text":"
    • tsrc calls git clone with --recurse-submodules when adding missing repositories
    • tsrc calls git submodule update --init --recursive when updating repositories
    "},{"location":"changelog/#misc","title":"Misc","text":"
    • Remove codecov usage
    • Rename default branch to main.
    "},{"location":"changelog/#221_2021-04-10","title":"2.2.1 (2021-04-10)","text":"

    • Project has been moved from TankerHQ organization to dmerejkowsky. New urls are:

      • github.com/dmerejkowsky/tsrc for the git repository
      • dmerejkowsky.github.io/tsrc for the documentation

    • Add CI jobs to check this project also works with Python 3.9

    • Drop Path Pie dependency
    • Minor internal fixes
    • Add more URLs in the metadata (and pypi.org project page)
    "},{"location":"changelog/#220_2020-07-17","title":"2.2.0 (2020-07-17)","text":""},{"location":"changelog/#add_symlink_support","title":"Add symlink support","text":"

    tsrc sync and tsrc init can now create symlinks as specified in the manifest file:

    repos:\n\n  - url: git@gitlab.local:proj1/app\n    dest: app\n    symlink:\n      - source: app/some_file\n        target: ../some_file\n

    In this case, a symlink will be created from <workspace>/app/some_file to <workspace>/some_file. (both source and target keys are relative to the repository's destination).

    "},{"location":"changelog/#changes_related_to_groups","title":"Changes related to groups","text":"
    • log, status, and sync all learned about the --group option and the --all-cloned options
    • foreach: remove the --groups-from-config options since this is now the default behavior
    "},{"location":"changelog/#misc_1","title":"Misc","text":"
    • Rework FAQ
    • Run black in lint.sh
    • Fix formatting of some messages
    • Update code manifesto to suggest using docstrings in tests
    "},{"location":"changelog/#210_2020-05-27","title":"2.1.0 (2020-05-27)","text":""},{"location":"changelog/#breaking_changes_1","title":"Breaking changes","text":""},{"location":"changelog/#change_in_manifest_syntax","title":"Change in manifest syntax","text":"

    It was discovered that the manifest syntax was confusing for newcomers, so we decided to update it.

    In particular, the src key meant both a relative path in the workspace when used in the repo config, and a relative path in the a repository when using in the repo.copy config.

    Starting with this release, repo.src becomes repo.dest and repo.copy.src becomes repo.copy.file.

    # Before (tsrc < 2.1.0)\nrepos:\n  url: \"https://acme.corp/foo\"\n  src: foo\n  copy:\n     src: some-file\n     dest: some-file\n
    # After (tsrc >= 2.1.0)\nrepos:\n  url: \"https://acme.corp/foo\"\n  dest : foo\n  copy:\n     file: some-file\n     dest: some-file\n

    This should make it clearer what tsrc does because:

    • dest now always refers to a relative path in the workspace (both in repo and copy).
    • By using repo.copy.file it's obvious that tsrc only supports copying files, not directories.
    "},{"location":"changelog/#supported_python_versions","title":"Supported Python versions","text":"

    Drop support for Python 3.5

    "},{"location":"changelog/#new_features","title":"New features","text":"
    • tsrc init learned a -r, --remote option that pins the remote with the given name as the only remote to be used for cloning and syncing. tsrc expects this remote to be present in the manifest for all repositories. This is useful if you use the same workspace in different physical locations, and one of the remotes is behind a VPN for instance. Patch by @tronje.
    "},{"location":"changelog/#bug_fixes_1","title":"Bug fixes","text":"
    • Fix #217: Preserves file attributes during the copy statements in repos
    "},{"location":"changelog/#other_1","title":"Other","text":"
    • The whole test suite now runs without errors on Windows - and Windows support is now part of the GitHub actions checks.
    • The tests now run faster and with more readable output (this was done by using libgit2 instead of running git commands in the tests helpers).
    • Add a scheduled GitHub action to run safety
    • Remove usage of deprecated API of the path library.
    • Run tests and linters for external pull requests too.
    "},{"location":"changelog/#v200_-_2020-04-06","title":"v2.0.0 - (2020-04-06)","text":"

    • Remove the tsrc push command and all review automation features. Please use hub, lab, or repo instead. See #207 for the discussion leading to this removal.

    • Implement small improvements on tsrc output messages.

    • Add tsrc apply-manifest, to apply changes in a manifest file locally, without having to make a commit and push to a server first.
    "},{"location":"changelog/#v103_-_2020-02-05","title":"v1.0.3 - (2020-02-05)","text":"
    • Use poetry for dependency management and packaging.
    "},{"location":"changelog/#v102_-_2020-01-29","title":"v1.0.2 - (2020-01-29)","text":"
    • Fix python_requires value in project metadata
    "},{"location":"changelog/#v101_-_2020-01-21","title":"v1.0.1 - (2020-01-21)","text":"
    • Fix #196: Do not attempt file copies for non-cloned repositories when using tsrc init with a list of groups.
    "},{"location":"changelog/#v100_-_2020-01-09","title":"v1.0.0 - (2020-01-09)","text":"

    Starting the new year with a stable release, at last!

    "},{"location":"changelog/#revamp_group_ux","title":"Revamp group UX","text":"

    The changes below in the configuration file and command line syntax allow for better UX regarding groups. See the corresponding milestone for the full list.

    "},{"location":"changelog/#new_configuration_file","title":"New configuration file","text":"

    Previously, tsrc stored its permanent configuration in .tsrc/manifest.yml and the file was not supposed to be edited by hand. Instead, users could use tsrc init to modify it, for instance with the --branch argument.

    Starting with this release, the command tsrc init can only be run once per workspace, and you must edit the .tsrc/config.yml file instead.

    "},{"location":"changelog/#changes_in_command_line_syntax","title":"Changes in command line syntax","text":"
    • tsrc init: remove --file option.
    • tsrc foreach: instead of repeating the --group option, you can use --groups with a list of groups:
    # before\ntsrc init --group foo --group bar\n\n# after\ntsrc init --groups foo bar\n

    • tsrc init learned a --clone-all-repos option to clone all repositories from the manifest, regardless of the groups. Fix #181

    • Remove --file option from tsrc init.

    • tsrc foreach learned a --groups-from-config option to use the groups configured in the workspace. Fix #178, #179.

    • tsrc push learned a -o, --origin option to specify a remote name different from \"origin\". Fix #170

    "},{"location":"changelog/#other_fixes","title":"Other fixes","text":"
    • Try and check that GitLab installation support required features before using them - typically, using tsrc push --approvers on GitLab Community Edition. (#165)
    • reported by @irizzant.
    • Switch to GitHub actions for running tests and linters. Also, publish documentation automatically when something is pushed to the master branch.
    • tsrc status : add information when local branch does not match manifest configuration. (#190). Feature suggested by @janjachnick
    "},{"location":"changelog/#v092_-_2019-09-30","title":"v0.9.2 - (2019-09-30)","text":"
    • Additional bug fix for #165 - the fix in 0.9.1 was incomplete
    • Improve error message when trying to use non-supported GitLab features (like using tsrc push --reviewer on GitLab Community Edition)
    "},{"location":"changelog/#v091_-_2019-09-23","title":"v0.9.1 - (2019-09-23)","text":"
    • Improve error message when tsrc foreach fails to start the process. Suggested by @dlewis-ald in #163
    • Fix crash when finding reviewers for a GitLab project not in a group. Reported by @irizzant in #165
    "},{"location":"changelog/#v090_-_2019-08-13","title":"v0.9.0 - (2019-08-13)","text":"
    • Add support for GitHub Enterprise: patch by @sdavids13.
    • Improve error message when using creating a merge request in a GitLab repository when the token cannot be found in the tsrc configuration file. Fix #158
    • Fix crash when running tsrc status on a workspace with missing repositories (#160) - reported by @blastrock
    "},{"location":"changelog/#v080_-_2019-08-12","title":"v0.8.0 - (2019-08-12)","text":"
    • Implement tsrc sync --force. Currently all it does is running git fetch --force on all repositories. Use with caution. See #152 for details.
    "},{"location":"changelog/#v071_-_2019-08-02","title":"v0.7.1 - (2019-08-02)","text":"
    • Fix crash in tsrc sync when the repo configuration in the manifest contained neither an URL nor a remote. tsrc now aborts as soon as the misconfiguration of the manifest is detected (Reported by @jongep86)
    "},{"location":"changelog/#v070_2019-07-08","title":"v0.7.0 (2019-07-08)","text":"
    • Add a --file option to tsrc init so that manifest can be read from a custom path in the file system
    • Remove support for Python 3.4
    • Switch from xdg to pyxdg
    • Format the code with black
    "},{"location":"changelog/#v066_2019-04-02","title":"v0.6.6 (2019-04-02)","text":"
    • Remove raw HTML from README.rst
    "},{"location":"changelog/#v065_2019-04-0","title":"v0.6.5 (2019-04-0)","text":"
    • Use codecov.io to measure coverage
    • Prettify README
    "},{"location":"changelog/#v064_2019-01-07","title":"v0.6.4 (2019-01-07)","text":"
    • Remove support for Python 3.3.
    • Use new and shiny cli-ui package instead of old python-cli-ui.
    "},{"location":"changelog/#v063_2018-11-04","title":"v0.6.3 (2018-11-04)","text":"
    • GitHub organization is now TankerHQ
    • We now use dmenv for dependencies management
    "},{"location":"changelog/#v062_2018-10-19","title":"v0.6.2 (2018-10-19)","text":"

    Fix crash when using tsrc push on a GitHub repository for the first time.

    "},{"location":"changelog/#v061_2018-10-10","title":"v0.6.1 (2018-10-10)","text":"

    Fix weird output when configuring remotes.

    "},{"location":"changelog/#v060_2018-10-09","title":"v0.6.0 (2018-10-09)","text":""},{"location":"changelog/#add_support_for_multiple_remotes","title":"Add support for multiple remotes","text":"
    # still valid (implicit 'origin' remote)\nsrc: foo\nurl: git@github.com/foo\n\n# also valid (two explicit remotes)\nsrc: foo\nremotes:\n  - { name: origin, url: git@github.com:john/foo }\n  - { name: upstream, url: git@github.com:foo/foo}\n\n# not valid (ambiguous)\nsrc: foo\nurl: git@github.com:john/foo\nremotes:\n   - { name: upstream, url: git@github.com:foo/foo }\n

    Thanks @tst2005 and @cgestes for their help with the configuration format.

    "},{"location":"changelog/#tsrc_foreach","title":"tsrc foreach","text":"
    • tsrc foreach: add a --group option to select the repositories to run the command on. Fix #40
    "},{"location":"changelog/#other_fixes_1","title":"Other fixes","text":"
    • Fix #113: do not hide branch when showing tag status.
    • Add support for Python 3.7
    "},{"location":"changelog/#v050_2018-08-14","title":"v0.5.0 (2018-08-14)","text":"
    • Add support for setting approvers with the -r,--approvers option in tsrc push (GitLab Enterprise Edition only).
    "},{"location":"changelog/#v041_2018-04-27","title":"v0.4.1 (2018-04-27)","text":"
    • Fixed regression: tsrc push was no longer able to create a merge request on GitLab if --target was not set.
    "},{"location":"changelog/#v040_2018-04-26","title":"v0.4.0 (2018-04-26)","text":""},{"location":"changelog/#highlights_2","title":"Highlights","text":"
    • Preliminary GitHub support
    • tsrc push: new features and bug fixes
    • Improved fixed reference handling
    • Support for shallow clones

    See below for the details.

    "},{"location":"changelog/#preliminary_github_support","title":"Preliminary GitHub support","text":"
    • Added support for creating merge requests on GitHub. No configuration required. Just make sure you are using tsrc from a repository which has a URL starting with git@github.com.

    tsrc will prompt you once for your login and password and then store an API token.

    Afterwards, you'll be able to use tsrc push to:

    • Create a pull request (or update it if it already exists)
    • Assign people to the request (with the -a/--assignee option)
    • Request reviewers (with the --reviewers option)
    • Merge the pull request (with the --merge option)

    This change has no impact if you were already using GitLab.

    "},{"location":"changelog/#tsrc_push_new_features_and_bug_fixes","title":"tsrc push: new features and bug fixes","text":"
    • Add --close option.
    • Breaking change: -m/--message option is gone, use --title instead. There's a concept of \"description\" or \"message\" for pull requests and merge requests, but the value of the option was only used to update the title, so it had to be renamed.
    • Do not assume local and remote tracking branch have the same name.
    • Allow using tsrc push <local>:<remote> to explicitly specify local and remote branch names.
    • Fix bugs when target is not specified on the command line. See this commit for details.
    • Fix missing merge requests in tsrc push (see issue #80). Patch by @maximerety.
    "},{"location":"changelog/#improve_fixed_reference_handling","title":"Improve fixed reference handling","text":"

    Breaking change: Instead of using fixed_ref in the manifest, you should now use tag or sha1:

    old: 

    repos:\n  - src: git@example.com/foo\n    fixed_ref: 42a70\n

    new: 

    repos:\n  - src: git@example.com/foo\n    tag: v0.1\n

    See the dedicated section about manifest format and the #57 pull request discussion for the details.

    This allow us to implement different behaviors depending on whether or not the fixed ref is a tag or just a sha1.

    "},{"location":"changelog/#support_for_shallow_clones","title":"Support for shallow clones","text":"

    To save time and space, you can use tsrc init --shallow to only have shallow clones in your workspace.

    Note that due to limitations in git itself, the shallow option cannot be used with a fixed SHA1. If you need this, prefer using a tag instead.

    "},{"location":"changelog/#misc_2","title":"Misc","text":"

    • Organization TankerApp was renamed to TankerHQ. New urls are:

      • github.com/TankerHQ/tsrc for the git repository
      • TankerHQ.github.io/tsrc for the documentation

    • We now use pipenv for dependency handling.

    "},{"location":"changelog/#v032_2017-11-02","title":"v0.3.2 (2017-11-02)","text":"
    • Improve tsrc status to handle tags. Patch by @arnaudgelas.
    • Fix crash when running tsrc version.
    "},{"location":"changelog/#v031_2017-10-06","title":"v0.3.1 (2017-10-06)","text":"
    • Improve tsrc status output. Now also shows number of commits ahead and behind, and display a short SHA-1 when not on any branch. Initial patch by @arnaudgelas.
    "},{"location":"changelog/#v030_2017-09-22","title":"v0.3.0 (2017-09-22)","text":"

    Breaking change: Add support for groups (#30). Reported by @arnaudgelas.

    See the dedicated section about manifest format for details.

    Upgrading from v0.2.4:

    To upgrade from an older version of tsrc, you should re-run tsrc init with the correct url:

    # Check manifest URL:\n$ cd <workspace>/.tsrc/manifest\n$ git remote get-url origin\n# Note the url, for instance ssh://git@example.com:manifest.git\n$ cd <workspace>\n$ tsrc init <manifest-url>\n

    This is required to create the <workspace>/.tsrc/manifest.yml file which is later used by tsrc sync and other commands.

    "},{"location":"changelog/#v024_2017-07-13","title":"v0.2.4 (2017-07-13)","text":"
    • tsrc push --assignee: fix when there are more than 50 GitLab users (#25). Reported by @arnaudgelas
    "},{"location":"changelog/#v023_2017-09-01","title":"v0.2.3 (2017-09-01)","text":"

    • Split user interface functionality into its own project: python-cli-ui.

    • Add --quiet and --color global options.

    "},{"location":"changelog/#v022_2017-08-22","title":"v0.2.2 (2017-08-22)","text":"

    Bug fix release.

    • tsrc init: Fix crash when a repository is empty (#17). Reported by @nicolasbrechet
    • tsrc push: Fix rude message when credentials are missing (#20). Reported by @cgestes
    "},{"location":"changelog/#v021_2017-08-10","title":"v0.2.1 (2017-08-10)","text":"

    Packaging fixes.

    "},{"location":"changelog/#v020_2017-08-09","title":"v0.2.0 (2017-08-09)","text":"
    • Support for specifying custom branches in the manifest
    • Support for specifying fixed refs (tags or hashes) in the manifest

    New syntax is:

    repos:\n  - src: foo\n    url: git@gitlab.com:proj/foo\n    branch: next\n\n  - src: bar\n    url: git@gitlab.com:proj/bar\n    branch: master\n    fixed_ref: v0.1\n

    Note that branch is still required.

    • You can now skip the dest part of the copy section if src and dest are equal:
    copy:\n  - src:foo\n\n# same thing as\ncopy:\n - src: foo\n   dest: foo\n
    "},{"location":"changelog/#v014_2017-08-04","title":"v0.1.4 (2017-08-04)","text":"

    Support for Python 3.3, 3.4, 3.5 and 3.6

    "},{"location":"changelog/#v011_2017-08-02","title":"v0.1.1 (2017-08-02)","text":"

    First public release

    "},{"location":"code-manifesto/","title":"Code Manifesto","text":""},{"location":"code-manifesto/#basics","title":"Basics","text":"

    We use black to enforce a coding style matching PEP8.

    In addition, every text file must be pushed using UNIX line endings. (On Windows, you are advised to set core.autocrlf to true in your git config file.)

    "},{"location":"code-manifesto/#pet_peeves","title":"Pet peeves","text":"
    • Prefer double quotes for string literals:
    # Yes\ndef bar():\n   \"\"\" bar stuff \"\"\"\n   a = \"foo\"\n\n\n# No\ndef bar():\n   ''' bar stuff '''\n   a = 'foo'\n\n# Exception\nmy_str = 'It contains some \"quotes\" inside'\n

    • Use the fact that empty data structures are falsy: 

      # Yes\nif not errors:\n    ...\n# No\nif len(errors) == 0:\n    ...\n

    • Avoid using double negatives: 

      # Yes\ndef make_coffee(sugar=False):\n    if sugar:\n        print(\"with sugar\")\n\n# No\ndef make_coffee(without_sugar=True):\n    if not without_sugar:\n        print(\"with sugar\")\n

    • Prefer using \"f-strings\" if possible, + may also work in some contexts.

    # Yes\nmessage = f\"Welcome {name}!\"\n\n# No\nmessage = \"Welcome, {}!\".format(name)\nmessage = \"Welcome, %s!\" % name\nmessage = \"Welcome, \" + name + \"!\"\n\n# Okayish\nwith_ext = name + \".txt\"\n
    • Use textwrap.dedent() to build nice-looking multi-lines strings:
    # Yes\ndef foo():\n    long_message = textwrap.dedent(\"\"\"\\\n        first line\n        second line\n        third line\"\"\")\n\n# No\ndef foo():\n    long_message = \"\"\"\\\nfirst line\nsecond line\nthird line\n\"\"\"\n
    • Do not initialize several variables on the same line, unless they come from a tuple (for instance the return of a function, or a iteration on a directory)
    # Yes\nok, mess  = run_command()\n\nfor test_result in test_results:\n    outcome, message = res\n\n# No\nfoo, bar = False, \"\"\n\nclass Foo:\n    self.bar, self.baz = None, True\n
    • Do not use conditional expressions. The order is not the same as the ternary operator in C++ and JavaScript, so it should be avoided:
    # Yes\nif foo:\n   a = \"ok\"\nelse:\n   a = \"nope\"\n\n\n# No:\na = \"ok\" if foo else \"nope\"\n
    • Use if ... in ... when you can:
    # Yes\nif value in (\"option1\", \"option2\"):\n   ...\n\n# No\nif value == \"option1\" or value == \"option2\"\n  ...\n
    "},{"location":"code-manifesto/#doc_strings_and_comments_in_production_code","title":"Doc strings and comments in production code","text":"

    First off, bad comments are worse that no comments.

    Also note that you should use comments to explain why, never what. If the what is no clear, it means the behavior of the function or method cannot be easily understood by reading implementation, and so you should fix the implementation instead.

    In conclusion, use comments and doc strings sparingly: that way, they will not rot and they will stay useful.

    Note: this does not apply for tests (see below).

    "},{"location":"code-manifesto/#collections","title":"Collections","text":"
    • Use .extend() instead of += to concatenate lists:

    # Yes\nlist_1.extend(list_2)\n\n# No\nlist_1 += list_2\n
    * Only use list() and dict() to convert a value to a list or dict. Prefer literals when possible

    # Yes\nmy_list = []\nmy_dict = {}\n\n# Also yes:\nmy_list = list(yield_stuff())\n\n# No\nmy_list = list()\nmy_dict = dict()\n
    • Also use explicit call to list() in order to make a copy:
    # Yes\nmy_copy = list(my_list)\n\n# Also yes:\nmy_copy = copy.copy(my_list)\n\n# No\nmy_copy = my_list[:]\n
    • Use list comprehensions instead of loops and \"functional\" methods:
    # Yes\nmy_list = [foo(x) for x in other_list]\n\n# No\nmy_list = list()\nfor x in other_list:\n     x.append(foo(x))\n\n# Also no\nmy_list = map(foo, other_list)\n\n# Yes\neven_nums = [x for x in nums if is_even(x)]\n\n# No\neven_nums = filter(is_even, nums)\n
    • Use iterable syntax instead of building an explicit list:
    # Yes\nmax(len(x) for x in my_iterable)\n\n# No\nmax([len(x) for x in my_iterable])\n
    • Use plural names for collections. This has the nice benefit of allowing you to have meaningful loop names:
    for result in results:\n   # do something with result\n
    "},{"location":"code-manifesto/#functions","title":"Functions","text":"

    Prefer using keyword-only parameters when possible:

    # Yes\n# If the parameter needs a default value:\ndef foo(bar, *, spam=True):\n    ...\n\n# If it does not:\ndef foo(bar, *, spam):\n    ...\n\n\n# No\ndef foo(bar, spam=True):\n    ...\n

    If you use the last form, Python will let you use foo(42, False), and set spam to False. This can cause problems if someone ever changes the foo function and adds a new optional argument before spam:

    def foo(bar, eggs=False, spam=True):\n    ...\n
    After such a change, the line foo(42, False) which used to call foo with spam=False now calls foo with bar=False and spam=True, leading to all kinds of interesting bugs.

    Exception to this rule: when the keyword is obvious and will not change: 

    def get(value, default=None):\n  ...\n

    "},{"location":"code-manifesto/#imports","title":"Imports","text":"

    For any foo.py file, import foo must never fail, unless there is a necessary module that could not be found. Do not catch ImportError unless it is necessary, for instance to deal with optional dependencies.

    import required_module\n\nHAS_NICE_FEATURE = True\ntry:\n    import nice_lib\nexcept ImportError:\n    HAS_NICE_FEATURE = False\n\n#...\n\nif HAS_NICE_FEATURE:\n    #....\n

    • Importing Python files should never cause side effects. It's OK to initialize global variables, but you should never call functions outside a if __name__ == main() block.

    • Prefer using fully-qualified imports and names:

    # Yes\nimport foo.bar\nmy_bar = foo.bar.Bar()\n\n# No\nfrom foo import bar\nmy_bar = bar.Bar()\n

    Note

    We allow a few exceptions like from pathlib import Path or importing classes directory in tests. Use your best judgment.

    "},{"location":"code-manifesto/#classes","title":"Classes","text":"
    • When you want to make sure a class follows an interface, use abc.ABCMeta instead of raising NotImplementedError. This way you get the error when the class is instantiated instead of when the method is called.
    # Yes\nclass AbstractFoo(metaclass=abc.ABCMeta):\n    @abc.abstractmethod\n    def foo(self):\n        pass\n\n\n# No\nclass AbstractFoo:\n     def foo(self):\n        raise NotImplementedError()\n
    • Make sure to use properties when relevant, instead of get_ methods.
    # Yes\nclass Person:\n      def __init__(self, first_name, last_name):\n            self.first_name = first_name\n            self.last_name = last_name\n\n      @property\n      def full_name(self):\n          return f\"{self.first_name} {self.last_name}\"\n\n\n# No:\nclass Foo:\n      def __init__(self, first_name, last_name):\n            self.first_name = first_name\n            self.last_name = last_name\n            self.full_name = f\"{self.first_name} {self.last_name}\"\n

    For instance, here:

    • full_name is read-only
    • The attribute is automatically updated if first_name changes after the object is initialized.

    Note that get_ methods are OK if they do more than simple computations (expensive in time or size, throwing exceptions ...)

    "},{"location":"code-manifesto/#file_paths","title":"File paths","text":"
    • If you are manipulating filenames, use the path.pylibrary and suffix the variable by _path. Avoid using os.path or shutil methods when path.py is better.
    # Yes\nwork_path = Path(\"foo/work\")\nwork_path.mkdir_p()\nfoo_path = work_path / \"foo.txt\"\nfoo_path.write_text(\"this is bar\")\n\n# No\nwork_path = os.path.join(foo, \"work\")\nos.path.mkdir(work_path, exist_ok=True)\nfoo_path = os.path.join(work_path, \"foo.txt\")\nwith open(foo_path, \"w\") as fileobj:\n    fileobj.write(\"this is foo\")\n
    "},{"location":"code-manifesto/#error_handling","title":"Error handling","text":"
    • All exceptions raised from within tsrc should derive from tsrc.Error.
    • When using external code (from the standard library or a third-party library), you should catch the exceptions and optionally re-raise them.
    "},{"location":"code-manifesto/#output_messages_to_the_user","title":"Output messages to the user","text":"

    Do not use print, use python-cli-ui functions instead. This makes it easier to distinguish between real messages and the throw-away print statements you add for debugging.

    Also, using \"high-level\" methods such as ui.info_1() or ui.warning() will make it easier to have a consistent user interface.

    "},{"location":"code-manifesto/#tests","title":"Tests","text":""},{"location":"code-manifesto/#docstrings","title":"Docstrings","text":"

    If you think the test implementation is complex, add a human-readable description of the test scenario in the doc string.

    For instance:

    def test_sync_with_errors(...):\n    \"\"\"\" Scenario:\n    * Create a manifest with two repos (foo and bar)\n    * Initialize a workspace from this manifest\n    * Push a new file to the foo repo\n    * Create a merge conflict in the foo repo\n    * Run `tsrc sync`\n    * Check that the command fails and produces the proper error message\n    \"\"\"\n
    "},{"location":"code-manifesto/#assertions_with_lists","title":"Assertions with lists","text":"
    • Use tuple unpacking to write shorter assertions:
    # Yes\nactual_list = function_that_returns_list()\n(first, second) = actual_list\nassert first == something\nassert second == something_else\n\n# NO\nactual_list = function_that_returns_list()\nassert len(actual_list) == 2\nfirst = actual_list[0]\nsecond = actual_list[1]\nassert first == something\nassert second == something_else\n
    "},{"location":"code-manifesto/#assertion_order","title":"Assertion order","text":"

    When writing assertions, use the form assert <actual> == <expected>:

    # Yes\ndef test_foo():\n    assert foo(42) == True\n\ndef test_big_stuff():\n    actual_result = ...\n    expected_result = ...\n\n    assert actual_result == expected_result\n\n\n# No\ndef test_foo():\n    assert True == foo(42)\n\n\ndef test_big_stuff():\n    actual_result = ...\n    expected_result = ...\n\n    assert expected_result == actual_result\n

    Rationale:

    • The assert(expected, actual) convention comes from JUnit but we are not writing Java code, and besides, the assert(actual, expected) convention also exists in other tools.
    • pytest does not really care, but we prefer being consistent in all tests.
    • It's a bit closer to what you would say in English: \"Assert that the result of foo() is 42\".
    "},{"location":"faq/","title":"FAQ","text":""},{"location":"faq/#what_does_the_name_mean","title":"What does the name mean?","text":"

    The t stands for tool and src for sources.

    If you speak French, you can also remember the name as \"tes sources\".

    "},{"location":"faq/#why_not_repo","title":"Why not repo?","text":"

    We used repo for a while, but found that tsrc had both a better command line API and a nicer output.

    On a less subjective level:

    • Good support for Windows (no need for Cygwin or anything like that)

    • Also, tsrc tries hard to never do any destructive operation or unexpected actions.

      For instance, tsrc never puts you in a \"detached HEAD\" state, nor does automatic rebase. It also never touches dirty repositories.

      This is achieved by using mostly 'porcelain' commands from git, instead of relying on plumbings internals.

    Also (and this matters a lot if you think about contribution):

    • Uses PEP8 coding style, enforced with black
    • Comprehensive test suite
    • Fully type-checked with mypy

    Note that there are a few features present in repo that are missing from tsrc (but may be implemented in the future). Feel free to open a feature request if needed!

    "},{"location":"faq/#why_not_git-subrepo_mu-repo_or_gr","title":"Why not git-subrepo, mu-repo, or gr?","text":"

    All this projects are fine but did not match our needs:

    • git-subrepo squashes commits, and we prefer having normal clones everywhere.
    • mu-repo is nice and contains an interesting dependency management feature, but currently we do not need this complexity.

    In any case, now that the whole team is using tsrc all the time, it's likely we'll keep using tsrc in the future.

    "},{"location":"faq/#why_not_git_submodule","title":"Why not git submodule?","text":"

    It's all about workflow.

    With git-submodule, you have a 'parent' repository and you freeze the state of the 'children' repositories to a specific commit.

    It's useful when you want to re-build a library you've forked when you build your main project, or when you have a library or build tools you want to factorize across repositories: this means that each 'parent' repository can have its children on any commit they want.

    With tsrc, all repositories are equal, and what you do instead is to make sure all the branches (or tags) are consistent across repositories.

    For instance, if you have foo and bar, you are going to make sure the 'master' branch of foo is always compatible to the 'master' branch of bar.

    Or if you want to go back to the state of the '0.42' release, you will run: tsrc foreach -- git reset --hard v0.42.

    Note that since tsrc 0.2 you can also freeze the commits of some of the repositories.

    Last but not least, if you really need to use fixed references, you may do so by adding a sha1 or tag line to the manifest. See the relevant guide for more details.

    "},{"location":"faq/#why_not_using_pygit2_or_similar_instead_of_running_git_commands","title":"Why not using pygit2 or similar instead of running git commands?","text":"

    First off, we do use pygit2, but only for tests.

    Second, the pygit2 package depends on a 3rd party C library (libgit2) - and that can cause problems in certain cases. If we can, we prefer using pure-Python libraries for the production code.

    Finally, we prefer calling git \"porcelain\" commands, both for readability of the source code and ease of debugging (see below).

    "},{"location":"faq/#why_do_you_hide_which_git_commands_are_run","title":"Why do you hide which git commands are run?","text":"

    It's mainly a matter of not cluttering the output. We take care of keeping the output of tsrc both concise, readable and informative.

    That being said:

    • In case a git command fails, we'll display the full command that was run.
    • If you still need to see all the git commands that are run, we provide a --verbose flag, like so: tsrc --verbose sync
    "},{"location":"faq/#why_yaml","title":"Why YAML?","text":"

    It's nice to read and write, and we use the excellent ruamel.yaml which even has round-trip support.

    Also, being Python fans, we don't mind that white space is part of the syntax.

    "},{"location":"faq/#why_do_i_have_to_create_a_separate_git_repo_with_just_one_file_in_it","title":"Why do I have to create a separate git repo with just one file in it?","text":"

    See #235 for why you can't have multiple manifest files in the same repository.

    Also, note that you can put other files in the repo - for instance, add a CI script that verifies the yaml syntax and checks that all the repos in the manifest can be cloned.

    "},{"location":"getting-started/","title":"Getting started","text":""},{"location":"getting-started/#requirements","title":"Requirements","text":"

    Python 3.7 or later

    "},{"location":"getting-started/#installing_tsrc","title":"Installing tsrc","text":"

    The recommended way to install tsrc is to use pipx. This is because pipx automatically creates isolated environment for each app, so you won't get into dependencies versions conflicts and won't have to deal with manual virtualenvs management.

    pip will also work, but it will not give you these benefits.

    Recommended: 

    pipx install tsrc\n

    Acceptable, if you know what you are doing: 

    pip install tsrc\n

    "},{"location":"getting-started/#checking_tsrc_installation","title":"Checking tsrc installation","text":"

    Run:

    $ tsrc --version\n
    "},{"location":"getting-started/#creating_a_repository_for_the_manifest","title":"Creating a repository for the manifest","text":"

    Let's say you are working for the ACME company and you have many git repositories.

    You need a tool to track them, so that if a new repository is created, all developers can get a clone on their development machine quickly, without having to look up its URL or even know it exists.

    Also, you need to make sure the repos are cloned in a certain way, so that you can for instance refer a repo from an other one by using a relative path.

    This is where tsrc comes in.

    The first step is to create a dedicated repository for the manifest. I know it may sound wasteful (\"I have already 100 repositories to manage, and you want me to create yet an other one?\"), but, trust me, it's worth it.

    So, if your company uses a GitLab instance at gitlab.acme.com and you want to crate a manifest for your team, you may start by creating a new repository at https://gitlab.acme.com/your-team/manifest.

    Inside this repository, create a file named manifest.yml looking like this:

    repos:\n  - dest: foo\n    url: git@gitlab.acme.com/your-team/foo\n\n  - dest: bar\n    url: git@gitlab.acme.com/your-team/bar\n

    Note that this approach works with any king of Git Hosting system, not just a custom GitLab instance. Just replace gitlab.acme.com/your-team with the correct suffix (like github.com/your-name/ if you want to track some repositories from your GitHub account).

    "},{"location":"getting-started/#creating_a_new_workspace","title":"Creating a new workspace","text":"

    Create a new, empty directory and then run tsrc init from it, using the URL of the manifest created in the above step:

    $ mkdir work\n$ cd work\n$ tsrc init git@gitlab.acme.com/your-team/manifest.git\n

    You should see something like this:

    :: Configuring workspace in /path/to/work\nCloning into 'manifest'...\n...\n=> Cloning missing repos\n* (1/2) Cloning foo\nCloning into 'foo'...\n...\n* (2/2) Cloning bar\nCloning into 'bar'...\n...\n=> Cloned repos:\n* foo cloned from gt@gitlab.acme.com/your-team/foo' (on master)\n* bar cloned from gt@gitlab.acme.com/your-team/bar' (on master)\n=> Configuring remotes\n=> Workspace initialized\n=> Configuration written in /path/to/work/.tsrc/config.yml\n

    You will notice that:

    • The foo ad bar repositories have been cloned into their respective destination
    • A workspace configuration file has been created in /path/to/work/.tsrc/config.yml. This file can be edited by hand to customize tsrc behavior. Follow the relevant guide, or read the workspace configuration file reference for more details.
    "},{"location":"getting-started/#updating_a_new_workspace","title":"Updating a new workspace","text":"

    Now let's assume that Alice created a new commit in foo, and Bob a new commit it bar, and that they both pushed them to the master branch of the respective repositories.

    Now that you have a workspace configured with tsrc, you can use tsrc sync to retrieve all the changes in one go:

    $ cd work\n$ tsrc sync\n

    This time, you should see the following output:

    :: Using workspace in /path/to/work\n=> Updating manifest\n...\n=> Cloning missing repos\n=> Configuring remotes\n=> Synchronizing repos\n* (1/2) Synchronizing foo\n* Fetching origin\n...\n   f20af74..aca6c35  master     -> origin/master\n* Updating branch: master\nUpdating f20af74..aca6c35\nFast-forward\n new.txt | 1 +\n 1 file changed, 1 insertion(+)\n create mode 100644 new.txt\n* (2/2) Synchronizing bar\n* Fetching origin\n...\n   f20af74..02cfef6  master     -> origin/master\n* Updating branch: master\nUpdating f20af74..02cfef6\nFast-forward\n spam.py | 1 +\n 1 file changed, 1 insertion(+)\n create mode 100644 spam.py\n:: Workspace synchronized\n

    Note: tsrc sync does not call git pull on every repository. The precise algorithm is described in the reference documentation

    "},{"location":"getting-started/#adding_a_new_repo_to_the_manifest","title":"Adding a new repo to the manifest","text":"

    Let's say your team now needs a third repository (for instance, at gitlab.acme.com/your-team/baz).

    Start by making a commit in the manifest repository that adds the new repository:

    --- a/manifest.yml\n+++ b/manifest.yml\n@@ -1,2 +1,3 @@\n repos:\n   - dest: foo\n     url: git@gitlab.acme.com/your-team/foo\n\n   - dest: bar\n     url: git@gitlab.acme.com/your-team/baz\n\n+  - dest: baz\n+    url: git@gitlab.acme.com/your-team/baz\n

    Then push this commit to the master branch of the manifest.

    This time when you run tsrc sync:

    • the manifest repository will get updated
    • the baz repo will be cloned in /path/to/work/baz
    $ tsrc sync\n
    :: Using workspace in /path/to/work\n=> Updating manifest\nremote: Enumerating objects: 5, done.\n...\nUnpacking objects: 100% (3/3), 354 bytes | 354.00 KiB/s, done.\nFrom gitlab.acme.com/your-team/manifest\n   63f12d4..bbcd4d9  master     -> origin/master\nReset branch 'master'\n...\nHEAD is now at bbcd4d9 add baz\n\n=> Cloning missing repos\n* (1/1) Cloning baz\nCloning into 'baz'...\n...\nReceiving objects: 100% (3/3), done.\n=> Cloned repos:\n* baz cloned from git@gitlab.acme.com/bas (on master)\n=> Configuring remotes\n=> Synchronizing repos\n* (1/3) Synchronizing foo\n* Fetching origin\n* Updating branch: master\nAlready up to date.\n* (2/3) Synchronizing bar\n* Fetching origin\n* Updating branch: master\nAlready up to date.\n* (3/3) Synchronizing baz\n* Fetching origin\n* Updating branch: master\nAlready up to date.\n:: Workspace synchronized\n
    "},{"location":"getting-started/#going_further","title":"Going further","text":"

    In this tutorial, we made a lot of assumptions:

    • Every repository is using the master as the main development branch
    • Each repository as only one git remote (the one from gitlab.acme.com in our example)
    • You're using a manifest just for your team, not the whole company
    • ...

    tsrc can handle all of this use cases, and more. See the other guides for more details.

    "},{"location":"contrib/dev/","title":"Suggesting changes","text":"

    All the development happens on GitHub.

    You are free to open a pull request for anything you want to change on tsrc.

    In particular, pull requests that implement a prototype for a new feature are welcome, having \"real code\" to look at can provide useful insight, even if the code is not merged after all.

    That being said, if you want your pull request to be merged, we'll ask that:

    • The code follows the indications from the code manifesto
    • All existing linters pass
    • All existing tests run
    • The new feature comes with appropriate tests
    • The Git History is easy to review

    See the GitHub actions workflows to see what exactly what commands are run and the Python versions we support.

    Also, if relevant, you will need to:

    • update the changelog (in docs/changelog.md)
    • update the documentation if required

    Finally, feel free to add your name in the THANKS file ;)

    "},{"location":"contrib/dev/#checking_your_changes","title":"Checking your changes","text":"
    • Install latest poetry version.
    • Install development and documentation dependencies:
    $ poetry install\n
    • Run linters and tests:
    $ poetry run invoke lint\n$ poetry run pytest -n auto\n
    "},{"location":"contrib/dev/#adding_documentation","title":"Adding documentation","text":"
    • Follow the steps from the above section to setup your python environment
    • Launch the development server locally:
    $ poetry run mkdocs serve\n
    • Edit the markdown files from the docs/ folder and review the changes in your browser
    • Finally, submit your changes by opening a pull request on GitHub
    "},{"location":"contrib/issues/","title":"Using the issue tracker","text":"

    Reporting bugs and requesting new features is done one the tsrc issue tracker on GitHub.

    "},{"location":"contrib/issues/#reporting_bugs","title":"Reporting bugs","text":"

    If you are reporting a bug, please provide the following information:

    • tsrc version
    • Details about your environment (operating system, Python version)
    • The exact command you run
    • The full output

    Doing so will ensure we can investigate your bug right away.

    "},{"location":"contrib/issues/#suggesting_new_features","title":"Suggesting new features","text":"

    If you think tsrc is lacking a feature, please provide the following information:

    • What exactly is your use case?
    • Do you need a new command-line option or even a new command?
    • Do you need changes in the configuration files?

    Note that changingtsrc behavior can get tricky.

    First off, we want to avoid data loss following a tsrc command above

    Second, we want to keep tsrc behavior as least surprising as possible, so that it can be used without having to read (too much of) documentation.

    To that end, and keeping in mind tsrc needs to accommodate a large variety of use cases, we want to keep the code:

    • easy to read and,
    • easy to maintain,
    • and very well tested.

    The best way to achieve all of this is to keep it simple.

    This means we'll be very cautious before implementing a new feature, so don't hesitate to open an issue for discussion before jumping into the development of a new feature.

    "},{"location":"guide/ci/","title":"Using tsrc with Continuous Integration (CI)","text":""},{"location":"guide/ci/#github_actions","title":"GitHub Actions","text":"

    Let suppose you have a private GitHub organization holding several private repositories and tsrc to synchronize them using the SSH protocol. Let suppose you want to use GitHub Actions to download the code source of your organization, compile it and run some non regression tests. What to write to achieve this with tsrc?

    "},{"location":"guide/ci/#step_1_your_tsrc_manifest","title":"Step 1: Your tsrc manifest","text":"

    Your tsrc manifest.yml looks something like this:

    repos:\n  - url: git@github.com:project1/foo\n    dest: foo\n

    The git@ means SSH protocol.

    "},{"location":"guide/ci/#step_2_create_your_github_workflows_file","title":"Step 2: Create your GitHub workflows file","text":"

    In your private GitHub repository holding the GitHub workflows files, create the folder .github/workflows and your yaml file with the desired name and the following content. For more information about GitHub actions syntax see this video:

    name: tsrc with private github repos\non:\n  workflow_dispatch:\n    branches:\n      - main\n\njobs:\n  export_linux:\n    runs-on: ubuntu-latest\n    steps:\n    - name: Installing tsrc tool\n      run: |\n        sudo apt-get update\n        sudo apt-get install -y python3\n        python -m pip install tsrc\n\n    - name: Cloning private github repos\n      run: |\n        git config --global url.\"https://${{ secrets.ACCESS_TOKEN }}@github.com/\".insteadOf git@github.com:\n        export WORKSPACE=$GITHUB_WORKSPACE/your_project\n        mkdir -p $WORKSPACE\n        cd $WORKSPACE\n        tsrc init git@github.com:yourorganisation/manifest.git\n        tsrc sync\n

    This script will run on the latest Ubuntu Docker and triggers steps: - The first step named Installing tsrc tool allows to install python3 and then tsrc. - The second step named Cloning private github repos creates a folder named your_project for your workspace and call the initialization and synchronization of your repositories.

    The important command is: 

    git config --global url.\"https://${{ secrets.ACCESS_TOKEN }}@github.com/\".insteadOf git@github.com:\n

    which allows to replace the SSH syntax by the HTTPs syntax on your GitHub repository names.

    "},{"location":"guide/ci/#step_3_create_the_github_secret","title":"Step 3: Create the GitHub secret","text":"

    For GitHub organization one member of the team has the responsibility to hold a Personal access tokens for the organization. Go https://github.com/settings/tokens and click on the button Generate new token then click on repo checkbox then click on the button Generate token.

    Now, this token shall be saved into an action secret named ACCESS_TOKEN inside the GitHub repository holding the GitHub workflows files.

    "},{"location":"guide/ci/#step_4_enjoy","title":"Step 4: Enjoy","text":"

    In the menu Actions of your repository you can trig the workflow. In this example we used workflow_dispatch to perform manual triggers. So click on the button to start the process. Once this step done with success, you can update your workflow yaml to complete your CI work: compilation of your project, run non regression tests, etc.

    "},{"location":"guide/fixed-refs/","title":"Using fixed git references","text":"

    By default, tsrc sync synchronize projects using branches names.

    Usually, one would use the same branch name for several git repositories, like this:

    repos:\n  - dest: foo\n    url: git@gitlab.acme.com/your-team/foo\n    branch: main\n\n  - dest: bar\n    url: git@gitlab.acme.com/your-team/bar\n    branch: main\n

    The assumption here is that foo and bar evolve \"at the same time\", so when the main branch of foo is updated, the main branch of bar much change too.

    Sometimes though, this will not be the case. For instance, the main branch of the bar repo needs a specific, fixed version of foo in order to work.

    "},{"location":"guide/fixed-refs/#using_a_tag","title":"Using a tag","text":"

    One way to solve this is to push a v1.0 tag in the foo repository, and change the manifest too look like this:

    repos:\n  - dest: foo\n    url: git@gitlab.acme.com/your-team/foo\n-    branch: main\n+    tag: v1.0\n
    "},{"location":"guide/fixed-refs/#using_a_sha1","title":"Using a sha1","text":"

    An other way is to put the SHA1 of the relevant git commit in the foo repository in the manifest:

    repos:\n  - dest: foo\n    url: git@gitlab.acme.com/your-team/foo\n    branch: main\n+    sha1: ad2b68539c78e749a372414165acdf2a1bb68203\n
    "},{"location":"guide/fixed-refs/#cloning_repos_using_fixed_refs","title":"Cloning repos using fixed refs","text":"
    • If the repo is configured with a tag, tsrc will call git clone --branch <tag> (which is valid)
    • Otherwise, tsrc will call git clone, followed by git reset --hard <sha1>

    This is because you cannot tell git to use an arbitrary git reference as start branch when cloning (tags are fine, but sha1s are not).

    This also explain why you need both branch and sha1 in the configuration.

    "},{"location":"guide/fixed-refs/#synchronizing_repos_using_fixed_refs","title":"Synchronizing repos using fixed refs","text":"

    Here's what tsrc sync will do when trying to synchronize a repo configured with a fixed ref:

    • Run git fetch --tags --prune
    • Check if the repository is clean
    • If so, run git reset --hard <tag or sha1>
    "},{"location":"guide/foreach/","title":"Running a command for each repo in the workspace","text":"

    tsrc comes with a foreach command that allows you to run the same command for each repo in the workspace.

    This can be used for several things. For instance, if you are building an artifact from a group of repositories, you may want to put a tag on each repo that was used to produce it:

    $ tsrc foreach git tag v1.2\n
    :: Using workspace in /path/to/work\n:: Running `git tag v1.1` on 2 repos\n/path/to/work/foo $ git tag v1.2\n/path/to/work/bar $ git tag v1.2\n/path/to/work/baz $ git tag v1.2\nOK \u2713\n
    "},{"location":"guide/foreach/#caveats","title":"Caveats","text":"
    • If the command you want to run contains arguments starting with\u00a0-: you need to call foreach like this:
    $ tsrc foreach -- some-command --with-option\n
    • By default, the command is passed \"as is\", without starting a shell. If you want to use a shell, use the -c option:
    $ tsrc foreach -c  'echo $PWD'\n

    Note that we need single quotes here to prevent the shell from expanding the PWD environment variable when tsrc is run.

    "},{"location":"guide/foreach/#using_repo_and_manifest_data","title":"Using repo and manifest data","text":"

    The current tsrc implementation may not contain all the features your organization needs.

    The good news is that you can extend tsrc's feature set by using tsrc foreach.

    Let's take an example, where you have a manifest containing foo and bar and both repos are configured to use a master branch.

    Here's what happens if you run tsrc sync with bar on the correct branch (master), and foo on an incorrect branch (devel):

    $ tsrc sync\n
    :: Using workspace in /path/to/work\n=> Updating manifest\n...\n=> Cloning missing repos\n=> Configuring remotes\n=> Synchronizing repos\n* (1/2) Synchronizing foo\n* Fetching origin\n* Updating branch: devel\nUpdating 702f428..2e4fb45\nFast-forward\n...\n* (2/2) Synchronizing bar\n* Fetching origin\n* Updating branch: master\nAlready up to date.\nError: Failed to synchronize the following repos:\n* foo : Current branch: 'devel' does not match expected branch: 'master'\n

    If this happens with multiple repos, you may want a command to checkout the correct branch automatically.

    Here's one way to do it:

    $ tsrc foreach -c 'git checkout $TSRC_PROJECT_MANIFEST_BRANCH'\n

    Here we take advantage of the fact that tsrc sets the TSRC_PROJECT_MANIFEST_BRANCH environment variable correctly for each repository before running the command.

    Here's the whole list:

    Variable Description TSRC_WORKSPACE_PATH Full path of the workspace root TSRC_MANIFEST_BRANCH Branch of the manifest TSRC_MANIFEST_URL URL of the manifest TSRC_PROJECT_CLONE_URL URL used to clone the repo TSRC_PROJECT_DEST Relative path of the repo in the workspace TSRC_PROJECT_MANIFEST_BRANCH Branch configured in the manifest for this repo TSRC_PROJECT_REMOTE_<NAME> URL of the remote named 'NAME' TSRC_PROJECT_STATUS_DIRTY Set to true if the project is dirty, otherwise unset TSRC_PROJECT_STATUS_AHEAD Number of commits ahead of the remote ref TSRC_PROJECT_STATUS_BEHIND Number of commits behind the remote ref TSRC_PROJECT_STATUS_BRANCH Current branch of the repo TSRC_PROJECT_STATUS_SHA1 SHA1 of the current branch TSRC_PROJECT_STATUS_STAGED Number of files that are staged but not committed TSRC_PROJECT_STATUS_NOT_STAGED Number of files that are changed but not staged TSRC_PROJECT_STATUS_UNTRACKED Number of files that are untracked

    You can implement more complex behavior using the environment variables above, for instance:

    #!/bin/bash\n# in switch-and-pull\nif [[ \"${TSRC_PROJECT_STATUS_DIRTY}\" = \"true\" ]]; then\n  echo Error: project is dirty\n  exit 1\nfi\n\ngit switch $TSRC_PROJECT_MANIFEST_BRANCH\ngit pull\n
    $ tsrc foreach switch-and-pull\n:: Running `switch-and-pull` on 2 repos\n* (1/2) foo\n/path/to/foo $ switch-and-pull\nSwitched to branch 'master'\nYour branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.\n  (use \"git pull\" to update your local branch)\nUpdating 9e7a8e4..5f9bbd4\nFast-forward\n* (2/2) bar\n/path/to/bar $ switch-and-pull\nError: project is dirty\nError: Command failed for 1 repo(s)\n* bar\n

    Of course, feel free to use your favorite programming language here :)

    "},{"location":"guide/fs/","title":"Performing file system operations","text":""},{"location":"guide/fs/#introduction","title":"Introduction","text":"

    When using tsrc, it is assumed that repositories are put in non-overlapping file system hierarchies, like this:

    workspace/\n  project_1/\n   CMakeLists.txt\n    foo.cpp\n    bar.cpp\n  project_2/\n    CMakeLists.txt\n    spam.cpp\n    eggs.cpp\n

    Not like that, where project_2 is inside a sub-directory of project_1:

    workspace/\n  project_1/\n    CMakeLists.txt\n    foo.cpp\n    bar.cpp\n    project_2/\n      CMakeLists.txt\n      spam.cpp\n      eggs.cpp\n

    Note

    if you really need project_2 to be a sub-directory of project_1, consider using git submodules instead.

    This is usually fine, except when project_1 and project_2 share some common configuration.

    For instance, you may want to use clang-format for both project_1 and project_2.

    "},{"location":"guide/fs/#copying_a_file","title":"Copying a file","text":"

    One solution is to put the .clang-format configuration file in a repo named common and then tell tsrc to copy it at the root of the workspace:

    repos:\n  - dest: project_1\n    url: git@acme.com:team/project_1\n\n  - dest: project_2\n    url: git@acme.com:team/project_2\n\n  - dest: common\n    url: git@acme.com:team/commont\n    copy:\n    - file: clang-format\n      dest: .clang-format\n
    $ tsrc sync\n=> Cloning missing repos\n* (1/1) Cloning common\nCloning into 'common'...\n...\n=> Performing filesystem operations\n* (1/1) Copy /path/to/work/common/clang-format -> /path/to/work/.clang-format\n

    Notes:

    • copy only works with files, not directories.
    • The source path for a copy link is relative to associated repos destination, whereas the destination path of the copy is relative to the workspace root.
    "},{"location":"guide/fs/#creating_a_symlink","title":"Creating a symlink","text":"

    The above method works fine if the file does not change too often - if not, you may want to create a symbolic link instead:

    repos:\n  - dest: project_1\n    url: git@acme.com:team/project_1\n\n  - dest: project_2\n    url: git@acme.com:team/project_2\n\n  - dest: common\n    url: git@acme.com:team/commont\n    symlink:\n    - source: .clang-format\n      target: common/clang-format\n
    $ tsrc sync\n=> Cloning missing repos\n...\n=> Performing filesystem operations\n* (1/1) Lint /path/to/work/.clang-format -> common/.clang-format\n

    Notes:

    • The source path for a symbolic link is relative to the top-level <workspace>, whereas each target path is then relative to the associated source. (This path relationship is essentially identical to how ln -s works on the command line in Unix-like environments.) Multiple symlinks can be specified; each must specify a source and target.

    • Symlink creation is supported on all operating systems, but creation of NTFS symlinks on Windows requires that the current user have appropriate security policy permission (SeCreateSymbolicLinkPrivilege). By default, only administrators have that privilege set, although newer versions of Windows 10 support a Developer Mode that permits unprivileged accounts to create symlinks. Note that Cygwin running on Windows defaults to creating links via Windows shortcuts, which do not require any special privileges. (Cygwin's symlink behavior can be user controlled with the winsymlinks setting in the CYGWIN environment variable.)

    "},{"location":"guide/groups/","title":"Using groups","text":"

    Sometimes it can be necessary to create groups of repositories, especially if the number of repositories grows and if you have people in different teams work on different repositories.

    "},{"location":"guide/groups/#defining_groups_in_the_manifest","title":"Defining groups in the manifest","text":"

    The first step is to edit the manifest.yml file to describe the groups. Here's an example.

    repos:\n  - {url: git@gitlab.local:acme/one,   dest: one}\n  - {url: git@gitlab.local:acme/two,   dest: two}\n  - {url: git@gitlab.local:acme/three, dest: three}\n\ngroups:\n  default:\n    repos: []\n  g1:\n    repos:\n      - one\n      - two\n  g2:\n    repos:\n      - three\n

    Here we define a g1 group that contains repositories named one and two, and a g2 group that contains the repository named three.

    "},{"location":"guide/groups/#using_groups_in_tsrc_init","title":"Using groups in tsrc init","text":"

    If you only need the repositories in the g1 group you can run:

    tsrc init git@gitlab.local:acme/manifest --group g1\n
    "},{"location":"guide/groups/#filtering_repositories_in_groups_with_regular_expressions","title":"Filtering repositories in groups with regular expressions","text":"

    You can utilize inclusive regular expression with the -r-flag and exclusive regular expression with the -i-flag. This allows you to filter repositories within a group or a set of groups for the given action.

    To include all repositories in the group g1 matching \"config\" and excluding \"template\", you can do the following:

    tsrc init git@gitlab.local:acme/manifest --group g1 -r config -i template\n
    "},{"location":"guide/groups/#updating_workspace_configuration","title":"Updating workspace configuration","text":"

    Alternatively, you can edit the .tsrc/config.yml file, like this:

    manifest_url: git@gitlab.local:acme/manifest.git\nmanifest_branch: master\nrepo_groups:\n- g1   # <- specify the list of groups to use\n

    You can use this technique to change the groups used in a given workspace - the above method using init only works to create new workspaces.

    The config file contains other configuration options, which are described in the workspace configuration documentation.

    "},{"location":"guide/manifest/","title":"Editing the manifest safely","text":""},{"location":"guide/manifest/#introduction_when_things_go_wrong","title":"Introduction: when things go wrong","text":"

    Let's assume you've successfully implemented tsrc for your organization - now need to make sure to not break anyone's workflow.

    Let's see what could go wrong if you make mistakes while editing the manifest, using a branch called broken for the sake of the example).

    First, let's see what happens if you break the YAML syntax:

    commit 1633c5a6 (HEAD -> broken, origin/broken)\n\n    Break the manifest syntax\n\ndiff --git a/manifest.yml b/manifest.yml\nindex fe74142..068c35e 100644\n--- a/manifest.yml\n+++ b/manifest.yml\n@@ -1,4 +1,4 @@\n-repos:\n+repos\n   - url: git@github.com:your-tools/bar.git\n     dest: bar\n

    After this change is push, anyone using the broken branch of the manifest will be faced with this kind of error message:

    $ tsrc sync\n
    => Updating manifest\nReset branch 'broken'\nYour branch is up to date with 'origin/broken'.\nBranch 'broken' set up to track remote branch 'broken' from 'origin'.\nHEAD is now at 1633c5a Break the manifest syntax\nError: /path/to/work/.tsrc/manifest/manifest.yml: mapping values are\nnot allowed here :\n\n      - url: git@gitlab.acme.com:your-team/foo\n           ^ (line: 2)\n

    Similarly, if you put an invalid URL in the manifest, like this:

    commit ccfb902 (HEAD -> broken, origin/broken)\n\n    Use invalid URL for bar repo\n\ndiff --git a/manifest.yml b/manifest.yml\nindex fe74142..068c35e 100644\n--- a/manifest.yml\n+++ b/manifest.yml\n@@ -1,4 +1,4 @@\nrepos:\n-  - url: git@gitlab.acme.com:your-team/bar\n+  - url: git@gitlab.acme.com:your-team/invalid\n     dest: bar\n

    Users will get:

    $ tsrc sync\n
    :: Using workspace in /path/to/work\n=> Updating manifest\n...\nHEAD is now at ccfb902 Use invalid URL\n=> Cloning missing repos\n=> Configuring remotes\n* bar: Update remote origin to new url: (git@acme.com:your-team/invalid.git)\n...\n=> Synchronizing repos\n* (1/2) Synchronizing bar\n* Fetching origin\nERROR: Repository not found.\nfatal: Could not read from remote repository.\n\nPlease make sure you have the correct access rights\nand the repository exists.\nError: fetch from 'origin' failed\n* (2/2) Synchronizing foo\n ...\nError: Failed to synchronize the following repos:\n* bar : fetch from 'origin' failed\n

    This will probably not be a huge problem for you, dear reader, because you know about tsrc's manifest and its syntax.

    It will, however, be a problem for people who are just using tsrc without knowledge of how it is implemented, because those error messages will definitely confuse them.

    "},{"location":"guide/manifest/#using_the_apply-manifest_command_to_avoid_breaking_developers_workflow","title":"Using the apply-manifest command to avoid breaking developers workflow","text":"

    If you have a file on your machine containing the manifest changes, you can use tsrc apply-manifest to check those changes against your own workspace:

    $ cd /path/to/work\n$ tsrc apply-manifest /path/to/manifest-repo/manifest.yml\n# Check that the changes are OK\n# If so, commit and push manifest changes:\n$ cd path/to/manifest-repo\n$ git commit -a -m \"...\"\n$ git push\n# Now you know that everyone can safely run `tsrc sync`\n
    "},{"location":"guide/manifest/#additional_notes","title":"Additional notes","text":"

    • It is not advised to edit the file in .tsrc/manifest/manifest.yml directly, because tsrc sync will silently undo any local changes made to this file. This is a known bug, see #279 for details.

    • It is common to place the manifest repo itself in the manifest - so it's easy to edit or read:

    # In acme.com:your-team/manifest - manifest.yml\nrepos:\n  - url: git@acme.com:your-team/manifest\n     dest: manifest\n\n  - url: git@acme.com:your-team/foo\n     dest: foo\n\n  - url: git@acme.com:your-team/bar\n     dest: bar\n

    In that case, you would use:

    $ tsrc apply-manifest <workspace>/manifest/manifest.yml\n

    to check changes before pushing them.

    "},{"location":"guide/remotes/","title":"Using several remotes","text":"

    When you specify a repository in the manifest with just an URL, tsrc assumes you want a remote named origin:

    repos:\n  - dest: foo\n    url: git@gitlab.acme.com/your-team/foo\n\n  - dest: bar\n    url: git@gitlab.acme.com/your-team/bar\n

    But sometimes you need several remotes. Let's see a few use cases.

    "},{"location":"guide/remotes/#mirroring_open-source_projects","title":"Mirroring open-source projects","text":"

    If you want some repos in your organization to be open source, you may need:

    • a remote named 'origin' containing for the private repository on your GitLab instance
    • a remote named 'github' for the public repository on GitHub

    In that case, you can use an alternative syntax:

    repos:\n  # foo is open source and thus needs two remotes:\n  - dest: foo\n  - remotes:\n    - name: origin\n      url: git@gitlab.acme.com/your-team/foo\n    - name: github\n      url: git@github.com/your-team/foo\n\n  # bar is closed source and thus only needs the\n  # default, 'origin' remote:\n  - dest: bar\n    url: gitlab.acme.com/your-team/bar\n

    After this change, when running tsrc init or tsrc sync, both the origin and github remotes will be created in the foo repo if they don't exist, and both remotes will be fetched when using tsrc sync.

    "},{"location":"guide/remotes/#using_a_vpn","title":"Using a VPN","text":"

    Sometimes you will need two remotes, because depending the physical location of your developers, they need to use either:

    • a 'normal' remote, when they are in the office
    • a 'vpn' remote, when they are working at home

    In that case, you can create a manifest looking like this:

    repos:\n  - dest: foo\n  - remotes:\n    - name: origin\n      url: git@gitlab.local/your-team/foo\n    - name: vpn\n      url: git@myvpn.com/gitlab/your-team/foo\n\n  - dest: bar\n  - remotes:\n    - name: origin\n      url: git@gitlab.local/your-team/bar\n    - name: vpn\n      url: git@myvpn.com/gitlab/your-team/bar\n

    Developers can then use the -r, --singular-remote option to either use the origin or vpn when running tsrc init (to create a workspace), or tsrc sync (to synchronize it), depending on their physical location:

    # Init the workspace using the 'vpn' remote\n$ tsrc init -r vpn\n# Bring back the computer in the office\n# Synchronize using the 'origin' remote:\n$ tsrc sync -r origin\n

    Note

    When using this option, tsrc expects the remote to be present in the manifest for all repositories.

    "},{"location":"guide/workspace-config/","title":"Editing workspace configuration","text":""},{"location":"guide/workspace-config/#creation","title":"Creation","text":"

    The configuration file created by tsrc init contains the whole list of available settings, with their default value, and is located at </path/to/workspace/.tsrc/manifest.yml>.

    Note that if you use command-line options when using tsrc init, those will be written in the .tsrc/config.yml.

    For instance:

    tsrc init git@github.com:dmerejkowsky/dummy-manifest\n

    generates this file:

    manifest_url: git@github.com:dmerejkowsky/dummy-manifest\nmanifest_branch: master\nrepo_groups: []\nshallow_clones: false\nclone_all_repos: false\nsingular_remote:\n

    But

    tsrc init git@github.com:dmerejkowsky/dummy-manifest --branch main\n

    generates this instead:

    manifest_url: git@github.com:dmerejkowsky/dummy-manifest\nmanifest_branch: main\nrepo_groups: []\nshallow_clones: false\nclone_all_repos: false\nsingular_remote:\n
    "},{"location":"guide/workspace-config/#editing","title":"Editing","text":"

    You can edit the workspace configuration as you please, for instance if you need to switch the manifest branch.

    If you do so, note that your changes will be taken into account next time you run tsrc sync.

    "},{"location":"ref/cli/","title":"Command line usage","text":""},{"location":"ref/cli/#important_note","title":"Important note","text":"

    We use the argparse library to parse command line arguments, so the --help messages are always up-to-date, probably more so than this documentation :)

    "},{"location":"ref/cli/#general","title":"General","text":"

    tsrc uses the same \"subcommand\" pattern as git does.

    Options common to all commands are placed right before the command name.

    Options after the command name only apply to this command.

    For instance:

    $ tsrc --verbose sync\n$ tsrc init MANIFEST_URL\n
    "},{"location":"ref/cli/#goodies","title":"Goodies","text":"

    First, note that like git, tsrc will walk up the folders hierarchy looking for a .tsrc folder, which means you can run tsrc commands anywhere in your workspace, not just at the top.

    Second, almost all commands run the operation in parallel. For instance, tsrc sync by default will use as many jobs as the number of CPUs available on the current machine to synchronize the repos in your workspace. If this behavior is not desired, you can specify a greater (or lower) number of jobs using something like tsrc sync -j2, or disable the parallelism completely with -j1. You can also set the default number of jobs by using the TSRC_PARALLEL_JOBS environment variable.

    "},{"location":"ref/cli/#global_options","title":"Global options","text":"--verbose show verbose messages -q, --quiet hide everything except errors and warnings --color [always|never|auto] control using color for messages (default 'auto', on if stdout is a terminal)"},{"location":"ref/cli/#usage","title":"Usage","text":"tsrc init MANIFEST_URL [--group GROUP1, GROUP2] [--singular-remote SINGULAR_REMOTE]

    Initializes a new workspace.

    MANIFEST_URL should be a git URL containing a valid manifest.yml file.

    The -g,--groups option can be used to specify a list of groups to use when cloning repositories.

    The -r \"inclusive regular expression\" and -i \"exclusive regular expression\" options can be combined with the group option to filter for repositories within a group. -r takes precedence if both options are present.

    The -s,--shallow option can be used to make shallow clone of all repositories.

    If you want to add or remove a group in your workspace, you can edit the configuration file in <workspace>/.tsrc/config.yml

    The -r,--singular-remote option can be used to set a fixed remote to use when cloning and syncing the repositories. If this flag is set, the remote from the manifest with the given name will be used for all repos. It is an error if a repo does not have this remote specified.

    tsrc foreach -- command --opt1 arg1

    Runs command --opt1 arg1 in every repository, and report failures at the end.

    Note the -- token to separate options for command from options for tsrc.

    tsrc foreach -c 'command --opt1 arg1' Ditto, but uses a shell (/bin/sh on Linux or macOS, cmd.exe on Windows). tsrc log --from FROM [--to TO]

    Display a summary of all changes since FROM (should be a tag), to TO (defaulting to master).

    Note that if no changes are found, the repository will not be displayed at all.

    tsrc status

    Displays a summary of the status of your workspace:

    • Shows dirty repositories
    • Shows repositories not on the expected branch
    tsrc sync [--correct-branch/-c] Updates all the repositories and shows a summary at the end. If any of the repositories is not on the configured branch, but it is clean and the --correct-branch/-c flag is set, then the branch is changed to the configured one and then the repository is updated. Otherwise that repository will not be not updated. tsrc version Displays tsrc version number, along additional data if run from a git clone. tsrc apply-manifest PATH Apply changes from the manifest file located at PATH. Useful to check changes in the manifest before publishing them to the manifest repository."},{"location":"ref/manifest-config/","title":"Manifest configuration","text":"

    The manifest configuration must be stored in a file named manifest.yml, using YAML syntax.

    It is always parsed as a mapping. Here's an example:

    repos:\n  - url: git@gitlab.local:proj1/foo\n    dest: foo\n    branch: next\n\n  - remotes:\n      - name: origin\n        url: git@gitlab.local:proj1/bar\n      - name: upstream\n        url: git@github.com:user/bar\n    dest: bar\n    branch: master\n    sha1: ad2b68539c78e749a372414165acdf2a1bb68203\n\n  - url: git@gitlab.local:proj1/app\n    dest: app\n    tag: v0.1\n    copy:\n      - file: top.cmake\n        dest: CMakeLists.txt\n      - file: .clangformat\n    symlink:\n      - source: app/some_file\n        target: ../foo/some_file\n

    In this example:

    • First, proj1/foo will be cloned into <workspace>/foo using the next branch.
    • Then, proj1/bar will be cloned into <workspace>/bar using the master branch, and reset to ad2b68539c78e749a372414165acdf2a1bb68203.
    • Finally:
      • proj1/app will be cloned into <workspace>/app using the v0.1 tag,
      • top.cmake will be copied from proj1/app/top.cmake to <workspace>/CMakeLists.txt,
      • .clang-format will be copied from proj1/app/ to <workspace>/, and
      • a symlink will be created from <workspace>/app/some_file to <workspace>/foo/some_file.
    "},{"location":"ref/manifest-config/#top_fields","title":"Top fields","text":"
    • repos (required): list of repositories to clone
    • groups (optional): list of groups
    "},{"location":"ref/manifest-config/#repos","title":"repos","text":"

    Each repository is also a mapping, containing:

    • Either:
      • url if you just need one remote named origin
      • A list of remotes with a name and url. In that case, the first remote will be used for cloning the repository.
    • dest (required): relative path of the repository in the workspace
    • branch (optional): The branch to use when cloning the repository (defaults to master)
    • tag (optional):
      • When running tsrc init: Project will be cloned at the provided tag.
      • When running tsrc sync: If the project is clean, project will be reset to the given tag, else a warning message will be printed.
    • sha1 (optional):
      • When running tsrc init: Project will be cloned, and then reset to the given sha1.
      • When running tsrc sync: If the project is clean, project will be reset to the given sha1, else a warning message will be printed.
    • ignore_submodules (optional, default=false):
      • When running tsrc init: if ignore_submodules is true, do not recursively clone submodules.
      • When running tsrc sync: if ignore_submodules is true, do not initialize or update submodules. to the given sha1, else a warning message will be printed.
    • copy (optional): A list of mappings with file and dest keys.
    • symlink (optional): A list of mappings with source and target keys.

    See the Using fixed references and the Performing file system operations guides for details about how and why you would use the tag, sha1, copy or symlink fields.

    "},{"location":"ref/manifest-config/#groups","title":"groups","text":"

    The groups section lists the groups by name. Each group should have a repos field containing a list of repositories (only repositories defined in the repos section are allowed).

    The groups can optionally include other groups, with a includes field which should be a list of existing group names.

    The group named default, if it exists, will be used to know which repositories to clone when using tsrc init and the --group command line argument is not used.

    Example:

    repos:\n  - dest: a\n    url: ..\n  - dest: b\n    url: ..\n  - dest: bar\n    url: ..\n  - dest: baz\n    url: ..\n\ngroups:\n  default:\n    repos: [a, b]\n  foo:\n    repos: [bar, baz]\n    includes: [default]\n
    $ tsrc init <manifest_url>\n# Clones a, b\n$ tsrc init <manifest_url> --group foo\n# Clones a, b, bar and baz\n

    Note that tsrc init records the names of the groups it was invoked with, so that tsrc sync re-uses them later on. This means that if you want to change the groups used, you must re-run tsrc init with the new group list.

    Note

    More information about how to use groups is available in the relevant guide.

    "},{"location":"ref/sync/","title":"Sync algorithm","text":"

    You may have noticed that tsrc sync does not just calls git pull on every repository.

    Here's the algorithm that is used:

    • Run git fetch --tags --prune
    • Check if the repository is on a branch
    • Check if the currently checked out branch matches the one configured in the manifest. If it does not but the --correct-branch flag is set and the repository is clean, the branch is changed to the configured one.
    • Check if the repository is dirty
    • Try and run a fast-forward merge

    Note that:

    • git fetch is always called so that local refs are up-to-date
    • tsrc will simply print an error and move on to the next repository if the fast-forward merge is not possible. That's because tsrc cannot guess what the correct action is, so it prefers doing nothing. It's up to the user to run something like git merge or git rebase.
    • in case the repository is on an incorrect branch, the fast-forward merge will still be attempted, but an error message will be show in the end
    "},{"location":"ref/workspace-config/","title":"Workspace configuration","text":"

    The workspace configuration lies in <workspace>/.tsrc/config.yml. It is created by tsrc init then read by tsrc sync and other commands. It can be freely edited by hand.

    Here's an example:

    manifest_url: git@acme.corp:manifest.git\nmanifest_branch: master\nshallow_clones: false\nrepo_groups:\n- default\nclone_all_repos: false\nsingular_remote:\n
    • manifest_url: an git URL containing a manifest.yml file
    • manifest_branch: the branch to use when updating the local manifest (e.g, the first step of tsrc sync)
    • shallow_clones: whether to use only shallow clones when cloning missing repositories
    • repo_groups: the list of groups to use - every mentioned group must be present in the manifest.yml file (see above)
    • clone_all_repos: whether to ignore groups entirely and clone every repository from the manifest instead
    • singular_remote: if set to <remote-name>, behaves as if tsrc sync and tsrc init were called with --singular-remote <remote-name> option. See the Using remotes guide for details.
    "}]} \ No newline at end of file +{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Home","text":""},{"location":"#tsrc_-_managing_groups_of_git_repositories","title":"tsrc - managing groups of git repositories","text":""},{"location":"#what_it_does","title":"What it does","text":"

    tsrc is a command-line tool that helps you manage groups of git repositories.

    It works by listing the repositories in a file called manifest.yml that looks like this:

    repos:\n  - dest: foo\n    url: git@example.com:foo.git\n\n  - dest: bar\n    url: git@example.com:bar.git\n

    You can then use:

    • tsrc init <manifest url> to create a workspace containing the foo and bar repository

    • tsrc sync to synchronize all repos in the workspace.

    • ... and many more commands. Run tsrc help to list them, or read the command line reference

    "},{"location":"#tutorial","title":"Tutorial","text":"

    Interested in using tsrc in your own organization?

    Proceed to the getting started tutorial!

    "},{"location":"#guides","title":"Guides","text":"

    Once you've learn how to setup tsrc for your organization, feel free to read the following guides - tsrc supports a variety of use cases beyond just listing git repositories to be cloned or synchronized and are described here:

    • Editing the manifest safely
    • Editing workspace configuration
    • Using groups
    • Using several remotes
    • Using fixed git references
    • Performing file system operations
    • Running a command for each repo in the workspace
    • Using tsrc with continuous integration
    "},{"location":"#reference","title":"Reference","text":"
    • Command line interface
    • Sync algorithm
    • Manifest configuration
    • Workspace configuration
    "},{"location":"#contributing","title":"Contributing","text":"
    • Using the issue tracker
    • Suggesting changes
    • Code Manifesto
    "},{"location":"#useful_links","title":"Useful links","text":"
    • FAQ
    • Changelog
    "},{"location":"changelog/","title":"Changelog","text":""},{"location":"changelog/#271_2022-05-30","title":"2.7.1 (2022-05-30)","text":"
    • In addition to preserving the order repos are listed in the manifest, tsrc now makes sure repos included via groups are processed before the other repos. See #356 for details. Thanks to @raabf for the bug report and code review!
    "},{"location":"changelog/#270_2022-05-14","title":"2.7.0 (2022-05-14)","text":"
    • Show which git commands are run by default.
    • tsrc init: fix order of operations - clone the local manifest before writing the workspace configuration. Fixes #344, where users could not run init a second time if the previous call failed. Bug report by @cgestes.
    • tsrc init: do not assume the default branch of the manifest is master. Note that master is still hard-coded in a few places. See #347 for details.
    • Breaking: When using -j 1, do not sort repositories by lexical order of destination, but preserve the order in which they were specified in the manifest. Suggested by @raabf.
    "},{"location":"changelog/#260_2022-03-27","title":"2.6.0 (2022-03-27)","text":"
    • Allow to use python -m tsrc in addition to just tsrc
    • Documentation updates
    • Bump mypy
    "},{"location":"changelog/#250_2022-12-21","title":"2.5.0 (2022-12-21)","text":""},{"location":"changelog/#highlights","title":"Highlights","text":"
    • Bump minimum supported version to Python 3.7.
    • Introduce ignore_submodules repository option - Patch by Thomas Hiscock.
    • When -j is not used, try getting the default jobs value from the TSRC_PARALLEL_JOBS environment variable. Patch by Marcin Jaworski.
    "},{"location":"changelog/#other","title":"Other","text":"
    • Bump mkdocs from 1.2.2 to 1.2.3
    • Bump pygit2
    • Bump linters (flake8, mypy, black)
    • Remove dependency on attr
    "},{"location":"changelog/#241_2021-11-05","title":"2.4.1 (2021-11-05)","text":"
    • Ad more guides in the documentation
    • Display absolute paths when performing file system operations
    • tsrc sync now uses parallel jobs by default. Use -j1 to force sequential processing. Patch by @gdubicki
    "},{"location":"changelog/#240_2021-08-22","title":"2.4.0 (2021-08-22)","text":""},{"location":"changelog/#highlights_1","title":"Highlights","text":"

    • All of tsrc commands can now be run in parallel. Try for instance tsrc sync -j auto.

    • tsrc foreach now sets a bunch of environment variables. This allows developers to add new behaviors to tsrc without having to change its source code. See the relevant guide for more information.

    • Augment documentation with more use cases and examples (still a work in progress)

    "},{"location":"changelog/#breaking_changes","title":"Breaking changes","text":"

    • Remove tsrc version - Use tsrc --version instead.

    • The 'parallel' feature caused the output of some commands like foreach or log to change slightly. Hopefully tsrc output is now more consistent.

    "},{"location":"changelog/#bug_fixes","title":"Bug fixes","text":"

    • Fix crash when running tsrc without any arguments

    • Fix crash when trying to clone repositories in some rare corner cases (like the destination existing but not being a directory)

    "},{"location":"changelog/#internal_changes","title":"Internal changes","text":"
    • Make all tsrc imports consistent
    • Fix error when calling repr on tsrc Errors.
    "},{"location":"changelog/#231_2021-06-28","title":"2.3.1 (2021-06-28)","text":""},{"location":"changelog/#bug_fixes_and_small_improvements","title":"Bug fixes and small improvements","text":"
    • Fix #268: tsrc apply-manifest now performs file system operation
    • Always display workspace path at the beginning of any action
    • Skip \"performing file system operations\" message if there is no work to be done
    "},{"location":"changelog/#internal_changes_1","title":"Internal changes","text":"
    • Bump linters and formatters (black, mypy, isort ...)
    • Use copier to simplify maintenance of tools configuration
    "},{"location":"changelog/#230_2021-05-31","title":"2.3.0 (2021-05-31)","text":""},{"location":"changelog/#repo_selection","title":"Repo selection","text":"
    • Added -r (regex) and -i (inverse regex) params for filtering repos. Patch by @xzr
    "},{"location":"changelog/#add_support_for_submodules","title":"Add support for submodules","text":"
    • tsrc calls git clone with --recurse-submodules when adding missing repositories
    • tsrc calls git submodule update --init --recursive when updating repositories
    "},{"location":"changelog/#misc","title":"Misc","text":"
    • Remove codecov usage
    • Rename default branch to main.
    "},{"location":"changelog/#221_2021-04-10","title":"2.2.1 (2021-04-10)","text":"

    • Project has been moved from TankerHQ organization to dmerejkowsky. New urls are:

      • github.com/dmerejkowsky/tsrc for the git repository
      • dmerejkowsky.github.io/tsrc for the documentation

    • Add CI jobs to check this project also works with Python 3.9

    • Drop Path Pie dependency
    • Minor internal fixes
    • Add more URLs in the metadata (and pypi.org project page)
    "},{"location":"changelog/#220_2020-07-17","title":"2.2.0 (2020-07-17)","text":""},{"location":"changelog/#add_symlink_support","title":"Add symlink support","text":"

    tsrc sync and tsrc init can now create symlinks as specified in the manifest file:

    repos:\n\n  - url: git@gitlab.local:proj1/app\n    dest: app\n    symlink:\n      - source: app/some_file\n        target: ../some_file\n

    In this case, a symlink will be created from <workspace>/app/some_file to <workspace>/some_file. (both source and target keys are relative to the repository's destination).

    "},{"location":"changelog/#changes_related_to_groups","title":"Changes related to groups","text":"
    • log, status, and sync all learned about the --group option and the --all-cloned options
    • foreach: remove the --groups-from-config options since this is now the default behavior
    "},{"location":"changelog/#misc_1","title":"Misc","text":"
    • Rework FAQ
    • Run black in lint.sh
    • Fix formatting of some messages
    • Update code manifesto to suggest using docstrings in tests
    "},{"location":"changelog/#210_2020-05-27","title":"2.1.0 (2020-05-27)","text":""},{"location":"changelog/#breaking_changes_1","title":"Breaking changes","text":""},{"location":"changelog/#change_in_manifest_syntax","title":"Change in manifest syntax","text":"

    It was discovered that the manifest syntax was confusing for newcomers, so we decided to update it.

    In particular, the src key meant both a relative path in the workspace when used in the repo config, and a relative path in the a repository when using in the repo.copy config.

    Starting with this release, repo.src becomes repo.dest and repo.copy.src becomes repo.copy.file.

    # Before (tsrc < 2.1.0)\nrepos:\n  url: \"https://acme.corp/foo\"\n  src: foo\n  copy:\n     src: some-file\n     dest: some-file\n
    # After (tsrc >= 2.1.0)\nrepos:\n  url: \"https://acme.corp/foo\"\n  dest : foo\n  copy:\n     file: some-file\n     dest: some-file\n

    This should make it clearer what tsrc does because:

    • dest now always refers to a relative path in the workspace (both in repo and copy).
    • By using repo.copy.file it's obvious that tsrc only supports copying files, not directories.
    "},{"location":"changelog/#supported_python_versions","title":"Supported Python versions","text":"

    Drop support for Python 3.5

    "},{"location":"changelog/#new_features","title":"New features","text":"
    • tsrc init learned a -r, --remote option that pins the remote with the given name as the only remote to be used for cloning and syncing. tsrc expects this remote to be present in the manifest for all repositories. This is useful if you use the same workspace in different physical locations, and one of the remotes is behind a VPN for instance. Patch by @tronje.
    "},{"location":"changelog/#bug_fixes_1","title":"Bug fixes","text":"
    • Fix #217: Preserves file attributes during the copy statements in repos
    "},{"location":"changelog/#other_1","title":"Other","text":"
    • The whole test suite now runs without errors on Windows - and Windows support is now part of the GitHub actions checks.
    • The tests now run faster and with more readable output (this was done by using libgit2 instead of running git commands in the tests helpers).
    • Add a scheduled GitHub action to run safety
    • Remove usage of deprecated API of the path library.
    • Run tests and linters for external pull requests too.
    "},{"location":"changelog/#v200_-_2020-04-06","title":"v2.0.0 - (2020-04-06)","text":"

    • Remove the tsrc push command and all review automation features. Please use hub, lab, or repo instead. See #207 for the discussion leading to this removal.

    • Implement small improvements on tsrc output messages.

    • Add tsrc apply-manifest, to apply changes in a manifest file locally, without having to make a commit and push to a server first.
    "},{"location":"changelog/#v103_-_2020-02-05","title":"v1.0.3 - (2020-02-05)","text":"
    • Use poetry for dependency management and packaging.
    "},{"location":"changelog/#v102_-_2020-01-29","title":"v1.0.2 - (2020-01-29)","text":"
    • Fix python_requires value in project metadata
    "},{"location":"changelog/#v101_-_2020-01-21","title":"v1.0.1 - (2020-01-21)","text":"
    • Fix #196: Do not attempt file copies for non-cloned repositories when using tsrc init with a list of groups.
    "},{"location":"changelog/#v100_-_2020-01-09","title":"v1.0.0 - (2020-01-09)","text":"

    Starting the new year with a stable release, at last!

    "},{"location":"changelog/#revamp_group_ux","title":"Revamp group UX","text":"

    The changes below in the configuration file and command line syntax allow for better UX regarding groups. See the corresponding milestone for the full list.

    "},{"location":"changelog/#new_configuration_file","title":"New configuration file","text":"

    Previously, tsrc stored its permanent configuration in .tsrc/manifest.yml and the file was not supposed to be edited by hand. Instead, users could use tsrc init to modify it, for instance with the --branch argument.

    Starting with this release, the command tsrc init can only be run once per workspace, and you must edit the .tsrc/config.yml file instead.

    "},{"location":"changelog/#changes_in_command_line_syntax","title":"Changes in command line syntax","text":"
    • tsrc init: remove --file option.
    • tsrc foreach: instead of repeating the --group option, you can use --groups with a list of groups:
    # before\ntsrc init --group foo --group bar\n\n# after\ntsrc init --groups foo bar\n

    • tsrc init learned a --clone-all-repos option to clone all repositories from the manifest, regardless of the groups. Fix #181

    • Remove --file option from tsrc init.

    • tsrc foreach learned a --groups-from-config option to use the groups configured in the workspace. Fix #178, #179.

    • tsrc push learned a -o, --origin option to specify a remote name different from \"origin\". Fix #170

    "},{"location":"changelog/#other_fixes","title":"Other fixes","text":"
    • Try and check that GitLab installation support required features before using them - typically, using tsrc push --approvers on GitLab Community Edition. (#165)
    • reported by @irizzant.
    • Switch to GitHub actions for running tests and linters. Also, publish documentation automatically when something is pushed to the master branch.
    • tsrc status : add information when local branch does not match manifest configuration. (#190). Feature suggested by @janjachnick
    "},{"location":"changelog/#v092_-_2019-09-30","title":"v0.9.2 - (2019-09-30)","text":"
    • Additional bug fix for #165 - the fix in 0.9.1 was incomplete
    • Improve error message when trying to use non-supported GitLab features (like using tsrc push --reviewer on GitLab Community Edition)
    "},{"location":"changelog/#v091_-_2019-09-23","title":"v0.9.1 - (2019-09-23)","text":"
    • Improve error message when tsrc foreach fails to start the process. Suggested by @dlewis-ald in #163
    • Fix crash when finding reviewers for a GitLab project not in a group. Reported by @irizzant in #165
    "},{"location":"changelog/#v090_-_2019-08-13","title":"v0.9.0 - (2019-08-13)","text":"
    • Add support for GitHub Enterprise: patch by @sdavids13.
    • Improve error message when using creating a merge request in a GitLab repository when the token cannot be found in the tsrc configuration file. Fix #158
    • Fix crash when running tsrc status on a workspace with missing repositories (#160) - reported by @blastrock
    "},{"location":"changelog/#v080_-_2019-08-12","title":"v0.8.0 - (2019-08-12)","text":"
    • Implement tsrc sync --force. Currently all it does is running git fetch --force on all repositories. Use with caution. See #152 for details.
    "},{"location":"changelog/#v071_-_2019-08-02","title":"v0.7.1 - (2019-08-02)","text":"
    • Fix crash in tsrc sync when the repo configuration in the manifest contained neither an URL nor a remote. tsrc now aborts as soon as the misconfiguration of the manifest is detected (Reported by @jongep86)
    "},{"location":"changelog/#v070_2019-07-08","title":"v0.7.0 (2019-07-08)","text":"
    • Add a --file option to tsrc init so that manifest can be read from a custom path in the file system
    • Remove support for Python 3.4
    • Switch from xdg to pyxdg
    • Format the code with black
    "},{"location":"changelog/#v066_2019-04-02","title":"v0.6.6 (2019-04-02)","text":"
    • Remove raw HTML from README.rst
    "},{"location":"changelog/#v065_2019-04-0","title":"v0.6.5 (2019-04-0)","text":"
    • Use codecov.io to measure coverage
    • Prettify README
    "},{"location":"changelog/#v064_2019-01-07","title":"v0.6.4 (2019-01-07)","text":"
    • Remove support for Python 3.3.
    • Use new and shiny cli-ui package instead of old python-cli-ui.
    "},{"location":"changelog/#v063_2018-11-04","title":"v0.6.3 (2018-11-04)","text":"
    • GitHub organization is now TankerHQ
    • We now use dmenv for dependencies management
    "},{"location":"changelog/#v062_2018-10-19","title":"v0.6.2 (2018-10-19)","text":"

    Fix crash when using tsrc push on a GitHub repository for the first time.

    "},{"location":"changelog/#v061_2018-10-10","title":"v0.6.1 (2018-10-10)","text":"

    Fix weird output when configuring remotes.

    "},{"location":"changelog/#v060_2018-10-09","title":"v0.6.0 (2018-10-09)","text":""},{"location":"changelog/#add_support_for_multiple_remotes","title":"Add support for multiple remotes","text":"
    # still valid (implicit 'origin' remote)\nsrc: foo\nurl: git@github.com/foo\n\n# also valid (two explicit remotes)\nsrc: foo\nremotes:\n  - { name: origin, url: git@github.com:john/foo }\n  - { name: upstream, url: git@github.com:foo/foo}\n\n# not valid (ambiguous)\nsrc: foo\nurl: git@github.com:john/foo\nremotes:\n   - { name: upstream, url: git@github.com:foo/foo }\n

    Thanks @tst2005 and @cgestes for their help with the configuration format.

    "},{"location":"changelog/#tsrc_foreach","title":"tsrc foreach","text":"
    • tsrc foreach: add a --group option to select the repositories to run the command on. Fix #40
    "},{"location":"changelog/#other_fixes_1","title":"Other fixes","text":"
    • Fix #113: do not hide branch when showing tag status.
    • Add support for Python 3.7
    "},{"location":"changelog/#v050_2018-08-14","title":"v0.5.0 (2018-08-14)","text":"
    • Add support for setting approvers with the -r,--approvers option in tsrc push (GitLab Enterprise Edition only).
    "},{"location":"changelog/#v041_2018-04-27","title":"v0.4.1 (2018-04-27)","text":"
    • Fixed regression: tsrc push was no longer able to create a merge request on GitLab if --target was not set.
    "},{"location":"changelog/#v040_2018-04-26","title":"v0.4.0 (2018-04-26)","text":""},{"location":"changelog/#highlights_2","title":"Highlights","text":"
    • Preliminary GitHub support
    • tsrc push: new features and bug fixes
    • Improved fixed reference handling
    • Support for shallow clones

    See below for the details.

    "},{"location":"changelog/#preliminary_github_support","title":"Preliminary GitHub support","text":"
    • Added support for creating merge requests on GitHub. No configuration required. Just make sure you are using tsrc from a repository which has a URL starting with git@github.com.

    tsrc will prompt you once for your login and password and then store an API token.

    Afterwards, you'll be able to use tsrc push to:

    • Create a pull request (or update it if it already exists)
    • Assign people to the request (with the -a/--assignee option)
    • Request reviewers (with the --reviewers option)
    • Merge the pull request (with the --merge option)

    This change has no impact if you were already using GitLab.

    "},{"location":"changelog/#tsrc_push_new_features_and_bug_fixes","title":"tsrc push: new features and bug fixes","text":"
    • Add --close option.
    • Breaking change: -m/--message option is gone, use --title instead. There's a concept of \"description\" or \"message\" for pull requests and merge requests, but the value of the option was only used to update the title, so it had to be renamed.
    • Do not assume local and remote tracking branch have the same name.
    • Allow using tsrc push <local>:<remote> to explicitly specify local and remote branch names.
    • Fix bugs when target is not specified on the command line. See this commit for details.
    • Fix missing merge requests in tsrc push (see issue #80). Patch by @maximerety.
    "},{"location":"changelog/#improve_fixed_reference_handling","title":"Improve fixed reference handling","text":"

    Breaking change: Instead of using fixed_ref in the manifest, you should now use tag or sha1:

    old: 

    repos:\n  - src: git@example.com/foo\n    fixed_ref: 42a70\n

    new: 

    repos:\n  - src: git@example.com/foo\n    tag: v0.1\n

    See the dedicated section about manifest format and the #57 pull request discussion for the details.

    This allow us to implement different behaviors depending on whether or not the fixed ref is a tag or just a sha1.

    "},{"location":"changelog/#support_for_shallow_clones","title":"Support for shallow clones","text":"

    To save time and space, you can use tsrc init --shallow to only have shallow clones in your workspace.

    Note that due to limitations in git itself, the shallow option cannot be used with a fixed SHA1. If you need this, prefer using a tag instead.

    "},{"location":"changelog/#misc_2","title":"Misc","text":"

    • Organization TankerApp was renamed to TankerHQ. New urls are:

      • github.com/TankerHQ/tsrc for the git repository
      • TankerHQ.github.io/tsrc for the documentation

    • We now use pipenv for dependency handling.

    "},{"location":"changelog/#v032_2017-11-02","title":"v0.3.2 (2017-11-02)","text":"
    • Improve tsrc status to handle tags. Patch by @arnaudgelas.
    • Fix crash when running tsrc version.
    "},{"location":"changelog/#v031_2017-10-06","title":"v0.3.1 (2017-10-06)","text":"
    • Improve tsrc status output. Now also shows number of commits ahead and behind, and display a short SHA-1 when not on any branch. Initial patch by @arnaudgelas.
    "},{"location":"changelog/#v030_2017-09-22","title":"v0.3.0 (2017-09-22)","text":"

    Breaking change: Add support for groups (#30). Reported by @arnaudgelas.

    See the dedicated section about manifest format for details.

    Upgrading from v0.2.4:

    To upgrade from an older version of tsrc, you should re-run tsrc init with the correct url:

    # Check manifest URL:\n$ cd <workspace>/.tsrc/manifest\n$ git remote get-url origin\n# Note the url, for instance ssh://git@example.com:manifest.git\n$ cd <workspace>\n$ tsrc init <manifest-url>\n

    This is required to create the <workspace>/.tsrc/manifest.yml file which is later used by tsrc sync and other commands.

    "},{"location":"changelog/#v024_2017-07-13","title":"v0.2.4 (2017-07-13)","text":"
    • tsrc push --assignee: fix when there are more than 50 GitLab users (#25). Reported by @arnaudgelas
    "},{"location":"changelog/#v023_2017-09-01","title":"v0.2.3 (2017-09-01)","text":"

    • Split user interface functionality into its own project: python-cli-ui.

    • Add --quiet and --color global options.

    "},{"location":"changelog/#v022_2017-08-22","title":"v0.2.2 (2017-08-22)","text":"

    Bug fix release.

    • tsrc init: Fix crash when a repository is empty (#17). Reported by @nicolasbrechet
    • tsrc push: Fix rude message when credentials are missing (#20). Reported by @cgestes
    "},{"location":"changelog/#v021_2017-08-10","title":"v0.2.1 (2017-08-10)","text":"

    Packaging fixes.

    "},{"location":"changelog/#v020_2017-08-09","title":"v0.2.0 (2017-08-09)","text":"
    • Support for specifying custom branches in the manifest
    • Support for specifying fixed refs (tags or hashes) in the manifest

    New syntax is:

    repos:\n  - src: foo\n    url: git@gitlab.com:proj/foo\n    branch: next\n\n  - src: bar\n    url: git@gitlab.com:proj/bar\n    branch: master\n    fixed_ref: v0.1\n

    Note that branch is still required.

    • You can now skip the dest part of the copy section if src and dest are equal:
    copy:\n  - src:foo\n\n# same thing as\ncopy:\n - src: foo\n   dest: foo\n
    "},{"location":"changelog/#v014_2017-08-04","title":"v0.1.4 (2017-08-04)","text":"

    Support for Python 3.3, 3.4, 3.5 and 3.6

    "},{"location":"changelog/#v011_2017-08-02","title":"v0.1.1 (2017-08-02)","text":"

    First public release

    "},{"location":"code-manifesto/","title":"Code Manifesto","text":""},{"location":"code-manifesto/#basics","title":"Basics","text":"

    We use black to enforce a coding style matching PEP8.

    In addition, every text file must be pushed using UNIX line endings. (On Windows, you are advised to set core.autocrlf to true in your git config file.)

    "},{"location":"code-manifesto/#pet_peeves","title":"Pet peeves","text":"
    • Prefer double quotes for string literals:
    # Yes\ndef bar():\n   \"\"\" bar stuff \"\"\"\n   a = \"foo\"\n\n\n# No\ndef bar():\n   ''' bar stuff '''\n   a = 'foo'\n\n# Exception\nmy_str = 'It contains some \"quotes\" inside'\n

    • Use the fact that empty data structures are falsy: 

      # Yes\nif not errors:\n    ...\n# No\nif len(errors) == 0:\n    ...\n

    • Avoid using double negatives: 

      # Yes\ndef make_coffee(sugar=False):\n    if sugar:\n        print(\"with sugar\")\n\n# No\ndef make_coffee(without_sugar=True):\n    if not without_sugar:\n        print(\"with sugar\")\n

    • Prefer using \"f-strings\" if possible, + may also work in some contexts.

    # Yes\nmessage = f\"Welcome {name}!\"\n\n# No\nmessage = \"Welcome, {}!\".format(name)\nmessage = \"Welcome, %s!\" % name\nmessage = \"Welcome, \" + name + \"!\"\n\n# Okayish\nwith_ext = name + \".txt\"\n
    • Use textwrap.dedent() to build nice-looking multi-lines strings:
    # Yes\ndef foo():\n    long_message = textwrap.dedent(\"\"\"\\\n        first line\n        second line\n        third line\"\"\")\n\n# No\ndef foo():\n    long_message = \"\"\"\\\nfirst line\nsecond line\nthird line\n\"\"\"\n
    • Do not initialize several variables on the same line, unless they come from a tuple (for instance the return of a function, or a iteration on a directory)
    # Yes\nok, mess  = run_command()\n\nfor test_result in test_results:\n    outcome, message = res\n\n# No\nfoo, bar = False, \"\"\n\nclass Foo:\n    self.bar, self.baz = None, True\n
    • Do not use conditional expressions. The order is not the same as the ternary operator in C++ and JavaScript, so it should be avoided:
    # Yes\nif foo:\n   a = \"ok\"\nelse:\n   a = \"nope\"\n\n\n# No:\na = \"ok\" if foo else \"nope\"\n
    • Use if ... in ... when you can:
    # Yes\nif value in (\"option1\", \"option2\"):\n   ...\n\n# No\nif value == \"option1\" or value == \"option2\"\n  ...\n
    "},{"location":"code-manifesto/#doc_strings_and_comments_in_production_code","title":"Doc strings and comments in production code","text":"

    First off, bad comments are worse that no comments.

    Also note that you should use comments to explain why, never what. If the what is no clear, it means the behavior of the function or method cannot be easily understood by reading implementation, and so you should fix the implementation instead.

    In conclusion, use comments and doc strings sparingly: that way, they will not rot and they will stay useful.

    Note: this does not apply for tests (see below).

    "},{"location":"code-manifesto/#collections","title":"Collections","text":"
    • Use .extend() instead of += to concatenate lists:

    # Yes\nlist_1.extend(list_2)\n\n# No\nlist_1 += list_2\n
    * Only use list() and dict() to convert a value to a list or dict. Prefer literals when possible

    # Yes\nmy_list = []\nmy_dict = {}\n\n# Also yes:\nmy_list = list(yield_stuff())\n\n# No\nmy_list = list()\nmy_dict = dict()\n
    • Also use explicit call to list() in order to make a copy:
    # Yes\nmy_copy = list(my_list)\n\n# Also yes:\nmy_copy = copy.copy(my_list)\n\n# No\nmy_copy = my_list[:]\n
    • Use list comprehensions instead of loops and \"functional\" methods:
    # Yes\nmy_list = [foo(x) for x in other_list]\n\n# No\nmy_list = list()\nfor x in other_list:\n     x.append(foo(x))\n\n# Also no\nmy_list = map(foo, other_list)\n\n# Yes\neven_nums = [x for x in nums if is_even(x)]\n\n# No\neven_nums = filter(is_even, nums)\n
    • Use iterable syntax instead of building an explicit list:
    # Yes\nmax(len(x) for x in my_iterable)\n\n# No\nmax([len(x) for x in my_iterable])\n
    • Use plural names for collections. This has the nice benefit of allowing you to have meaningful loop names:
    for result in results:\n   # do something with result\n
    "},{"location":"code-manifesto/#functions","title":"Functions","text":"

    Prefer using keyword-only parameters when possible:

    # Yes\n# If the parameter needs a default value:\ndef foo(bar, *, spam=True):\n    ...\n\n# If it does not:\ndef foo(bar, *, spam):\n    ...\n\n\n# No\ndef foo(bar, spam=True):\n    ...\n

    If you use the last form, Python will let you use foo(42, False), and set spam to False. This can cause problems if someone ever changes the foo function and adds a new optional argument before spam:

    def foo(bar, eggs=False, spam=True):\n    ...\n
    After such a change, the line foo(42, False) which used to call foo with spam=False now calls foo with bar=False and spam=True, leading to all kinds of interesting bugs.

    Exception to this rule: when the keyword is obvious and will not change: 

    def get(value, default=None):\n  ...\n

    "},{"location":"code-manifesto/#imports","title":"Imports","text":"

    For any foo.py file, import foo must never fail, unless there is a necessary module that could not be found. Do not catch ImportError unless it is necessary, for instance to deal with optional dependencies.

    import required_module\n\nHAS_NICE_FEATURE = True\ntry:\n    import nice_lib\nexcept ImportError:\n    HAS_NICE_FEATURE = False\n\n#...\n\nif HAS_NICE_FEATURE:\n    #....\n

    • Importing Python files should never cause side effects. It's OK to initialize global variables, but you should never call functions outside a if __name__ == main() block.

    • Prefer using fully-qualified imports and names:

    # Yes\nimport foo.bar\nmy_bar = foo.bar.Bar()\n\n# No\nfrom foo import bar\nmy_bar = bar.Bar()\n

    Note

    We allow a few exceptions like from pathlib import Path or importing classes directory in tests. Use your best judgment.

    "},{"location":"code-manifesto/#classes","title":"Classes","text":"
    • When you want to make sure a class follows an interface, use abc.ABCMeta instead of raising NotImplementedError. This way you get the error when the class is instantiated instead of when the method is called.
    # Yes\nclass AbstractFoo(metaclass=abc.ABCMeta):\n    @abc.abstractmethod\n    def foo(self):\n        pass\n\n\n# No\nclass AbstractFoo:\n     def foo(self):\n        raise NotImplementedError()\n
    • Make sure to use properties when relevant, instead of get_ methods.
    # Yes\nclass Person:\n      def __init__(self, first_name, last_name):\n            self.first_name = first_name\n            self.last_name = last_name\n\n      @property\n      def full_name(self):\n          return f\"{self.first_name} {self.last_name}\"\n\n\n# No:\nclass Foo:\n      def __init__(self, first_name, last_name):\n            self.first_name = first_name\n            self.last_name = last_name\n            self.full_name = f\"{self.first_name} {self.last_name}\"\n

    For instance, here:

    • full_name is read-only
    • The attribute is automatically updated if first_name changes after the object is initialized.

    Note that get_ methods are OK if they do more than simple computations (expensive in time or size, throwing exceptions ...)

    "},{"location":"code-manifesto/#file_paths","title":"File paths","text":"
    • If you are manipulating filenames, use the path.pylibrary and suffix the variable by _path. Avoid using os.path or shutil methods when path.py is better.
    # Yes\nwork_path = Path(\"foo/work\")\nwork_path.mkdir_p()\nfoo_path = work_path / \"foo.txt\"\nfoo_path.write_text(\"this is bar\")\n\n# No\nwork_path = os.path.join(foo, \"work\")\nos.path.mkdir(work_path, exist_ok=True)\nfoo_path = os.path.join(work_path, \"foo.txt\")\nwith open(foo_path, \"w\") as fileobj:\n    fileobj.write(\"this is foo\")\n
    "},{"location":"code-manifesto/#error_handling","title":"Error handling","text":"
    • All exceptions raised from within tsrc should derive from tsrc.Error.
    • When using external code (from the standard library or a third-party library), you should catch the exceptions and optionally re-raise them.
    "},{"location":"code-manifesto/#output_messages_to_the_user","title":"Output messages to the user","text":"

    Do not use print, use python-cli-ui functions instead. This makes it easier to distinguish between real messages and the throw-away print statements you add for debugging.

    Also, using \"high-level\" methods such as ui.info_1() or ui.warning() will make it easier to have a consistent user interface.

    "},{"location":"code-manifesto/#tests","title":"Tests","text":""},{"location":"code-manifesto/#docstrings","title":"Docstrings","text":"

    If you think the test implementation is complex, add a human-readable description of the test scenario in the doc string.

    For instance:

    def test_sync_with_errors(...):\n    \"\"\"\" Scenario:\n    * Create a manifest with two repos (foo and bar)\n    * Initialize a workspace from this manifest\n    * Push a new file to the foo repo\n    * Create a merge conflict in the foo repo\n    * Run `tsrc sync`\n    * Check that the command fails and produces the proper error message\n    \"\"\"\n
    "},{"location":"code-manifesto/#assertions_with_lists","title":"Assertions with lists","text":"
    • Use tuple unpacking to write shorter assertions:
    # Yes\nactual_list = function_that_returns_list()\n(first, second) = actual_list\nassert first == something\nassert second == something_else\n\n# NO\nactual_list = function_that_returns_list()\nassert len(actual_list) == 2\nfirst = actual_list[0]\nsecond = actual_list[1]\nassert first == something\nassert second == something_else\n
    "},{"location":"code-manifesto/#assertion_order","title":"Assertion order","text":"

    When writing assertions, use the form assert <actual> == <expected>:

    # Yes\ndef test_foo():\n    assert foo(42) == True\n\ndef test_big_stuff():\n    actual_result = ...\n    expected_result = ...\n\n    assert actual_result == expected_result\n\n\n# No\ndef test_foo():\n    assert True == foo(42)\n\n\ndef test_big_stuff():\n    actual_result = ...\n    expected_result = ...\n\n    assert expected_result == actual_result\n

    Rationale:

    • The assert(expected, actual) convention comes from JUnit but we are not writing Java code, and besides, the assert(actual, expected) convention also exists in other tools.
    • pytest does not really care, but we prefer being consistent in all tests.
    • It's a bit closer to what you would say in English: \"Assert that the result of foo() is 42\".
    "},{"location":"faq/","title":"FAQ","text":""},{"location":"faq/#what_does_the_name_mean","title":"What does the name mean?","text":"

    The t stands for tool and src for sources.

    If you speak French, you can also remember the name as \"tes sources\".

    "},{"location":"faq/#why_not_repo","title":"Why not repo?","text":"

    We used repo for a while, but found that tsrc had both a better command line API and a nicer output.

    On a less subjective level:

    • Good support for Windows (no need for Cygwin or anything like that)

    • Also, tsrc tries hard to never do any destructive operation or unexpected actions.

      For instance, tsrc never puts you in a \"detached HEAD\" state, nor does automatic rebase. It also never touches dirty repositories.

      This is achieved by using mostly 'porcelain' commands from git, instead of relying on plumbings internals.

    Also (and this matters a lot if you think about contribution):

    • Uses PEP8 coding style, enforced with black
    • Comprehensive test suite
    • Fully type-checked with mypy

    Note that there are a few features present in repo that are missing from tsrc (but may be implemented in the future). Feel free to open a feature request if needed!

    "},{"location":"faq/#why_not_git-subrepo_mu-repo_or_gr","title":"Why not git-subrepo, mu-repo, or gr?","text":"

    All this projects are fine but did not match our needs:

    • git-subrepo squashes commits, and we prefer having normal clones everywhere.
    • mu-repo is nice and contains an interesting dependency management feature, but currently we do not need this complexity.

    In any case, now that the whole team is using tsrc all the time, it's likely we'll keep using tsrc in the future.

    "},{"location":"faq/#why_not_git_submodule","title":"Why not git submodule?","text":"

    It's all about workflow.

    With git-submodule, you have a 'parent' repository and you freeze the state of the 'children' repositories to a specific commit.

    It's useful when you want to re-build a library you've forked when you build your main project, or when you have a library or build tools you want to factorize across repositories: this means that each 'parent' repository can have its children on any commit they want.

    With tsrc, all repositories are equal, and what you do instead is to make sure all the branches (or tags) are consistent across repositories.

    For instance, if you have foo and bar, you are going to make sure the 'master' branch of foo is always compatible to the 'master' branch of bar.

    Or if you want to go back to the state of the '0.42' release, you will run: tsrc foreach -- git reset --hard v0.42.

    Note that since tsrc 0.2 you can also freeze the commits of some of the repositories.

    Last but not least, if you really need to use fixed references, you may do so by adding a sha1 or tag line to the manifest. See the relevant guide for more details.

    "},{"location":"faq/#why_not_using_pygit2_or_similar_instead_of_running_git_commands","title":"Why not using pygit2 or similar instead of running git commands?","text":"

    First off, we do use pygit2, but only for tests.

    Second, the pygit2 package depends on a 3rd party C library (libgit2) - and that can cause problems in certain cases. If we can, we prefer using pure-Python libraries for the production code.

    Finally, we prefer calling git \"porcelain\" commands, both for readability of the source code and ease of debugging (see below).

    "},{"location":"faq/#why_do_you_hide_which_git_commands_are_run","title":"Why do you hide which git commands are run?","text":"

    It's mainly a matter of not cluttering the output. We take care of keeping the output of tsrc both concise, readable and informative.

    That being said:

    • In case a git command fails, we'll display the full command that was run.
    • If you still need to see all the git commands that are run, we provide a --verbose flag, like so: tsrc --verbose sync
    "},{"location":"faq/#why_yaml","title":"Why YAML?","text":"

    It's nice to read and write, and we use the excellent ruamel.yaml which even has round-trip support.

    Also, being Python fans, we don't mind that white space is part of the syntax.

    "},{"location":"faq/#why_do_i_have_to_create_a_separate_git_repo_with_just_one_file_in_it","title":"Why do I have to create a separate git repo with just one file in it?","text":"

    See #235 for why you can't have multiple manifest files in the same repository.

    Also, note that you can put other files in the repo - for instance, add a CI script that verifies the yaml syntax and checks that all the repos in the manifest can be cloned.

    "},{"location":"getting-started/","title":"Getting started","text":""},{"location":"getting-started/#requirements","title":"Requirements","text":"

    Python 3.7 or later

    "},{"location":"getting-started/#installing_tsrc","title":"Installing tsrc","text":"

    The recommended way to install tsrc is to use pipx. This is because pipx automatically creates isolated environment for each app, so you won't get into dependencies versions conflicts and won't have to deal with manual virtualenvs management.

    pip will also work, but it will not give you these benefits.

    Recommended: 

    pipx install tsrc\n

    Acceptable, if you know what you are doing: 

    pip install tsrc\n

    "},{"location":"getting-started/#checking_tsrc_installation","title":"Checking tsrc installation","text":"

    Run:

    $ tsrc --version\n
    "},{"location":"getting-started/#creating_a_repository_for_the_manifest","title":"Creating a repository for the manifest","text":"

    Let's say you are working for the ACME company and you have many git repositories.

    You need a tool to track them, so that if a new repository is created, all developers can get a clone on their development machine quickly, without having to look up its URL or even know it exists.

    Also, you need to make sure the repos are cloned in a certain way, so that you can for instance refer a repo from an other one by using a relative path.

    This is where tsrc comes in.

    The first step is to create a dedicated repository for the manifest. I know it may sound wasteful (\"I have already 100 repositories to manage, and you want me to create yet an other one?\"), but, trust me, it's worth it.

    So, if your company uses a GitLab instance at gitlab.acme.com and you want to crate a manifest for your team, you may start by creating a new repository at https://gitlab.acme.com/your-team/manifest.

    Inside this repository, create a file named manifest.yml looking like this:

    repos:\n  - dest: foo\n    url: git@gitlab.acme.com/your-team/foo\n\n  - dest: bar\n    url: git@gitlab.acme.com/your-team/bar\n

    Note that this approach works with any king of Git Hosting system, not just a custom GitLab instance. Just replace gitlab.acme.com/your-team with the correct suffix (like github.com/your-name/ if you want to track some repositories from your GitHub account).

    "},{"location":"getting-started/#creating_a_new_workspace","title":"Creating a new workspace","text":"

    Create a new, empty directory and then run tsrc init from it, using the URL of the manifest created in the above step:

    $ mkdir work\n$ cd work\n$ tsrc init git@gitlab.acme.com/your-team/manifest.git\n

    You should see something like this:

    :: Configuring workspace in /path/to/work\nCloning into 'manifest'...\n...\n=> Cloning missing repos\n* (1/2) Cloning foo\nCloning into 'foo'...\n...\n* (2/2) Cloning bar\nCloning into 'bar'...\n...\n=> Cloned repos:\n* foo cloned from gt@gitlab.acme.com/your-team/foo' (on master)\n* bar cloned from gt@gitlab.acme.com/your-team/bar' (on master)\n=> Configuring remotes\n=> Workspace initialized\n=> Configuration written in /path/to/work/.tsrc/config.yml\n

    You will notice that:

    • The foo ad bar repositories have been cloned into their respective destination
    • A workspace configuration file has been created in /path/to/work/.tsrc/config.yml. This file can be edited by hand to customize tsrc behavior. Follow the relevant guide, or read the workspace configuration file reference for more details.
    "},{"location":"getting-started/#updating_a_new_workspace","title":"Updating a new workspace","text":"

    Now let's assume that Alice created a new commit in foo, and Bob a new commit it bar, and that they both pushed them to the master branch of the respective repositories.

    Now that you have a workspace configured with tsrc, you can use tsrc sync to retrieve all the changes in one go:

    $ cd work\n$ tsrc sync\n

    This time, you should see the following output:

    :: Using workspace in /path/to/work\n=> Updating manifest\n...\n=> Cloning missing repos\n=> Configuring remotes\n=> Synchronizing repos\n* (1/2) Synchronizing foo\n* Fetching origin\n...\n   f20af74..aca6c35  master     -> origin/master\n* Updating branch: master\nUpdating f20af74..aca6c35\nFast-forward\n new.txt | 1 +\n 1 file changed, 1 insertion(+)\n create mode 100644 new.txt\n* (2/2) Synchronizing bar\n* Fetching origin\n...\n   f20af74..02cfef6  master     -> origin/master\n* Updating branch: master\nUpdating f20af74..02cfef6\nFast-forward\n spam.py | 1 +\n 1 file changed, 1 insertion(+)\n create mode 100644 spam.py\n:: Workspace synchronized\n

    Note: tsrc sync does not call git pull on every repository. The precise algorithm is described in the reference documentation

    "},{"location":"getting-started/#adding_a_new_repo_to_the_manifest","title":"Adding a new repo to the manifest","text":"

    Let's say your team now needs a third repository (for instance, at gitlab.acme.com/your-team/baz).

    Start by making a commit in the manifest repository that adds the new repository:

    --- a/manifest.yml\n+++ b/manifest.yml\n@@ -1,2 +1,3 @@\n repos:\n   - dest: foo\n     url: git@gitlab.acme.com/your-team/foo\n\n   - dest: bar\n     url: git@gitlab.acme.com/your-team/baz\n\n+  - dest: baz\n+    url: git@gitlab.acme.com/your-team/baz\n

    Then push this commit to the master branch of the manifest.

    This time when you run tsrc sync:

    • the manifest repository will get updated
    • the baz repo will be cloned in /path/to/work/baz
    $ tsrc sync\n
    :: Using workspace in /path/to/work\n=> Updating manifest\nremote: Enumerating objects: 5, done.\n...\nUnpacking objects: 100% (3/3), 354 bytes | 354.00 KiB/s, done.\nFrom gitlab.acme.com/your-team/manifest\n   63f12d4..bbcd4d9  master     -> origin/master\nReset branch 'master'\n...\nHEAD is now at bbcd4d9 add baz\n\n=> Cloning missing repos\n* (1/1) Cloning baz\nCloning into 'baz'...\n...\nReceiving objects: 100% (3/3), done.\n=> Cloned repos:\n* baz cloned from git@gitlab.acme.com/bas (on master)\n=> Configuring remotes\n=> Synchronizing repos\n* (1/3) Synchronizing foo\n* Fetching origin\n* Updating branch: master\nAlready up to date.\n* (2/3) Synchronizing bar\n* Fetching origin\n* Updating branch: master\nAlready up to date.\n* (3/3) Synchronizing baz\n* Fetching origin\n* Updating branch: master\nAlready up to date.\n:: Workspace synchronized\n
    "},{"location":"getting-started/#going_further","title":"Going further","text":"

    In this tutorial, we made a lot of assumptions:

    • Every repository is using the master as the main development branch
    • Each repository as only one git remote (the one from gitlab.acme.com in our example)
    • You're using a manifest just for your team, not the whole company
    • ...

    tsrc can handle all of this use cases, and more. See the other guides for more details.

    "},{"location":"contrib/dev/","title":"Suggesting changes","text":"

    All the development happens on GitHub.

    You are free to open a pull request for anything you want to change on tsrc.

    In particular, pull requests that implement a prototype for a new feature are welcome, having \"real code\" to look at can provide useful insight, even if the code is not merged after all.

    That being said, if you want your pull request to be merged, we'll ask that:

    • The code follows the indications from the code manifesto
    • All existing linters pass
    • All existing tests run
    • The new feature comes with appropriate tests
    • The Git History is easy to review

    See the GitHub actions workflows to see what exactly what commands are run and the Python versions we support.

    Also, if relevant, you will need to:

    • update the changelog (in docs/changelog.md)
    • update the documentation if required

    Finally, feel free to add your name in the THANKS file ;)

    "},{"location":"contrib/dev/#checking_your_changes","title":"Checking your changes","text":"
    • Install latest poetry version.
    • Install development and documentation dependencies:
    $ poetry install\n
    • Run linters and tests:
    $ poetry run invoke lint\n$ poetry run pytest -n auto\n
    "},{"location":"contrib/dev/#adding_documentation","title":"Adding documentation","text":"
    • Follow the steps from the above section to setup your python environment
    • Launch the development server locally:
    $ poetry run mkdocs serve\n
    • Edit the markdown files from the docs/ folder and review the changes in your browser
    • Finally, submit your changes by opening a pull request on GitHub
    "},{"location":"contrib/issues/","title":"Using the issue tracker","text":"

    Reporting bugs and requesting new features is done one the tsrc issue tracker on GitHub.

    "},{"location":"contrib/issues/#reporting_bugs","title":"Reporting bugs","text":"

    If you are reporting a bug, please provide the following information:

    • tsrc version
    • Details about your environment (operating system, Python version)
    • The exact command you run
    • The full output

    Doing so will ensure we can investigate your bug right away.

    "},{"location":"contrib/issues/#suggesting_new_features","title":"Suggesting new features","text":"

    If you think tsrc is lacking a feature, please provide the following information:

    • What exactly is your use case?
    • Do you need a new command-line option or even a new command?
    • Do you need changes in the configuration files?

    Note that changingtsrc behavior can get tricky.

    First off, we want to avoid data loss following a tsrc command above

    Second, we want to keep tsrc behavior as least surprising as possible, so that it can be used without having to read (too much of) documentation.

    To that end, and keeping in mind tsrc needs to accommodate a large variety of use cases, we want to keep the code:

    • easy to read and,
    • easy to maintain,
    • and very well tested.

    The best way to achieve all of this is to keep it simple.

    This means we'll be very cautious before implementing a new feature, so don't hesitate to open an issue for discussion before jumping into the development of a new feature.

    "},{"location":"guide/ci/","title":"Using tsrc with Continuous Integration (CI)","text":""},{"location":"guide/ci/#github_actions","title":"GitHub Actions","text":"

    Let suppose you have a private GitHub organization holding several private repositories and tsrc to synchronize them using the SSH protocol. Let suppose you want to use GitHub Actions to download the code source of your organization, compile it and run some non regression tests. What to write to achieve this with tsrc?

    "},{"location":"guide/ci/#step_1_your_tsrc_manifest","title":"Step 1: Your tsrc manifest","text":"

    Your tsrc manifest.yml looks something like this:

    repos:\n  - url: git@github.com:project1/foo\n    dest: foo\n

    The git@ means SSH protocol.

    "},{"location":"guide/ci/#step_2_create_your_github_workflows_file","title":"Step 2: Create your GitHub workflows file","text":"

    In your private GitHub repository holding the GitHub workflows files, create the folder .github/workflows and your yaml file with the desired name and the following content. For more information about GitHub actions syntax see this video:

    name: tsrc with private github repos\non:\n  workflow_dispatch:\n    branches:\n      - main\n\njobs:\n  export_linux:\n    runs-on: ubuntu-latest\n    steps:\n    - name: Installing tsrc tool\n      run: |\n        sudo apt-get update\n        sudo apt-get install -y python3\n        python -m pip install tsrc\n\n    - name: Cloning private github repos\n      run: |\n        git config --global url.\"https://${{ secrets.ACCESS_TOKEN }}@github.com/\".insteadOf git@github.com:\n        export WORKSPACE=$GITHUB_WORKSPACE/your_project\n        mkdir -p $WORKSPACE\n        cd $WORKSPACE\n        tsrc init git@github.com:yourorganisation/manifest.git\n        tsrc sync\n

    This script will run on the latest Ubuntu Docker and triggers steps: - The first step named Installing tsrc tool allows to install python3 and then tsrc. - The second step named Cloning private github repos creates a folder named your_project for your workspace and call the initialization and synchronization of your repositories.

    The important command is: 

    git config --global url.\"https://${{ secrets.ACCESS_TOKEN }}@github.com/\".insteadOf git@github.com:\n

    which allows to replace the SSH syntax by the HTTPs syntax on your GitHub repository names.

    "},{"location":"guide/ci/#step_3_create_the_github_secret","title":"Step 3: Create the GitHub secret","text":"

    For GitHub organization one member of the team has the responsibility to hold a Personal access tokens for the organization. Go https://github.com/settings/tokens and click on the button Generate new token then click on repo checkbox then click on the button Generate token.

    Now, this token shall be saved into an action secret named ACCESS_TOKEN inside the GitHub repository holding the GitHub workflows files.

    "},{"location":"guide/ci/#step_4_enjoy","title":"Step 4: Enjoy","text":"

    In the menu Actions of your repository you can trig the workflow. In this example we used workflow_dispatch to perform manual triggers. So click on the button to start the process. Once this step done with success, you can update your workflow yaml to complete your CI work: compilation of your project, run non regression tests, etc.

    "},{"location":"guide/fixed-refs/","title":"Using fixed git references","text":"

    By default, tsrc sync synchronize projects using branches names.

    Usually, one would use the same branch name for several git repositories, like this:

    repos:\n  - dest: foo\n    url: git@gitlab.acme.com/your-team/foo\n    branch: main\n\n  - dest: bar\n    url: git@gitlab.acme.com/your-team/bar\n    branch: main\n

    The assumption here is that foo and bar evolve \"at the same time\", so when the main branch of foo is updated, the main branch of bar much change too.

    Sometimes though, this will not be the case. For instance, the main branch of the bar repo needs a specific, fixed version of foo in order to work.

    "},{"location":"guide/fixed-refs/#using_a_tag","title":"Using a tag","text":"

    One way to solve this is to push a v1.0 tag in the foo repository, and change the manifest too look like this:

    repos:\n  - dest: foo\n    url: git@gitlab.acme.com/your-team/foo\n-    branch: main\n+    tag: v1.0\n
    "},{"location":"guide/fixed-refs/#using_a_sha1","title":"Using a sha1","text":"

    An other way is to put the SHA1 of the relevant git commit in the foo repository in the manifest:

    repos:\n  - dest: foo\n    url: git@gitlab.acme.com/your-team/foo\n    branch: main\n+    sha1: ad2b68539c78e749a372414165acdf2a1bb68203\n
    "},{"location":"guide/fixed-refs/#cloning_repos_using_fixed_refs","title":"Cloning repos using fixed refs","text":"
    • If the repo is configured with a tag, tsrc will call git clone --branch <tag> (which is valid)
    • Otherwise, tsrc will call git clone, followed by git reset --hard <sha1>

    This is because you cannot tell git to use an arbitrary git reference as start branch when cloning (tags are fine, but sha1s are not).

    This also explain why you need both branch and sha1 in the configuration.

    "},{"location":"guide/fixed-refs/#synchronizing_repos_using_fixed_refs","title":"Synchronizing repos using fixed refs","text":"

    Here's what tsrc sync will do when trying to synchronize a repo configured with a fixed ref:

    • Run git fetch --tags --prune
    • Check if the repository is clean
    • If so, run git reset --hard <tag or sha1>
    "},{"location":"guide/foreach/","title":"Running a command for each repo in the workspace","text":"

    tsrc comes with a foreach command that allows you to run the same command for each repo in the workspace.

    This can be used for several things. For instance, if you are building an artifact from a group of repositories, you may want to put a tag on each repo that was used to produce it:

    $ tsrc foreach git tag v1.2\n
    :: Using workspace in /path/to/work\n:: Running `git tag v1.1` on 2 repos\n/path/to/work/foo $ git tag v1.2\n/path/to/work/bar $ git tag v1.2\n/path/to/work/baz $ git tag v1.2\nOK \u2713\n
    "},{"location":"guide/foreach/#caveats","title":"Caveats","text":"
    • If the command you want to run contains arguments starting with\u00a0-: you need to call foreach like this:
    $ tsrc foreach -- some-command --with-option\n
    • By default, the command is passed \"as is\", without starting a shell. If you want to use a shell, use the -c option:
    $ tsrc foreach -c  'echo $PWD'\n

    Note that we need single quotes here to prevent the shell from expanding the PWD environment variable when tsrc is run.

    "},{"location":"guide/foreach/#using_repo_and_manifest_data","title":"Using repo and manifest data","text":"

    The current tsrc implementation may not contain all the features your organization needs.

    The good news is that you can extend tsrc's feature set by using tsrc foreach.

    Let's take an example, where you have a manifest containing foo and bar and both repos are configured to use a master branch.

    Here's what happens if you run tsrc sync with bar on the correct branch (master), and foo on an incorrect branch (devel):

    $ tsrc sync\n
    :: Using workspace in /path/to/work\n=> Updating manifest\n...\n=> Cloning missing repos\n=> Configuring remotes\n=> Synchronizing repos\n* (1/2) Synchronizing foo\n* Fetching origin\n* Updating branch: devel\nUpdating 702f428..2e4fb45\nFast-forward\n...\n* (2/2) Synchronizing bar\n* Fetching origin\n* Updating branch: master\nAlready up to date.\nError: Failed to synchronize the following repos:\n* foo : Current branch: 'devel' does not match expected branch: 'master'\n

    If this happens with multiple repos, you may want a command to checkout the correct branch automatically.

    Here's one way to do it:

    $ tsrc foreach -c 'git checkout $TSRC_PROJECT_MANIFEST_BRANCH'\n

    Here we take advantage of the fact that tsrc sets the TSRC_PROJECT_MANIFEST_BRANCH environment variable correctly for each repository before running the command.

    Here's the whole list:

    Variable Description TSRC_WORKSPACE_PATH Full path of the workspace root TSRC_MANIFEST_BRANCH Branch of the manifest TSRC_MANIFEST_URL URL of the manifest TSRC_PROJECT_CLONE_URL URL used to clone the repo TSRC_PROJECT_DEST Relative path of the repo in the workspace TSRC_PROJECT_MANIFEST_BRANCH Branch configured in the manifest for this repo TSRC_PROJECT_REMOTE_<NAME> URL of the remote named 'NAME' TSRC_PROJECT_STATUS_DIRTY Set to true if the project is dirty, otherwise unset TSRC_PROJECT_STATUS_AHEAD Number of commits ahead of the remote ref TSRC_PROJECT_STATUS_BEHIND Number of commits behind the remote ref TSRC_PROJECT_STATUS_BRANCH Current branch of the repo TSRC_PROJECT_STATUS_SHA1 SHA1 of the current branch TSRC_PROJECT_STATUS_STAGED Number of files that are staged but not committed TSRC_PROJECT_STATUS_NOT_STAGED Number of files that are changed but not staged TSRC_PROJECT_STATUS_UNTRACKED Number of files that are untracked

    You can implement more complex behavior using the environment variables above, for instance:

    #!/bin/bash\n# in switch-and-pull\nif [[ \"${TSRC_PROJECT_STATUS_DIRTY}\" = \"true\" ]]; then\n  echo Error: project is dirty\n  exit 1\nfi\n\ngit switch $TSRC_PROJECT_MANIFEST_BRANCH\ngit pull\n
    $ tsrc foreach switch-and-pull\n:: Running `switch-and-pull` on 2 repos\n* (1/2) foo\n/path/to/foo $ switch-and-pull\nSwitched to branch 'master'\nYour branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.\n  (use \"git pull\" to update your local branch)\nUpdating 9e7a8e4..5f9bbd4\nFast-forward\n* (2/2) bar\n/path/to/bar $ switch-and-pull\nError: project is dirty\nError: Command failed for 1 repo(s)\n* bar\n

    Of course, feel free to use your favorite programming language here :)

    "},{"location":"guide/fs/","title":"Performing file system operations","text":""},{"location":"guide/fs/#introduction","title":"Introduction","text":"

    When using tsrc, it is assumed that repositories are put in non-overlapping file system hierarchies, like this:

    workspace/\n  project_1/\n   CMakeLists.txt\n    foo.cpp\n    bar.cpp\n  project_2/\n    CMakeLists.txt\n    spam.cpp\n    eggs.cpp\n

    Not like that, where project_2 is inside a sub-directory of project_1:

    workspace/\n  project_1/\n    CMakeLists.txt\n    foo.cpp\n    bar.cpp\n    project_2/\n      CMakeLists.txt\n      spam.cpp\n      eggs.cpp\n

    Note

    if you really need project_2 to be a sub-directory of project_1, consider using git submodules instead.

    This is usually fine, except when project_1 and project_2 share some common configuration.

    For instance, you may want to use clang-format for both project_1 and project_2.

    "},{"location":"guide/fs/#copying_a_file","title":"Copying a file","text":"

    One solution is to put the .clang-format configuration file in a repo named common and then tell tsrc to copy it at the root of the workspace:

    repos:\n  - dest: project_1\n    url: git@acme.com:team/project_1\n\n  - dest: project_2\n    url: git@acme.com:team/project_2\n\n  - dest: common\n    url: git@acme.com:team/commont\n    copy:\n    - file: clang-format\n      dest: .clang-format\n
    $ tsrc sync\n=> Cloning missing repos\n* (1/1) Cloning common\nCloning into 'common'...\n...\n=> Performing filesystem operations\n* (1/1) Copy /path/to/work/common/clang-format -> /path/to/work/.clang-format\n

    Notes:

    • copy only works with files, not directories.
    • The source path for a copy link is relative to associated repos destination, whereas the destination path of the copy is relative to the workspace root.
    "},{"location":"guide/fs/#creating_a_symlink","title":"Creating a symlink","text":"

    The above method works fine if the file does not change too often - if not, you may want to create a symbolic link instead:

    repos:\n  - dest: project_1\n    url: git@acme.com:team/project_1\n\n  - dest: project_2\n    url: git@acme.com:team/project_2\n\n  - dest: common\n    url: git@acme.com:team/commont\n    symlink:\n    - source: .clang-format\n      target: common/clang-format\n
    $ tsrc sync\n=> Cloning missing repos\n...\n=> Performing filesystem operations\n* (1/1) Lint /path/to/work/.clang-format -> common/.clang-format\n

    Notes:

    • The source path for a symbolic link is relative to the top-level <workspace>, whereas each target path is then relative to the associated source. (This path relationship is essentially identical to how ln -s works on the command line in Unix-like environments.) Multiple symlinks can be specified; each must specify a source and target.

    • Symlink creation is supported on all operating systems, but creation of NTFS symlinks on Windows requires that the current user have appropriate security policy permission (SeCreateSymbolicLinkPrivilege). By default, only administrators have that privilege set, although newer versions of Windows 10 support a Developer Mode that permits unprivileged accounts to create symlinks. Note that Cygwin running on Windows defaults to creating links via Windows shortcuts, which do not require any special privileges. (Cygwin's symlink behavior can be user controlled with the winsymlinks setting in the CYGWIN environment variable.)

    "},{"location":"guide/groups/","title":"Using groups","text":"

    Sometimes it can be necessary to create groups of repositories, especially if the number of repositories grows and if you have people in different teams work on different repositories.

    "},{"location":"guide/groups/#defining_groups_in_the_manifest","title":"Defining groups in the manifest","text":"

    The first step is to edit the manifest.yml file to describe the groups. Here's an example.

    repos:\n  - {url: git@gitlab.local:acme/one,   dest: one}\n  - {url: git@gitlab.local:acme/two,   dest: two}\n  - {url: git@gitlab.local:acme/three, dest: three}\n\ngroups:\n  default:\n    repos: []\n  g1:\n    repos:\n      - one\n      - two\n  g2:\n    repos:\n      - three\n

    Here we define a g1 group that contains repositories named one and two, and a g2 group that contains the repository named three.

    "},{"location":"guide/groups/#using_groups_in_tsrc_init","title":"Using groups in tsrc init","text":"

    If you only need the repositories in the g1 group you can run:

    tsrc init git@gitlab.local:acme/manifest --group g1\n
    "},{"location":"guide/groups/#filtering_repositories_in_groups_with_regular_expressions","title":"Filtering repositories in groups with regular expressions","text":"

    You can utilize inclusive regular expression with the -r-flag and exclusive regular expression with the -i-flag. This allows you to filter repositories within a group or a set of groups for the given action.

    To include all repositories in the group g1 matching \"config\" and excluding \"template\", you can do the following:

    tsrc init git@gitlab.local:acme/manifest --group g1 -r config -i template\n
    "},{"location":"guide/groups/#updating_workspace_configuration","title":"Updating workspace configuration","text":"

    Alternatively, you can edit the .tsrc/config.yml file, like this:

    manifest_url: git@gitlab.local:acme/manifest.git\nmanifest_branch: master\nrepo_groups:\n- g1   # <- specify the list of groups to use\n

    You can use this technique to change the groups used in a given workspace - the above method using init only works to create new workspaces.

    The config file contains other configuration options, which are described in the workspace configuration documentation.

    "},{"location":"guide/manifest/","title":"Editing the manifest safely","text":""},{"location":"guide/manifest/#introduction_when_things_go_wrong","title":"Introduction: when things go wrong","text":"

    Let's assume you've successfully implemented tsrc for your organization - now need to make sure to not break anyone's workflow.

    Let's see what could go wrong if you make mistakes while editing the manifest, using a branch called broken for the sake of the example).

    First, let's see what happens if you break the YAML syntax:

    commit 1633c5a6 (HEAD -> broken, origin/broken)\n\n    Break the manifest syntax\n\ndiff --git a/manifest.yml b/manifest.yml\nindex fe74142..068c35e 100644\n--- a/manifest.yml\n+++ b/manifest.yml\n@@ -1,4 +1,4 @@\n-repos:\n+repos\n   - url: git@github.com:your-tools/bar.git\n     dest: bar\n

    After this change is push, anyone using the broken branch of the manifest will be faced with this kind of error message:

    $ tsrc sync\n
    => Updating manifest\nReset branch 'broken'\nYour branch is up to date with 'origin/broken'.\nBranch 'broken' set up to track remote branch 'broken' from 'origin'.\nHEAD is now at 1633c5a Break the manifest syntax\nError: /path/to/work/.tsrc/manifest/manifest.yml: mapping values are\nnot allowed here :\n\n      - url: git@gitlab.acme.com:your-team/foo\n           ^ (line: 2)\n

    Similarly, if you put an invalid URL in the manifest, like this:

    commit ccfb902 (HEAD -> broken, origin/broken)\n\n    Use invalid URL for bar repo\n\ndiff --git a/manifest.yml b/manifest.yml\nindex fe74142..068c35e 100644\n--- a/manifest.yml\n+++ b/manifest.yml\n@@ -1,4 +1,4 @@\nrepos:\n-  - url: git@gitlab.acme.com:your-team/bar\n+  - url: git@gitlab.acme.com:your-team/invalid\n     dest: bar\n

    Users will get:

    $ tsrc sync\n
    :: Using workspace in /path/to/work\n=> Updating manifest\n...\nHEAD is now at ccfb902 Use invalid URL\n=> Cloning missing repos\n=> Configuring remotes\n* bar: Update remote origin to new url: (git@acme.com:your-team/invalid.git)\n...\n=> Synchronizing repos\n* (1/2) Synchronizing bar\n* Fetching origin\nERROR: Repository not found.\nfatal: Could not read from remote repository.\n\nPlease make sure you have the correct access rights\nand the repository exists.\nError: fetch from 'origin' failed\n* (2/2) Synchronizing foo\n ...\nError: Failed to synchronize the following repos:\n* bar : fetch from 'origin' failed\n

    This will probably not be a huge problem for you, dear reader, because you know about tsrc's manifest and its syntax.

    It will, however, be a problem for people who are just using tsrc without knowledge of how it is implemented, because those error messages will definitely confuse them.

    "},{"location":"guide/manifest/#using_the_apply-manifest_command_to_avoid_breaking_developers_workflow","title":"Using the apply-manifest command to avoid breaking developers workflow","text":"

    If you have a file on your machine containing the manifest changes, you can use tsrc apply-manifest to check those changes against your own workspace:

    $ cd /path/to/work\n$ tsrc apply-manifest /path/to/manifest-repo/manifest.yml\n# Check that the changes are OK\n# If so, commit and push manifest changes:\n$ cd path/to/manifest-repo\n$ git commit -a -m \"...\"\n$ git push\n# Now you know that everyone can safely run `tsrc sync`\n
    "},{"location":"guide/manifest/#additional_notes","title":"Additional notes","text":"

    • It is not advised to edit the file in .tsrc/manifest/manifest.yml directly, because tsrc sync will silently undo any local changes made to this file. This is a known bug, see #279 for details.

    • It is common to place the manifest repo itself in the manifest - so it's easy to edit or read:

    # In acme.com:your-team/manifest - manifest.yml\nrepos:\n  - url: git@acme.com:your-team/manifest\n     dest: manifest\n\n  - url: git@acme.com:your-team/foo\n     dest: foo\n\n  - url: git@acme.com:your-team/bar\n     dest: bar\n

    In that case, you would use:

    $ tsrc apply-manifest <workspace>/manifest/manifest.yml\n

    to check changes before pushing them.

    "},{"location":"guide/remotes/","title":"Using several remotes","text":"

    When you specify a repository in the manifest with just an URL, tsrc assumes you want a remote named origin:

    repos:\n  - dest: foo\n    url: git@gitlab.acme.com/your-team/foo\n\n  - dest: bar\n    url: git@gitlab.acme.com/your-team/bar\n

    But sometimes you need several remotes. Let's see a few use cases.

    "},{"location":"guide/remotes/#mirroring_open-source_projects","title":"Mirroring open-source projects","text":"

    If you want some repos in your organization to be open source, you may need:

    • a remote named 'origin' containing for the private repository on your GitLab instance
    • a remote named 'github' for the public repository on GitHub

    In that case, you can use an alternative syntax:

    repos:\n  # foo is open source and thus needs two remotes:\n  - dest: foo\n  - remotes:\n    - name: origin\n      url: git@gitlab.acme.com/your-team/foo\n    - name: github\n      url: git@github.com/your-team/foo\n\n  # bar is closed source and thus only needs the\n  # default, 'origin' remote:\n  - dest: bar\n    url: gitlab.acme.com/your-team/bar\n

    After this change, when running tsrc init or tsrc sync, both the origin and github remotes will be created in the foo repo if they don't exist, and both remotes will be fetched when using tsrc sync.

    "},{"location":"guide/remotes/#using_a_vpn","title":"Using a VPN","text":"

    Sometimes you will need two remotes, because depending the physical location of your developers, they need to use either:

    • a 'normal' remote, when they are in the office
    • a 'vpn' remote, when they are working at home

    In that case, you can create a manifest looking like this:

    repos:\n  - dest: foo\n  - remotes:\n    - name: origin\n      url: git@gitlab.local/your-team/foo\n    - name: vpn\n      url: git@myvpn.com/gitlab/your-team/foo\n\n  - dest: bar\n  - remotes:\n    - name: origin\n      url: git@gitlab.local/your-team/bar\n    - name: vpn\n      url: git@myvpn.com/gitlab/your-team/bar\n

    Developers can then use the -r, --singular-remote option to either use the origin or vpn when running tsrc init (to create a workspace), or tsrc sync (to synchronize it), depending on their physical location:

    # Init the workspace using the 'vpn' remote\n$ tsrc init -r vpn\n# Bring back the computer in the office\n# Synchronize using the 'origin' remote:\n$ tsrc sync -r origin\n

    Note

    When using this option, tsrc expects the remote to be present in the manifest for all repositories.

    "},{"location":"guide/workspace-config/","title":"Editing workspace configuration","text":""},{"location":"guide/workspace-config/#creation","title":"Creation","text":"

    The configuration file created by tsrc init contains the whole list of available settings, with their default value, and is located at </path/to/workspace/.tsrc/manifest.yml>.

    Note that if you use command-line options when using tsrc init, those will be written in the .tsrc/config.yml.

    For instance:

    tsrc init git@github.com:dmerejkowsky/dummy-manifest\n

    generates this file:

    manifest_url: git@github.com:dmerejkowsky/dummy-manifest\nmanifest_branch: master\nrepo_groups: []\nshallow_clones: false\nclone_all_repos: false\nsingular_remote:\n

    But

    tsrc init git@github.com:dmerejkowsky/dummy-manifest --branch main\n

    generates this instead:

    manifest_url: git@github.com:dmerejkowsky/dummy-manifest\nmanifest_branch: main\nrepo_groups: []\nshallow_clones: false\nclone_all_repos: false\nsingular_remote:\n
    "},{"location":"guide/workspace-config/#editing","title":"Editing","text":"

    You can edit the workspace configuration as you please, for instance if you need to switch the manifest branch.

    If you do so, note that your changes will be taken into account next time you run tsrc sync.

    "},{"location":"ref/cli/","title":"Command line usage","text":""},{"location":"ref/cli/#important_note","title":"Important note","text":"

    We use the argparse library to parse command line arguments, so the --help messages are always up-to-date, probably more so than this documentation :)

    "},{"location":"ref/cli/#general","title":"General","text":"

    tsrc uses the same \"subcommand\" pattern as git does.

    Options common to all commands are placed right before the command name.

    Options after the command name only apply to this command.

    For instance:

    $ tsrc --verbose sync\n$ tsrc init MANIFEST_URL\n
    "},{"location":"ref/cli/#goodies","title":"Goodies","text":"

    First, note that like git, tsrc will walk up the folders hierarchy looking for a .tsrc folder, which means you can run tsrc commands anywhere in your workspace, not just at the top.

    Second, almost all commands run the operation in parallel. For instance, tsrc sync by default will use as many jobs as the number of CPUs available on the current machine to synchronize the repos in your workspace. If this behavior is not desired, you can specify a greater (or lower) number of jobs using something like tsrc sync -j2, or disable the parallelism completely with -j1. You can also set the default number of jobs by using the TSRC_PARALLEL_JOBS environment variable.

    "},{"location":"ref/cli/#global_options","title":"Global options","text":"--verbose show verbose messages -q, --quiet hide everything except errors and warnings --color [always|never|auto] control using color for messages (default 'auto', on if stdout is a terminal)"},{"location":"ref/cli/#usage","title":"Usage","text":"tsrc init MANIFEST_URL [--group GROUP1, GROUP2] [--singular-remote SINGULAR_REMOTE]

    Initializes a new workspace.

    MANIFEST_URL should be a git URL containing a valid manifest.yml file.

    The -g,--groups option can be used to specify a list of groups to use when cloning repositories.

    The -r \"inclusive regular expression\" and -i \"exclusive regular expression\" options can be combined with the group option to filter for repositories within a group. -r takes precedence if both options are present.

    The -s,--shallow option can be used to make shallow clone of all repositories.

    If you want to add or remove a group in your workspace, you can edit the configuration file in <workspace>/.tsrc/config.yml

    The -r,--singular-remote option can be used to set a fixed remote to use when cloning and syncing the repositories. If this flag is set, the remote from the manifest with the given name will be used for all repos. It is an error if a repo does not have this remote specified.

    tsrc foreach -- command --opt1 arg1

    Runs command --opt1 arg1 in every repository, and report failures at the end.

    Note the -- token to separate options for command from options for tsrc.

    tsrc foreach -c 'command --opt1 arg1' Ditto, but uses a shell (/bin/sh on Linux or macOS, cmd.exe on Windows). tsrc log --from FROM [--to TO]

    Display a summary of all changes since FROM (should be a tag), to TO (defaulting to master).

    Note that if no changes are found, the repository will not be displayed at all.

    tsrc status

    Displays a summary of the status of your workspace:

    • Shows dirty repositories
    • Shows repositories not on the expected branch
    tsrc sync [--no-correct-branch] Updates all the repositories and shows a summary at the end. If any of the repositories is not on the configured branch, but it is clean and the --no-correct-branch flag is NOT set, then the branch is changed to the configured one and then the repository is updated. Otherwise that repository will not be not updated. tsrc version Displays tsrc version number, along additional data if run from a git clone. tsrc apply-manifest PATH Apply changes from the manifest file located at PATH. Useful to check changes in the manifest before publishing them to the manifest repository."},{"location":"ref/manifest-config/","title":"Manifest configuration","text":"

    The manifest configuration must be stored in a file named manifest.yml, using YAML syntax.

    It is always parsed as a mapping. Here's an example:

    repos:\n  - url: git@gitlab.local:proj1/foo\n    dest: foo\n    branch: next\n\n  - remotes:\n      - name: origin\n        url: git@gitlab.local:proj1/bar\n      - name: upstream\n        url: git@github.com:user/bar\n    dest: bar\n    branch: master\n    sha1: ad2b68539c78e749a372414165acdf2a1bb68203\n\n  - url: git@gitlab.local:proj1/app\n    dest: app\n    tag: v0.1\n    copy:\n      - file: top.cmake\n        dest: CMakeLists.txt\n      - file: .clangformat\n    symlink:\n      - source: app/some_file\n        target: ../foo/some_file\n

    In this example:

    • First, proj1/foo will be cloned into <workspace>/foo using the next branch.
    • Then, proj1/bar will be cloned into <workspace>/bar using the master branch, and reset to ad2b68539c78e749a372414165acdf2a1bb68203.
    • Finally:
      • proj1/app will be cloned into <workspace>/app using the v0.1 tag,
      • top.cmake will be copied from proj1/app/top.cmake to <workspace>/CMakeLists.txt,
      • .clang-format will be copied from proj1/app/ to <workspace>/, and
      • a symlink will be created from <workspace>/app/some_file to <workspace>/foo/some_file.
    "},{"location":"ref/manifest-config/#top_fields","title":"Top fields","text":"
    • repos (required): list of repositories to clone
    • groups (optional): list of groups
    "},{"location":"ref/manifest-config/#repos","title":"repos","text":"

    Each repository is also a mapping, containing:

    • Either:
      • url if you just need one remote named origin
      • A list of remotes with a name and url. In that case, the first remote will be used for cloning the repository.
    • dest (required): relative path of the repository in the workspace
    • branch (optional): The branch to use when cloning the repository (defaults to master)
    • tag (optional):
      • When running tsrc init: Project will be cloned at the provided tag.
      • When running tsrc sync: If the project is clean, project will be reset to the given tag, else a warning message will be printed.
    • sha1 (optional):
      • When running tsrc init: Project will be cloned, and then reset to the given sha1.
      • When running tsrc sync: If the project is clean, project will be reset to the given sha1, else a warning message will be printed.
    • ignore_submodules (optional, default=false):
      • When running tsrc init: if ignore_submodules is true, do not recursively clone submodules.
      • When running tsrc sync: if ignore_submodules is true, do not initialize or update submodules. to the given sha1, else a warning message will be printed.
    • copy (optional): A list of mappings with file and dest keys.
    • symlink (optional): A list of mappings with source and target keys.

    See the Using fixed references and the Performing file system operations guides for details about how and why you would use the tag, sha1, copy or symlink fields.

    "},{"location":"ref/manifest-config/#groups","title":"groups","text":"

    The groups section lists the groups by name. Each group should have a repos field containing a list of repositories (only repositories defined in the repos section are allowed).

    The groups can optionally include other groups, with a includes field which should be a list of existing group names.

    The group named default, if it exists, will be used to know which repositories to clone when using tsrc init and the --group command line argument is not used.

    Example:

    repos:\n  - dest: a\n    url: ..\n  - dest: b\n    url: ..\n  - dest: bar\n    url: ..\n  - dest: baz\n    url: ..\n\ngroups:\n  default:\n    repos: [a, b]\n  foo:\n    repos: [bar, baz]\n    includes: [default]\n
    $ tsrc init <manifest_url>\n# Clones a, b\n$ tsrc init <manifest_url> --group foo\n# Clones a, b, bar and baz\n

    Note that tsrc init records the names of the groups it was invoked with, so that tsrc sync re-uses them later on. This means that if you want to change the groups used, you must re-run tsrc init with the new group list.

    Note

    More information about how to use groups is available in the relevant guide.

    "},{"location":"ref/sync/","title":"Sync algorithm","text":"

    You may have noticed that tsrc sync does not just calls git pull on every repository.

    Here's the algorithm that is used:

    • Run git fetch --tags --prune
    • Check if the repository is on a branch
    • Check if the currently checked out branch matches the one configured in the manifest. If it does not, the repository is clean and the --no-correct-branch flag is NOT set, the branch is changed to the configured one.
    • Check if the repository is dirty
    • Try and run a fast-forward merge

    Note that:

    • git fetch is always called so that local refs are up-to-date
    • tsrc will simply print an error and move on to the next repository if the fast-forward merge is not possible. That's because tsrc cannot guess what the correct action is, so it prefers doing nothing. It's up to the user to run something like git merge or git rebase.
    • in case the repository is on an incorrect branch, the fast-forward merge will still be attempted, but an error message will be show in the end
    "},{"location":"ref/workspace-config/","title":"Workspace configuration","text":"

    The workspace configuration lies in <workspace>/.tsrc/config.yml. It is created by tsrc init then read by tsrc sync and other commands. It can be freely edited by hand.

    Here's an example:

    manifest_url: git@acme.corp:manifest.git\nmanifest_branch: master\nshallow_clones: false\nrepo_groups:\n- default\nclone_all_repos: false\nsingular_remote:\n
    • manifest_url: an git URL containing a manifest.yml file
    • manifest_branch: the branch to use when updating the local manifest (e.g, the first step of tsrc sync)
    • shallow_clones: whether to use only shallow clones when cloning missing repositories
    • repo_groups: the list of groups to use - every mentioned group must be present in the manifest.yml file (see above)
    • clone_all_repos: whether to ignore groups entirely and clone every repository from the manifest instead
    • singular_remote: if set to <remote-name>, behaves as if tsrc sync and tsrc init were called with --singular-remote <remote-name> option. See the Using remotes guide for details.
    "}]} \ No newline at end of file diff --git a/sitemap.xml.gz b/sitemap.xml.gz index 832e3981c4ddae6ab9620fdc034c6916a8db4974..66c9e2c74561ea8ee903198160a8c49b4bbadee8 100644 GIT binary patch delta 12 Tcmb=gXOr*d;1E1Ak*yK{7bOFv delta 12 Tcmb=gXOr*d;COy`B3mT@8&3pz