From f68c26fc0ec1780dd18e635877d09fafe536de8d Mon Sep 17 00:00:00 2001 From: Andrew Halberstadt Date: Fri, 13 Oct 2023 15:59:55 -0400 Subject: [PATCH] docs: create 'concepts' page on scopes --- docs/concepts/index.rst | 1 + docs/concepts/scopes.rst | 221 +++++++++++++++++++++++++++++++++++++++ docs/glossary.rst | 5 + 3 files changed, 227 insertions(+) create mode 100644 docs/concepts/scopes.rst diff --git a/docs/concepts/index.rst b/docs/concepts/index.rst index ae8dee604..8354b4667 100644 --- a/docs/concepts/index.rst +++ b/docs/concepts/index.rst @@ -22,3 +22,4 @@ create the tasks. loading transforms optimization + Scopes diff --git a/docs/concepts/scopes.rst b/docs/concepts/scopes.rst new file mode 100644 index 000000000..8da3f5d73 --- /dev/null +++ b/docs/concepts/scopes.rst @@ -0,0 +1,221 @@ +How Scopes Work +=============== + +Access to resources in Taskcluster are controlled by :term:`scopes `. +While they are a simple and effective way of providing access control, they can +be difficult to understand. + +This article aims to de-mystify scopes by explaining how they work. + +Scope Components +---------------- + +There are four pieces that work together to provide security guarantees from +scopes: + +1. Task definition +2. Deployment configuration +3. Auth and Queue services +4. Scope consumer + +Let's look at each piece in more detail and go over how they interact with one +another. + +Task Definition +~~~~~~~~~~~~~~~ + +A task declares which scopes it requires `in its definition`_. For example: + +.. code-block:: yaml + + scopes: + - my:awesome:scope + - another-scope + # .. rest of task definition + +It is a list of strings, where each string denotes a scope that the task +will require to execute its payload. + +.. note:: + + Scopes are typically delimited by ``:`` to break it up into logical + segments, but this is only a convention. + +.. _in its definition: https://docs.taskcluster.net/docs/reference/platform/queue/task-schema#scopes + + +Deployment Configuration +~~~~~~~~~~~~~~~~~~~~~~~~ + +Each Taskcluster deployment has associated configuration that defines the +worker pools, secrets, hooks and yes, scopes. + +In its simplest form, this configuration can be input in the Taskcluster web +interface. But for more complicated instances, configuration is managed via +repository. + +.. note:: + + Here are some configuration repositories for known Taskcluster instances: + + * `Firefox-CI configuration`_ (managed by the Release Engineering team in `#firefox-ci`_) + * `Community configuration`_ (managed by the Taskcluster team in `#taskcluster`_) + +Each deployment will create a number of `roles`_ and determine which scopes those +roles have access to. `In this example`_ from the Firefox-CI instance, the scope +``hooks:trigger-hook:project-releng/cron-task-mozilla-mobile-firefox-android/nightly`` +is being granted to a role that is assumed by tasks that run on the ``main`` branch +of the `Firefox Android`_ repo. + +The deployment configuration is full of rules like that. It acts as the access control +list for a specific Taskcluster deployment. + +.. _Firefox-CI configuration: https://hg.mozilla.org/ci/ci-configuration/ +.. _Community configuration: https://github.com/taskcluster/community-tc-config +.. _#firefox-ci: https://matrix.to/#/#firefox-ci:mozilla.org +.. _#taskcluster: https://matrix.to/#/#taskcluster:mozilla.org +.. _roles: https://docs.taskcluster.net/docs/reference/platform/auth/roles +.. _In this example: https://hg.mozilla.org/ci/ci-configuration/file/c7423cfdb54c554101e4c42e1b79a3a6aef520be/grants.yml#l1478 +.. _Firefox Android: https://github.com/mozilla-mobile/firefox-android + +Queue and Auth Services +~~~~~~~~~~~~~~~~~~~~~~~ + +Given the requested scopes in a task definition and the access control list in +the deployment configuration, Taskcluster can now make a decision about whether +a task is authorized to use the scopes it requests or not. This is where the `queue`_ +and `auth`_ services come in. + +The queue service: + +1. Inspects the requested scopes in the task's definition. +2. `Queries the auth service`_ to get the current set of `expanded scopes`_. +3. Determines whether the current scopes `satisfy`_ the task's requested + scopes. If the scopes are satisfied the task is created, otherwise an error + occurs. + +Where do *Current Scopes* Come From? +.................................... + +When the `createTask`_ API is used, the caller must have a superset of the +scopes required by the task. But where do the caller's scopes come from? In +Taskgraph, the :term:`Decision Task` is responsible for creating tasks, so it +must contain a superset of the scopes all the tasks it creates need. But in +that case, where do the Decision task's scopes come from? + +Typically the Decision task gets its scopes from Taskcluster's `Github +service`_. Specifically the Github service `assumes a role`_ based on the +webhook event generated by Github and the deployment configuration defines which +scopes are attached to that :term:`role `. + +.. note:: + + The Firefox-CI instance has additional tasks not triggered by Github. Namely + tasks running in response to events from ``hg.mozilla.org``, ``cron`` and + ``action`` tasks. The role assumed by these tasks is defined by the + `build-decision package`_. + + +.. _queue: https://docs.taskcluster.net/docs/reference/platform/queue +.. _auth: https://docs.taskcluster.net/docs/reference/platform/auth +.. _Queries the auth service: https://docs.taskcluster.net/docs/reference/platform/auth/api#currentScopes +.. _expanded scopes: https://docs.taskcluster.net/docs/reference/platform/auth/roles#roles-and-role-expansion +.. _satisfy: https://docs.taskcluster.net/docs/reference/platform/auth/scopes#satisfaction +.. _createTask: https://docs.taskcluster.net/docs/reference/platform/queue/api#createTask +.. _Github service: https://docs.taskcluster.net/docs/reference/integrations/github +.. _assumes a role: https://github.com/taskcluster/taskcluster/blob/b1386bbda818e1de01ccfcc0ce07b951cd67a87e/services/github/src/tc-yaml.js#L43 +.. _build-decision package: https://hg.mozilla.org/ci/ci-configuration/file/tip/build-decision + +Scope Consumer +~~~~~~~~~~~~~~ + +So far we have a set of requested scopes defined in the task definition, an +access control list defined in the deployment configuration, and the queue / +auth services have validated the task is allowed to contain the scopes. + +So far so good! But you may have noticed there is still something missing! How do +the scopes actually influence task execution? Take the following task definition: + +.. code-block:: yaml + + scopes: + - secrets:get:my-secret + +How does adding this scope actually grant access to ``my-secret``? More +importantly, how does omitting it deny access to ``my-secret``? + +This is where things get a bit ad-hoc. For every scope, there is some code +somewhere that operates on a task definition. For the purposes of this article, +let's call this the *scope consumer*. The scope consumer needs to validate that +the task contains the appropriate scope necessary to perform the sensitive +action. + +For the example above, the *scope consumer* is the `secrets service`_. The secrets +service has code that will return a ``403`` response if you attempt to download a secret +without the proper scope. The secrets service doesn't need to worry about whether the +task is authorized to use the scope, because the ``queue`` and ``auth`` services have +already done so. + +It's worth noting that anything can be a scope consumer. Internal Taskcluster +services, external services, workers, packages published to package managers and even +simple scripts. + +.. _secrets service: https://docs.taskcluster.net/docs/reference/core/secrets + +Implicit Scopes +--------------- + +So far, we've only discussed *explicit scopes*. That is scopes that are defined +directly in the task definition. But there are also *implicit scopes*. + +If you've worked with Taskcluster long enough, you've likely encountered an +error that looked something like this: + +.. code-block:: text + + ERROR - Client ID task-client/Vf_Z9695QM6CUFQ-3DvBlg/0/on/us-west1/1068496344932690688/until/1697225520.15 does not have sufficient scopes and is missing the following scopes: + { + "AnyOf": [ + "queue:create-task:highest:gecko-t/t-linux-large-gcp", + "queue:create-task:very-high:gecko-t/t-linux-large-gcp", + "queue:create-task:high:gecko-t/t-linux-large-gcp", + "queue:create-task:medium:gecko-t/t-linux-large-gcp", + "queue:create-task:low:gecko-t/t-linux-large-gcp" + ] + } + +If you happened to look at the generated definition for the failing task, you +might notice that the task doesn't declare any ``queue:create-task:`` scopes! +So what gives, why are you getting errors about it? + +The ``queue:create-task:`` scopes are an example of *implicit scopes*. That is, +the queue service derives the scopes from other parts of the task definition. +In this case, from the task's `taskQueueId`_ key. Since the ``taskQueueId`` is +already known and we know that using it requires a scope, forcing task authors +to define both ``taskQueueId`` and a scope with the same name would be adding +unnecessary busy work. + +.. _taskQueueId: https://docs.taskcluster.net/docs/reference/platform/queue/task-schema#taskQueueId + +Implicit Scopes with External Scope Consumers +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Implicit scopes are typically always ones where the *scope consumer* is an +internal Taskcluster service. The reason is that if scope satisfaction fails, +the task can be prevented from running. + +It's theoretically possible for external scope consumers to use implicit scopes +as well. They could: + +1. Derive scopes from other parts of a task's definition. +2. Query the ``auth`` service in a similar manner as the ``queue`` service does. +3. Abort the operation if scopes are not satisfied. + +The major downside to this approach is that the satisfaction check happens +*during* task execution instead of before the task is created. This means, not +only does this task waste resources, but also its dependencies and potentially +every task in the entire task group is wasted (depending how important the task +in question is). + +It's ok for internal Taskcluster services to do this because they run *before* +the task is created, not after. diff --git a/docs/glossary.rst b/docs/glossary.rst index f95325b4c..2f139d220 100644 --- a/docs/glossary.rst +++ b/docs/glossary.rst @@ -37,6 +37,11 @@ Glossary used to specify whether the generation is for a pull request or a push. See :doc:`/reference/parameters` for a list of supported parameters. + Role + An alias for a set of scopes. Roles consist of a name, description and a + list of scopes. Granting something the special scope `assume:` grants + access to all scopes defined within that role. + Scope Taskcluster permission required to perform a particular action. Each task has a set of these permissions determining what it can do.