-
Notifications
You must be signed in to change notification settings - Fork 42
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs: create 'concepts' page on scopes
Showing
3 changed files
with
227 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -22,3 +22,4 @@ create the tasks. | |
loading | ||
transforms | ||
optimization | ||
Scopes <scopes> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,221 @@ | ||
How Scopes Work | ||
=============== | ||
|
||
Access to resources in Taskcluster are controlled by :term:`scopes <scope>`. | ||
While they are a simple and effective way of providing access control, they can | ||
be difficult to understand. | ||
|
||
This article aims to de-mystify scopes by explaining how they work. | ||
|
||
Scope Components | ||
---------------- | ||
|
||
There are four pieces that work together to provide security guarantees from | ||
scopes: | ||
|
||
1. Task definition | ||
2. Deployment configuration | ||
3. Auth and Queue services | ||
4. Scope consumer | ||
|
||
Let's look at each piece in more detail and go over how they interact with one | ||
another. | ||
|
||
Task Definition | ||
~~~~~~~~~~~~~~~ | ||
|
||
A task declares which scopes it requires `in its definition`_. For example: | ||
|
||
.. code-block:: yaml | ||
scopes: | ||
- my:awesome:scope | ||
- another-scope | ||
# .. rest of task definition | ||
It is a list of strings, where each string denotes a scope that the task | ||
will require to execute its payload. | ||
|
||
.. note:: | ||
|
||
Scopes are typically delimited by ``:`` to break it up into logical | ||
segments, but this is only a convention. | ||
|
||
.. _in its definition: https://docs.taskcluster.net/docs/reference/platform/queue/task-schema#scopes | ||
|
||
|
||
Deployment Configuration | ||
~~~~~~~~~~~~~~~~~~~~~~~~ | ||
|
||
Each Taskcluster deployment has associated configuration that defines the | ||
worker pools, secrets, hooks and yes, scopes. | ||
|
||
In its simplest form, this configuration can be input in the Taskcluster web | ||
interface. But for more complicated instances, configuration is managed via | ||
repository. | ||
|
||
.. note:: | ||
|
||
Here are some configuration repositories for known Taskcluster instances: | ||
|
||
* `Firefox-CI configuration`_ (managed by the Release Engineering team in `#firefox-ci`_) | ||
* `Community configuration`_ (managed by the Taskcluster team in `#taskcluster`_) | ||
|
||
Each deployment will create a number of `roles`_ and determine which scopes those | ||
roles have access to. `In this example`_ from the Firefox-CI instance, the scope | ||
``hooks:trigger-hook:project-releng/cron-task-mozilla-mobile-firefox-android/nightly`` | ||
is being granted to a role that is assumed by tasks that run on the ``main`` branch | ||
of the `Firefox Android`_ repo. | ||
|
||
The deployment configuration is full of rules like that. It acts as the access control | ||
list for a specific Taskcluster deployment. | ||
|
||
.. _Firefox-CI configuration: https://hg.mozilla.org/ci/ci-configuration/ | ||
.. _Community configuration: https://github.com/taskcluster/community-tc-config | ||
.. _#firefox-ci: https://matrix.to/#/#firefox-ci:mozilla.org | ||
.. _#taskcluster: https://matrix.to/#/#taskcluster:mozilla.org | ||
.. _roles: https://docs.taskcluster.net/docs/reference/platform/auth/roles | ||
.. _In this example: https://hg.mozilla.org/ci/ci-configuration/file/c7423cfdb54c554101e4c42e1b79a3a6aef520be/grants.yml#l1478 | ||
.. _Firefox Android: https://github.com/mozilla-mobile/firefox-android | ||
|
||
Queue and Auth Services | ||
~~~~~~~~~~~~~~~~~~~~~~~ | ||
|
||
Given the requested scopes in a task definition and the access control list in | ||
the deployment configuration, Taskcluster can now make a decision about whether | ||
a task is authorized to use the scopes it requests or not. This is where the `queue`_ | ||
and `auth`_ services come in. | ||
|
||
The queue service: | ||
|
||
1. Inspects the requested scopes in the task's definition. | ||
2. `Queries the auth service`_ to get the current set of `expanded scopes`_. | ||
3. Determines whether the current scopes `satisfy`_ the task's requested | ||
scopes. If the scopes are satisfied the task is created, otherwise an error | ||
occurs. | ||
|
||
Where do *Current Scopes* Come From? | ||
.................................... | ||
|
||
When the `createTask`_ API is used, the caller must have a superset of the | ||
scopes required by the task. But where do the caller's scopes come from? In | ||
Taskgraph, the :term:`Decision Task` is responsible for creating tasks, so it | ||
must contain a superset of the scopes all the tasks it creates need. But in | ||
that case, where do the Decision task's scopes come from? | ||
|
||
Typically the Decision task gets its scopes from Taskcluster's `Github | ||
service`_. Specifically the Github service `assumes a role`_ based on the | ||
webhook event generated by Github and the deployment configuration defines which | ||
scopes are attached to that :term:`role <Role>`. | ||
|
||
.. note:: | ||
|
||
The Firefox-CI instance has additional tasks not triggered by Github. Namely | ||
tasks running in response to events from ``hg.mozilla.org``, ``cron`` and | ||
``action`` tasks. The role assumed by these tasks is defined by the | ||
`build-decision package`_. | ||
|
||
|
||
.. _queue: https://docs.taskcluster.net/docs/reference/platform/queue | ||
.. _auth: https://docs.taskcluster.net/docs/reference/platform/auth | ||
.. _Queries the auth service: https://docs.taskcluster.net/docs/reference/platform/auth/api#currentScopes | ||
.. _expanded scopes: https://docs.taskcluster.net/docs/reference/platform/auth/roles#roles-and-role-expansion | ||
.. _satisfy: https://docs.taskcluster.net/docs/reference/platform/auth/scopes#satisfaction | ||
.. _createTask: https://docs.taskcluster.net/docs/reference/platform/queue/api#createTask | ||
.. _Github service: https://docs.taskcluster.net/docs/reference/integrations/github | ||
.. _assumes a role: https://github.com/taskcluster/taskcluster/blob/b1386bbda818e1de01ccfcc0ce07b951cd67a87e/services/github/src/tc-yaml.js#L43 | ||
.. _build-decision package: https://hg.mozilla.org/ci/ci-configuration/file/tip/build-decision | ||
|
||
Scope Consumer | ||
~~~~~~~~~~~~~~ | ||
|
||
So far we have a set of requested scopes defined in the task definition, an | ||
access control list defined in the deployment configuration, and the queue / | ||
auth services have validated the task is allowed to contain the scopes. | ||
|
||
So far so good! But you may have noticed there is still something missing! How do | ||
the scopes actually influence task execution? Take the following task definition: | ||
|
||
.. code-block:: yaml | ||
scopes: | ||
- secrets:get:my-secret | ||
How does adding this scope actually grant access to ``my-secret``? More | ||
importantly, how does omitting it deny access to ``my-secret``? | ||
|
||
This is where things get a bit ad-hoc. For every scope, there is some code | ||
somewhere that operates on a task definition. For the purposes of this article, | ||
let's call this the *scope consumer*. The scope consumer needs to validate that | ||
the task contains the appropriate scope necessary to perform the sensitive | ||
action. | ||
|
||
For the example above, the *scope consumer* is the `secrets service`_. The secrets | ||
service has code that will return a ``403`` response if you attempt to download a secret | ||
without the proper scope. The secrets service doesn't need to worry about whether the | ||
task is authorized to use the scope, because the ``queue`` and ``auth`` services have | ||
already done so. | ||
|
||
It's worth noting that anything can be a scope consumer. Internal Taskcluster | ||
services, external services, workers, packages published to package managers and even | ||
simple scripts. | ||
|
||
.. _secrets service: https://docs.taskcluster.net/docs/reference/core/secrets | ||
|
||
Implicit Scopes | ||
--------------- | ||
|
||
So far, we've only discussed *explicit scopes*. That is scopes that are defined | ||
directly in the task definition. But there are also *implicit scopes*. | ||
|
||
If you've worked with Taskcluster long enough, you've likely encountered an | ||
error that looked something like this: | ||
|
||
.. code-block:: text | ||
ERROR - Client ID task-client/Vf_Z9695QM6CUFQ-3DvBlg/0/on/us-west1/1068496344932690688/until/1697225520.15 does not have sufficient scopes and is missing the following scopes: | ||
{ | ||
"AnyOf": [ | ||
"queue:create-task:highest:gecko-t/t-linux-large-gcp", | ||
"queue:create-task:very-high:gecko-t/t-linux-large-gcp", | ||
"queue:create-task:high:gecko-t/t-linux-large-gcp", | ||
"queue:create-task:medium:gecko-t/t-linux-large-gcp", | ||
"queue:create-task:low:gecko-t/t-linux-large-gcp" | ||
] | ||
} | ||
If you happened to look at the generated definition for the failing task, you | ||
might notice that the task doesn't declare any ``queue:create-task:`` scopes! | ||
So what gives, why are you getting errors about it? | ||
|
||
The ``queue:create-task:`` scopes are an example of *implicit scopes*. That is, | ||
the queue service derives the scopes from other parts of the task definition. | ||
In this case, from the task's `taskQueueId`_ key. Since the ``taskQueueId`` is | ||
already known and we know that using it requires a scope, forcing task authors | ||
to define both ``taskQueueId`` and a scope with the same name would be adding | ||
unnecessary busy work. | ||
|
||
.. _taskQueueId: https://docs.taskcluster.net/docs/reference/platform/queue/task-schema#taskQueueId | ||
|
||
Implicit Scopes with External Scope Consumers | ||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
|
||
Implicit scopes are typically always ones where the *scope consumer* is an | ||
internal Taskcluster service. The reason is that if scope satisfaction fails, | ||
the task can be prevented from running. | ||
|
||
It's theoretically possible for external scope consumers to use implicit scopes | ||
as well. They could: | ||
|
||
1. Derive scopes from other parts of a task's definition. | ||
2. Query the ``auth`` service in a similar manner as the ``queue`` service does. | ||
3. Abort the operation if scopes are not satisfied. | ||
|
||
The major downside to this approach is that the satisfaction check happens | ||
*during* task execution instead of before the task is created. This means, not | ||
only does this task waste resources, but also its dependencies and potentially | ||
every task in the entire task group is wasted (depending how important the task | ||
in question is). | ||
|
||
It's ok for internal Taskcluster services to do this because they run *before* | ||
the task is created, not after. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters