Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New runs fundamentals page #1401

Merged
merged 9 commits into from
Oct 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added docs/images/run-lifecycle.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/run-with-batchTriggerAndWait().png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/run-with-delay.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/run-with-retries.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/run-with-triggerAndWait().png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/run-with-ttl.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions docs/mint.json
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,7 @@
"pages": ["tasks/overview", "tasks/scheduled"]
},
"triggering",
"runs-and-attempts",
"apikeys",
{
"group": "Configuration",
Expand Down
187 changes: 187 additions & 0 deletions docs/runs-and-attempts.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,187 @@
---
title: "Runs & attempts"
description: "Understanding the lifecycle of task execution in Trigger.dev"
---

In Trigger.dev, the concepts of runs and attempts are fundamental to understanding how tasks are executed and managed. This article explains these concepts in detail and provides insights into the various states a run can go through during its lifecycle.

## What are runs?

A run is created when you trigger a task (e.g. calling `yourTask.trigger({ foo: "bar" })`). It represents a single instance of a task being executed and contains the following key information:

- A unique run ID
- The current status of the run
- The payload (input data) for the task
- Lots of other metadata

## The run lifecycle

A run can go through **various** states during its lifecycle. The following diagram illustrates a typical state transition where a single run is triggered and completes successfully:

![Run Lifecycle](/images/run-lifecycle.png)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

Run Lifecycle image is missing

Please add the run-lifecycle.png image to the repository to ensure the documentation is complete.

🔗 Analysis chain

Verify the existence of the Run Lifecycle image

Please ensure that the referenced image file exists and is correctly placed in the repository:

![Run Lifecycle](/images/run-lifecycle.png)

If the image is missing, please add it to maintain the documentation's completeness.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Verify the existence of the Run Lifecycle image
if [ -f "public/images/run-lifecycle.png" ]; then
    echo "Run Lifecycle image exists."
else
    echo "Run Lifecycle image is missing."
fi

Length of output: 126


Script:

#!/bin/bash
# Verify the existence of the Run Lifecycle image
if [ -f "images/run-lifecycle.png" ]; then
    echo "Run Lifecycle image exists."
else
    echo "Run Lifecycle image is missing."
fi

Length of output: 110


Runs can also find themselves in lots of other states depending on what's happening at any given time. The following sections describe all the possible states in more detail.

### Initial States

<Icon icon="rectangle-history" iconType="solid" color="#FBBF24" size={17} /> **Waiting for deploy**: If a task is triggered before it has been deployed, the run enters this state and waits for the task to be deployed.

<Icon icon="clock" iconType="solid" color="#878C99" size={17} /> **Delayed**: When a run is triggered with a delay, it enters this state until the specified delay period has passed.

<Icon icon="rectangle-history" iconType="solid" color="#878C99" size={17} /> **Queued**: The run is ready to be executed and is waiting in the queue.

### Execution States

<Icon icon="spinner-third" iconType="duotone" color="#3B82F6" size={17} /> **Executing**: The task is currently running.

<Icon icon="arrows-rotate" iconType="solid" color="#3B82F6" size={17} /> **Reattempting**: The task has failed and is being retried.

<Icon icon="snowflake" iconType="solid" color="#68BAF2" size={17} /> **Frozen**: Task has been frozen and is waiting to be resumed.

### Final States

<Icon icon="circle-check" iconType="solid" color="#28BF5C" size={17} /> **Completed**: The task has successfully finished execution.

<Icon icon="ban" iconType="solid" color="#878C99" size={17} /> **Canceled**: The run was manually canceled by the user.

<Icon icon="circle-xmark" iconType="solid" color="#E11D48" size={17} /> **Failed**: The task has failed to complete successfully.

<Icon icon="alarm-exclamation" iconType="solid" color="#E11D48" size={17} /> **Timed out**: Task has failed because it exceeded its `maxDuration`.

<Icon icon="fire" iconType="solid" color="#E11D48" size={17} /> **Crashed**: The worker process crashed during execution (likely due to an Out of Memory error).

<Icon icon="bolt-slash" iconType="solid" color="#E11D48" size={17} /> **Interrupted**: In development mode, when the CLI is disconnected.

<Icon icon="bug" iconType="solid" color="#E11D48" size={17} /> **System failure**: An unrecoverable system error has occurred.

<Icon icon="trash-can" iconType="solid" color="#878C99" size={17} /> **Expired**: The run's Time-to-Live (TTL) has passed before it could start executing.

## Attempts

An attempt represents a single execution of a task within a run. A run can have one or more attempts, depending on the task's retry settings and whether it fails. Each attempt has:

- A unique attempt ID
- A status
- An output (if successful) or an error (if failed)

When a task fails, it will be retried according to its retry settings, creating new attempts until it either succeeds or reaches the retry limit.

![Run with retries](/images/run-with-retries.png)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

Run with retries image is missing.

Please add the run-with-retries.png image to the /public/images/ directory to ensure the documentation is complete and the image reference is valid.

🔗 Analysis chain

Verify the existence of the Run with retries image

Please ensure that the referenced image file exists and is correctly placed in the repository:

![Run with retries](/images/run-with-retries.png)

If the image is missing, please add it to maintain the documentation's completeness.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Verify the existence of the Run with retries image
if [ -f "public/images/run-with-retries.png" ]; then
    echo "Run with retries image exists."
else
    echo "Run with retries image is missing."
fi

Length of output: 126


## Run completion

A run is considered finished when:

1. The last attempt succeeds, or
2. The task has reached its retry limit and all attempts have failed

At this point, the run will have either an output (if successful) or an error (if failed).

## Advanced run features

### Idempotency Keys

When triggering a task, you can provide an idempotency key to ensure the task is executed only once, even if triggered multiple times. This is useful for preventing duplicate executions in distributed systems.

```javascript
yourTask.trigger({ foo: "bar" }, { idempotencyKey: "unique-key" });
```

- If a run with the same idempotency key is already in progress, the new trigger will be ignored.
- If the run has already finished, the previous output or error will be returned.

### Canceling runs

You can cancel an in-progress run using the API or the dashboard:

```ts
runs.cancel(runId);
```

When a run is canceled:

– The task execution is stopped

– The run is marked as canceled

– The task will not be retried

– Any in-progress child runs are also canceled

### Time-to-live (TTL)

You can set a TTL when triggering a run:

```ts
yourTask.trigger({ foo: "bar" }, { ttl: "10m" });
```

If the run hasn't started within the specified TTL, it will automatically expire. This is useful for time-sensitive tasks. Note that dev runs automatically have a 10-minute TTL.

![Run with TTL](/images/run-with-ttl.png)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

Run with TTL image is missing

The referenced image ![Run with TTL](/images/run-with-ttl.png) does not exist in the repository. Please add the image to maintain the documentation's completeness.

🔗 Analysis chain

Verify the existence of the Run with TTL image

Please ensure that the referenced image file exists and is correctly placed in the repository:

![Run with TTL](/images/run-with-ttl.png)

If the image is missing, please add it to maintain the documentation's completeness.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Verify the existence of the Run with TTL image
if [ -f "public/images/run-with-ttl.png" ]; then
    echo "Run with TTL image exists."
else
    echo "Run with TTL image is missing."
fi

Length of output: 322


Script:

#!/bin/bash
# Verify the existence of the Run with TTL image
if [ -f "public/images/run-with-ttl.png" ]; then
    echo "Run with TTL image exists."
else
    echo "Run with TTL image is missing."
fi

Length of output: 114


### Delayed runs

You can schedule a run to start after a specified delay:

```ts
yourTask.trigger({ foo: "bar" }, { delay: "1h" });
```

This is useful for tasks that need to be executed at a specific time in the future.

![Run with delay](/images/run-with-delay.png)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

Run with delay image is missing in the repository

  • The image file public/images/run-with-delay.png is missing.

Please add the run-with-delay.png image to the public/images/ directory to maintain the documentation's completeness.

🔗 Analysis chain

Verify the existence of the Run with delay image

Please ensure that the referenced image file exists and is correctly placed in the repository:

![Run with delay](/images/run-with-delay.png)

If the image is missing, please add it to maintain the documentation's completeness.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Verify the existence of the Run with delay image
if [ -f "public/images/run-with-delay.png" ]; then
    echo "Run with delay image exists."
else
    echo "Run with delay image is missing."
fi

Length of output: 322


Script:

#!/bin/bash
# Verify the existence of the Run with delay image
if [ -f "public/images/run-with-delay.png" ]; then
    echo "Run with delay image exists."
else
    echo "Run with delay image is missing."
fi

Length of output: 120


### Replaying runs

You can create a new run with the same payload as a previous run:

```ts
runs.replay(runId);
```

This is useful for re-running a task with the same input, especially for debugging or recovering from failures. The new run will use the latest version of the task.

You can also replay runs from the dashboard using the same or different payload. Learn how to do this [here](/replaying).

### Waiting for runs

#### triggerAndWait()

The `triggerAndWait()` function triggers a task and then lets you wait for the result before continuing. [Learn more about triggerAndWait()](/triggering#yourtask-triggerandwait).

![Run with triggerAndWait](/images/run-with-triggerAndWait().png)

#### batchTriggerAndWait()

Similar to `triggerAndWait()`, the `batchTriggerAndWait()` function lets you batch trigger a task and wait for all the results [Learn more about batchTriggerAndWait()](/triggering#yourtask-batchtriggerandwait).

![Run with batchTriggerAndWait](/images/run-with-batchTriggerAndWait().png)

### Runs API

The runs API provides methods to interact with and manage runs:

```ts
// List all runs
runs.list();

// Get a specific run by ID
runs.retrieve(runId);

// Replay a run
runs.replay(runId);

// Reschedule a run
runs.reschedule(runId, delay);

// Cancel a run
runs.cancel(runId);
```

These methods allow you to access detailed information about runs and their attempts, including payloads, outputs, parent runs, and child runs.

### Triggering runs for undeployed tasks

It's possible to trigger a run for a task that hasn't been deployed yet. The run will enter the "Waiting for deploy" state until the task is deployed. Once deployed, the run will be queued and executed normally.
This feature is particularly useful in CI/CD pipelines where you want to trigger tasks before the deployment is complete.
12 changes: 6 additions & 6 deletions docs/triggering.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,19 +6,19 @@ description: "Tasks need to be triggered in order to run."
Trigger tasks **from your backend**:

| Function | This works | What it does |
| ------------------------ | ---------- | --------------------------------------------------------------------------------------------------------------------------- |
| :----------------------- | :--------- | :-------------------------------------------------------------------------------------------------------------------------- |
| `tasks.trigger()` | Anywhere | Triggers a task and gets a handle you can use to fetch and manage the run. [Read more](#tasks-trigger) |
| `tasks.batchTrigger()` | Anywhere | Triggers a task multiple times and gets a handle you can use to fetch and manage the runs. [Read more](#tasks-batchtrigger) |
| `tasks.triggerAndPoll()` | Anywhere | Triggers a task and then polls the run until it’s complete. [Read more](#tasks-triggerandpoll) |

Trigger tasks **from inside a run**:

| Function | This works | What it does |
| -------------------------------- | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `yourTask.trigger()` | Anywhere | Triggers a task and gets a handle you can use to monitor and manage the run. It does not wait for the result. [Read more](#task-trigger) |
| `yourTask.batchTrigger()` | Anywhere | Triggers a task multiple times and gets a handle you can use to monitor and manage the runs. It does not wait for the results. [Read more](#task-batchtrigger) |
| `yourTask.triggerAndWait()` | Inside task | Triggers a task and then waits until it's complete. You get the result data to continue with. [Read more](#task-triggerandwait) |
| `yourTask.batchTriggerAndWait()` | Inside task | Triggers a task multiple times in parallel and then waits until they're all complete. You get the resulting data to continue with. [Read more](#task-batchtriggerandwait) |
| :------------------------------- | :---------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `yourTask.trigger()` | Anywhere | Triggers a task and gets a handle you can use to monitor and manage the run. It does not wait for the result. [Read more](#yourtask-trigger) |
| `yourTask.batchTrigger()` | Anywhere | Triggers a task multiple times and gets a handle you can use to monitor and manage the runs. It does not wait for the results. [Read more](#yourtask-batchtrigger) |
| `yourTask.triggerAndWait()` | Inside task | Triggers a task and then waits until it's complete. You get the result data to continue with. [Read more](#yourtask-triggerandwait) |
| `yourTask.batchTriggerAndWait()` | Inside task | Triggers a task multiple times in parallel and then waits until they're all complete. You get the resulting data to continue with. [Read more](#yourtask-batchtriggerandwait) |

Additionally, [scheduled tasks](/tasks/scheduled) get **automatically** triggered on their schedule and webhooks when receiving a webhook.

Expand Down