Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update taskdef.md #308

Merged
merged 2 commits into from
Nov 14, 2024
Merged

Update taskdef.md #308

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
51cb609
Update taskdef.md
RizaFarheen Nov 13, 2024
bf6a379
Update taskdef.md
RizaFarheen Nov 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 11 additions & 11 deletions docs/documentation/configuration/taskdef.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,21 +10,21 @@ This should not be confused with [*Task Configurations*](./workflowdef/index.md#
| Field | Type | Description | Notes |
| :-------------------------- | :----------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | :------------------------------------------------------------------------------- |
| name | string | Task Name. Unique name of the Task that resonates with its function. | Must be unique |
| description | string | Description of the task | Optional |
| retryCount | number | Number of retries to attempt when a Task is marked as failure | Defaults to 3 with maximum allowed capped at 10 |
| retryLogic | string (enum) | Mechanism for the retries | See [Retry Logic](#retry-logic) |
| retryDelaySeconds | number | Time to wait before retries | Defaults to 60 seconds |
| timeoutPolicy | string (enum) | Task's timeout policy | Defaults to `TIME_OUT_WF`; See [Timeout Policy](#timeout-policy) |
| timeoutSeconds | number | Time in seconds, after which the task is marked as `TIMED_OUT` if not completed after transitioning to `IN_PROGRESS` status for the first time | No timeouts if set to 0 |
| description | string | Description of the task. | Optional |
| retryCount | number | Number of retries to attempt when a Task is marked as failure. | Defaults to 3 with maximum allowed capped at 10 |
| retryLogic | string (enum) | Mechanism for the retries. | See [Retry Logic](#retry-logic) |
| retryDelaySeconds | number | Time to wait before retries. | Defaults to 60 seconds |
| timeoutPolicy | string (enum) | Task's timeout policy. | Defaults to `TIME_OUT_WF`; See [Timeout Policy](#timeout-policy) |
| timeoutSeconds | number | Time in seconds, after which the task is marked as `TIMED_OUT` if it has not reached a terminal state after transitioning to `IN_PROGRESS` status for the first time. | No timeouts if set to 0 |
| responseTimeoutSeconds | number | If greater than 0, the task is rescheduled if not updated with a status after this time (heartbeat mechanism). Useful when the worker polls for the task but fails to complete due to errors/network failure. | Defaults to 3600 |
| pollTimeoutSeconds | number | Time in seconds, after which the task is marked as `TIMED_OUT` if not polled by a worker | No timeouts if set to 0 |
| pollTimeoutSeconds | number | Time in seconds, after which the task is marked as `TIMED_OUT` if not polled by a worker. | No timeouts if set to 0 |
| inputKeys | array of string(s) | Array of keys of task's expected input. Used for documenting task's input. | Optional. See [Using inputKeys and outputKeys](#using-inputkeys-and-outputkeys). |
| outputKeys | array of string(s) | Array of keys of task's expected output. Used for documenting task's output. | Optional. See [Using inputKeys and outputKeys](#using-inputkeys-and-outputkeys). |
| inputTemplate | object | Define default input values. | Optional. See [Using inputTemplate](#using-inputtemplate) |
| concurrentExecLimit | number | Number of tasks that can be executed at any given time | Optional |
| concurrentExecLimit | number | Number of tasks that can be executed at any given time. | Optional |
| rateLimitFrequencyInSeconds | number | Sets the rate limit frequency window. | Optional. See [Task Rate limits](#task-rate-limits) |
| rateLimitPerFrequency | number | Sets the max number of tasks that can be given to workers within window. | Optional. See [Task Rate limits](#task-rate-limits) below |
| ownerEmail | string | Email address of the team that owns the task | Required |
| ownerEmail | string | Email address of the team that owns the task. | Required |

### Retry Logic

Expand Down Expand Up @@ -57,13 +57,13 @@ You have 1000 task executions waiting in the queue, and 1000 workers polling thi
**Example**
Let's set `rateLimitFrequencyInSeconds = 5`, and `rateLimitPerFrequency = 12`. This means our frequency window is of 5 seconds duration, and for each frequency window, Conductor would only give 12 tasks to workers. So, in a given minute, Conductor would only give 12*(60/5) = 144 tasks to workers irrespective of the number of workers that are polling for the task.

Note that unlike `concurrentExecLimit`, rate limiting doesn't take into account tasks already in progress/completed. Even if all the previous tasks are executed within 1 sec, or would take a few days, the new tasks are still given to workers at configured frequency, 144 tasks per minute in above example.
Note that unlike `concurrentExecLimit`, rate limiting doesn't take into account tasks already in progress or a terminal state. Even if all the previous tasks are executed within 1 sec, or would take a few days, the new tasks are still given to workers at configured frequency, 144 tasks per minute in above example.


### Using `inputKeys` and `outputKeys`

* `inputKeys` and `outputKeys` can be considered as parameters and return values for the Task.
* Consider the task Definition as being represented by an interface: ```(value1, value2 .. valueN) someTaskDefinition(key1, key2 .. keyN);```
* Consider the task Definition as being represented by an interface: ```(value1, value2 .. valueN) someTaskDefinition(key1, key2 .. keyN);```.
* However, these parameters are not strictly enforced at the moment. Both `inputKeys` and `outputKeys` act as a documentation for task re-use. The tasks in workflow need not define all of the keys in the task definition.
* In the future, this can be extended to be a strict template that all task implementations must adhere to, just like interfaces in programming languages.

Expand Down
Loading