Skip to content

Commit

Permalink
add notes about cassandra schema
Browse files Browse the repository at this point in the history
  • Loading branch information
taylanisikdemir committed Mar 20, 2024
1 parent 3bd1b3e commit 2ac1756
Showing 1 changed file with 15 additions and 6 deletions.
21 changes: 15 additions & 6 deletions docs/flow.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
# Flow of tasks
# Flow

## Workflow processing flow

Below diagram demonstrates how a workflow is processed by Cadence workers and backend components.
Note that this is a partial diagram that only highlights selected components and their interactions. There is more happening behind the scenes.
Expand All @@ -16,18 +18,25 @@ flowchart TD
WaitUntilTaskFound --> |6. Return task to worker| Worker
Worker -->|7. Generate decisions\n& respond| Frontend
Frontend -->|8. Respond decisions| History[[\nCadence History\n\n]]
History -->|9. Mutable state update| ExecutionsTable[(Executions Table)]
History -->|9. Mutable state update| ExecutionsStore[(Executions Store)]
History -->|10.a. Notify| HistoryQueue(History Queues - per shard)
HistoryQueue -->|Periodic task fetch| ExecutionsTable
HistoryQueue -->|Periodic task fetch| ExecutionsStore
HistoryQueue -->|11.b. Execute transfer task| TransferTaskExecutor(Transfer Task Executor)
HistoryQueue -->|11.a. Execute timer task| TimerTaskExecutor(Timer Task Executor)
HistoryQueue -->|Periodic per-shard offset save| ShardsTable[(Shards Table)]
HistoryQueue -->|Periodic per-shard offset save| ShardsStore[(Shards Store)]
TransferTaskExecutor -->|12.b. Add task| Matching
TimerTaskExecutor -->|12.a. Add task| Matching
Matching -->|13. Add task| TaskListManager
TaskListManager -->|14. Add task| SyncMatch{Check sync match}
SyncMatch -->|15.b. Found poller - Do sync match| SyncMatched
SyncMatch -->|15.a. No pollers - Save task| TasksTable[(Tasks Table)]
TaskListManager -->|Periodic task fetch| TasksTable
SyncMatch -->|15.a. No pollers - Save task| TasksStore[(Tasks Store)]
TaskListManager -->|Periodic task fetch| TasksStore
```

### Cassandra Notes:

In order to leverage [Cassandra Light Weight Transactions](https://www.yugabyte.com/blog/apache-cassandra-lightweight-transactions-secondary-indexes-tunable-consistency/), Cadence stores multiple types of records in the same `executions` table. ([ref](https://github.com/uber/cadence/blob/51758676ce9d9609e736c64f94dc387ed2c75b7c/schema/cassandra/cadence/schema.cql#L338))
- **Shards Store**: Shard records are stored in `executions` table with type=0.
- **Executions Store**: Execution records are stored in `executions` table with type=1
- **Tasks Store**: Task records are stored in `executions` table with type={2,3,4,5}

0 comments on commit 2ac1756

Please sign in to comment.