Skip to content

taco-xyz/tacoq

Repository files navigation

TacoQ(ueue)

What is TacoQ?

TacoQ's mission is to be:

  • Specifically a very smart task queue (not a bloated workflow engine!)
  • Developer friendly (nice documentation, lots of examples, easily extendible)
  • High-quality SDKs (async support, hot reloading, use language-specific features)
  • Fault tolerant (transactional outbox support, worker failure handling)
  • Observable (built-in logging, metrics, and tracing support)
  • Performant (extremely low task execution latency)

SDKs

TacoQ also supports multiple SDKs for communicating with the manager:

Language Worker Publisher Client Notes
Python Async support, hot reloading
Rust Soon 1
TypeScript Soon 2
Go Soon 3

To avoid having lazy implementations, we are conservative with how many languages we officially support. Workers also take up 90% of the effort, so publisher clients might get added faster than their worker counterparts.

Tech Stack

TacoQ consists of the following

  • Task Manager - A central service built in Rust that handles worker registration, task assignment, and information retrieval.
  • Application <-> Manager Communication: An HTTP API with optional gRPC support coming soon.
  • Workers <-> Manager Communication: RabbitMQ
  • Database: Postgres

Developer Notes

SQLX Type Checking

To be able to type check sqlx queries, you need to have the database running:

  1. Start up the Postgres database
docker compose up postgres
  1. Set the DATABASE_URL environment variable to the connection string for the postgres database:
DATABASE_URL="postgresql://user:password@localhost:5432/tacoq"
  1. Run the following command to update the database:
cargo sqlx migrate --source src/services/libs/db-common

NOTE: You might need to restart your rust-analyzer to stop giving type errors!

Testing

To run the tests, your database and RabbitMQ instances must be running.

Rust Tests:

cargo test

Python Client Tests:

The Python client requires UV to be installed. To install UV, run the following command:

# On macOS and Linux.
curl -LsSf https://astral.sh/uv/install.sh | sh
# On Windows.
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

To run the tests, navigate to the python client directory and run the following command:

cd clients/client_sdks/python
uv run pytest