Imaging you are building a simple Order Management System for an E-Commerce Store. The system should be able to receive and provision new orders from a store website. The provisioning process should be able to process all orders, handle payments, as well as process shipments.
For high availability and high performance, you architect the system using event-driven architecture (EDA), and build microservice apps to handle store frontend, order management, payment processing, and shipment management. You deploy the whole system in a cloud environment. To handle high workloads, you leverage a messaging system to buffer the loads, and scale up multiple instances of microservices. The architecture could look similar to:
While each microservice is acting on its own event channels, EventMesh plays a crucial role of doing Event Orchestration.
We use CNCF Serverless Workflow to describe this Event Workflow Orchestration.
CNCF Serverless Workflow defines a vendor-neutral, open-source, and fully community-driven ecosystem for defining and running DSL-based workflows that target the Serverless technology domain.
Serverless Workflow defines a Domain Specific Language (DSL) to describe stateful and stateless workflow-based orchestrations of serverless functions and microservices.
More about this can be found in its official github site
We leverage Serverless Workflow DSL to describe the EventMesh workflow. Based on its spec, the workflow is consists of a series of workflow states used to describe the control-flow logic. At this time we only support event related workflow states. See the supported states in Workflow DSL Design.
A workflow state can include applicable actions, or services/functions that should be invoked during workflow execution.
These actions can reference reusable function definitions which define how these functions/services should be invoked.
They can also reference events that trigger event-based service invocations, and events to wait for that denote completion of
such event-based service invocation completion.
In EDA solution, we usually defined our event-driven microservice using AsyncAPI.
Serverless workflow function definitions support defining invocation semantics using AsyncAPI.
See Using Funtions for AsyncAPI Service
for more information.
AsyncAPI is an open source initiative that seeks to improve the current state of Event-Driven Architectures (EDA). Our long-term goal is to make working with EDAs as easy as it is to work with REST APIs. That goes from documentation to code generation, discovery to event management. Most of the processes you apply to your REST APIs nowadays would be applicable to your event-driven/asynchronous APIs too.
See AsyncAPI detail in the official site
In this example, we build the event-driven workflow of the Order management system above.
First, we need to define AsyncAPI definitions for our microservice apps.
- Online Store App
asyncapi: 2.2.0
info:
title: Online Store application
version: '0.1.0'
channels:
store/order:
subscribe:
operationId: newStoreOrder
message:
$ref : '#/components/NewOrder'
- Order Service
asyncapi: 2.2.0
info:
title: Order Service
version: '0.1.0'
channels:
order/inbound:
publish:
operationId: sendOrder
message:
$ref : '#/components/Order'
order/outbound:
subscribe:
operationId: processedOrder
message:
$ref : '#/components/Order'- Payment Service
asyncapi: 2.2.0
info:
title: Payment Service
version: '0.1.0'
channels:
payment/inbound:
publish:
operationId: sendPayment
message:
$ref : '#/components/OrderPayment'
payment/outbound:
subscribe:
operationId: paymentReceipt
message:
$ref : '#/components/OrderPayment'- Shipment Service
asyncapi: 2.2.0
info:
title: Shipment Service
version: '0.1.0'
channels:
shipment/inbound:
publish:
operationId: sendShipment
message:
$ref : '#/components/OrderShipment'Once that is defined, we define the order workflow that describes our Order Management business logic.
id: storeorderworkflow
version: '1.0'
specVersion: '0.8'
name: Store Order Management Workflow
states:
- name: Receive New Order Event
type: event
onEvents:
- eventRefs:
- NewOrderEvent
actions:
- eventRef:
triggerEventRef: OrderServiceSendEvent
resultEventRef: OrderServiceResultEvent
- eventRef:
triggerEventRef: PaymentServiceSendEvent
resultEventRef: PaymentServiceResultEvent
transition: Check Payment Status
- name: Check Payment Status
type: switch
dataConditions:
- name: Payment Successfull
condition: "${ .payment.status == 'success' }"
transition: Send Order Shipment
- name: Payment Denied
condition: "${ .payment.status == 'denied' }"
end: true
defaultCondition:
end: true
- name: Send Order Shipment
type: operation
actions:
- eventRef:
triggerEventRef: ShipmentServiceSendEvent
end: true
events:
- name: NewOrderEvent
source: file://onlineStoreApp.yaml#newStoreOrder
type: asyncapi
kind: consumed
- name: OrderServiceSendEvent
source: file://orderService.yaml#sendOrder
type: asyncapi
kind: produced
- name: OrderServiceResultEvent
source: file://orderService.yaml#processedOrder
type: asyncapi
kind: consumed
- name: PaymentServiceSendEvent
source: file://paymentService.yaml#sendPayment
type: asyncapi
kind: produced
- name: PaymentServiceResultEvent
source: file://paymentService.yaml#paymentReceipt
type: asyncapi
kind: consumed
- name: ShipmentServiceSendEvent
source: file://shipmentService.yaml#sendShipment
type: asyncapi
kind: producedThe corresponding workflow diagram is the following:
In the following architecture diagram, the EventMesh Catalog, EventMesh Workflow Engine and EventMesh Runtime are running in three different processors.
The steps running the workflow is the followings:
-
Deploy the Publisher and Subscriber Apps in the environment. Describe the App APIs using AsyncAPI, generate the asyncAPI yaml. Register the Publisher and Subscriber Apps in EventMesh Catalog using AsyncAPI.
-
Register the Serverless Workflow DSL in EventMesh Workflow Engine.
-
EventMesh Workflow Engine query the EventMesh Catalog for Publisher and Subscribers required in Workflow DSL
function -
Event-driven Apps are publish events to EventMesh Runtime to trigger the Workflow. EventMesh Workflow Engine also publish and subscribe events for orchestrating the events.
EventMesh Catalog store the Publisher, Subscriber and Channel metadata. consists of the following modules:
-
AsyncAPI Parser
Using the SDK provided by AsyncAPI community (see tool list), parse and validated the AsyncAPI yaml inputs, and generate the AsyncAPI definition.
-
Publisher, Channel, Subscriber Modules
From the AsyncAPI definition store the Publisher, Subscriber and Channel information.
EventMesh Workflow Engine consists of the following modules:
-
Workflow Parser
Using the SDK provided by Serverless Workflow community (see supported SDKs), parse and validated the workflow DSL inputs, and generate workflow definition.
-
Workflow Module
It manages a workflow instance life cycle, from create, start, stop to destroy.
-
State Module
It manages workflow state life cycle. We support the event-related states, and the supported state list below is Work-in-Progress.
Workflow State Description Operation Execute the AsyncAPI functions defined in the Actions Event Check if the defined Event matched, if so execute the defined AsyncAPI functions Switch Check the event is matched with the event-conditions, and execute teh defined AsyncAPI functions Parallel Execute the defined AsyncAPI functions in parallel ForEach Iterate the inputCollection and execute the defined AsyncAPI functions -
Action Module
It managed the functions inside the action.
-
Function Module
It manages the AsyncAPI functions by creating the publisher and/or subscriber in EventMesh Runtime, and manage the publisher/subscriber life cycle.
AsyncAPI Operation EventMesh Runtime Publish Publisher Subscribe Subscriber -
Event Module
It manages the CloudEvents data model, including event filter, correlation and transformation using the rules defined in the workflow DSL.
-
Retry Module
It manages the retry logic of the event publishing into EventMesh Runtime.