What are serverless workflows?

There are no shortage of workflow specifications: open source and commercial, services, platforms and software. Approaches have evolved rapidly with cloud-native development, microservice orchestration and serverless functions. The CNCF (Cloud Native Computing Foundation) sought to solve this problem with a vendor-neutral, open-source specification: serverless workflow.

The Serverless Workflow is a Domain Specific Language (DSL) for defining workflow logic. It decouples the definition of your workflow from the implementation (the runtime), allowing you to write your business logic once and run it on any cloud or platform. Serverless workflows are designed to be transportable and interoperable.

Why do we need them?

Business logic within complex systems and interconnectivity between microservices are often built using either GRPC or REST APIs for internal calls. For simple use cases this works, but once you start orchestrating flows that depend on multiple services to do the actual work (notify, create, modify etc.) if a part of that workflow fails recovering from failed state becomes increasingly challenging. How do we notify up-stream and down-stream components of this failure to rollback changes? Can we guarantee different teams responsible for different services are co-ordinated to support these capabilities?

Without a dedicated orchestration layer, spaghetti code is almost guaranteed: workflow logic will end up directly embedded into functions or services. This leads to tight coupling, difficult debugging, and a lack of visibility into the overall business process and general all round misery, unhappiness and compounding complexity.

The Serverless Workflows DSL provides a standard way to support:

• Orchestration
  Complex chains of events and service calls.
• Portability
  Avoid vendor lock-in by using a standard specification rather than proprietary tools like AWS Step Functions or Azure Logic Apps.
• Standardisation
  Use a common format (YAML or JSON) that developers across different teams and organizations can understand.

How? Meet the DSL (Domain Specific Language)

Serverless Workflows are defined using a declarative YAML or JSON syntax. A workflow definition typically consists of three main parts:

1. Document
   Metadata about the workflow (ID, version, name).
2. Use
   Definitions of reusable resources like functions, events, and secrets.
3. Do
   The control flow logic, defined as a sequence of states. Including events and sub-workflows

Anatomy of a Workflow

Let's take a look at a simple "Hello World" example of a Serverless Workflow:

document:
  dsl: "1.0.0-alpha5"
  namespace: "thand-test"
  name: "hello-world"
  version: "1.0.0"
  do:
  - greet:
      set:
        greeting: ${ "Hello, " + (.name // "World") + "!" }

In this example, the primary workflow sets and outputs “Hello World”. What's cool about this example is a JSON greeting output: we can use simply use jq to expressively take the name from the input and place it in our output string. Otherwise, use World. More on that later.

Serverless workflow Features

Event-Driven Architecture

Serverless Workflow is designed from the ground up to be event-driven. It natively supports CloudEvents, allowing workflows to be triggered by events (e.g., a file upload, a user signup) and can emit events during execution.

Advanced Control Flow

The DSL supports complex logic beyond simple sequences:

• Switch
  If/else logic to branch execution paths.
• Parallel
  Execute multiple branches concurrently.
• Foreach
  Iterate over a dataset.

Error Handling

In distributed systems, failure is inevitable and expected. The specification includes robust error handling mechanisms like `onErrors` and `retries`, allowing you to define backoff strategies and fallback logic directly in the workflow definition.

Task Types

Serverless Workflow defines a rich set of tasks to control the flow of execution. Here is how common concepts map to the specification:

• CALL
  The workhorse of the workflow. It executes a specified function, such as calling a REST API (HTTP), a gRPC service, or an OpenAPI operation.
• DO
  Executes a sequence of subtasks. This is useful for grouping actions together.
• EMIT
  Publishes events to event brokers or messaging systems, enabling event-driven communication.
• FOR
  Iterates over a collection of data, executing a set of actions for each item.
• FORK
  Splits execution into multiple concurrent branches, waiting for all (or some) to complete before proceeding.
• SWITCH
  Evaluates conditions to determine the next state transition (If/Else logic).
• LISTEN
  Waits for external events to occur before proceeding.
• RUN
  Executes external processes like containers, shell commands, scripts, or other workflows.

Data Manipulation with jq

Serverless Workflow relies heavily on jq (or JSONPath) to manipulate data as it flows between states. This allows you to transform inputs and outputs without writing custom code.

• Input Filtering
  Use the `input` property with a `jq` expression to select only the necessary parts of the incoming data for the current task.
• Output Filtering
  Use the `output` property with a `jq` expression to shape the data passed to the next task.
• Expressions
  You can use `jq` expressions within your workflow definition (e.g., `${ .user.id }`) to dynamically reference data in function arguments or state transitions.

Workflow Types & Patterns

Understanding the different types of workflows is key to leveraging the full power of the specification.

The SDK

For Go developers, the Serverless Workflow Go SDK is the starting point. It provides a comprehensive library for: Parsing, Validation and Diagram Generation of workflows.

While the SDK is excellent for working with the definitions, it is not a full-featured runtime. It can execute simple workflows in-memory, but it lacks the durability required for long-running business processes, or handling signals. At Thand, we handle this by executing workflows with Temporal.

Thand - Durable Execution with Temporal

Thand supports complex authorisation workflows, which - as you can imagine - makes extensive use of pre-defined approval workflows. The Thand agent leverages the serverless workflow SDK to provide a production-grade runtime for Serverless Workflows, backed by Temporal.io.

Why Temporal?

Temporal is a durable execution platform that ensures your workflows run to completion, even in the presence of failures. By mapping Serverless Workflow definitions to Temporal workflows, Thand offers:

• Durability
  If your service crashes or restarts, the workflow resumes exactly where it left off. State is persisted automatically.
• Long-Running Processes
  Workflows can sleep for days, wait for human approval, or listen for external signals without consuming resources.
• Signaling
  You can interact with running workflows. For example, you can "signal" a workflow to update its state or trigger a new path (e.g., an admin approving a request).
• Scalability
  Temporal handles the heavy lifting of distributing workflow execution across a cluster.
• Scheduling
  Temporal handles the scheduling of workflows for a given point in time.

The Thand SDK

With the Thand SDK, you can define your business logic using the standard Serverless Workflow DSL, but execute it with the power of Temporal. This gives you the best of both worlds: the portability and readability of a standard spec, with the reliability and power of a world-class orchestration engine.

For example, we try and make second-person authorisation as simple as modern 2fa, so handling a complex approval process involving emails, Slack notifications, and timeouts becomes a matter of defining the states in YAML. Thand and Temporal ensure execution (grants or revocation of access, or monitoring of activity) happen consistently: signals are received and processed correctly, no matter how long the user takes to respond or how long access is granted.

Ready to get started?
Check out the Serverless Workflow Specification and the Thand Documentation to learn more.