This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

Workflows (Flows) — Workflow Definitions & Deployment

The Flows component contains all workflow definitions, connections, policies, and configurations for task automation and integration.

What are Flows?

Flows represent the application logic layer of MeshFlows:

Workflow Definitions — YAML files describing execution steps
Connections — Secure credential storage (HTTP, OAuth, FTP, SSH)
Policies — Execution rules and access control
Schedules — CronJob trigger configurations
Settings — Runtime configuration overrides

Flows are version controlled in Git and can be:

Deployed alongside the Engine (integrated deployment)
Deployed independently (as ConfigMaps or directly)
Updated without restarting other Engine services

Workflow Types

HTTP-Triggered Workflows

User or external system initiates via HTTP:

curl -X POST http://meshflows/v1/run/my-workflow

Scheduled Workflows

Run automatically on a schedule:

cron: "0 2 * * *"  # Daily at 2 AM

Event-Driven Workflows

Triggered by storage changes or RabbitMQ messages.

Start Here

Build a New Workflow

Integrate External Systems

Run on Schedule or Events

I’m a Developer — What Do I Need to Know?

Getting Started?

First Workflow — Create your first workflow in 5 minutes
Connections & Credentials — Store credentials securely
Deployment Methods — Deploy to the Engine

Writing Workflows?

Workflow YAML Syntax — Complete reference
Step Types — All available step types
Step Contract — Required/optional fields and validation rules per step
Context & Variables — Passing data between steps
Conditions & Path Expressions — IF branches, JSON/XML paths, service output
Transformations — XSLT, Liquid, JSON conversions
Examples & Recipes — Real-world examples

Testing & Debugging?

Local Testing — Test locally before deploying
Viewing Traces — Debug execution with Tempo (Forgejo mirror: Observability)

Key Concepts

Workflow Structure

name: my-workflow
invocation:
  type: [http] # use explicit list, e.g. [http, schedule]

steps:
  - name: step_1
    type: http
    method: GET
    url: https://api.example.com/data
    
  - name: step_2
    type: storage_write
    key: result
    body_from: step_1

Step Types

Type	Purpose
`http`	HTTP GET/POST/PUT/DELETE
`transform`	XSLT, xml2json, json2xml, Liquid
`storage_read`	Read key-value or documents
`storage_write`	Write key-value or documents
`if`	Conditional execution
`for_each`	Loop over collections
`parallel`	Execute steps concurrently
`wait`	Delay execution
`log`	Log and capture output
`cancel`	Cancel workflow
`compensation`	Rollback on failure

Connections (Secure Credentials)

type: http
name: my_api
properties:
  base_url: https://api.example.com
  headers:
    Authorization: "Bearer {secret-token}"

Policies (Execution Rules)

name: require_auth_header
type: required_headers
apply_to:
  - "sensitive-*"
apply_to_triggers:
  - http
headers:
  - Authorization
on_violation: reject

Common Workflows

1. Data Integration

Fetch data from external API, transform, store locally:

HTTP GET → XML2JSON → Storage Write

2. File Processing

Upload file, process, send results:

HTTP GET (file) → FTP Upload → Storage Write

3. Scheduled Reports

Generate reports on schedule, email results:

CronJob Trigger → Transform Data → HTTP POST (email service)

4. Event-Driven Processing

React to storage changes or messages:

Storage Change Event → Process → RabbitMQ Publish

5. Multi-Step Integration

Complex workflow with branching logic:

HTTP GET → IF (check status) → Parallel (notify teams) → Compensate (on error)

Deployment Methods

1. ConfigMap (Development)

Quick deployment for testing:

kubectl create configmap workflows --from-file=workflows/
kubectl create configmap connections --from-file=connections/

2. GitOps (Recommended for Production)

Workflows stored in Git, CI/CD manages deployment:

Push workflow YAML to Git
Forgejo Actions validates and builds
GitOps controller syncs changes to the cluster

3. API Upload (Advanced)

curl -X POST http://storage:8080/internal/upload/workflows/demo \
  -H "Authorization: Bearer <runtime-admin-token>" \
  -H "Content-Type: application/yaml" \
  --data-binary @workflows/demo.yaml

Other single-item uploads:

POST /internal/upload/connections/{name}
POST /internal/upload/policies/{name} (limited to required_headers)
POST /internal/upload/schedules/{name}
POST /internal/upload/config (runtime settings YAML)

After uploading a workflow, trigger a reload in orchestrator (POST /admin/reload).

Workflow Lifecycle

Edit YAML
   ↓
Local Testing
   ↓
Push to Git
   ↓
CI/CD Validation
   ↓
Deploy to Cluster
   ↓
Monitor in Tempo/Dashboard
   ↓
Update if needed

Common Tasks

Next Steps

First Time?: Start with First Workflow
Learning: Check out Examples & Recipes
Production: Review Deployment Methods and Security (Forgejo mirror: Security) (Forgejo mirror: Security)
Advanced: Explore Error Handling & Compensation