feat: CSV-transform capability (RFC 4180 quoting) (#106) #113

Merged
Marc merged 2 commits from feat/issue-106-csv-transform into main 2026-08-14 12:17:40 +00:00
Contributor

Summary

  • Voegt transformers.csv_to_json (POST /csv2json) en transformers.json_to_csv (POST /json2csv) toe op de bestaande transformers-service.
  • RFC 4180 quoting: een waarde als "meshflows:runs" blijft één veld als de delimiter : is. Unquoted delimiter in een waarde is een kolombreuk; extra niet-lege cellen t.o.v. de header geven 400.
  • Headers: eerste rij wordt object-keys tenzij has_header: false (dan column_1 …). Lege/duplicate headers worden genormaliseerd.
  • Demo-workflow csv_transform_demo + wiki-tabellen. Geen nieuwe orchestrator-stap, geen VERSION-bump.

Closes #106

Test plan

  • Pytest: quoted : in waarde, unquoted : → 400, has_header true/false, duplicate headers, round-trip, demo-workflow load
  • Branch CI (push feat/issue-106-csv-transform @ bc7bd98): combined status success (30/30)
## Summary - Voegt `transformers.csv_to_json` (`POST /csv2json`) en `transformers.json_to_csv` (`POST /json2csv`) toe op de bestaande transformers-service. - RFC 4180 quoting: een waarde als `"meshflows:runs"` blijft één veld als de delimiter `:` is. Unquoted delimiter in een waarde is een kolombreuk; extra niet-lege cellen t.o.v. de header geven 400. - Headers: eerste rij wordt object-keys tenzij `has_header: false` (dan `column_1` …). Lege/duplicate headers worden genormaliseerd. - Demo-workflow `csv_transform_demo` + wiki-tabellen. Geen nieuwe orchestrator-stap, geen VERSION-bump. Closes #106 ## Test plan - [x] Pytest: quoted `:` in waarde, unquoted `:` → 400, `has_header` true/false, duplicate headers, round-trip, demo-workflow load - [x] Branch CI (push `feat/issue-106-csv-transform` @ `bc7bd98`): combined status **success** (30/30)
Add CSV transform capabilities with RFC 4180 quoting.
All checks were successful
Build & Push Docker Images / Trigger Deploy DEV (dev) (push) Successful in 0s
Build & Push Docker Images / Trigger Deploy TST/LAB (lab) (push) Has been skipped
Build & Push Docker Images / Trigger Deploy TST/LAB (tst) (push) Has been skipped
Deploy to Cluster / Deploy Workflows (push) Successful in 29s
Build & Push Docker Images / Runner Preflight (pull_request) Successful in 1s
Build & Push Docker Images / Validate JavaScript Syntax (pull_request) Successful in 4s
Test & Validate / Validate Repository Structure (pull_request) Successful in 4s
Test & Validate / Validate Workflow YAML (pull_request) Successful in 14s
Test & Validate / Validate Kubernetes Manifests (pull_request) Successful in 5s
Build & Push Docker Images / Detect Changed Components (pull_request) Successful in 9s
Build & Push Docker Images / Validate Repository Structure (pull_request) Successful in 3s
Test & Validate / Test Engine Smoke Suite (pytest) (pull_request) Successful in 42s
Build & Push Docker Images / Validate Workflow YAML (pull_request) Successful in 12s
Build & Push Docker Images / Validate Kubernetes Manifests (pull_request) Successful in 4s
Build & Push Docker Images / Build Forgejo Runner Image (pull_request) Has been skipped
Build & Push Docker Images / Build Wiki (MkDocs) (pull_request) Successful in 4s
Build & Push Docker Images / Sync compact pointer to Forgejo Wiki (pull_request) Has been skipped
Build & Push Docker Images / Test Engine Smoke Suite (pytest) (pull_request) Successful in 39s
Build & Push Docker Images / Build (orchestrator) Engine Services [./engine/services/orchestrator] (orchestrator, ./engine/services/orchestrator) (pull_request) Successful in 22s
Test & Validate / Test Summary (pull_request) Successful in 1s
Build & Push Docker Images / Test Summary (pull_request) Successful in 0s
Build & Push Docker Images / Test Gate (pull_request) Successful in 0s
Build & Push Docker Images / Build (transformers) Engine Services [./engine/services/transformers] (transformers, ./engine/services/transformers) (pull_request) Successful in 24s
Build & Push Docker Images / Retag Product Version (pull_request) Successful in 0s
Build & Push Docker Images / Publish Release Bundle (pull_request) Successful in 1s
Build & Push Docker Images / Resolve Auto-Deploy Targets (pull_request) Successful in 1s
Build & Push Docker Images / Build Summary (pull_request) Successful in 0s
Build & Push Docker Images / Trigger Deploy DEV (dev) (pull_request) Successful in 0s
Build & Push Docker Images / Trigger Deploy TST/LAB (lab) (pull_request) Has been skipped
Build & Push Docker Images / Trigger Deploy TST/LAB (tst) (pull_request) Has been skipped
bc7bd98a12
Expose csv_to_json and json_to_csv on transformers so a quoted value like meshflows:runs stays one field when the delimiter is colon, and the first row becomes object keys unless has_header is false.

Co-authored-by: Cursor <cursoragent@cursor.com>
Merge branch 'main' into feat/issue-106-csv-transform
All checks were successful
Build & Push Docker Images / Build (transformers) Engine Services [./engine/services/transformers] (transformers, ./engine/services/transformers) (push) Successful in 23s
Build & Push Docker Images / Build Forgejo Runner Image (pull_request) Has been skipped
Build & Push Docker Images / Sync compact pointer to Forgejo Wiki (pull_request) Has been skipped
Build & Push Docker Images / Build Wiki (MkDocs) (pull_request) Successful in 4s
Build & Push Docker Images / Build (orchestrator) Engine Services [./engine/services/orchestrator] (orchestrator, ./engine/services/orchestrator) (pull_request) Successful in 23s
Build & Push Docker Images / Test Summary (pull_request) Successful in 1s
Build & Push Docker Images / Test Gate (pull_request) Successful in 0s
Build & Push Docker Images / Build (transformers) Engine Services [./engine/services/transformers] (transformers, ./engine/services/transformers) (pull_request) Successful in 22s
Build & Push Docker Images / Retag Product Version (push) Successful in 0s
Deploy to Cluster / Resolve Environment (push) Successful in 0s
Build & Push Docker Images / Retag Product Version (pull_request) Successful in 0s
Deploy to Cluster / Resolve Release Bundle (push) Has been skipped
Deploy to Cluster / List Recent Release Bundles (push) Has been skipped
Build & Push Docker Images / Publish Release Bundle (push) Successful in 1s
Deploy to Cluster / Deploy Platform Data (push) Has been skipped
Build & Push Docker Images / Publish Release Bundle (pull_request) Successful in 2s
Deploy to Cluster / Deploy Wiki (push) Has been skipped
Build & Push Docker Images / Resolve Auto-Deploy Targets (pull_request) Successful in 0s
Build & Push Docker Images / Resolve Auto-Deploy Targets (push) Successful in 1s
Deploy to Cluster / Deploy Engine (push) Has been skipped
Build & Push Docker Images / Build Summary (push) Successful in 0s
Build & Push Docker Images / Trigger Deploy DEV (dev) (push) Successful in 0s
Deploy to Cluster / Rollback on Deploy Failure (push) Has been skipped
Build & Push Docker Images / Build Summary (pull_request) Successful in 0s
Build & Push Docker Images / Trigger Deploy DEV (dev) (pull_request) Successful in 0s
Build & Push Docker Images / Trigger Deploy TST/LAB (lab) (push) Has been skipped
Build & Push Docker Images / Trigger Deploy TST/LAB (tst) (push) Has been skipped
Build & Push Docker Images / Trigger Deploy TST/LAB (tst) (pull_request) Has been skipped
Build & Push Docker Images / Trigger Deploy TST/LAB (lab) (pull_request) Has been skipped
Deploy to Cluster / Deploy Workflows (push) Successful in 26s
78e4b1750c
Keep CSV transform and JSON Schema validation side by side after #114 landed on main.

Co-authored-by: Cursor <cursoragent@cursor.com>
Marc merged commit c5b3c082b6 into main 2026-08-14 12:17:40 +00:00
Sign in to join this conversation.
No description provided.