9 Automation and CI/CD Pipelines

Our platform includes end-to-end automation pipelines connecting scheduled orchestration, continuous integration testing, and documentation publication.

9.1 Scheduled Workflows with n8n

We automate recurring model evaluations using n8n:

Schedule (Sundays 02:00) / Webhook Trigger
                    ↓
       python3 scripts/run_all_benchmarks.py
                    ↓
       python3 scripts/make_all_reports.py
                    ↓
       git commit artifacts & push
                    ↓
       Post-Run Webhook Notification

9.1.1 Workflow Specification

Configuration File: n8n/uc12-benchmark-workflow.json
Default Schedule: Weekly execution on Sundays at 02:00 local time.
On-Demand Webhook: POST http://localhost:5678/webhook/uc12-benchmark-trigger

To trigger an immediate evaluation run without waiting for the scheduled cron:

curl -X POST http://localhost:5678/webhook/uc12-benchmark-trigger

9.1.2 Environment Settings in n8n

Configure operational settings within n8n under Settings -> Variables:

Variable	Value Description	Purpose
`WEBHOOK_NOTIFY_URL`	Outbound webhook URL	Dispatches completion summaries to Slack, Teams, or email
`LINKEDIN_ACCESS_TOKEN`	OAuth bearer token	Enables direct automated posting of generated research briefs

9.2 Continuous Integration via GitHub Actions

Our repository maintains automated workflows in .github/workflows/:

9.2.1 1. Script Validation (`validate.yml`)

Runs on every push to main and on pull requests affecting scripts/**: - Installs Python 3.11 and dependencies (matplotlib, numpy, requests). - Executes python -m py_compile across all evaluation scripts. - Parses abstract syntax trees (AST) to verify import integrity without triggering compute overhead.

9.2.2 2. Quarto Book Publishing (`deploy-book.yml`)

Deploys this interactive documentation book to GitHub Pages: - Installs the Quarto publishing CLI via quarto-dev/quarto-actions/setup@v2. - Renders the complete book into static HTML, JavaScript, and SVG assets. - Pushes the rendered bundle to the gh-pages branch via peaceiris/actions-gh-pages@v4.

9.3 Docker Containerization

To run reproducible evaluations in isolated container environments, the platform provides Docker definitions:

# Launch CPU-bound evaluation stack
docker compose up -d

# Launch GPU-accelerated evaluation stack with NVIDIA container runtime
docker compose -f docker-compose.gpu.yml up -d

The container stack mounts local model weights and output directories while exposing ChainForge on port 8765.