CI/CD Pipeline Guide

Jubiloop uses GitHub Actions for automated continuous integration and deployment workflows.

Overview

Application environment and infrastructure configuration is centralized in infra/deploy/env.deploy.yml. This file defines:

• Infrastructure settings (Terraform variables, backend config)
• Environment variables for all services
• Secret references (auto-discovered by scripts)
• Variable references (non-sensitive config)

CI-only credentials and controls, including Discord webhooks, role IDs, and notification switches, are GitHub repository secrets and variables rather than application configuration.

Configuration Generation Scripts

The deployment system is powered by two main scripts that dynamically process env.deploy.yml.

generate-env-files.js

Generates environment configuration for all services.

Usage:

# Generate .env files for server services (default)
node scripts/generate-env-files.js DEV

# Generate shell exports for web app
node scripts/generate-env-files.js DEV --web-app

# Generate shell exports for marketing
node scripts/generate-env-files.js PROD --marketing

# Preview without writing files
node scripts/generate-env-files.js DEV --dry-run

# List required secrets
node scripts/generate-env-files.js DEV --list-secrets

Features:

• Auto-discovers services from env.deploy.yml
• Handles from_secrets pattern (generates secret names like DEV_SERVER_APP_KEY)
• Supports explicit secrets (see Secret Management section)
• Validates environments and services

extract-config.js

Master configuration extractor for all deployment needs.

Usage:

# Generate Terraform variables
node scripts/extract-config.js terraform-vars DEV-QA

# Generate Terraform backend config
node scripts/extract-config.js backend-hcl

# Generate secret export script
node scripts/extract-config.js secret-exports

# Generate Ansible inventory
node scripts/extract-config.js ansible-inventory dev

# Generate documentation
node scripts/extract-config.js documentation

# List deployment configuration secrets
node scripts/extract-config.js list-secrets

# List all variables
node scripts/extract-config.js list-variables

GitHub Actions Workflows

terraform-deploy.yml

• Purpose: Manages infrastructure with Terraform
• Trigger: Manual dispatch only
• Environments: dev-qa or prod (both enabled)
• Actions: plan or apply
• Plan/apply behavior: An apply run normally creates and applies a fresh saved plan in the same run. Plan-only artifacts are not inputs to later runs. initial_apply uses targeted bootstrap applies before a full apply.
• Approval input: action=apply also requires auto_approve=true; otherwise the non-interactive job exits instead of waiting for approval. Use GitHub environment protection for human approval before the job starts.

deploy-server.yml

• Purpose: Builds and deploys AdonisJS backend
• Automatic triggers:
  • develop branch → dev environment
  • qa branch → qa environment
  • main branch → production environment
• Process: Builds Docker image, pushes to registry, deploys via Ansible

deploy-web-app.yml

• Purpose: Deploys React app to Cloudflare Pages
• Triggers: Changes in apps/webapp/ or manual dispatch
• Target: Cloudflare Pages

deploy-marketing.yml

• Purpose: Deploys the Next.js marketing site through OpenNext using the matching Terraform-provisioned Cloudflare project name
• Triggers: Changes in apps/marketing/ or manual dispatch
• Target: Cloudflare Workers
• Studio deployment: After the Worker succeeds, resolves the environment-specific *_SANITY_STUDIO_DEPLOY_TOKEN and runs pnpm exec sanity deploy -y
• Required marketing config: SANITY_STUDIO_API_TOKEN is generated into the marketing environment because apps/marketing/src/sanity/env.ts currently requires it

Manual Sanity Studio deployment is a fallback, not the normal release path. See Sanity CMS setup for the fallback command and credential distinction.

release.yml

• Purpose: Publishes a CalVer GitHub Release after a release PR merges
• Triggers: Merged release PRs targeting qa or main
• Target: QA prerelease or production release

Discord Deployment Notifications

GitHub Actions routes operational status to three private Discord channels:

Channel	Purpose
🛠️・ops-dev	PR, CI, review, and development deployment status
🧪・ops-qa	QA release and deployment status for builds ready to verify
🚀・ops-production	Production release and deployment status, including urgent failures

Terraform and documentation deployments aren't included. The application deployment set is the server, web app, and marketing site.

Development PR status

.github/workflows/discord-develop-pr-status.yml coordinates one persistent, searchable Discord message for each non-draft PR targeting develop. The message title includes the PR number and links to the PR. It shows the current head SHA and updates in place as the PR changes, rather than posting a new message for each event.

The current snapshot includes:

• Lifecycle and current head after PR updates, including merge
• Aggregate status for applicable server, web app, marketing, and Discord notification test workflows on the current head
• Latest effective review decision per reviewer: approved, changes requested, or commented
• Development deployment status for applicable server, web app, and marketing workflows after merge

Review status is based on submitted reviews. A later comment does not replace an existing approval or changes request. A comment-only review, including a submitted review with inline comments, appears as Commented. Ordinary PR conversation comments and standalone inline review-comment events don't trigger the coordinator, and thread counts aren't shown. A bot-authored marker comment on the GitHub PR stores the Discord message ID, current head and aggregate states, review signature, deployment SHA, and sent failure-alert keys.

Draft PRs don't create Discord messages. A ready_for_review event can create one when the relevant notification is enabled. Existing messages continue to update for events that maintain their lifecycle. A development deployment without an associated merged PR targeting develop writes an Actions warning and skips the Discord PR update.

All development notification switches are explicit opt-ins. An unset value behaves as false:

Repository variable	Controls
DISCORD_DEV_NOTIFY_PR_OPENED	Initial PR status message
DISCORD_DEV_NOTIFY_REVIEW_STATUS	Updates after submitted review state changes
DISCORD_DEV_NOTIFY_CI_SUCCESS	Successful applicable CI aggregate updates
DISCORD_DEV_NOTIFY_CI_FAILURE	Failed or cancelled CI aggregate updates
DISCORD_DEV_NOTIFY_DEPLOY_SUCCESS	Successful development deployment updates
DISCORD_DEV_NOTIFY_DEPLOY_FAILURE	Failed or cancelled development deploy updates
DISCORD_DEV_MENTION_REVIEWERS_ON_PR_OPENED	Reviewers role mention when the message starts
DISCORD_DEV_MENTION_REVIEWERS_ON_FAILURE	Reviewers role alert for later failures

DISCORD_DEV_REVIEWERS_ROLE_ID contains the numeric Discord role ID. The opened mention and later failure mentions are independent. Editing the persistent message does not generate a new mention, so a newly observed CI or development deployment failure posts one separate, concise role alert. The marker's per-head alert keys deduplicate that alert; details remain in the persistent message.

QA and production status

.github/workflows/discord-qa-main-pr-status.yml coordinates one persistent message for each trusted same-repository PR targeting qa or main. The card is posted to the matching environment channel and updates in place throughout the PR lifecycle. It includes the current head, applicable CI, submitted reviews, release publication when the PR is a release PR, and server, web app, and marketing deployment status after merge.

Successful deployment rows include the available application, container image, Cloudflare deployment, Worker target, version, and dashboard links. Release PR cards also link the published GitHub Release. Deployment workflows continue to update the marked deployment table in the Release body without replacing release notes.

When a deployment is matched to a trusted qa or main PR, the persistent card replaces the standalone service notification. Any unmatched deployment retains its standalone in-progress and final message. This includes manual deployments that do not resolve to a trusted merged PR, direct pushes without an associated merged PR, and deployments associated only with a fork PR. Notification delivery is best-effort, so a coordinator failure doesn't change the deployment result.

Release association first checks the release's target commit. If that field does not contain the merge SHA, the workflows resolve lightweight or annotated release tags and compare the referenced commit. Standalone deployment notifications retry briefly while release publication propagates.

Persistent card edits never contain role mentions. A newly observed CI, release, or deployment failure posts a separate concise alert in the same QA or production channel, mentions DISCORD_PROD_MAINTAINERS_ROLE_ID, and links directly to the failed Actions run. Successful and cancelled aggregate states don't mention the role. Existing standalone production deployment cancellation alerts remain enabled.

Delivery and credentials

The repository secrets are:

• DISCORD_DEV_NOTIFICATIONS_WEBHOOK_URL
• DISCORD_QA_DEPLOYMENTS_WEBHOOK_URL
• DISCORD_PROD_DEPLOYMENTS_WEBHOOK_URL

Each workflow injects its webhook secret into the action step as the DISCORD_WEBHOOK_URL environment variable. Webhook URLs aren't action inputs or application runtime configuration, and they must not be added to infra/deploy/env.deploy.yml.

Notification delivery is best-effort. A missing webhook, unresolved association, or Discord API error produces an Actions warning without changing the PR check, deployment, or release result. A hard workflow cancellation can stop an update, leaving stale status in Discord. Workflow rows link to Actions so the actual result remains available.

Discord setup

1. Create private 🛠️・ops-dev, 🧪・ops-qa, and 🚀・ops-production channels.
2. Create a separate incoming webhook in each channel. Name all three Jubiloop Ops and use the same Jubiloop operations avatar. Notification payloads don't override the webhook name, so later changes made in Discord apply without code changes.
3. Create a reviewers role for 🛠️・ops-dev and an operations maintainers role for 🧪・ops-qa and 🚀・ops-production. Allow each role to view its channels and make it mentionable, or grant the channel permission required for webhook mentions.
4. Add the three webhook repository secrets listed above under Settings > Secrets and variables > Actions.
5. Add DISCORD_DEV_REVIEWERS_ROLE_ID, DISCORD_PROD_MAINTAINERS_ROLE_ID, and the eight explicit development opt-in variables as repository variables. Use numeric role IDs without the <@&...> wrapper.

Keep each webhook scoped to its environment. If a webhook URL is exposed, delete it in Discord, create a replacement, and update the matching GitHub secret. Do not add these URLs to infra/deploy/env.deploy.yml; they are CI credentials, not application runtime configuration.

Secret Management

Two patterns for secrets:

1. Auto-generated: from_secrets creates names like {ENV}_{SERVICE}_{KEY}

   • Example: DEV.server.APP_KEY: from_secrets becomes DEV_SERVER_APP_KEY

2. Explicit: GitHub secrets syntax uses exact secret name

   • Example: PROD.server.DB_URL: '\$\{\{ secrets.NEON_DATABASE_URL }\}\'

Adding secrets:

1. Define in env.deploy.yml
2. Add to GitHub Settings → Secrets and variables → Actions
3. Verify with node scripts/extract-config.js list-secrets

Environment Structure

Example env.deploy.yml structure:

INFRASTRUCTURE:
  dev-qa:
    droplet_size: 's-1vcpu-1gb'
    region: 'nyc3'

DEV:
  server:
    PORT: 3333
    NODE_ENV: 'development'
    DATABASE_URL: from_secrets
  webapp:
    VITE_API_URL: 'https://dev-api.jubiloop.ca'

PROD:
  server:
    PORT: 3333
    NODE_ENV: 'production'
    DATABASE_URL: '${{ secrets.DO_DB_CONNECTION_STRING }}' # DigitalOcean Managed PostgreSQL

Deployment Process

1. Configuration: Define application and infrastructure settings in env.deploy.yml
2. Secret Discovery: Scripts find all secret references
3. GitHub Actions: Workflows generate configs dynamically
4. Infrastructure: Terraform provisions resources
5. Deployment: Ansible deploys applications

Local Testing

cd infra/deploy

# Test server configs
node scripts/generate-env-files.js DEV --server --dry-run

# Check required secrets
node scripts/extract-config.js list-secrets

# Generate documentation
node scripts/extract-config.js documentation

Troubleshooting

Common issues:

1. Secret not found: Check list-secrets output matches GitHub secrets
2. Files not generated: Check YAML syntax, use --dry-run
3. Terraform issues: Verify R2 credentials and backend.hcl

Best Practices

1. Never hardcode secrets
2. Test with --dry-run first
3. Keep service names consistent
4. Use variables for non-sensitive config
5. Keep application and infrastructure config in env.deploy.yml; keep CI-only credentials and controls in GitHub repository settings

Detailed Script Documentation

generate-env-files.js Features

The script supports multiple environments and modes:

Environment Support:

• Single environments: DEV, QA, PROD
• Combined environment: DEV-QA (generates files for both)

Service Discovery:

• Automatically finds all services in env.deploy.yml
• No hardcoded service lists
• Validates service availability per environment

Output Formats:

• Server mode: Creates .env.{service} files
• Web mode: Outputs shell export commands

Secret Handling:

• from_secrets: Auto-generates as {ENV}_{SERVICE}_{KEY}
• Explicit references: Uses exact secret name from GitHub

extract-config.js Commands

terraform-vars:

• Extracts INFRASTRUCTURE section from env.deploy.yml
• Creates terraform.tfvars with all variables
• Supports environment-specific configurations

backend-hcl:

• Generates Terraform backend configuration
• Configures Cloudflare R2 for state storage
• Uses credentials from env.deploy.yml

secret-exports:

• Discovers all secrets and variables
• Generates shell script for GitHub Actions
• Exports secrets dynamically during workflow

ansible-inventory:

• Reads Terraform state output
• Creates dynamic inventory with server IPs
• Groups servers by environment

documentation:

• Generates markdown documentation
• Lists all secrets and variables
• Shows environment configurations

Adding New Services

1. Update env.deploy.yml:

   DEV:
     my_new_service:
       PORT: 8080
       API_KEY: from_secrets

2. Add Docker Compose configuration

3. Test configuration:

   node scripts/generate-env-files.js DEV --dry-run
   node scripts/extract-config.js list-secrets

4. Add required secrets to GitHub

Script Documentation Details

How the Scripts Work Together

The deployment system is built around two core scripts that work in tandem:

1. extract-config.js reads env.deploy.yml and extracts specific configurations:

   • Infrastructure variables for Terraform
   • Secret and variable references for GitHub Actions
   • Dynamic inventory generation from Terraform state
   • Complete documentation of all settings

2. generate-env-files.js reads env.deploy.yml and generates environment files:

   • Discovers all services dynamically (no hardcoded lists)
   • Handles secret substitution from GitHub Actions environment
   • Outputs appropriate format based on deployment target

Script Execution in GitHub Actions

Terraform and application deployment are separate workflows. The relevant commands are:

# Manual Terraform workflow for dev/QA
node scripts/extract-config.js terraform-vars DEV-QA > terraform/dev-qa/terraform.tfvars
node scripts/extract-config.js backend-hcl > terraform/dev-qa/backend.hcl
cd terraform/dev-qa
terraform init -backend-config=backend.hcl
terraform plan -out=tfplan -input=false
terraform apply -auto-approve -input=false tfplan

# Server application workflow for development
cd ../..
node scripts/generate-env-files.js DEV-QA --server
node scripts/extract-config.js ansible-inventory dev > ansible/inventory.ini
ansible-playbook -i ansible/inventory.ini ansible/playbook.yml --extra-vars 'env=dev'

These commands illustrate the generated artifacts and positional environment arguments. The actual workflows also inject GitHub secrets and variables, resolve the droplet address from remote state, and pass deployment credentials and role variables to Ansible. Do not use the abbreviated example as a substitute for the reviewed workflow when deploying an environment.

Environment Variable Substitution

The scripts handle three types of values:

1. Static values: Copied as-is

   PORT: 3333 # Becomes PORT=3333

2. Auto-generated secrets: from_secrets

   APP_KEY: from_secrets # Becomes APP_KEY=$DEV_SERVER_APP_KEY

3. Explicit references: GitHub secrets/variables

   DATABASE_URL: '${{ secrets.PROD_DB_URL }}' # Becomes DATABASE_URL=$PROD_DB_URL

Multi-Environment Support

The scripts support both single and combined environments:

• Single: DEV, QA, PROD - generates files for one environment
• Combined: DEV-QA - generates files for both dev and qa simultaneously

This allows the shared dev-qa server to be deployed with a single command.

Application deploy workflows do not plan or apply infrastructure; server deployment initializes Terraform and reads its state to find the target droplet. No workflow automatically rolls back a failed deployment. Infrastructure changes use the separate manual Terraform workflow. Recovery requires an inspected fresh plan, a forward fix, or a Git revert and explicit redeployment.

Support

For deployment issues:

1. Check generated configuration locally
2. Review GitHub Actions workflow logs
3. Contact team member directly
4. Check 1Password for secret documentation