Jubiloop Docs

Appearance

Environment Management ​

This document covers environment URLs, infrastructure details, configurations, and management practices.

Environment URLs ​

Development ​

QA ​

Production ​

Infrastructure Costs ​

Monthly Breakdown ​

  • DigitalOcean Droplets: $12 total
    • Dev/QA shared: $6/month (s-1vcpu-1gb)
    • Production: $6/month (s-1vcpu-1gb)
  • DigitalOcean Managed PostgreSQL: ~$15/month (db-s-1vcpu-1gb, production only)
  • Cloudflare: $0 (free tier)
  • Domains: ~$2/month
  • Total: ~$29/month

Droplet Specifications ​

  • Size: 1 vCPU, 2GB RAM, 50GB SSD
  • OS: Ubuntu 22.04 LTS
  • Region: Toronto (tor1)

Service Locations ​

Backend (DigitalOcean) ​

  • AdonisJS API
  • Redis cache
  • PostgreSQL (dev/qa only - local container)
  • Caddy proxy

Frontend (Cloudflare Pages) ​

  • React web app
  • Next.js marketing site

Database ​

Development/QA ​

  • Local PostgreSQL container
  • Separate databases per environment

Production (DigitalOcean Managed PostgreSQL) ​

  • DigitalOcean Managed PostgreSQL 16 (db-s-1vcpu-1gb, region: tor1)
  • Automatic daily backups with 7-day retention
  • PgBouncer connection pooling (transaction mode, pool size 14)
  • Firewall rules restrict access to production droplet only

Environment Variables ​

All environment configuration is managed through env.deploy.yml:

yaml
# Common pattern for all environments
{ ENV }:
  server:
    PORT: 3333
    DATABASE_URL: from_secrets # Becomes {ENV}_SERVER_DATABASE_URL
    NODE_ENV: '{environment}'
  webapp:
    API_URL: 'https://{env}-api.jubiloop.ca'

Environment-Specific Settings ​

Development:

  • Debug logging enabled
  • Relaxed rate limits (1000/hour)
  • CORS allows localhost
  • Verbose error messages

QA:

  • Production-like settings
  • Standard rate limits (500/hour)
  • CORS restricted to QA domains
  • Structured error logging

Production:

  • Minimal logging
  • Strict rate limits (100/hour)
  • CORS limited to production domains
  • Generic error messages
  • DigitalOcean Managed PostgreSQL connection string
  • Backups enabled on droplet

External Service Configuration ​

Payment Processing (Stripe) ​

EnvironmentModeWebhook Endpoint
DevelopmentTest modedev-api.jubiloop.ca/webhooks/stripe
QATest modeqa-api.jubiloop.ca/webhooks/stripe
ProductionLive modeapi.jubiloop.ca/webhooks/stripe

Email Service ​

The platform uses a comprehensive Email Infrastructure with:

EnvironmentResend (Transactional)Brevo (Marketing)
DevelopmentSandbox modeTest lists
QALimited sendingSandbox campaigns
ProductionFull sendingLive campaigns

File Storage (Cloudflare R2) ​

Not yet implemented

Cloudflare R2 is planned for application file storage (photos, documents), but no buckets, upload logic, or signed URL infrastructure exists in the codebase yet. R2 is currently used only for Terraform state storage (jubiloop-terraform-state bucket).

Database Management ​

Development & QA ​

  • Shared PostgreSQL container on same droplet
  • Separate databases: jubiloop_dev and jubiloop_qa
  • Can be reset anytime
  • Test data seeding scripts available

Production ​

  • DigitalOcean Managed PostgreSQL 16 (db-s-1vcpu-1gb)
  • Automated daily backups with 7-day retention
  • Point-in-time recovery
  • PgBouncer connection pooling (transaction mode, pool size 14)
  • Firewall rules restrict access to production droplet only

Environment Promotion ​

Development → QA ​

  1. Code Freeze: Feature complete on develop
  2. Create PR: From develop to qa
  3. Automated Checks:
    • Tests must pass
    • Linting must pass
    • Build must succeed
  4. Manual Review: Code review required
  5. Merge: Triggers automatic deployment

QA → Production ​

  1. QA Sign-off: All test cases passed
  2. Create PR: From qa to main
  3. Final Checks:
    • No critical bugs
    • Performance acceptable
    • Security scan passed
  4. Approval: Requires admin approval
  5. Deploy: Merge triggers deployment
  6. Verify: Post-deployment checks

Rollback Procedures ​

Quick Rollback ​

No commit SHA input exists

The deploy workflows do not accept a commit SHA. You cannot target a specific previous commit directly from the Actions UI. The correct rollback method is a git revert.

Option A — Git revert (recommended, keeps full history):

bash
# Revert the bad commit on the relevant branch
git revert <bad-commit-sha>
git push origin main   # or develop/qa

# This triggers automatic deployment of the reverted commit.

Option B — Manual re-trigger from the Actions UI (re-deploys HEAD):

If the branch is already in a good state (e.g. the bad commit was already reverted by someone else):

  1. Go to the Actions tab in GitHub
  2. Select the relevant workflow: Deploy Server, Deploy Web App, or Deploy Marketing Site
  3. Click Run workflow → select the branch → click Run workflow

This deploys the current HEAD of the selected branch — it does not accept a commit SHA input.

Each app has its own deploy workflow. If you need to roll back all three (server + webapp + marketing), you must trigger each one separately.

Database Rollback ​

For database changes:

  1. Migrations are reversible
  2. Run rollback migration
  3. Deploy previous code version

Environment Isolation ​

Network Level ​

  • No cross-environment communication
  • Separate Redis instances
  • Independent service deployments

Data Level ​

  • No production data in dev/QA
  • Separate secret keys
  • Different API credentials

Access Control ​

  • Production app requires Zero Trust auth (temporary)
  • Production marketing site is public
  • QA requires Zero Trust auth for all frontend apps
  • Dev requires Zero Trust auth for all frontend apps
  • API endpoints protected by session auth (not Zero Trust)

Monitoring & Observability ​

What exists today ​

CapabilityDetail
Health endpointGET /health — checks disk space, memory heap, and PostgreSQL connectivity. Returns sanitized { status, isHealthy } by default; pass x-monitoring-secret header for full diagnostic report.
Docker healthchecksAll server containers poll http://localhost:3333/health every 30s. Containers are automatically restarted if unhealthy.
CI deployment verificationGitHub Actions curls /health after every server deployment (3 retries). Deploy fails if the endpoint doesn't return healthy.
Deployment failure alertsGitHub Actions posts a PR comment when a deployment job fails. No other alert channel is configured.
DigitalOcean metrics agentmonitoring = true on all droplets — provides basic CPU, memory, disk, and bandwidth graphs in the DO console. No alert policies are defined.
Docker log rotationjson-file driver, 10MB max per file, 3 files retained. View via docker compose logs.

Not yet implemented ​

  • Error tracking — Sentry is planned but not installed (SENTRY_DSN is commented out in env.deploy.yml)
  • Uptime monitoring — no external service configured (no UptimeRobot, Pingdom, Better Uptime, etc.)
  • Runtime alerts — no Slack, PagerDuty, email, or webhook notifications for errors or downtime
  • APM / performance tracking — no Datadog, New Relic, or OpenTelemetry
  • Metrics dashboards — no Prometheus or Grafana

Common Tasks ​

Refresh Dev/QA Database ​

bash
# SSH to dev-qa server
ssh deploy@dev-qa.jubiloop.ca

# Backup current data (optional)
docker exec postgres pg_dump jubiloop_dev > backup.sql

# Reset database
docker exec postgres psql -c "DROP DATABASE jubiloop_dev;"
docker exec postgres psql -c "CREATE DATABASE jubiloop_dev;"

# Run migrations
docker exec server npm run migration:run

Check Environment Health ​

bash
# Check all services
curl https://dev-api.jubiloop.ca/health
curl https://qa-api.jubiloop.ca/health
curl https://api.jubiloop.ca/health

View Logs ​

bash
# Development/QA
ssh deploy@dev-qa.jubiloop.ca
docker logs server-dev -f
docker logs server-qa -f

# Production
ssh deploy@<production-ip>  # Get IP from terraform output
docker logs server -f

Troubleshooting ​

Environment Not Deploying ​

  1. Check GitHub Actions for errors
  2. Verify branch protection rules
  3. Check secrets are set correctly
  4. Ensure Docker registry is accessible

Wrong Environment Variables ​

  1. Check env.deploy.yml is correct
  2. Run npm run generate:env locally
  3. Verify GitHub Secrets match
  4. Restart containers after changes

Database Connection Issues ​

  1. Check DATABASE_URL is correct
  2. Verify firewall rules
  3. Check connection pool limits
  4. Review PostgreSQL logs

Built with ❤️ by the Jubiloop team

Copyright © 2025 Jubiloop