spicy-automation/docs/COST_OPTIMIZATION_TESTING.md

# Cost Optimization for Testing Environments

Barebones setup guide for VPC → Cluster → Service with blue/green deployment and internet-facing ALB.

## CloudFormation Import Pattern

All examples use the **CloudFormation import pattern** for minimal configuration:

- **VPC Stack**: Only `numberOfAzs` needs to be specified (exports `${vpcStackName}-NumberOfAZs`)
- **Cluster Stack**: Uses `vpcStackName` to auto-import VPC ID, CIDR, subnets, and `numberOfAzs`. Exports `${clusterStackName}-VPCStackName` for other stacks.
- **Service Stack**: Uses `clusterName` to import VPC stack name from cluster export, then imports VPC details from VPC stack
- **ALB Stack**: Uses `clusterName` to import VPC stack name from cluster export, then imports VPC details from VPC stack

**Key benefit**: `numberOfAzs` only needs to be specified once (in the VPC stack), and all other stacks automatically import it.

## Cost Breakdown

### Optimized Testing Setup

- **VPC**: 2 AZs, 2 NAT Gateways = ~$64/month (NAT) + ~$2/month (EIPs)
- **Cluster**: 1 x t3.small instance (Spot) = ~$5-8/month, OR Fargate-only (0 instances) = $0/month
- **ALB**: Individual ALB per service = ~$16/month
- **Service**: 1 task = minimal additional cost
- **Total**: ~$87-90/month (with EC2) or ~$82-84/month (Fargate-only)

**Savings:** ~70-80% reduction vs production setup

## Barebones Blue/Green Setup

### Complete Configuration

```groovy
@Library(["spicy-automation@main"]) _

// 1. VPC (2 AZs, minimal cost)
spicyVPC(
    jenkinsAwsCredentialsId: "aws-credentials",
    region: "ca-central-1",
    stackName: "test-vpc",
    ownerTag: "Test",
    productTag: "test",
    componentTag: "vpc",
    numberOfAzs: 2,                          // 2 AZs instead of 4 (saves 2 NAT Gateways)
    createAdditionalPrivateSubnets: false,   // Skip NACL subnets
)

// 2. Cluster (with Fargate enabled for cheaper option)
spicyECSCluster(
    jenkinsAwsCredentialsId: "aws-credentials",
    region: "ca-central-1",
    stackName: "test-cluster",
    vpcStackName: "test-vpc",                 // Auto-imports VPC ID, CIDR, subnets, numberOfAzs
    ownerTag: "Test",
    productTag: "test",
    componentTag: "cluster",
    environment: "test",

    // Option A: EC2 with Spot (cheaper than on-demand)
    instanceType: "t3.small",                // Minimal viable instance
    minClusterSize: 1,                       // Single instance
    maxClusterSize: 1,                       // No scaling
    spotEnabled: true,                       // Use Spot for 60-70% savings
    onDemandPercentage: 0,                   // 100% Spot
    enableFargate: true,                    // Enable Fargate capacity providers

    // Option B: Fargate-only (no EC2 costs)
    // enableFargate: true,
    // minClusterSize: 0,                    // No EC2 instances
    // maxClusterSize: 0,                    // No EC2 instances

    containerInsights: false,                // Disable Container Insights (saves CloudWatch costs)
    createExternalLoadBalancer: false,       // Service creates its own ALB
    createInternalLoadBalancer: false,
)

// 3. Service with Blue/Green (pipeline auto-creates ALB stack)
spicyECSService(
    jenkinsAwsCredentialsId: "aws-credentials",
    region: "ca-central-1",
    stackName: "test-service",               // Pipeline adds -blue/-green suffixes
    serviceName: "test-api",
    clusterName: "test-cluster",             // Auto-imports VPC from cluster stack
    vpcStackName: "test-vpc",                 // Auto-imports subnets, numberOfAzs from VPC stack

    // Container (minimal resources)
    image: "nexus.kodeniks.com/docker-hosted/test-api:latest",
    containerPort: 3000,
    desiredCount: 1,                         // Single task
    cpu: 256,                                // Minimal CPU (0.25 vCPU)
    memory: 512,                             // Minimal memory (512 MiB)

    // Use Fargate Spot if cluster has no EC2 instances (Option B above)
    // capacityProviderStrategy: [
    //     [capacityProvider: "FARGATE_SPOT", weight: 1],
    // ],

    // Blue/Green Configuration
    blueGreen: true,
    hostName: "api-test.example.com",        // Auto-generates active/inactive hostnames
    // bgHostedZoneId: "Z1234567890ABCDEF", // Route53 hosted zone (optional, for Route53 DNS)

    // ALB Configuration (pipeline auto-creates ALB stack if not exists)
    useClusterAlb: false,                    // Use individual ALB (not cluster ALB)
    albScheme: "internet-facing",
    certificateArn: "arn:aws:acm:ca-central-1:123456789:certificate/xxx",
    redirectHttpToHttps: true,
    healthCheckPath: "/health",

    // Tags
    ownerTag: "Test",
    productTag: "test",
    componentTag: "api",
    environment: "test",

    // Test hooks
    blueGreenTest: { args, buildInfo ->
        sh "curl -f https://${buildInfo.inactiveHostname}/health"
    },
    smokeTest: { args, buildInfo ->
        sh "curl -f https://${buildInfo.activeHostname}/health"
    },
)
```

**What happens:**

1. **VPC Stack**: Creates 2 AZs with 2 NAT Gateways (~$66/mo)
2. **Cluster Stack**: Creates 1 Spot instance (~$5-8/mo) OR 0 instances with Fargate (~$0/mo)
3. **ALB Stack**: Pipeline automatically creates `test-service-alb` stack (~$16/mo)
4. **Service Stack**: Pipeline creates `test-service-green` (inactive) referencing the ALB
5. **DNS**: Route53 records created automatically if `bgHostedZoneId` provided, OR manual Cloudflare setup (see below)
6. **Swap**: Pipeline swaps ALB listener rules (no DNS changes)

**Cost Breakdown:**

| Component                 | Cost (EC2)     | Cost (Fargate) |
| ------------------------- | -------------- | -------------- |
| VPC (2 AZs, 2 NAT)        | $66/mo         | $66/mo         |
| Cluster (1 Spot instance) | $5-8/mo        | $0/mo          |
| Individual ALB            | $16/mo         | $16/mo         |
| Service (1 task)          | Minimal        | $2-3/mo        |
| **Total**                 | **~$87-90/mo** | **~$82-84/mo** |

**Key Simplifications:**

- ✅ Uses `vpcStackName` and `clusterName` for auto-imports (no manual subnet IDs)
- ✅ Pipeline automatically creates ALB stack (`test-service-alb`)
- ✅ Uses `hostName` parameter (auto-generates active/inactive hostnames)
- ✅ Minimal resources: 1 Spot instance (or Fargate), 1 task, 256 CPU / 512 MB memory
- ✅ Route53 DNS managed automatically (if `bgHostedZoneId` provided)

## Cloudflare DNS Setup

If your DNS is hosted with Cloudflare (not Route53), you'll need to manually configure DNS records after deployment:

1. **Get ALB DNS name** from ALB stack outputs (ALB stack name is `{serviceStackName}-alb`):

   ```bash
   # Get ALB DNS name from ALB stack
   aws cloudformation describe-stacks \
     --stack-name test-service-alb \
     --query 'Stacks[0].Outputs[?OutputKey==`LoadBalancerDNS`].OutputValue' \
     --output text
   ```

   Or find it in the AWS Console: EC2 → Load Balancers → Your ALB → DNS name

2. **Create Cloudflare DNS records** (set once, never change):

   **Active hostname:**

   - **Type**: CNAME
   - **Name**: `api-test` (or your subdomain)
   - **Target**: `{alb-dns-name}` (e.g., `test-service-alb-123456789.ca-central-1.elb.amazonaws.com`)
   - **Proxy**: Enabled (orange cloud) for Cloudflare protection
   - **TTL**: Auto

   **Inactive hostname:**

   - **Name**: `inactive-api-test`
   - **Target**: Same ALB DNS name (both point to same ALB, routing is by hostname)

   **Note:** These DNS records point to the ALB and never change. The ALB handles routing internally based on the Host header.

3. **SSL/TLS Settings in Cloudflare**:
   - Set to **Full** or **Full (strict)** mode
   - This ensures HTTPS works end-to-end (Cloudflare → ALB → Service)

### How Blue/Green Works

```text
┌──────────────────────────────────────────────────────────┐
│                    Cloudflare DNS                        │
│  ┌──────────────────┐  ┌───────────────────────────┐     │
│  │ api-test.example │  │ inactive-api-test.example │     │
│  │     .com         │  │         .com              │     │
│  └────────┬─────────┘  └──────────────┬────────────┘     │
└───────────┼───────────────────────────┼──────────────────┘
            │                           │
            │  (Both point to same ALB) │
            └──────────────┬────────────┘
                           ▼
            ┌──────────────────────────────┐
            │   AWS ALB (Individual)       │
            │  ┌─────────────┐ ┌─────────┐ │
            │  │ Host: api-  │ │ Host:   │ │
            │  │ test...     │ │ inactive│ │
            │  │ → BLUE      │ │ → GREEN │ │
            │  └─────┬───────┘ └────┬────┘ │
            └────────┼──────────────┼──────┘
                     │              │
            ┌────────▼───────┐ ┌────▼──────────┐
            │  BLUE Service  │ │ GREEN Service │
            │  (Active)      │ │ (Inactive)    │
            └────────────────┘ └───────────────┘
```

**Deployment Flow:**

1. Deploy new version to inactive (GREEN) service
2. Test via `inactive-api-test.example.com` in Cloudflare
3. Pipeline swaps ALB hostname routing (BLUE ↔ GREEN)
4. Active traffic now goes to new version
5. Old version stays running for rollback window

**Important:** The hostname does NOT change during deployments. DNS records point to the ALB (set once, never change), and the ALB routes traffic based on the Host header. The pipeline swaps ALB listener rules (not DNS), so rollback is instant (~30 seconds).

## Rollback

The pipeline handles ALB routing swaps automatically. For manual rollback:

```groovy
@Library(["spicy-automation@main"]) _

spicyRollback(
    jenkinsAwsCredentialsId: "aws-credentials",
    region: "ca-central-1",
    stackName: "test-service",
    serviceName: "test-api",
    // ... same config as deploy ...
    activeHostname: "api-test.example.com",
    inactiveHostname: "inactive-api-test.example.com",
    // If using Route53:
    // bgHostedZoneId: "Z1234567890ABCDEF",
)
```

This swaps the hostname routing in the ALB (not DNS records), so rollback is instant (~30 seconds). Your DNS records (Route53 or Cloudflare) never change.

## Important Notes

1. **ACM Certificate**: You'll need an ACM certificate in the same region as your ALB

   - Request via AWS Console or CLI
   - Must validate domain ownership
   - Can use DNS validation (add CNAME to Cloudflare)

2. **Cloudflare Proxy**: Keep proxy enabled (orange cloud) for:

   - DDoS protection
   - CDN caching (if desired)
   - SSL termination at Cloudflare edge

3. **Hostname Routing**: The ALB routes by `Host` header, so both Cloudflare DNS records point to the same ALB DNS name

4. **Testing**: Use the inactive hostname for integration tests before swapping

5. **Fargate vs EC2**: For cheapest option, use Fargate-only (set `minClusterSize: 0` and `maxClusterSize: 0` in cluster, then use `FARGATE_SPOT` capacity provider in service)

## Cleanup

Remember to destroy test stacks when done:

```bash
# Via Jenkins or AWS CLI
aws cloudformation delete-stack --stack-name test-service-blue
aws cloudformation delete-stack --stack-name test-service-green
aws cloudformation delete-stack --stack-name test-service-alb
aws cloudformation delete-stack --stack-name test-cluster
aws cloudformation delete-stack --stack-name test-vpc
```

## Summary

**Barebones blue/green setup:**

- VPC: 2 AZs, 2 NAT Gateways
- Cluster: 1 Spot instance (or Fargate-only with 0 instances)
- Service: 1 task with blue/green deployment
- ALB: Individual internet-facing ALB (auto-created by pipeline)
- DNS: Route53 (automatic) or Cloudflare (manual)
- **Total: ~$87-90/month (EC2) or ~$82-84/month (Fargate)**