Skip to content

Terraform Environment Compatibility

This page describes what the Tracker Terraform code needs in order to work cleanly in both staging and production when those environments live in different AWS accounts:

  • Staging account: 802732539686
  • Production account: 951665295205

The short version:

  • Staging can keep using the self-hosted data tier already present in modules/database.
  • Production should be treated as a separate root state in the production account.
  • Production must reuse the existing TimescaleDB cluster instead of creating a new database host.
  • Production can create a new managed Valkey/MemoryDB cluster in the production account's tracker VPC if that simplifies deployment.

1. Account Boundary

Terraform resources do not cross AWS account boundaries by default. That means each environment needs its own:

  • AWS provider configuration
  • remote backend
  • state file
  • KMS keys
  • Secrets Manager secrets
  • VPC, subnets, route tables, and security groups
  • ECS cluster and services
  • ALB, WAF, ACM certificates, and log groups

Nothing is shared between the staging and production accounts. Any resource that lives outside the current Terraform state should be treated as an external dependency and passed in as data, not created implicitly by the same state.

2. What Staging Owns Today

The current staging stack owns the full application tier and the full data tier:

  • ECS services and jobs
  • ALB and WAF
  • a project VPC
  • an EC2 host that runs PostgreSQL, TimescaleDB, PostGIS, and Valkey

That design is fine for staging because it is self-contained. It is not the right shape for production, where the database already exists elsewhere.

3. What Production Must Reuse

Production should reuse the existing TimescaleDB cluster in the production account and existing production VPC, rather than provisioning a new EC2 database host.

That implies Terraform must support:

  • a database endpoint that is supplied externally
  • a read/write port mapping that may point at PgBouncer or HAProxy
  • authentication credentials supplied from the production account
  • network reachability from the tracker VPC to the existing database VPC
  • the existing production TimescaleDB VPC CIDR and database security group for the peering link

If the live database sits behind HAProxy and PgBouncer, the safest integration point is usually the HAProxy/PgBouncer layer rather than the Patroni nodes themselves.

4. What Production Can Own for Valkey

Valkey is a separate decision from TimescaleDB.

Two valid production patterns are:

  1. Create a new managed AWS Valkey/MemoryDB cluster in the production account's tracker VPC.
  2. Reuse an existing managed cache only if the network and account ownership model make that practical.

The repo already contains a memorydb module, so Terraform can provision a managed Valkey cluster when the production root wires that module in.

5. Network Requirements for Cross-VPC Database Access

If the tracker VPC and the existing production TimescaleDB VPC are in different VPCs inside the production account, Terraform must account for the network hop explicitly.

At minimum, you need:

  • VPC peering or Transit Gateway connectivity between the VPCs
  • route table entries in the tracker VPC and in the existing production VPC, managed by the owning teams
  • security group or CIDR-based ingress rules that allow the tracker application to reach the database endpoint
  • DNS resolution for the database hostname if you want to keep using private names instead of raw IPs

In practice, the tracker Terraform can create the tracker-side peering attachment and route entry, but the existing production VPC route-table change still needs to be submitted to the team that owns that VPC.

The current Terraform modules are written around VPC-local resources. They do not yet model a cross-VPC database attachment as a first-class module input.

6. What Production Terraform Needs as Inputs

To make the production root truly environment-aware, the stack should accept these values as inputs rather than hardcoding staging assumptions:

  • account-specific provider profile / assume-role configuration
  • aws_region
  • vpc_cidr
  • public_hostname
  • admin_hostname
  • secret_names or secret ARNs for the production account
  • external PostgreSQL/TimescaleDB endpoint details for the existing production VPC, typically the HAProxy FQDN in front of TimescaleDB
  • external PostgreSQL read port if PgBouncer or HAProxy sits in front
  • a pre-created JSON Secrets Manager secret for the production database connection values, managed outside Terraform
  • that database secret must be encrypted with the production tracker CMK, because the ECS task execution role only gets kms:Decrypt on the tracker stack key
  • external Valkey endpoint details if the cache is not created by this stack
  • image tags for the account where the images are deployed
  • the existing production VPC ID and CIDR
  • the existing production TimescaleDB security group ID

7. App Runtime Expectations

The application and worker code already supports the main production requirements:

  • PostgreSQL connection settings are fully environment-driven
  • Redis/Valkey can run in standalone or cluster mode
  • TLS can be enabled for managed cache services

For production Valkey, the important runtime changes are typically:

  • REDIS_HOST points at the MemoryDB endpoint
  • REDIS_PORT is the cluster port
  • REDIS_CLUSTER_MODE=true
  • REDIS_TLS_ENABLED=true
  • REDIS_TLS_CERT_REQS is set to the level required by the managed service and your trust model

8. Current Gaps To Close

The current repo still has staging-first assumptions that need to be removed or parameterized before production is fully compliant:

  • there is no checked-in infra/envs/prod root yet
  • the current wiring still points cache variables at the self-hosted database host
  • the database module still assumes it owns the PostgreSQL and Valkey processes
  • the infrastructure docs currently describe the stack as if the data tier is always local

The cleanest long-term model is:

  • infra/envs/staging in the staging AWS account, self-contained and owning its own data tier
  • infra/envs/prod in the production AWS account, owning the tracker VPC, ECS, ALB, WAF, ECR, secrets, and managed cache
  • production TimescaleDB consumed as an existing dependency in the production account's existing VPC
  • production Valkey either created in-account by Terraform or reused explicitly as a managed dependency
  • bootstrap image tags default to sha-bootstrap, with an override file for later rollouts
  • temporary prod hostnames such as tracker.prod.glimpse.technology and tracker-admin.prod.glimpse.technology are used until switchover

That gives you a clear ownership boundary and keeps the two environments operationally independent.