AWS Cost Optimization

Audit + fixes started 2026-06-30 (Jun 2026 bill ~$397/mo). Each item below is written as Issue seen → Problem → Fix. Sections: code changes, console/infra changes, deferred.

---

Code changes (done)

1. Email-parse cron ran daily

• Issue: refreshUtilityInvoicesDaily ran every day at 02:00 and made 2 Sonnet (Bedrock) calls per bundled utility sub.
• Problem: Utility invoices don't change intra-week, so the daily cadence burned Claude tokens for no new data — a large share of the Bedrock bill.
• Fix: Cron changed to every 5 days (0 2 */5 * *), both Sonnet calls kept. ~80% fewer email-parse Claude calls. — src/jobs/jobs/user-subscription-utility.job.ts

2. Empathy text generated on every bank sync

• Issue: generateBundulEmpathyText (a Sonnet call) ran inside recomputeAndStoreBundulSummary, which fires on every PLAID_DATA_SYNCED webhook.
• Problem: Plaid webhooks fire multiple times per day per connected user, so we paid for a Claude call many times daily to produce a headline no longer shown in the app.
• Fix: Removed the Claude call; kept a static fallback string and the rest of the summary metrics. The Plaid trigger/listener stay (they still refresh the report metrics). — src/bundul-report/bundul-report.service.ts

3. Every-minute Airtable OTP cron

• Issue: airtable-otp.job polled Airtable every minute (1,440 runs/day).
• Problem: The Airtable OTP flow is retired, so the poll was pure wasted churn on the always-on task.
• Fix: Deleted the job file and its registration. (AirtableOtpService was kept at the time, but has since been removed along with the rest of the Airtable integration.) — src/jobs/jobs/airtable-otp.job.ts, src/jobs/jobs.module.ts

4. Dead plaid-maintenance job

• Issue: plaid-maintenance.job had its @Cron commented out since 2026-03-25 (commit c175db3).
• Problem: Dead code — a scheduled job that never ran, cluttering the jobs module.
• Fix: Deleted the job file and its registration. PlaidMaintenanceService kept (used elsewhere). — src/jobs/jobs/plaid-maintenance.job.ts, src/jobs/jobs.module.ts

5. N+1 queries in the "load-all" crons

• Issue: Several daily crons looped over all records doing per-record DB lookups: refreshUtilityInvoicesDaily (2 findById per sub), one-sub-reminder (a findOne user per sub), onesub-reconcile (one live Passport API call per sub, fully sequential).
• Problem: N+1 query patterns cause CPU/RAM spikes at run time, which is what would otherwise force a larger (costlier) Fargate task.
• Fix:
  • refreshUtilityInvoicesDaily: two batched $in queries + filter UTILITY in the DB. — src/subscriptions/user-subscriptions/user-subscription.service.ts
  • one-sub-reminder: single batched $in user lookup. — src/jobs/jobs/one-sub-reminder.job.ts
  • onesub-reconcile: bounded concurrency (8). The live Passport call is kept — it's the only thing that catches external charge-amount drift. — src/jobs/jobs/onesub-reconcile.job.ts

6. Fargate provisioned RAM for Chrome that never runs

• Issue: Dockerfile pinned NODE_OPTIONS=--max_old_space_size=4096; dev task-def was cpu 2048 / memory 4096 — larger than prod (1024 / 3072).
• Problem: The 4 GB heap and large dev task exist only for the Puppeteer/Chrome automation, which is off (all services manual). Fargate bills on provisioned CPU/RAM 24/7, so we paid for idle headroom — and dev cost more than prod.
• Fix: Removed the runtime max_old_space_size cap; set dev task-def to 1024 / 3072 to match prod. — Dockerfile, task-definition.json
• Gotcha (found in CI): that heap flag was also carrying the compile step — removing it entirely made pnpm build OOM-abort (SIGABRT / exit 134) on the CI runner (local builds passed because dev machines have a bigger default heap). Corrected by scoping the 4 GB heap to just the build command (RUN NODE_OPTIONS=--max_old_space_size=4096 pnpm build) so runtime stays uncapped but the compile has its headroom.

7. Dev images accumulate in ECR forever

• Issue: Dev CI tagged images with the bare git SHA (${sha}), a permanent tag.
• Problem: Permanent tags never become untagged, so an untagged-expiry lifecycle rule can't reach them and they pile up (1176 images observed). They also can't be prefix-targeted safely because prod shares the same repo.
• Fix: Dev image tag changed to dev-${sha} so an ECR rule (tagPrefixList: ["dev"], keep last N) can expire old dev images without ever matching the prod bundul-prod-api tag. — .github/workflows/main.yml

---

Console / infra changes

8. CloudWatch logs kept forever

• Issue: Both /ecs/bundul-* log groups had no retention set.
• Problem: Logs accumulate and bill indefinitely — and they're redundant since logs also ship to BetterStack.
• Fix: Retention set to 3 days on both groups. (Kept the awslogs driver — it's the only place startup crashes are visible.) — done

9. Redundant load balancer for a retired service

• Issue: 3 ALBs exist: bundul-api-prod-lb, bundul-api-dev-lb, and bundul-integrated-billing-dev-lb.
• Problem: The integrated-billing service is retired, so its ALB (~$18/mo base) is pure waste.
• Fix: Delete bundul-integrated-billing-dev-lb after confirming ~0 traffic (Monitoring tab) and no DNS pointing at it. Keep the other two. — pending

10. Untagged ECR images

• Issue: ~450 untagged images from :latest/moving-tag churn.
• Problem: Superseded image layers billed as storage.
• Fix: Rule 1 (Untagged · expire · days-since-created ≥ 1) applied. Matches only untagged images; does not touch any tagged image. — done

11. Historical dev image pile

• Issue: ~700 existing raw-SHA dev images (1176 total in repo).
• Problem: They're tagged (so rule 1 skips them) and lack the dev- prefix (so the future rule 2 skips them) — they linger forever.
• Fix (two parts):
  • One-time manual cleanup of the existing raw-SHA images (Images tab → sort by Created at → keep bundul-prod-api + most recent, delete the rest): done — over 1,000 old images cleared (2026-07-02).
  • After the dev- tag change (item 7) deploys, add ECR rule 2: tagPrefixList: ["dev"], imageCountMoreThan: 10, expire — so the pile doesn't rebuild. — pending

---

Deferred

12. Prod task still oversized + ships unused Chrome

• Issue: Prod task-def is 1024 / 3072 and the image installs full Chrome + Xvfb, unused while automation is off.
• Problem: Provisioned RAM and image size we don't currently need.
• Why deferred: One-way door — shrinking prod RAM and stripping Chrome must be done together, and both must be reverted (bump RAM, re-add Chrome) if automation is turned back on. Do it deliberately, not as a quick edit.

13. NAT Gateway data cost (VPC line)

• Issue: VPC line ~$39/mo, almost entirely a NAT Gateway (base + per-GB data processing).
• Problem: All outbound traffic (S3, ECR, Secrets Manager, Bedrock, Plaid) routes through NAT and pays data-processing fees.
• Why deferred: Adding VPC endpoints for S3/ECR/Secrets/Bedrock to bypass NAT is a networking change — meaningful but larger scope; scheduled for later.

14. virtual-card-topup loads all users (monthly)

• Issue: topUpVirtualCardForAllUsers loads all users into memory once a month.
• Problem: Same load-all/N+1 spike pattern as the fixed crons.
• Why deferred: It's a payment-critical path and only monthly — left untouched until it can be refactored (cursor streaming) with proper care.