Skip to main content

Runbook: Bootstrap Repository (spwti-bootstrap)

What this runbook covers

The spwti-bootstrap repo is the Day 0 foundation — it creates the S3 remote state backend, GitHub OIDC provider, and all GitHub Actions IAM roles that other repos depend on. This runbook covers the initial setup, making changes after the fact, and troubleshooting.


What spwti-bootstrap Manages

ResourceAWS NameDescription
S3 bucketspwti-terraform-stateRemote Terraform state for all repos (shared)
DynamoDB tablespwti-terraform-state-lockTerraform state locking (shared, PAY_PER_REQUEST)
OIDC providertoken.actions.githubusercontent.comTrusts GitHub Actions to assume AWS IAM roles
IAM rolespwti-terraformAssumed by spwti-infra-* repos on main
IAM rolespwti-ecr-pushAssumed by app repos (any branch) for ECR pushes
IAM rolespwti-s3-deployAssumed by app repos (main only) for S3/CloudFront deploys
IAM policiesspwti-terraform-* (8 policies)Scoped permissions for spwti-terraform role

Why it is local-only (no CI pipeline)

spwti-bootstrap runs manually from a local machine using the temporary vendor-bootstrap admin identity. It must never have a CI pipeline because:

  1. A CI pipeline would need AWS credentials to run — creating a circular dependency (the pipeline would need the roles it's trying to create)
  2. No automated process should be able to modify its own OIDC trust policy (privilege escalation risk)
  3. It is a one-time setup — day-2 infrastructure changes belong in spwti-infra-platform

Prerequisites

Before running, ensure the following are in place on your local machine:

  • AWS CLI v2 installed (aws --version)
  • Terraform >= 1.5.0 installed (terraform version)
  • vendor-bootstrap AWS profile configured — see setup below
  • Access to GitHub org SPW-HEALTHCARE-INNOVATIONS-Pvt-Ltd
  • spwti-bootstrap repo cloned locally

One-Time Local Machine Setup

The vendor-bootstrap AWS CLI profile uses temporary STS session tokens (not static access keys). Terraform's Go SDK cannot resolve these automatically — you need to configure credential_process in ~/.aws/config so the SDK delegates credential resolution to the AWS CLI.

Add this to ~/.aws/config under the vendor-bootstrap profile:

[profile vendor-bootstrap]
credential_process = aws configure export-credentials --profile vendor-bootstrap --format process
login_session = arn:aws:iam::170420138919:user/vendor-bootstrap-admin-sparkops
region = ap-south-1

Verify it works:

# Should return account 170420138919
aws sts get-caller-identity --profile vendor-bootstrap
Why credential_process works

aws configure export-credentials reads directly from the AWS CLI's internal credential cache (bypassing credential_process itself — no circular loop). It outputs credentials in the JSON format that Terraform's Go SDK accepts via credential_process.


Initial Bootstrap (First Time Ever)

This is a two-phase process because of the chicken-and-egg problem: we need to create the S3 bucket before we can use it as a backend.

Phase A — Create Remote State Backend

cd spwti-bootstrap/

# Set the AWS profile via Terraform variable
$env:TF_VAR_aws_profile = "vendor-bootstrap"

# Initialise with local backend (backend.tf is commented out at this point)
terraform init

# Review — should show 5 resources: S3 bucket + versioning + encryption + public access block + DynamoDB
terraform plan

# Apply — creates spwti-terraform-state and spwti-terraform-state-lock
terraform apply

Expected output:

Apply complete! Resources: 5 added, 0 changed, 0 destroyed.

Phase B — Migrate State to S3

  1. Open backend.tf and uncomment the terraform { backend "s3" { ... } } block
  2. Run:
terraform init -migrate-state

When prompted Do you want to copy existing state to the new backend?, type yes.

Expected output:

Successfully configured the backend "s3"!

State is now at: s3://spwti-terraform-state/bootstrap/terraform.tfstate

Phase C — Create OIDC Provider and IAM Roles

# Review — should show OIDC provider + 3 IAM roles + 8 policies + policy attachments
terraform plan

# Apply
terraform apply

Expected output:

Apply complete! Resources: 14 added, 0 changed, 0 destroyed.

After apply — add role ARNs to GitHub Actions Variables:

RepoVariable nameValue
spwti-infra-platformTERRAFORM_ROLE_ARNarn:aws:iam::170420138919:role/spwti-terraform
ward-mitraECR_PUSH_ROLE_ARNarn:aws:iam::170420138919:role/spwti-ecr-push
ward-mitraS3_DEPLOY_ROLE_ARNarn:aws:iam::170420138919:role/spwti-s3-deploy

Making Changes After Initial Setup (Day-2)

State is already in S3. No migration needed — just plan and apply.

cd spwti-bootstrap/
$env:TF_VAR_aws_profile = "vendor-bootstrap"

# Always plan first and get approval before applying
terraform plan

# After approval:
terraform apply
Always get plan reviewed before applying

Changes to OIDC trust policies or IAM role permissions have immediate security impact. Never apply without a plan review.


Adding a New App Repo

When a new application repo is created (e.g. field-assist), update its trust in the ECR push and S3 deploy roles:

  1. Edit variables.tf in spwti-bootstrap:
variable "app_repo_names" {
type = list(string)
default = ["ward-mitra", "field-assist"] # add new repo here
}
  1. Run plan and apply:
$env:TF_VAR_aws_profile = "vendor-bootstrap"
terraform plan # should show 2 IAM role trust policy updates
terraform apply

The spwti-ecr-push and spwti-s3-deploy roles will now also trust the new repo.


Adding a New Infra Repo

No changes needed. The spwti-terraform role uses a wildcard trust:

repo:SPW-HEALTHCARE-INNOVATIONS-Pvt-Ltd/spwti-infra-*:ref:refs/heads/main

Any new repo prefixed spwti-infra- can immediately assume the spwti-terraform role from its main branch. Just add the TERRAFORM_ROLE_ARN GitHub Actions variable to the new repo.


Troubleshooting

"No valid credential sources found"

Error: No valid credential sources found
with provider["registry.terraform.io/hashicorp/aws"]

Cause: The credential_process entry is missing from ~/.aws/config for the vendor-bootstrap profile.

Fix: Add to ~/.aws/config:

[profile vendor-bootstrap]
credential_process = aws configure export-credentials --profile vendor-bootstrap --format process

"EntityAlreadyExists: Provider with url https://token.actions.githubusercontent.com already exists"

Cause: The GitHub OIDC provider was already created manually in the AWS account (or by a previous run).

Fix: Import the existing provider into Terraform state:

$env:TF_VAR_aws_profile = "vendor-bootstrap"
terraform import aws_iam_openid_connect_provider.github \
arn:aws:iam::170420138919:oidc-provider/token.actions.githubusercontent.com

Then re-run terraform plan — should show no changes or minor tag updates only.


"ExpiredTokenException" or "Request has expired"

Cause: The temporary STS credentials for vendor-bootstrap have expired (they typically last a few hours).

Fix: Re-authenticate using whatever method was used to obtain the original session (re-login with the vendor-bootstrap tool/portal). Then retry.


State Lock Error

Error acquiring the state lock

Cause: A previous Terraform run was interrupted and left a lock in DynamoDB.

Fix:

terraform force-unlock <LOCK_ID>

The lock ID is shown in the error message. Only force-unlock if you are certain no other Terraform process is running.


What NOT To Do

danger
  • Never run from CI/CD. This repo is local-only by design.
  • Never add AdministratorAccess to spwti-terraform. Use the 8 scoped policies — add new permissions there if needed.
  • Never add GitHub OIDC trust for spwti-bootstrap itself. The bootstrap repo is the chicken — it cannot use the eggs it lays.
  • Never delete spwti-terraform-state bucket or spwti-terraform-state-lock table. All Terraform state for all repos lives here. Deleting it is unrecoverable without state backups.
  • Never commit AWS credentials (access keys, session tokens) to this repo.

File Reference

FilePurpose
versions.tfTerraform >= 1.5.0, AWS provider ~> 5.0
providers.tfAWS provider config — region ap-south-1, profile from var.aws_profile
variables.tfInput variables: aws_profile, aws_region, github_org, app_repo_names, state bucket/table names
backend.tfS3 remote backend configuration
state_backend.tfaws_s3_bucket + aws_dynamodb_table resource definitions
oidc.tfaws_iam_openid_connect_provider for GitHub Actions
iam_roles.tf3 IAM roles with OIDC trust policies
iam_policies.tf8 scoped IAM managed policies for spwti-terraform role
data.tfaws_caller_identity, aws_region data sources
main.tfPlaceholder — future bootstrap resources go here

SparkOps Advisory Services · April 2026 · Confidential — SPW Healthcare Innovations Pvt. Ltd.