Secrets Management in CI/CD Pipelines for Solo Federal App Developers
- kate frese
- May 26
- 7 min read
Executive Summary
Secrets—API keys, database passwords, encryption keys, tokens—are the crown jewels of your application. If they leak, an attacker has direct access to your data and systems. Yet many solo developers and small teams manage secrets carelessly: hardcoding them in source code, storing them in plaintext config files, or sharing them via email.
NIST 800-53 control SC-7 (Boundary Protection) and SC-12 (Cryptographic Key Management) require that secrets be protected, rotated, and never exposed. For developers working in CI/CD pipelines (where code is automatically tested and deployed), secrets management is especially critical because the pipeline itself is an attack surface.
This white paper explains how to implement secrets management in a CI/CD pipeline in a way that’s secure, auditable, and practical for solo developers and small teams.
Scope & Assumptions
Team size: Solo developer to 5-person team.
Application scope: Web apps, APIs, mobile app backends.
Environment: Federal (FISMA, FedRAMP, or agency-specific compliance).
CI/CD platform: GitHub Actions, GitLab CI, Jenkins, or cloud-native (AWS CodePipeline, Azure Pipelines, Google Cloud Build).
Constraints: Limited DevOps resources, no dedicated security team.
Threat & Failure Modes
Without proper secrets management, several risks emerge:
Hardcoded secrets in source code: A developer commits an API key to GitHub. The code is public (or becomes public). An attacker finds the key and uses it to access your systems. Even if you delete the key, it’s in the git history forever.
Secrets in environment variables (unencrypted): CI/CD logs are often visible to team members. If secrets are logged as plaintext environment variables, anyone with access to logs can see them.
Secrets in config files: A developer stores a database password in a config file. The file is checked into version control or copied to a server. An attacker who gains access to the server can read the password.
Secrets shared via email or Slack: A developer shares a password with a colleague via email. The email is forwarded, forwarded again, and eventually ends up in someone’s inbox who shouldn’t have it.
Secrets never rotated: A developer uses the same API key for years. If the key is ever compromised, the attacker has long-term access.
No audit trail: If a secret is leaked, you don’t know when, how, or who accessed it.
Secrets management solves all of these problems.
Architecture Pattern: Secrets Management in CI/CD
Components
Secrets vault (centralized storage)
A dedicated service that stores secrets encrypted at rest.
Examples: AWS Secrets Manager, Azure Key Vault, HashiCorp Vault, or cloud-native options.
Secrets are never stored in source code, config files, or environment variables.
Access control (least privilege)
Only the CI/CD pipeline (and authorized humans) can retrieve secrets.
Each secret has an access policy: “This secret can only be accessed by the CI/CD pipeline for the production deployment.”
Access is logged: every time a secret is retrieved, it’s recorded.
Secret injection (runtime retrieval)
At runtime, the CI/CD pipeline retrieves secrets from the vault and injects them as environment variables.
Secrets are never logged or printed; they’re passed directly to the application.
After the job completes, secrets are cleared from memory.
Rotation (periodic refresh)
Secrets are rotated on a schedule (e.g., every 90 days for API keys, every 30 days for passwords).
Rotation is automated: the vault generates a new secret, updates the application, and invalidates the old secret.
Old secrets are retained in the audit log but are no longer usable.
Audit logging (accountability)
Every secret access is logged: who accessed it, when, from where, and why.
Logs are immutable and retained for compliance.
Logs are reviewed regularly for anomalies.
Data Flow
Developer pushes code to GitHub
↓
CI/CD pipeline triggers
↓
Pipeline requests secrets from vault
↓
Vault checks permissions
├─ DENIED → Log attempt, fail job
└─ ALLOWED → Retrieve encrypted secret, decrypt, return to pipeline
↓
Pipeline injects secret as environment variable
↓
Application starts, reads environment variable
↓
Application uses secret (e.g., connects to database)
↓
Job completes, secrets are cleared from memory
↓
Vault logs the access
NIST 800-53 Control Mapping
Evidence & Audit Artifacts
When an auditor asks “how do you manage secrets?”, here’s what you produce:
Secrets management policy document
What secrets are managed (API keys, database passwords, encryption keys, tokens).
Where secrets are stored (vault name, region, encryption).
Who can access secrets (CI/CD pipeline, specific team members).
How secrets are rotated (schedule, process, automation).
How secrets are retired (when no longer needed, how they’re invalidated).
Vault configuration
Screenshots or config files showing vault is encrypted at rest.
Access control policies (who can read, write, rotate secrets).
Example: “Only the CI/CD pipeline service account can read the production database password.”
CI/CD pipeline configuration
Configuration file (GitHub Actions YAML, GitLab CI YAML, Jenkinsfile, etc.) showing how secrets are retrieved and injected.
Example: uses: aws-actions/configure-aws-credentials@v1 with secret retrieval.
Proof that secrets are never logged or printed.
Secrets access logs
50–100 log entries showing when secrets were accessed, by whom, and why.
Example: “2026-05-26 14:22:00 - CI/CD pipeline retrieved ‘prod-db-password’ for deployment job #1234.”
Redact sensitive data but keep enough detail to show the pattern.
Key rotation evidence
Schedule showing when keys are rotated (e.g., “API keys rotated quarterly”).
Evidence of past rotations: timestamps, old key IDs, new key IDs.
Automation proof: runbook or script showing rotation is automated.
Access control policy
Who has permission to read each secret.
Who has permission to rotate secrets.
Who has permission to retire secrets.
Example: “Only the CI/CD service account (arn:aws:iam::123456789:role/cicd-pipeline) can read the production database password.”
Incident response documentation
Runbook: “If a secret is compromised, do this.”
Evidence of past incidents: when a secret was compromised, how it was detected, what actions were taken.
Example: “2026-05-20 - Developer accidentally committed API key to GitHub. Detected via automated scanning. Key was rotated immediately. No unauthorized access detected in logs.”
Encryption configuration
Evidence that secrets are encrypted at rest (e.g., AWS Secrets Manager uses AWS KMS encryption).
Evidence that secrets are encrypted in transit (TLS/HTTPS).
Key management: how encryption keys are stored and rotated.
Implementation Checklist (Solo-Friendly)
Identify all secrets
List every secret your application uses: database passwords, API keys, encryption keys, tokens, certificates.
Categorize by sensitivity: critical (database passwords), high (API keys), medium (tokens).
Document where each secret is currently stored (hardcoded? config file? environment variable?).
Choose a secrets vault
If on AWS: use AWS Secrets Manager.
If on Azure: use Azure Key Vault.
If on Google Cloud: use Google Cloud Secret Manager.
If self-hosted: use HashiCorp Vault or a similar solution.
Ensure the vault supports encryption at rest, access control, audit logging, and rotation.
Set up the vault
Create a vault instance.
Enable encryption at rest (usually default).
Enable audit logging.
Create a service account for the CI/CD pipeline.
Grant the service account permission to read (but not write or delete) secrets.
Migrate secrets to the vault
For each secret, create an entry in the vault.
Remove the secret from source code, config files, and environment variables.
Update your application to read secrets from the vault at runtime.
Test: deploy the application and verify it can read secrets from the vault.
Configure CI/CD pipeline to retrieve secrets
Update your CI/CD configuration (GitHub Actions YAML, GitLab CI YAML, etc.) to retrieve secrets from the vault.
Example (GitHub Actions):
name: Retrieve secrets
uses: aws-actions/configure-aws-credentials@v1
with:
role-to-assume: arn:aws:iam::123456789:role/cicd-pipeline
name: Get database password
run: |
DB_PASSWORD=$(aws secretsmanager get-secret-value --secret-id prod-db-password --query SecretString --output text)
echo "::add-mask::$DB_PASSWORD"
echo "DB_PASSWORD=$DB_PASSWORD" >> $GITHUB_ENV
Ensure secrets are masked in logs (use ::add-mask:: in GitHub Actions, @sensitive in GitLab CI, etc.).
Set up secret rotation
For each secret, define a rotation schedule (e.g., API keys every 90 days, passwords every 30 days).
If the vault supports automatic rotation, enable it.
If not, create a scheduled job (cron, Lambda, Cloud Function) that rotates secrets.
Test rotation: rotate a secret and verify the application still works.
Enable audit logging
Ensure the vault logs all access: who accessed which secret, when, and why.
Configure log retention (e.g., 1 year for federal compliance).
Ensure logs are immutable (append-only).
Document the process
Write a 1-page secrets management policy: what secrets are managed, where, how they’re rotated, who can access them.
Write a 1-page runbook: “How to add a new secret,” “How to rotate a secret,” “How to retire a secret,” “How to investigate a compromised secret.”
Test the system
Deploy the application and verify it can read secrets from the vault.
Verify secrets are never logged or printed.
Verify secrets are masked in CI/CD logs.
Simulate a secret rotation and verify the application still works.
Simulate a secret compromise: rotate the secret and verify old secret is invalidated.
Set up monitoring and alerting
Monitor for suspicious access patterns: “If a secret is accessed 10+ times in 1 minute, alert.”
Monitor for failed access attempts: “If access to a secret is denied 5+ times, alert.”
Configure alerts to notify security team.
Establish a review cadence
Weekly: review secrets access logs for anomalies.
Monthly: review rotation schedule; ensure all secrets are rotated on time.
Quarterly: review access control policies; ensure only necessary people/services have access.
Prepare audit artifacts
Compile policy, runbook, vault config, CI/CD config, logs, rotation evidence into a folder.
Create a summary: “Secrets Management Implementation Overview” (1 page).
Train your team
Brief team on the new secrets management process.
Provide runbooks and examples.
Emphasize: never hardcode secrets, never commit secrets to version control, always use the vault.
Plan for scale
As your application grows, you’ll have more secrets.
Monitor vault performance (latency, throughput).
Plan for growth: “At current rate, we’ll need to increase vault capacity in 6 months.”
Prepare for audit
2–3 weeks before audit, compile all artifacts.
Do a dry run: walk through as if you’re the auditor.
Brief your team on the scope and timeline.
Conclusion
Secrets management isn’t optional in federal development—it’s a requirement. A well-designed secrets management system takes a few days to set up and minutes per week to maintain. The payoff is confidence that your secrets are protected, rotated, and auditable.
If you’re building or evaluating a federal app, see bluevioletapps.com for tools and templates that make secrets management straightforward.
BlueVioletApps LLC is an independent software company. This content is not affiliated with, endorsed by, or produced on behalf of the U.S. Navy, Department of Defense, NAVSUP, or any federal agency. Google LLC is not affiliated with this content.



Comments