Building scalable secrets management in hybrid cloud environments: Lessons from enterprise adoption – Source: www.csoonline.com

I’ll never forget the morning a few years ago, when a teammate accidentally pushed an AWS key to a public GitHub repo. It took less than 30 minutes before someone flagged the issue, and although we rotated the credentials quickly, that was our wake-up call. 

At the time, our organization was expanding into a hybrid cloud environment, with workloads running on AWS, Azure and on-prem infrastructure. Secrets were scattered across Jenkins, Kubernetes, CI/CD pipelines and dev laptops. No single tool or policy governed how secrets were stored or accessed. The sprawl was real, and it was risky.

That incident pushed us to take secrets management seriously. What followed was an 18-month journey to implement scalable, secure secrets management at the enterprise level. This is the story of how we did it, what we learned and what I’d recommend to anyone walking a similar path. We also realized that without a unified secrets management system, we were introducing audit gaps that could fail both internal controls and regulatory compliance reviews, something we couldn’t afford in a heavily regulated industry.

Why hybrid clouds magnify the secrets problem 

In today’s cloud-native world, developers spin up new apps, APIs and services with breathtaking speed. Each one requires access to sensitive information, such as database credentials, API keys and tokens, and each cloud provider has its method for managing these. AWS has Secrets Manager. Azure offers Key Vault. Kubernetes has its own in-cluster “Secrets” mechanism, which is essentially base64-encoded configuration data. 

This decentralized model worked until it didn’t. Secrets were duplicated across environments. Keys were hardcoded into scripts. Access logs were missing. There was no unified policy for rotation, and no alerting system in place when something was misused. Even worse, we had no real inventory of where secrets lived or how often they were updated, which meant that revoking a single credential could take hours and involve multiple teams. 

According to GitGuardian’s 2024 Secrets Sprawl report, 23 million secrets were leaked publicly in 2023 alone, and 70% of those were still valid months after being exposed. That stat stopped me cold. Our security posture had to change. 

The first decision was to consolidate secrets across clouds and environments. We needed a unified control plane. After evaluating cloud-native tools, we chose HashiCorp Vault for its multi-cloud flexibility, robust policy engine and enterprise support. We also piloted Akeyless, a SaaS-based alternative with zero-trust architecture and easier onboarding. Both brought strong capabilities, but we leaned on Vault for critical, tightly regulated workloads.

Lessons from integration: Identity, Kubernetes and CI/CD 

Choosing a secrets management tool is the easy part. Integrating it across an enterprise is where the work begins. 

We started with identity. Manual user provisioning was not an option. We integrated Vault with our SSO platform using OIDC and mapped groups to Vault policies based on least privilege. For machines, we used cloud-native IAM: AWS roles, Azure managed identities and Kubernetes service accounts. These identities became our new trust layer. 

For Kubernetes, we deployed the Vault Agent Injector. It injected secrets into pods at runtime without storing them in plaintext or hardcoding them into configs. Developers no longer needed to manage secrets manually; the platform handled them automatically. 

CI/CD was a bigger challenge. Secrets were embedded in GitLab CI variables, Jenkins configs and Terraform state files. We created Vault roles for each stage of our pipelines, scoped tightly to the environment. Build jobs authenticated to Vault using AppRole and retrieve secrets on the fly. We also created time-bound tokens for sensitive pipelines, further reducing the window of exposure. These measures became critical in protecting against supply chain attacks. 

The result? We eliminated over 90% of static secrets in pipelines. More importantly, we built trust into the workflow; developers didn’t need to break rules to ship fast. 

As a bonus, we piped Vault’s audit logs into our SIEM. Now, every secret request is logged, visible and correlated with user identity and system behavior. That visibility became critical during an incident months later, when a compromised service account was detected accessing secrets it shouldn’t have.

Culture shift: From manual to automated, from reactive to resilient 

Secrets management isn’t just a technical transformation; it’s a cultural one. Early on, developers resisted. They didn’t want to learn new tools or wait for access. So, we met them where they were: 

• Automated onboarding via Terraform 
• Self-service portals for teams to manage their namespaces 
• Secrets as environment variables injected during builds 

We also adopted rotation by default. Vault allowed us to issue short-lived, dynamic credentials for databases and cloud providers. Some secrets were valid for only 24 hours. That meant even if a leak occurred, the blast radius was small. 

This wasn’t just about security; it was about developer velocity. If secrets can be created, rotated and revoked in minutes without manual approval, teams move faster. 

Another lesson: plan for failure. Vault is a critical service. If its down, pipelines fail, apps crash and access to infrastructure grinds to a halt. We deployed it in HA mode, configured auto-unseal with AWS KMS and ran load tests regularly. After one too many outages caused by “noisy neighbor” workloads, we gave Vault its dedicated cluster. Problem solved.

We also developed a disaster recovery playbook. Backups were encrypted and tested quarterly. We simulated secret exfiltration events and practiced revoking tokens in real-time. That discipline made a difference during a real event when a partner system was compromised—our rotation policies and revocation workflows kicked in within minutes.

Secrets management is an investment

In a hybrid cloud world, where infrastructure spans data centers and clouds, secrets are ubiquitous and attackers are aware of this. You can’t secure what you can’t see, and you can’t scale what you can’t automate.

If there’s one takeaway from our journey, it’s this: secrets management is not a security project, it’s a platform investment. Get identity integration right. Prioritize automation. Enforce least privilege. And above all, make it easy for developers to do the right thing.

We didn’t get there overnight. But today, our secrets are rotated automatically, accessed securely, audited continuously and managed centrally. That peace of mind is worth every hour we spent getting there. As the complexity of cloud-native architectures increases, secrets management must evolve from a sidecar service to a foundational pillar of enterprise security.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?