Photo by regularguy.eth on Unsplash
π How-To Securely Work With Secrets During Development
Incorporating the features of enterprise-grade secret management systems into local development processes.
Table of contents
If you're working on any software projects that require talking to other services, chances are that you probably have to make use of secrets. The most common type of secrets you may have run into -- are passwords and API keys.
There are others as well, and Cyberark does a good job of defining a secret as a:
[...] private piece of information that acts as a key to unlock protected resources or sensitive information in tools, applications, containers, DevOps and cloud-native environments.
Some of the most common types of secrets include:
Privileged account credentials
Passwords
Certificates
SSH keys
API keys
Encryption keys
As you may have learned, perhaps the hard way through a security incident, secrets can't be treated like just any other type of data. Because of their ability to access resources; they must be handled with care and with security in mind.
π§ How Do You Store Secrets During Development?
If you've googled this question, you may have run into a couple of articles that suggest the following options:
π Provide Secrets as Shell Variables.
The assumption is that your code (i.e. run-server.sh
) is set up to look for a key as a shell variable. For example:
cat run-server.sh
#! /bin/bash
echo "OUTPUT: API Key is ${API_KEY}"
In this approach, you can provide the secret to your program at run-time by running: API_KEY=<my-api-key> run-server.sh
which provides the following output:
API_KEY=my-api-key ./run-server.sh
OUTPUT: API Key is my-api-key
βοΈThe drawback of this approach is that your key has now been included in plaintext, within your shell history. If an attacker were to ever compromise your machine, the secrets would be available by running a simple history
command.
1815 API_KEY=my-api-key ./run-server.sh
π² Provide Secrets as Environment Variables
For this approach, we do a bit better setting the secret as an environment variable, which our program can read from.
Setting the actual environment variable can be done in a number of ways, such as reading from a file on the system. (i.e. api-key-secret.txt
)
export API_KEY=$(cat api-key-secret.txt)
./run-server.sh
OUTPUT: API Key is my-env-var-api-key
While the secret is no longer printed out to shell history β , the secret is contained within a file that must now be secured appropriately βοΈ.
You need to ensure that the file containing the secret key isn't acidentally checked into Verison Control (i.e. Github)βοΈ.
π Use A
.env
File
Some programming languages recommend the pattern of using a dotenv file, where programs can be instructed to look up values from a specific file (i.e. development.env)
However, this approach has the same drawback as above.
π€ What's A Better Way To Store Secrets
A decent number of moons ago when I worked as a Security Engineer doing β¨ security β¨ things, one of the items in our portfolio was a Secret Management system named Conjur (coincidentally, also built by Cyberark).
As an enterprise-grade secret management solution, it had a lot of features such as audit logging and access control policies, but one of the coolest features was the ability to do dynamic secret retrieval.
Specifically, it meant that after engineers checked secrets in to Conjur, they could simply (1) refer to their secrets by their secret reference paths and (2) use the Conjur CLI to "wrap" the invocation of their programs, to pull those secrets at run-time.
I remember thinking that this was super cool, and wished I could do that for personal projects with my existing secret manager (1PW). Unfortunately, the main disqualifier for that at the time, outside of licensing costs, was the requirement to run and secure your own secrets server which wasn't really feasible for local development.
Well, fast forward about 5 years and things have now changed!
π€¨ What's A Better Way To Store Secrets in Development?
For me, the "development phase" in personal projects is the phase where I want to rapidly prototype my ideas. I don't want to get bogged down by tedious tasks, which is what secret management used to be. I could move fast by using env or shell variables, but I was only a single .gitignore
file mistake away from checking in the contents to version control.
I'd always hoped for a better way and thankfully a couple of weeks ago - I came across this fantastic blog post by 1Password when looking into how to access secrets programmatically.
https://blog.1password.com/1password-cli-2_0/
This blog post goes into using the 1Password CLI to essentially achieve what I wanted to do those years ago. It enables developers to:
Add secrets to their password vault
Get the references to those secrets
Fetch secrets dynamically at runtime.
How does it work?
Download & install the 1Password CLI + 1Pasword8
Add your secret(s) into 1Password. In my case my secret value for
my-api-key
is:secret-key
- Grab the references for the secrets you're interested in.
op item get my-api-key --format json
{
"id": "<<REDACTED>>",
"title": "my-api-key",
"version": 1,
"vault": {
"id": "<<REDACTED>>",
"name": "Blog-post-development"
},
"category": "API_CREDENTIAL",
"last_edited_by": "[...]",
"created_at": "2023-02-26T23:14:43Z",
"updated_at": "2023-02-26T23:14:43Z",
"fields": [
[...]
{
"id": "credential",
"type": "CONCEALED",
"label": "credential",
"value": "secret-key",
"reference": "op://Blog-post-development/my-api-key/credential"
},
Which in this case is: op://Blog-post-development/my-api-key/credential
- Include the secret reference and wrap your execution command with
op run
Before, I used to have a Makefile to run my server:
cat Makefile
run:
./run-server.sh
Now I simply wrap the run
target with op run
and my secret reference as follows:
cat Makefile
run:
API_KEY="op://Blog-post-development/my-api-key/credential" \
op run ./run-server.sh
When I execute with make run
will first:
- Prompt me for a fingerprint (via touchID on Mac, or master password on all others); and
- Retrieve the secret for use in the command.
make run
API_KEY="op://Blog-post-development/my-api-key/credential" \
op run ./run-server.sh
OUTPUT: API Key is <concealed by 1Password>
Note: that 1PW's CLI is smart / safe enough to conceal the password. If you want to override this option and display the secret, you can run the same command with the --no-masking
flag.
For example:
cat Makefile
run:
API_KEY="op://Blog-post-development/my-api-key/credential" \
op run --no-masking ./run-server.sh
make run
API_KEY="op://Blog-post-development/my-api-key/credential" \
op run --no-masking ./run-server.sh
OUTPUT: API Key is secret-key
Why is this better?
Secrets are stored and retrieved from a single place. β The single source of truth for your secret's value is in 1Password. This means that if you need to read this secret from multiple places, you only need to update it once. It also means that if your secret gets "popped", you only need to change or "rotate" it in one place.
Uses a password or biometric to authenticate access. β If you're using a mac, you'll be prompted for TouchID. Else, you'll be prompted for your master password anywhere else.
Secrets are not stored in code. β You don't have to worry about accidentally checking in secrets to version control (i.e. Github).
You don't have to worry about the security of where your secrets are stored. β That's managed by an organization that specializes in secret storage and has a vested interest security of your secrets (1PW).
You don't need to do any other lift β to get this working. No need to set up IAM with AWS or GCP, uses 1Password.
Further Extensibility
1Password can group secrets into vaults which means you can also extend this to your colleagues & partners. When you add a secret to a vault and add them as members to your vault, they can also use secrets in the same manner from their 1password accounts.
1Password also has other shell-plugins to securely authenticate with services right from your shell.
There's also a way to integrate CI/CD as well.
Conclusion
Overall, I am extremely excited to see that these sorts of features are slowly trickling down and becoming more accessible for more usecases. The more we do to make security "easier", the better that software becomes for all.