Using Golang Client to Operate Vault

Hello everyone, currently we’re going to try using golang vault client to create Write and Read secrets from vault.

Image credits to https://unsplash.com/photos/3wPJxh-piRw

For a brief introduction in the recent software engineering development, Vault is widely used for managing and storing sensitive data (ie: secret key, db password, etc). Most application has its configuration in order to define things how the app will run, for example let say we have simple application and it depends on this config.json

{
  "app_name": "myApp",
  "port": 8000,
  "debug": false,
  "db_username": "app_db_user",
  "db_password": "this_is!@MyP@@sw0R00"
}

instead of saving the file to your repository in which making the db credentials “public-ly” available to everyone / people in your organization (as per your repository only available to your organization). We can put it into Vault, in which has a very restriction of who can access the vault.

The idea as an example of this writing is to change it into something like this:

1. Removing the sensitive configuration from your config.json
2. Have your Vault server ready
3. Store the “secrets” using app / manually (only who authorized to use / access to Vault)
4. The app can access / read the secrets

Let’s run the vault server locally using Docker image (https://hub.docker.com/_/vault/), we won’t be discussing a lot about how to host the vault server here.

docker run --cap-add=IPC_LOCK -e 'VAULT_LOCAL_CONFIG={"storage": {"file": {"path": "/vault/file"}}, "listener": [{"tcp": { "address": "0.0.0.0:8200", "tls_disable": true}}], "default_lease_ttl": "168h", "max_lease_ttl": "720h", "ui": true}' -p 8200:8200 hashicorp/vault server

After running, it will show us some logs like this

Keep in mind the Root Token

Let’s save the root token for now and testing later. Then, you can access your localhost:8200, it will need Token to “login” into the vault, use the Root Token, then you’re in

You can see cubbyhole and secrets. For simplicity let’s just take it for granted now that commonly your “sensitive data” will be put in the “secret” folder above.

https://developer.hashicorp.com/vault/docs/secrets/cubbyhole if you want to read more about cubbyhole.

Then, let’s create a simple Go application that will do these things:

1. It has a usingToken function that will perform Write, and Read into the vault using Root Token mechanism
2. It has usingAppRole function that will perform Read operation into the vault using AppRole mechanism

In order for us to do something within the vault, we need to be authorized. Using Root Token will give us like a “sudo” privilege and we can do anything.

While on app level especially production, it’s more recommended to use AppRole mechanism instead of using root token / sudo like. In short, the AppRole is a role that we set to be allowed doing spesific operation, in the spesific path. In this golang example, we’ll mostly use code based on the github readme (https://github.com/hashicorp/vault-client-go) with a slight modification for simplicity.

package main

import (
 "context"
 "log"
 "time"

 "github.com/hashicorp/vault-client-go"
 "github.com/hashicorp/vault-client-go/schema"
)

func initVaultClient() *vault.Client {
   // prepare a client with the given base address
   client, err := vault.New(
   vault.WithAddress("http://0.0.0.0:8200"),
   vault.WithRequestTimeout(10*time.Second),
 )
 
  if err != nil {
    log.Fatal(err)
  }

   return client
}

func main() {
  usingToken()
}

func usingToken() {
 cl := initVaultClient()
 // example using Root Token
 err := cl.SetToken("hvs.K5aUowkGeDwAyAgbo1mmcgGS") // THIS SHOULD NOT BE HARDCODED IN REAL APP, use os.Getenv() as example of more secured way of storing the token
 if err != nil {
  log.Fatal(err)
 }

 _, err = cl.Write(context.Background(), "/secret/data/myapp-secret", map[string]interface{}{
  "data": map[string]interface{}{
   "username":    "username456",
   "my_password": "wow this is amazing",
  },
 })

 if err != nil {
  log.Fatal(err)
 }

 log.Println("secret written successfully")

 resp, err := cl.Read(context.Background(), "/secret/data/myapp-secret")
 if err != nil {
  log.Fatal(err)
 }

 data, ok := resp.Data["data"].(map[string]interface{})
 if !ok {
  log.Fatal("not map interface")
 }

 if data["username"] != nil {
  log.Println("my username is", data["username"].(string))
 }

 if data["my_password"] != nil {
  log.Println("my_password is", data["my_password"].(string))
 }
}

These app does:

1. initiate vault client
2. set the token with root token
3. Write key value secrets to the vault with path of “/secret/data/myapp-secret” that will exist in the vault as “/secret/myapp-secret”
4. Read the value from the path (making sure it’s correct)

Vault SDK Read function will return response that is mostly map[string]interface data type, therefore we need to manually parse each of its data.

Run the application and we’ll see

PS C:\Users\Adit\Documents\Code\go\src\basic-vault> go run .\main.go
2023/06/09 17:50:08 secret written successfully
2023/06/09 17:50:08 my username is username456
2023/06/09 17:50:08 my_password is wow this is amazing

Checking in into the vault, and we’re seeing

(Ignore the timestamp, I’ve done several times locally). As you can see even though we Write using the path of “secret/data/myapp-secret”, it’s stored in “secret/myapp-secret”

Ok, using root token everything is working. But now let’s do more proper way of authenticating our application using App Role mechanism.

In order for us to use AppRole mechanism, we’ll need to conduct several steps:

1. Enable the AppRole authentication method (in the vault)
2. Create ACL Policy, this is to set what are allowed for the role we want to create
3. Create a role that is bounded to the ACL policy below with some ttl configurations
4. Generate a secret_id (like a password) for that role
5. You’ll get a role_id, and secret_id that the app can use to authenticate

step 1

by default path is “approle/” you can change it if you want

Creating ACL policy to create, read, and update for those path only

Creating “role” with the name of “myapp” with token_policy (ACL) of “myapp” based on previous picture. More detail on the parameter option can be seen here https://developer.hashicorp.com/vault/api-docs/auth/approle

Since we’re not defining the parameter of secret_id_ttl , then our secret_id (password like for a role) should be never expires.

Get the role_id

Generate (get) the secret_id

Then let’s write the function of using AppRole to test it out.

// the rest of main function same as before

func usingAppRole() {
 cl := initVaultClient()

 resp, err := cl.Auth.AppRoleLogin(
  context.Background(),
  schema.AppRoleLoginRequest{
   RoleId:   "60b94e80-0089-83c2-9e8f-6ec274526b47", // DONT Hardcode for real app, use Environment Var 
   SecretId: "eb3053ea-42d3-dc6c-3b22-9682ce20700e", // DONT Hardcode for real app, use Environment Var
  },
 )
 if err != nil {
  log.Fatal(err)
 }

 if err := cl.SetToken(resp.Auth.ClientToken); err != nil {
  log.Fatal(err)
 }

 secretResp, err := cl.Read(context.Background(), "/secret/data/myapp-secret")
 if err != nil {
  log.Fatal(err)
 }

 data, ok := secretResp.Data["data"].(map[string]interface{})
 if !ok {
  log.Fatal("not map interface")
 }

 if data["username"] != nil {
  log.Println("my username is", data["username"].(string))
 }

 if data["my_password"] != nil {
  log.Println("my_password is", data["my_password"].(string))
 }
}

Run the app and we’ll see

PS C:\Users\Adit\Documents\Code\go\src\basic-vault> go run .\main.go
2023/06/09 18:11:06 my username is username456
2023/06/09 18:11:06 my_password is wow this is amazing

There you go, now you can Store your secrets more safely in the Vault and fetch it rather than storing it “unsecurely” in the configuration file on your repository. Happy trying :)