1. Docs
  2. Pulumi ESC
  3. Get started
  4. Retrieve secrets from external sources

Pulumi ESC: Retrieve Secrets from External Sources

Overview

With Pulumi ESC, you can safely store configuration values as plaintext or as secrets directly in the environment file. You also have the option to dynamically import secrets from external sources such as:

  • AWS Secrets Manager
  • Azure Key Vault
  • Google Secret Manager

This enables you to centralize both new and existing secrets and configurations and use them in multiple places without duplicating values across providers or environments.

Import external secrets

If you have not done so already, make sure you have configured OIDC connectivity between Pulumi and a supported provider listed below. During the configuration, you will need to make sure that you add the permissions necessary to interact with secrets in your chosen provider.

To retrieve secret values from AWS Secrets Manager, you must first:

Once that is complete, you will need to update your environment definition to add the aws-secrets provider configuration. To do this, add the following configuration to your environment definition, making sure to:

  • nest it under the aws second-level key
  • replace the value of <your-aws-region-here> with your secret’s region
  • replace the value of <your-secret-name-here> with the name of your secret
values:
  aws: # your existing oidc configuration
    login:
      ...
      ...
    secrets: # add the aws-secrets provider config
      fn::open::aws-secrets:
        region: <your-aws-region-here>
        login: ${aws.login}
        get:
          app-secret:
            secretId: <your-secret-name-here>
Copy

Your updated environment file should look similar to the following:

# example environment definition
values:
  aws:
    login:
      fn::open::aws-login:
        oidc:
          roleArn: arn:aws:iam::0123456789:role/esc-oidc
          sessionName: pulumi-environments-session
          duration: 1h
    secrets:
      fn::open::aws-secrets:
        region: us-east-1
        login: ${aws.login}
        get:
          app-secret:
            secretId: my-app-secret
Copy

You can validate this configuration by opening the environment via the ESC console, clicking the Open button and then clicking the Show secrets slider.

Alternatively, you can validate the configuration by opening the environment via the ESC CLI. Run the esc env open <your-org>/<your-project-name>/<your-environment-name> command, making sure to replace the values of <your-org>, <your-project-name>, and <your-environment-name> with the names of your Pulumi organization, ESC project, and environment respectively.

$ esc env open pulumi/demos/aws-secrets-example
{
  "aws": {
    "login": {
      "accessKeyId": "ASIA...",
      "secretAccessKey": "Jdt...",
      "sessionToken": "Fwo..."
    },
    "secrets": {
      "app-secret": "pulumi-esc-get-started-secret"
    }
  }
}
Copy

If you need to retrieve multiple Secrets Manager secrets, you can do so as shown below:

values:
  aws:
    login:
      ...
      ...
    secrets:
      fn::open::aws-secrets:
        region: us-east-1
        login: ${aws.login}
        get:
          app-secret:
            secretId: my-app-secret
          api-key: # additional AWS Secrets Manager secret
            secretId: my-api-key
Copy

To retrieve secret values from Azure Key Vault, you must first:

Once that is complete, you will need to update your environment definition to add the azure-secrets provider configuration. To do this, add the following configuration to your environment definition, making sure to:

  • nest it under the azure second-level key
  • replace the value of <your-vault-name-here> with the name of your vault
  • replace the value of <your-secret-name-here> with the name of your secret
values:
  azure: # your existing oidc configuration
    login:
      ...
      ...
    secrets: # add the azure-secrets provider config
      fn::open::azure-secrets:
        login: ${azure.login}
        vault: <your-vault-name-here>
        get:
          app-secret:
            name: <your-secret-name-here>
Copy

Your updated environment file should look similar to the following:

# example environment definition
values:
  azure:
    login:
      fn::open::azure-login:
        clientId: 99b...
        tenantId: 706...
        subscriptionId: 028...
        oidc: true
    secrets:
      fn::open::azure-secrets:
        login: ${azure.login}
        vault: pulumi-esc-vault
        get:
          app-secret:
            name: my-app-secret
Copy

You can validate this configuration by opening the environment via the ESC console, clicking the Open button and then clicking the Show secrets slider.

Alternatively, you can validate the configuration by opening the environment via the ESC CLI. Run the esc env open <your-org>/<your-project>/<your-environment-name> command, making sure to replace the values of <your-org>, <your-project-name>, and <your-environment-name> with the names of your Pulumi organization, ESC project, and environment respectively.

$ esc env open pulumi/demos/azure-secrets-example
{
  "azure": {
    "login": {
      "clientId": "99b...",
      "oidc": {
        "token": "eyJ..."
      },
      "subscriptionId": "028...",
      "tenantId": "706..."
    },
    "secrets": {
      "app-secret": "pulumi-esc-get-started-secret"
    }
  }
}
Copy

If you need to retrieve multiple Azure Key Vault secrets, you can do so as shown below:

values:
  azure:
    login:
      ...
      ...
    secrets:
      fn::open::azure-secrets:
        login: ${azure.login}
        vault: pulumi-esc-vault
        get:
          app-secret:
            name: my-app-secret
          api-key: # additional Azure Key Vault secret
            secretId: my-api-key
Copy

To retrieve secret values from Google Secret Manager, you must first:

Once that is complete, you will need to update your environment definition to add the gcp-secrets provider configuration. To do this, add the following configuration to your environment definition, making sure to:

  • nest it under the gcp second-level key
  • replace the value of <your-secret-name-here> with the name of your secret
values:
  gcp: # your existing oidc configuration
    login:
      ...
      ...
    secrets: # add the gcp-secrets provider config
      fn::open::gcp-secrets:
        login: ${gcp.login}
        access:
          app-secret:
            name: <your-secret-name-here>
Copy

Your updated environment file should look similar to the following:

# example environment definition
values:
  gcp:
    login:
      fn::open::gcp-login:
        project: 123456789
        oidc:
          workloadPoolId: pulumi-esc
          providerId: pulumi-esc
          serviceAccount: pulumi-esc@foo-bar-123456.iam.gserviceaccount.com
    secrets:
      fn::open::gcp-secrets:
        login: ${gcp.login}
        access:
          app-secret:
            name: my-app-secret
Copy

You can validate this configuration by opening the environment via the ESC console, clicking the Open button and then clicking the Show secrets slider.

Alternatively, you can validate the configuration by opening the environment via the ESC CLI. Run the esc env open <your-org>/<your-project-name>/<your-environment-name> command, making sure to replace the values of <your-org>, <your-project-name>, and <your-environment-name> with the names of your Pulumi organization, ESC project, and environment respectively.

$ esc env open pulumi/demos/gcp-secrets-example
{
  "gcp": {
    "login": {
      "accessToken": "ya29....",
      "expiry": "2024-01-25T12:52:15Z",
      "project": 123456789,
      "tokenType": "Bearer"
    },
    "secrets": {
      "app-secret": "pulumi-esc-get-started-secret"
    }
  }
}
Copy

If you need to retrieve multiple Google Secret Manager secrets, you can do so as shown below:

values:
  gcp:
    login:
      ...
      ...
    secrets:
      fn::open::gcp-secrets:
        login: ${gcp.login}
        access:
          api-key: # additional Google Secret Manager secret
            name: api-key
          app-secret:
            name: app-secret
Copy

Was this page helpful?

IDP Builder Course. Register Now.