Skip to content

Conversation

@busser
Copy link

@busser busser commented Sep 4, 2025

Related Issue

Fixes #437

Description

This PR implements an ephemeral resource that works like the existing data source. This allows running external programs without storing the result in Terraform's state, addressing concerns with sensitive data handling (closes #437).

Our use-case is running an external program that outputs credentials for use by a provider. Storing a Terraform user's credentials in a remote state backend would be a problem. Here's the code this PR allows us to write:

provider "google" {
  credentials = local.credentials
}

locals {
  credentials = jsonencode(ephemeral.external.credentials.result)
}

ephemeral "external" "credentials" {
  program = [
	# Program that fetches credentials on the user's workstation
  ]
}

The ephemeral resource's implementation is heavily copied from the data source's to ensure that the behavior is the same. Any difference would be confusing for users and a pain to document and maintain.

The ephemeral resource's tests check the same scenarios as the data source's. I've actually added a test case to the data source, that checks that the working_dir attribute works as expected.

I've run this provider on our Terraform codebase and it behaves as desired.

If you have any changes you want me to make, I am happy to make them.

Rollback Plan

  • If a change needs to be reverted, we will roll out an update to the code within 7 days.

Changes to Security Controls

Are there any changes to security controls (access controls, encryption, logging) in this pull request? If so, explain.

No.

This PR implements an ephemeral resource that works like the existing
data source. This allows running external programs without storing the
result in Terraform's state, addressing concerns with sensitive data
handling (closes hashicorp#437).

Our use-case is running an external program that outputs credentials for
use by a provider. Storing a Terraform user's credentials in a remote
state backend would be a problem. Here's the code this PR allows us to
write:

```hcl
provider "google" {
  credentials = local.credentials
}

locals {
  credentials = jsonencode(ephemeral.external.credentials.result)
}

ephemeral "external" "credentials" {
  program = [
	# Program that fetches credentials on the user's workstation
  ]
}
```

The ephemeral resource's implementation is heavily copied from the data
source's to ensure that the behavior is the same. Any difference would
be confusing for users and a pain to document and maintain.

The ephemeral resource's tests check the same scenarios as the data
source's, but take a different approach to inspecting the program's
output. The data source's tests run code with `output` blocks, but that
doesn't work with ephemeral resources. I created a separate test program
for the new tests. This program writes its output to a file specified by
the test. After running Terraform, the tests can read this file to check
that the content is as expected.

I've run this provider on our Terraform codebase and it behaves as
desired.

If you have any changes you want me to make, I am happy to make them.
@busser busser requested a review from a team as a code owner September 4, 2025 17:12
@busser busser mentioned this pull request Sep 5, 2025
1 task
@busser
Copy link
Author

busser commented Sep 17, 2025

Hey @austinvalle, sorry for the ping. Any chance you can take a look at this PR? 🙏

@busser
Copy link
Author

busser commented Oct 13, 2025

Hey @austinvalle and @stephybun 👋 I'm still waiting on a response for this PR. Do you see any work needed to get this merged? I'm more than happy to make any changes you bring up :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ephemeral External Resource

1 participant