ADR-0032: Secret providers: references resolved in memory, at the point of use¶
- Status: Accepted (2026-11-03). Built: the
secret_providercontract 0.1 inods-sdk, its conformance suite, the built-inenv:provider,SecretResolver, the fake inods-provider-fake, andOds::secret_providerfor custom builds, listed byods version,ods plugin listandods doctor. Not yet: a consumer (the first is Databricks authentication, #297, ADR-0021), a doctor check that references resolve, out-of-process secret providers (§5), and the dashboard's About page listing secret providers. - Date: 2026-11-03
- Issues: #126
- Deciders: @n1ckyb
Context¶
AGENTS.md rule 9: secrets are referenced, never persisted in configuration, state,
events or logs. ADR-0005 §4 made every credential in ods.toml a reference,
{ secret = "<scheme>:<name>" }, rejected plaintext (ODS-E0103), and left to #126
which schemes exist and how a reference becomes a value.
Nothing resolves a reference yet. The first code that must is Databricks authentication (#297, ADR-0021): a personal access token or a service principal's client secret, read from where the user keeps it, used for one HTTP request, and never written anywhere. #126 asks for environment variables (which is also how CI systems inject secrets), OS keychains, and cloud secret stores through plugins, with a provider-neutral contract and redaction in logs and errors.
Forces:
- Neutrality (rule 1): core and modules must not know Vault from a keychain. The
scheme in a reference is the only thing that picks a store.
- Conservative defaults (rule 3): a missing or empty secret must never be taken for
a credential, and an unknown scheme must never fall back to another store.
- I/O: keychains and cloud stores are I/O; AGENTS.md keeps async to I/O boundaries.
- Leaks: a value that reaches Debug, a log line, an error message, JSON output or
the state store is a leak. Engines and servers sometimes echo a credential back in
an error.
Options considered¶
Option A — Resolve in ods-config while loading¶
Pros: one place; every consumer gets values. Cons: the loaded Config would hold
secrets, which ADR-0005 rules out (it is shown by ods config explain, fingerprinted
and served by the dashboard); every command would read every store, even those it
doesn't need, and a locked keychain would break ods --help.
Option B — A contract in ods-sdk; resolve at the point of use (chosen)¶
A SecretProvider per scheme; the host resolves a reference only where a credential is
used, and holds the value only for that use. Pros: configuration stays reference-only;
stores are pluggable like every other provider; nothing is read that isn't used. Cons:
each consumer must ask, and handle the error.
Option C — Shell out to a helper (credential_process-style) for every scheme¶
Pros: any language. Cons: every secret crosses a process boundary and the plugin
protocol, which needs its own trust model (ADR-0031 §5); too much for the built-in
env: case. Kept as a later addition (§5).
For the value type: a hand-written wrapper, re-exporting secrecy's types, or a
newtype over secrecy::SecretString. The newtype keeps a 0.x third-party type out of
the SDK's public API while getting zeroize-on-drop and a redacting Debug from a
maintained crate (secrecy and zeroize are Apache-2.0 OR MIT).
Decision¶
1. The contract¶
ods_sdk::contracts::secrets, contract secret_provider 0.1 (SDK 0.10):
#[async_trait]
pub trait SecretProvider: Provider {
fn scheme(&self) -> &str;
async fn resolve(&self, reference: &SecretRef) -> Result<SecretValue, SecretError>;
}
- One provider serves one scheme, as
SecretRef::is_valid_schemeallows (lowercase letters, digits,-), and the scheme never changes. - Every provider must refuse an empty value (below); the resolver (§3) refuses one too, so an empty credential is never handed out whatever a provider does.
resolveonly reads. Async, because stores are I/O.- A reference with another scheme is
UnknownScheme; a missing secret isNotFound; one that exists but is empty isEmpty, never anOkwith an empty value. The store refusing isDenied; a store that can't be reached isUnavailable(retryable); a value that can't be used (not text) isInvalid. SecretErrormessages name the reference, assecret(<scheme>:<name>), never the value. The free-textreasonofUnavailableandInvalidis the provider's own, and keeping the value out of it is the provider's obligation. It converts intoProviderError(Unavailablestays retryable), so a provider that needs a credential can use?.
2. The value¶
SecretValue wraps secrecy::SecretString: wiped from memory when dropped; Debug
prints SecretValue([redacted]); no Display, Serialize or Clone (compile-fail
doctests keep it so), so it can't be printed, written out or copied by accident, and
is shared with Arc. expose() returns the text for the one place that needs it.
redact_in(text) replaces every occurrence of the value in a message, with
[secret removed], for errors that echo a credential (an HTTP 401 body, an engine
message).
3. Resolution¶
ods_sdk::secrets::SecretResolver maps each scheme to its one provider (a BTreeMap;
registering a second provider for a scheme, or an invalid scheme, is an error, so which
store is read never depends on registration order). resolve sends a reference to its
scheme's provider; an unknown scheme is UnknownScheme listing the schemes served. It
keeps no values: every call asks the provider again, so a rotated secret is picked up
and no value outlives its caller.
A host resolves a reference only where a credential is used, holds the SecretValue
for that use, and drops it. Resolved values never enter Config, state, run events,
the journal, logs, command output or the dashboard. ADR-0031's rule stands: a plugin is
never handed a resolved secret. How a warehouse plugin's own requests get a credential
(e.g. the Databricks provider's token for the SQL Statement API) is decided with #297,
in ADR-0021, within this ADR's rules.
4. Schemes¶
envis built in (EnvSecrets), reading the environmentodswas started with. It is how CI injects secrets (GitHub Actions secrets, GitLab CI variables, …), so CI needs nothing else. The host passes the environment in, so tests stay hermetic.- Keychains and cloud stores (
keychain:,vault:,aws-sm:, …) come from in-process plugins: a custom build adds one withOds::secret_provider(origin!(), provider), whichods version,ods plugin listandods doctor(capabilities.plugins) list assecret_providerplugins. ODS ships none: each OS keychain and each cloud SDK is a large, platform-specific dependency, and the contract lets them live outside the released binary.envis reserved: no plugin can serve it, even in a build without the built-in.
5. Not now: out-of-process secret providers¶
A secrets request on the script protocol (ADR-0031 §5) would let a plugin in any
language serve a scheme, but a secret would then cross a process boundary, and the
protocol, its trust (ods health trust) and its records were designed never to carry
one. It needs its own decision before it is built.
Consequences¶
- Positive:
- Configuration, state and output stay reference-only, as ADR-0005 decided; values exist only in memory, only while used.
-
297 can resolve
tokenandclient_secretwith one call and one error type.¶ - Third parties can add any store without a change to ODS, and run the conformance
suite (
ods_sdk::conformance::secrets) against it, asods-provider-fakedoes. - Negative / trade-offs:
- Each consumer must resolve, and handle
SecretError, itself. - The value is still in process memory while used; zeroize-on-drop shortens that,
but a core dump or a debugger can see it.
env:values are also in the process environment, andEnvSecretskeeps its copy of the variables, not zeroized, for as long as the resolver lives; only theSecretValues it hands out are wiped. redact_inonly removes exact occurrences; an engine that echoes a transformed credential (e.g. base64 in a header dump) isn't caught. Messages from engines also pass throughods_core::redact(rule 9), which removes quoted literals.- Follow-up issues:
-
297: the first consumer (Databricks PAT and M2M client secret).¶
- A doctor check that every configured reference resolves, without showing values; it needs commands to be handed the environment.
- The dashboard's About page listing secret providers.
- Out-of-process secret providers (§5).
Threat model¶
| Where a value could leak | Why it doesn't |
|---|---|
ods.toml, ods config explain, the dashboard's Settings |
Only references are allowed (ADR-0005 §4); resolution never writes back. |
| State store, run events, journal, ledger | Nothing resolved is passed to them; they take configuration, which holds references. |
Debug, tracing fields, panics |
SecretValue's Debug is redacted; it has no Display. |
| Error messages | SecretError names references only; a provider's free-text reasons are its obligation; redact_in for echoed values; conformance checks a provider's errors and info() never hold the value. |
| Child processes (dbt) | ODS passes dbt the environment it was given and adds no secret (ADR-0021 keeps dbt's own auth). |
| Plugins | Handed references, never values (ADR-0031). |