LLM API Key Protection

How OpenHands protects LLM API keys from agent access and exfiltration

This technical note explains how OpenHands protects LLM API keys—including LiteLLM virtual keys and Bring Your Own Key (BYOK) configurations—from being accessed or exfiltrated by the AI agent running in the sandbox.

Overview

When you use OpenHands, an AI agent executes code in a sandboxed environment. A natural security concern is: can the agent access and steal the LLM API key used to power it?

The short answer is no. OpenHands implements multiple layers of protection to ensure that LLM API keys are never exposed to the agent’s execution environment.

The LiteLLM Proxy Layer

Before diving into SDK-level protections, it’s important to understand the first layer of defense: the LiteLLM proxy.

How It Works

OpenHands routes all LLM API calls through a LiteLLM proxy server. This proxy holds the actual provider API keys (OpenAI, Anthropic, etc.) and issues virtual keys to users:

Component	Who Configures It	What It Holds
LiteLLM Proxy	OpenHands (SaaS) or Customer (Enterprise)	Master API keys for all LLM providers
Virtual Key	Generated per Organization/Personal Workspace	Reference to master key, with budget/usage tracking
Agent	N/A	Receives only the virtual key (if at all)

Master Key Configuration

• OpenHands SaaS: OpenHands configures master API keys in the LiteLLM proxy. Users never see or handle provider API keys directly.
• OpenHands Enterprise: Customers configure master API keys in their Helm values file or VM Installer. These keys are stored in the LiteLLM proxy, not in the application layer.

Virtual Keys Per Organization

When a user or organization is created, OpenHands generates a virtual key in LiteLLM:

# Simplified from enterprise/storage/lite_llm_manager.py

key = await LiteLlmManager._generate_key(
    client,
    keycloak_user_id,
    org_id,
    key_alias,
    max_budget,
)

# The virtual key is stored in user settings, NOT the master key
oss_settings.update({
    'agent_settings_diff': {
        'llm': {
            'model': get_default_litellm_model(),
            'api_key': key,  # Virtual key, not provider key
            'base_url': LITE_LLM_API_URL,  # Points to proxy
        }
    }
})

This virtual key:

• Cannot be used directly with provider APIs (OpenAI, Anthropic, etc.)
• Only works with the LiteLLM proxy that issued it
• Has budget limits enforced by the proxy
• Can be revoked without affecting other users

What About BYOK (Bring Your Own Key)?

When users provide their own API keys through the OpenHands settings UI, the behavior depends on the configuration:

BYOK Scenario	Goes Through LiteLLM?	Key Exposure
Custom base_url pointing to own LiteLLM	Yes (user’s proxy)	User’s proxy holds master key
Custom base_url pointing directly to provider	No	Key goes directly to provider
Only custom api_key (no custom base_url)	Yes (OpenHands proxy)	Key is passed to OpenHands LiteLLM proxy

In all cases, the API key (whether virtual or BYOK) is stored in Agent.llm.api_key and protected by the SDK-level mechanisms described below.

Architecture

OpenHands uses a split architecture where:

1. The Agent Server (Python process) holds sensitive credentials and makes LLM API calls
2. The Sandbox (isolated container or process) executes agent-requested commands without access to credentials

Protection Mechanisms

1. LLM API Key Isolation

The LLM’s api_key is stored in the Agent.llm.api_key field as a Pydantic SecretStr. This key is:

• Used only within the SDK’s Python process when making API calls via LiteLLM
• Never exported as an environment variable to the shell
• Never accessible via bash commands like echo $LLM_API_KEY

The agent cannot request to “print environment variables” or write code that reads the LLM API key because it simply doesn’t exist in the sandbox’s environment.

2. SESSION_API_KEY Stripping

The SESSION_API_KEY is a credential that grants access to user secrets via the OpenHands API. If an agent could read this, it could potentially access other sensitive data.

OpenHands explicitly strips this variable before any subprocess execution:

# From openhands-sdk/openhands/sdk/utils/command.py

_SENSITIVE_ENV_VARS = frozenset({"SESSION_API_KEY"})

def sanitized_env(env: Mapping[str, str] | None = None) -> dict[str, str]:
    """Return a copy of *env* with sanitized values.

    Sensitive environment variables (e.g., ``SESSION_API_KEY``) are stripped
    to prevent LLM-driven agents from accessing credentials via terminal
    commands.
    """
    base_env: dict[str, str]
    if env is None:
        base_env = dict(os.environ)
    else:
        base_env = dict(env)

    # Strip sensitive env vars to prevent agent access via bash commands
    for key in _SENSITIVE_ENV_VARS:
        base_env.pop(key, None)

    return base_env

This sanitized_env() function is called in:

• bash_service.py — before executing any bash command
• desktop_service.py — before starting desktop processes
• vscode_service.py — before launching VS Code
• skills_service.py — before running skill-related processes

3. Registered Secrets: On-Demand Injection with Masking

For secrets that are meant to be used by the agent (like GITHUB_TOKEN for git operations), OpenHands uses a controlled injection mechanism:

1. On-demand injection: Secrets are only added to the environment when the command text explicitly references them
2. Output masking: Any secret values that appear in command output are automatically replaced with <secret-hidden>

Understanding “Controlled”: LLM vs Agent Access

Registered secrets are accessible to the agent but hidden from the LLM:

Layer	Access to Secret Values
LLM (language model)	❌ Never sees actual values—masked as <secret-hidden> in conversation history
Agent (sandbox execution)	✅ Full access—can read, write to files, transmit over network

How it works: When a command outputs a secret value, it’s replaced with <secret-hidden> before being added to the conversation history. This prevents the secret from appearing in prompts sent to the LLM. However, the agent executing in the sandbox has full access to use the secret as needed.

Expected behavior: The agent will use registered secrets for legitimate tasks—writing to .git-credentials, including tokens in API headers, configuring services, etc. This is by design. Output masking keeps secrets out of conversation logs and the UI, but does not restrict how the agent uses them during execution.

Implementation Details

# From openhands-sdk/openhands/sdk/conversation/secret_registry.py

def get_secrets_as_env_vars(self, command: str) -> dict[str, str]:
    """Get secrets that should be exported as environment variables."""
    found_secrets = self.find_secrets_in_text(command)

    if not found_secrets:
        return {}

    env_vars = {}
    for key in found_secrets:
        source = self.secret_sources[key]
        value = source.get_value()
        if value:
            env_vars[key] = value
            # Track for masking
            self._exported_values[key] = value

    return env_vars

def mask_secrets_in_output(self, text: str) -> str:
    """Mask secret values in the given text."""
    masked_text = text
    for value in self._exported_values.values():
        masked_text = masked_text.replace(value, "<secret-hidden>")
    return masked_text

4. LookupSecret for Dynamic Tokens

For OAuth tokens and other credentials that may be refreshed, OpenHands uses LookupSecret which fetches tokens via authenticated HTTP requests at runtime:

# From openhands-sdk/openhands/sdk/secret/secrets.py

class LookupSecret(SecretSource):
    """A secret looked up from some external url"""

    url: str
    headers: dict[str, str] = Field(default_factory=dict)

    def get_value(self) -> str:
        response = httpx.get(self.url, headers=self.headers, timeout=30.0)
        response.raise_for_status()
        return response.text

This means tokens are never stored statically in the sandbox—they’re fetched fresh when needed, and the fetch URL/headers are also protected.

Security Testing

The SDK includes explicit security tests to verify these protections work:

# From tests/agent_server/test_terminal_service.py

@pytest.mark.asyncio
async def test_terminal_does_not_expose_session_api_key(bash_service, monkeypatch):
    """Verify SESSION_API_KEY is not accessible to bash commands.

    This is a security test: SESSION_API_KEY grants access to user secrets via
    the SaaS API. If an LLM-driven agent could read this env var via terminal
    commands, it could exfiltrate all user secrets.
    """
    secret_value = "super-secret-session-key-12345"
    monkeypatch.setenv("SESSION_API_KEY", secret_value)

    # Agent tries to read the env var
    request = ExecuteBashRequest(
        command='echo "SESSION_API_KEY=$SESSION_API_KEY"',
        cwd="/tmp",
    )
    command, task = await bash_service.start_bash_command(request)
    await task

    # The secret value should NOT appear in the output
    assert secret_value not in combined_stdout

What About BYOK (Bring Your Own Key)?

When users provide their own API keys through the OpenHands settings UI:

Secret Type	Exposed to LLM?	Exposed to Agent?	Notes
LLM API Key	❌ No	❌ No	Stored in Agent.llm.api_key, used only by SDK
LiteLLM Virtual Key	❌ No	❌ No	Same protection as direct API keys
GitHub/GitLab Tokens	❌ No	✅ Yes	Agent can use for git operations, write to files, etc.
Custom Secrets	❌ No	✅ Yes	Agent can use as needed for tasks

The LLM API key specifically is never injected into the agent environment, regardless of whether it’s a direct provider key, a LiteLLM virtual key, or an OpenHands-provided key.

Registered secrets (GitHub/GitLab tokens, custom secrets) are fully accessible to the agent by design—this is required for the agent to perform tasks like pushing code or calling APIs. Output masking ensures these values don’t appear in conversation history sent to the LLM.

Important: A user can instruct the agent to write secret values to files. While output masking prevents the secret from appearing in the tool call results sent to the LLM, once written to disk the agent could subsequently read the file and include its contents in a message, pass the value to an external LLM, or transmit it via network requests. The security mechanism is designed to protect secrets from the LLM—not from the end user who stored them originally. Users retain full control over their own secrets and can direct the agent to use them however they choose.

Potential Attack Vectors (and Mitigations)

Could an agent write a program to read env vars?

The agent could write Python code like:

import os
print(os.environ.get('LLM_API_KEY', 'not found'))

Mitigation: The variable doesn’t exist in the subprocess environment—it returns 'not found'.

Could an agent read the agent-server’s memory?

In theory, a malicious program could try to read /proc/<pid>/environ of the parent process.

Mitigation: The sandbox runs in an isolated container (Docker) with no access to the host’s process space. The agent-server process is outside the container.

Could an agent intercept LLM API calls?

The agent doesn’t make LLM calls—the agent server does. The agent only receives the LLM’s text responses, not the API request/response details.

Could secrets leak through error messages?

FastAPI validation errors could potentially echo back request bodies containing secrets.

Mitigation: OpenHands sanitizes all validation error responses:

# From openhands-agent-server/openhands/agent_server/api.py

def _sanitize_validation_errors(errors: Sequence[Any]) -> list[dict]:
    """Sanitize validation error details to remove sensitive input values."""
    sanitized: list[dict] = []
    for error in errors:
        error = dict(error)
        if "input" in error:
            error["input"] = sanitize_dict(error["input"])
        sanitized.append(error)
    return sanitized

Summary

OpenHands protects LLM API keys through defense-in-depth:

1. Architectural separation: Keys live in the agent server, not the sandbox
2. Environment stripping: Sensitive vars are removed before subprocess exec
3. On-demand injection: Only explicitly-needed secrets are injected
4. Output masking: Secret values are redacted from all output
5. Container isolation: Sandbox cannot access host process memory

This design ensures that even a malicious or manipulated agent cannot access or exfiltrate LLM API keys or LiteLLM virtual keys.

References

• OpenHands SDK Source Code
• Secret Registry Implementation
• Sanitized Environment Implementation
• Security Tests