EC2 `ModifyImageAttribute` rejects "Character sets beyond ASCII are not
supported" when registering the AMI description. The whole AMI rolls back
on this error. Replace em-dash with hyphen.
LIT-2878
Built by `packer build` against the BYOC PoC account (us-west-2). Customers
running their own BYOC account should re-run the Packer build and replace
this value.
LIT-2878
`creds` was reassigned from `AwsCreds` to `Optional[AwsCreds]` in the
fallback branch — rename the second binding so mypy can narrow the type.
LIT-2878
Avoids the PytestUnknownMarkWarning when collecting
`tests/test_litellm/proxy/agent_session_endpoints/vm_providers/test_ec2_provider_real.py`
without `-m slow`.
LIT-2878
The cnf-update-db post-invoke hook fails (exit 100) when we install a
non-default python3 alongside, because cnf-update-db imports apt_pkg which
is bound to /usr/bin/python3 -> python3.12. Disabling the hook makes apt
update + install idempotent for AMI builds.
Also stop remapping /usr/bin/python3 via update-alternatives — it breaks
Ubuntu's python-coupled apt tooling. Tools that need 3.13 invoke it
explicitly; the systemd unit already uses /usr/bin/python3.13.
LIT-2878
Validations #3 (real boot), #11 (real BYOC fail-fast), and the real-cloud
piece of #8. Skipped by default; enable with `pytest -m slow` when the
`LITELLM_AGENT_AWS_*` + `LITELLM_TEST_*` env vars are set.
Each test wraps RunInstances in try/finally with TerminateInstances and
installs a 60-min process watchdog (per the AWS safety boundary) so a
hung test cannot leak an instance.
LIT-2878
Mocked Prisma client driving each sweeper through its happy path plus the
optimistic-lock branch (sweeper skips a session whose status changed
underneath it).
LIT-2878
Covers validation #11 (no creds = fail-fast, no instance launched), #12
(two teams resolve to two distinct creds objects), and the env-var fallback
path used in local dev before the LiteLLM_AgentVMConfig table is populated.
LIT-2878
Covers: prerequisites (Packer + AWS profile), `packer init` + `packer
build` invocation, sharing the AMI cross-account via `ami_users`, the
`LITELLM_AGENT_MODE` boot-mode contract, the Epic C migration path, and
the leak-cleanup one-liner that targets `LitellmManagedBy=agent-vm-provider`.
LIT-2878
Behaviour:
- reads runtime config from systemd's EnvironmentFile
- session mode: bootstrap → heartbeat every 30s, exit cleanly on HTTP 410
- warm mode: idle (real warm-pool hydrate lands in B2)
- redacts JWT in log output
Zero non-system deps (only `requests`, installed by the AMI builder).
Replaced wholesale by Epic C.
LIT-2878
Reads runtime config from /etc/litellm-agent/runtime.env (written by
EC2 user-data, mode 600). Restart=on-failure with a 5s backoff so transient
network blips during bootstrap don't permanently kill the session.
LIT-2878
Provisioner script driven by `litellm-agent-runtime.pkr.hcl`. Each tool is
pinned to a specific version and verified against a SHA-256 sidecar where
upstream provides one (uv) — see CLAUDE.md "CI Supply-Chain Safety".
The bun installer has no checksum sidecar, so we pin a version and pull the
artifact directly (not the install script).
LIT-2878
Builds an Ubuntu 24.04 AMI with node 24, python 3.13, git, gh, uv, bun, and
the agent-runtime systemd unit autostarted on boot. The daemon honours
`LITELLM_AGENT_MODE` from EC2 user-data: `session` for cold-boot,
`warm` for warm-pool prewarming (B2).
Uses IMDSv2 only. Tags every resource Packer creates for easy cleanup.
`ami_users` lets us share the AMI cross-account without rebuilding.
Validation #2 (`packer build`) covers this file.
LIT-2878
Documents the agent_settings YAML shape consumed by `get_vm_provider`.
B0's AWS resource IDs are referenced via `default_ami_id: ami-CHANGEME`;
the user fills in the real AMI after running `packer build`.
LIT-2878
Three sweepers run on the same 30s tick:
- bootstrap_timeout — sessions stuck in `provisioning` past the timeout
- heartbeat_timeout — `ready` sessions whose daemon stopped checking in
- max_session_minutes — sessions older than the configured ceiling
Each sweeper:
- bounds its batch to 100 rows so a backlog doesn't stall the loop
- re-fetches the row (optimistic lock) before terminating so multiple
proxy replicas don't double-terminate
- treats terminate failures as non-fatal (retry next tick)
Uses Prisma model methods (`find_many` / `find_unique` / `update`); no
raw SQL per project rules.
Validations covered: #7 (max_session_minutes), #9 (bootstrap_timeout), #10
(heartbeat_loss).
LIT-2878
One EC2 per session, launched in the team's AWS account using the team's
BYOC creds (decrypted at use, never logged). Spot first, on-demand fallback
when capacity unavailable. Per-session tags (litellm-session-id,
litellm-team-id, litellm-agent-id) for cleanup.
Safety:
- `set_stream_logger('botocore', WARNING)` so SigV4 payloads can't leak the
access key into proxy logs (regression-tested in #13)
- creds enter via ProvisionContext, never leave this module
- `_safe_aws_error` formats ClientError without echoing the request payload
- `InvalidClientTokenId` / `SignatureDoesNotMatch` → fail-fast InvalidCredentialsError
- terminate is idempotent on already-gone instances
Validations covered: #5 (spot fallback), #8 (terminate idempotent), #11
(invalid creds fail-fast), #13 (creds never logged).
LIT-2878
Reads the team's BYOC AWS creds from `LiteLLM_AgentVMConfig` (decrypts
each field individually) and falls back to `LITELLM_AGENT_AWS_*` env vars
for local dev. Raises `InvalidCredentialsError` if neither path yields
creds (validation #11 fail-fast).
Falls back gracefully when the table doesn't exist yet (Epic G hasn't
shipped its migration), so this code can land before LIT-2891.
LIT-2878
Reads `agent_settings.vm_provider` and `agent_settings.<provider>` from
the loaded proxy config and builds the matching provider. Defaults to
`noop`. Unknown values raise `ValueError` listing the supported
providers so config typos surface fast.
Validation #1 (test_factory) covers this path. Validation #6 (provider
swap is config-only) is also exercised here.
LIT-2878
In-memory `AgentVMProvider` used by the unit tests and as the default when
`agent_settings.vm_provider` is unset. The factory returns `NoopProvider`
when no AWS-backed provider is configured so the proxy boots cleanly without
AWS credentials.
LIT-2878
The pluggable VM-provider ABC for agent sessions. Per-session VMs are
provisioned via this interface; v1 implementation is EC2 (BYOC AWS).
Key types: `ProvisionContext` carries the team's AWS creds + EC2 overrides
through to the provider. `AwsCreds.__repr__` redacts secrets so we cannot
accidentally print them. `InvalidCredentialsError` (400) and
`ProvisionError` (500) are the user-facing error types.
LIT-2878
Mirrors the schema in the published `litellm-proxy-extras` package so the
bundled migrations match what Prisma actually applies on proxy startup.
LIT-2878
Per-team BYOC AWS config consumed by the agent-session EC2 VM provider.
`aws_creds_enc` is a JSON blob with each field individually encrypted via
`encrypt_value_helper` so a partial DB leak doesn't expose the secret.
Owned by Epic G's Settings UI (LIT-2891); consumed by Epic B's EC2 provider.
LIT-2878
LIT-2890 / B2's long-poll heartbeat calls find_worker_by_jwt on every
request, which filters by worker_jwt_hash. Without an index that's a
full table scan per heartbeat once the worker pool grows past a handful
of rows. Adding the index now so it lands with the table create rather
than as a follow-up migration during B2 ramp-up.
Covers all four resolution paths and the security-critical default:
* LITELLM_CLOUD_AGENT_PROXY_BASE_URL wins regardless of forwarded
headers (and trailing slashes get stripped).
* X-Forwarded-Host is IGNORED unless LITELLM_TRUST_PROXY_HEADERS=1.
A forged header pointing at attacker.example must not show up.
* Trust-flag opt-in honors X-Forwarded-Host + X-Forwarded-Proto,
but x-forwarded-proto alone (no host) doesn't poison the response.
* Last-resort fallback to localhost:4000 when no Host header.
Resolution order for the URL embedded in the install one-liner is now:
1. LITELLM_CLOUD_AGENT_PROXY_BASE_URL env var (operator-configured,
fully trusted) — recommended for production.
2. X-Forwarded-Host / X-Forwarded-Proto, ONLY when the operator
opts in via LITELLM_TRUST_PROXY_HEADERS=1.
3. The request's direct Host header — safe by default because it
reflects the actual TCP destination, not an attacker-supplied hop.
Previously any authenticated caller could forge X-Forwarded-Host to
embed an attacker-controlled URL in the install command. If a second
operator ran that command, the worker would send its raw pair token to
the attacker's host, who could then call POST /v2/agent-workers/register
and gain a long-lived worker JWT.
Also adds structured logging on /register failures (invalid / replayed
/ expired tokens) so operators running the proxy behind a WAF / fail2ban
can detect abuse at the network layer (the proxy itself doesn't ship a
built-in per-IP limiter).
Locks the production-safe default for LITELLM_CLOUD_AGENT_MOCK_AWS so a
future revert to the unsafe "1" default trips CI. Also covers the
strict-string parsing — typos like "true" / "yes" must NOT silently
flip on the mock.
Previously the test-connection endpoint defaulted to mock-on, so a fresh
production proxy would silently return a synthetic success for any
non-empty AWS access key. Operators saving incorrect credentials would
only discover the failure later when VMs failed to launch.
Default is now "0" — operators must set LITELLM_CLOUD_AGENT_MOCK_AWS=1
explicitly to opt into the mock path during local development.
Also adds an inline comment on _build_update_payload's `is not None`
guard so a future contributor doesn't silently drop `False` / `0`
updates by switching to truthy comparison.
Adds three regression cases that fail if a future change drops the
shlex.quote() pass on proxy_url, raw_token, or install_script_url. The
existing simple-input cases still pass unchanged because shlex.quote
returns alnum/colon/slash/dot strings verbatim.
shlex.quote() the install_script_url, proxy_url, and raw_token before
interpolating into the curl-pipe-sh one-liner. Without quoting, a
proxy_url containing spaces or shell metacharacters (e.g. via a misconfig
or the X-Forwarded-Host issue Greptile also flagged) could produce a
malformed or exploitable command on the worker box.
