litellm

mirror of https://github.com/BerriAI/litellm.git synced 2026-09-13 23:11:40 +00:00

History

Marty Sullivan 60fe4e464c fix(bedrock): resolve the managed-batch output bucket on the inline accounting path too A third path reads a completed batch's output file, and it could not resolve the bucket either. When cost is accounted from the retrieve itself rather than from the poller, the batch success handler calls _handle_completed_batch, which fetches the output file through _extract_file_access_credentials. That helper forwarded a whitelist covering Azure and Vertex, gcs_bucket_name included, but nothing for Bedrock, and retrieve_batch built its litellm_params through get_litellm_params, whose fixed signature drops the trusted credential snapshot. So the snapshot never reached the file read and it failed with "S3 bucket_name is required" for a bucket the deployment had configured, leaving the batch's cost unrecorded. Adding s3_bucket_name to that whitelist would not have worked. The Bedrock file config deliberately resolves the bucket only from the immutable server-side snapshot or the environment, never from a request param, because the bucket is what managed file ids are validated against. The snapshot is therefore what has to flow, exactly as it already does for the model-routed and cost-poller paths. retrieve_batch now re-adds the snapshot after get_litellm_params, the same way the file operations already do, the whitelist forwards it, and the proxy attaches it for router-routed managed batches from the deployment behind the unified id. Verified against a live proxy reading a real completed Bedrock batch: the cost row appears within seconds of the retrieve carrying the batch's real spend and usage, where before the read raised and no row was written. Resolving those credentials is best effort. A batch whose deployment no longer resolves, which happens when a model group is removed while batches are in flight, still serves its status instead of failing the request on the lookup. This matters for the OSS and polling-disabled configurations, where the retrieve path is the only thing that accounts for a batch at all.	2026-08-14 01:17:56 -04:00
..
__init__.py	test(batches): add 1:1 test file scaffold for batches component paths (#30529 )	2026-06-29 09:22:58 +05:30
test_endpoints.py	fix(bedrock): resolve the managed-batch output bucket on the inline accounting path too	2026-08-14 01:17:56 -04:00

Marty Sullivan 60fe4e464c fix(bedrock): resolve the managed-batch output bucket on the inline accounting path too

A third path reads a completed batch's output file, and it could not resolve the
bucket either. When cost is accounted from the retrieve itself rather than from
the poller, the batch success handler calls _handle_completed_batch, which fetches
the output file through _extract_file_access_credentials. That helper forwarded a
whitelist covering Azure and Vertex, gcs_bucket_name included, but nothing for
Bedrock, and retrieve_batch built its litellm_params through get_litellm_params,
whose fixed signature drops the trusted credential snapshot. So the snapshot never
reached the file read and it failed with "S3 bucket_name is required" for a bucket
the deployment had configured, leaving the batch's cost unrecorded.

Adding s3_bucket_name to that whitelist would not have worked. The Bedrock file
config deliberately resolves the bucket only from the immutable server-side
snapshot or the environment, never from a request param, because the bucket is
what managed file ids are validated against. The snapshot is therefore what has to
flow, exactly as it already does for the model-routed and cost-poller paths.

retrieve_batch now re-adds the snapshot after get_litellm_params, the same way the
file operations already do, the whitelist forwards it, and the proxy attaches it
for router-routed managed batches from the deployment behind the unified id.
Verified against a live proxy reading a real completed Bedrock batch: the cost row
appears within seconds of the retrieve carrying the batch's real spend and usage,
where before the read raised and no row was written.

Resolving those credentials is best effort. A batch whose deployment no longer
resolves, which happens when a model group is removed while batches are in
flight, still serves its status instead of failing the request on the lookup.
This matters for the OSS and polling-disabled configurations, where the retrieve
path is the only thing that accounts for a batch at all.

2026-08-14 01:17:56 -04:00

__init__.py

test(batches): add 1:1 test file scaffold for batches component paths (#30529 )

2026-06-29 09:22:58 +05:30

test_endpoints.py

fix(bedrock): resolve the managed-batch output bucket on the inline accounting path too

2026-08-14 01:17:56 -04:00