mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-30 06:43:27 +00:00
57f05e56f6
* fix(bedrock): omit system prompt from audio transcription * fix(bedrock): report Converse token usage on audio transcription Bedrock transcriptions returned usage null and were billed at $0. Map Converse inputTokens, outputTokens and totalTokens into the transcription usage so spend uses the model's per-token price. input_token_details is optional because Converse does not split audio and text tokens. Also switch the updated request-shape test to rstest. * fix(bedrock): default transcription temperature to 0 and price ap-southeast-2 Voxtral regionally Voxtral wrapped some transcripts in quotes at its default sampling temperature. In direct Converse calls in ap-southeast-2, 0 of 48 calls at temperature 0 were framed against 7 of 72 at the default, and instruction wording made no clear difference, so the request now sends temperature 0 when the caller sets none, matching OpenAI's transcription default. Calls in ap-southeast-2 billed at the us-east-1 price. The cost map gains bedrock/ap-southeast-2 Voxtral rows with all four service tiers, the Bedrock transcription response carries the deployment region in its hidden params, and the transcription token cost uses the region-resolved model key. The bridge now builds the response with model_copy instead of assigning usage after construction, and the native wheel test checks the usage the compiled route returns. * fix(bedrock): forward transcription language, reject unsupported formats, and bill env and Tokyo/London regions get_optional_params_transcription had no Bedrock branch, so language, response_format and timestamp_granularities were dropped before the Rust transform ever saw them. Bedrock now forwards language, prompt and temperature, accepts response_format json or text (the only shapes it can return), and answers 400 for srt, vtt, verbose_json or timestamp_granularities unless drop_params is set. The billing region now falls back to AWS_REGION_NAME and then AWS_REGION when aws_region_name is unset, the same order the Rust resolver uses to pick the endpoint, so an env-configured region gets its regional price. Adds ap-northeast-1 and eu-west-2 Voxtral rows (standard, batch, flex and priority) from the AWS Bedrock offer files, the other two regions AWS serves Voxtral audio in. * fix(bedrock): send transcription temperature as a number and name the rejected param The proxy keeps multipart form values as strings, so -F temperature=0.2 reached Converse as "0.2" and Bedrock answered 400. The Bedrock mapper now converts temperature to a float and answers 400 (or drops it under drop_params) when it is not a finite number. get_optional_params_transcription copied prompt and temperature back from the raw call after the Bedrock branch ran, which undid that conversion and any drop. The Bedrock branch now owns every OpenAI transcription param. An unsupported param such as timestamp_granularities now gets a 400 that names it instead of the generic "user/encoding format" message. * refactor(bedrock): return unsupported transcription params as a value and raise at the param step * fix(bedrock): drop the retired max_tokens field from the regional voxtral price rows --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| _support | ||
| audio_tests | ||
| base_sdk_tests | ||
| benchmarks | ||
| code_coverage_tests | ||
| documentation_tests | ||
| e2e | ||
| e2e_harness | ||
| fixtures | ||
| harness_e2e | ||
| integration | ||
| rust-python-harness | ||
| test_litellm_rust | ||
| typing | ||
| unit | ||
| windows_tests | ||
| __init__.py | ||
| _fake_openai_endpoint_server.py | ||
| _flush_vcr_cache.py | ||
| _live_test_helpers.py | ||
| _master_key.py | ||
| _openai_record_replay_proxy.py | ||
| _process_helpers.py | ||
| _vcr_conftest_common.py | ||
| _vcr_redis_persister.py | ||
| _wait_helpers.py | ||
| _ws_vcr.py | ||
| AGENTS.md | ||
| capturing_transport.py | ||
| eval_swe_bench.py | ||
| fake_openai_endpoint.py | ||
| gettysburg.wav | ||
| large_text.py | ||
| openai_batch_completions.jsonl | ||
| pyrightconfig.json | ||
| README.MD | ||
| test_logging.conf | ||
| test_rust_python_harness.py | ||
| VCR_TESTING.md | ||
| white_100x100.png | ||
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/unit
This folder can only run mock tests.