Krish Dholakia
bfaab8ad7e
Merge pull request #14557 from timelfrink/fix/issue-14478-bedrock-count-tokens-endpoint
...
Implement AWS Bedrock CountTokens API support
2025-09-17 23:51:06 -07:00
Krish Dholakia
ff36dfdc76
Merge pull request #14637 from akraines/feature/middle-truncate-spend-logs
...
feat: implement middle-truncation for spend log payloads
2025-09-17 23:47:04 -07:00
Krish Dholakia
e7bc700b20
Merge pull request #14640 from Sameerlite/litellm_gardrail_default_latest_message
...
[Feat]Add last message as default in gaurdrail
2025-09-17 23:42:40 -07:00
Tim Elfrink
c234b13275
Apply code formatting and linting fixes
...
- Apply Black formatting to all Bedrock CountTokens files
- Clean up imports and remove unused variables in tests
- Fix indentation and simplify test structure
- Fix pyright type error with type ignore annotation
- All tests continue to pass after cleanup
2025-09-18 08:28:17 +02:00
Tim Elfrink
e74ac35b5d
Add comprehensive tests for Bedrock CountTokens functionality
...
- Add endpoint integration test in test_proxy_token_counter.py
- Add unit tests for transformation logic in bedrock/count_tokens/
- Test model extraction from request body vs endpoint path
- Test input format detection (converse vs invokeModel)
- Test request transformation from Anthropic to Bedrock format
- All tests follow existing codebase patterns and pass successfully
2025-09-18 08:16:56 +02:00
Sameerlite
edf95966c9
Handle consecutive user messages
2025-09-18 10:35:25 +05:30
Alexsander Hamir
8d96626044
fix: iscoroutine removed from hot path +50 RPS ( #14649 )
...
* fix: iscoroutine removed from hot path
* fix: replace all instances & separate concerns
1. Replaced all instances of iscoroutine with is_async_callable
2. Place the coroutine checker in its own file
* fix: PR comment changes
* fix: missing config setting declaration
* fix: revert non-performance related changes
* fix: revert to initial implementation
* fix: remove dead const
2025-09-17 19:01:43 -07:00
Krrish Dholakia
00d8dedd16
test: update test
2025-09-17 17:44:14 -07:00
Krish Dholakia
44e0c730b9
Merge pull request #14653 from ARajan1084/in-memory-guardrail-fix
...
fix: In Memory Guardrail fails to update
2025-09-17 17:42:23 -07:00
=
268b984cb6
added test for updating in memory guardrails
2025-09-17 17:16:49 -07:00
Mubashir Osmani
8b804303ed
fix: ci/cd tests + lint errors ( #14646 )
...
* fix: lint errors + tests
* fixed ci tests
* fixed tests
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2025-09-17 17:06:43 -07:00
Krish Dholakia
f51003538a
Merge pull request #14650 from BerriAI/litellm_dev_09_17_2025_p1
...
Bedrock Guardrails - support setting bedrock runtime endpoint + Protect `/health/test_connect` to prevent users without model creation permissions from calling it
2025-09-17 16:55:53 -07:00
Krrish Dholakia
1954bbffc0
test: fix test on ci/cd
2025-09-17 16:53:07 -07:00
Krish Dholakia
895c41efa3
Merge pull request #14619 from BerriAI/litellm_dev_09_16_2025_p1
...
UI - allow team member to view service account keys they create + Anthropic - include cache creation tokens in prompt token total (separate out during cost tracking)
2025-09-17 15:43:04 -07:00
Krrish Dholakia
530662d735
test: add unit testing
2025-09-17 14:54:44 -07:00
Sameer Kankute
69c01488bd
remove not needed names ( #14641 )
2025-09-17 14:26:48 -07:00
Krrish Dholakia
08ba38adeb
test: update unit test
2025-09-17 14:23:13 -07:00
Sameerlite
1371abf880
add last message as default in gaurdrail
2025-09-17 21:08:19 +05:30
Carlos Marchal
e168161e64
Feat/add posthog observability ( #14610 )
...
* feat: add posthog observability
* docs: add posthog logging docs
* docs: posthog integration in proxy mode
2025-09-17 08:24:04 -07:00
Akiva Kraines
115a3e9ded
feat: implement middle-truncation for spend log payloads
...
- Change truncation strategy from head-only to middle-truncation (35% start, 65% end)
- Preserve both beginning and end of long strings for better debugging context
- Apply same sanitization to response payloads when store_prompts_in_spend_logs is enabled
- Increase default MAX_STRING_LENGTH_PROMPT_IN_DB from 1000 to 2048 characters
- Update tests to verify new truncation behavior with 35%-65% split
This provides better diagnostic value by keeping the more important end context
while still maintaining storage limits.
2025-09-17 16:30:25 +03:00
Krish Dholakia
bf0dd4a284
Merge pull request #14418 from iabhi4/deep-copy-issue
...
fix: avoid deepcopy crash with non-pickleables in Gemini/Vertex
2025-09-16 22:55:31 -07:00
Krish Dholakia
2563c0ab2d
Merge pull request #14534 from hula-la/fix/response-api-bugfixes
...
fix: improve response api handling and cold storage configuration
2025-09-16 22:46:03 -07:00
Krish Dholakia
fcf84027e8
Merge pull request #14555 from mubashir1osmani/dd_spend_metric
...
DataDog shows spend metrics
2025-09-16 22:44:47 -07:00
Krish Dholakia
635dc72211
Merge pull request #14604 from Sameerlite/litellm_gemini_api_base_update
...
Litellm gemini api base update
2025-09-16 22:38:44 -07:00
Krish Dholakia
6bccdcd530
Merge pull request #14620 from BerriAI/litellm_dev_09_16_2025_p2
...
(feat) Anthropic - document pricing for cache creation tokens above 1hr
2025-09-16 22:33:11 -07:00
Krrish Dholakia
8273630e08
test: update tests
2025-09-16 22:30:40 -07:00
Mubashir Osmani
0fde408e35
added langfuse logging for responses api ( #14597 )
...
* added langfuse logging for responses api
* tests added
2025-09-16 21:43:19 -07:00
mubashir1osmani
e56b11cb54
added tests
2025-09-17 00:19:03 -04:00
Krrish Dholakia
0e747aaaf1
test: fix test
2025-09-16 19:20:12 -07:00
Ishaan Jaff
93524cf19b
[Feat] Batches - Add bedrock retrieve endpoint support ( #14618 )
...
* feat: add bedrock retrieve endpoint
* feat: feat: add bedrock retrieve endpoint
* test: batches mocked transform
* ruff fix
* refactor
* fix transform
* fix: parse_timestamp
2025-09-16 19:19:02 -07:00
Krrish Dholakia
22d3e492f9
fix(test_utils.py): fix test
2025-09-16 19:17:44 -07:00
Krrish Dholakia
e735808614
test(test_utils.py): fix cache creation tokens
2025-09-16 19:03:16 -07:00
Krrish Dholakia
1c855385c9
build(model_cost): add cache_creation_input_token_cost_above_1hr pricing
2025-09-16 18:43:57 -07:00
Krrish Dholakia
0341e7fc09
fix: fix test
2025-09-16 18:34:24 -07:00
Krrish Dholakia
e488312873
fix(utils.py): log cache_creation_tokens in prompt token details
...
Closes LIT-907
2025-09-16 18:24:10 -07:00
Krrish Dholakia
1162c52692
fix(anthropic/chat/transformation.py): include cache creation token count in final prompt token value
...
Closes LIT-907
2025-09-16 17:52:12 -07:00
Krrish Dholakia
32f634e305
test: add unit testing
2025-09-16 16:54:03 -07:00
Sameer Kankute
ab1fb2b2e7
Add Support for Bedrock Guardrails to supportive selective Guarding ( #14575 )
...
* Add Support for Bedrock Guardrails to supportive selective Guarding
* Add method for better handling
* Add guarded_text content type
* Add guarded_text content type
* Update Dockerfile
* Update Dockerfile
2025-09-16 16:43:34 -07:00
Ishaan Jaffer
7de8811c4c
test
2025-09-16 16:33:10 -07:00
Alexsander Hamir
02db2e8ae8
[Performance] RPS Improvement +500 RPS when sending the user field ( #14616 )
...
* perf tool
* fix: cache type issue
* fix: exception hanging & cache setting
1. Removed unhandled exceptions
2. Set cache value to dict
2025-09-16 16:18:23 -07:00
Ishaan Jaffer
68105ce1a7
fix type
2025-09-16 15:41:52 -07:00
Sameerlite
f08fc45a0f
add base url support for gemini
2025-09-16 15:15:24 +05:30
Krish Dholakia
f34bbd1f35
Merge pull request #14569 from talesofai/volcengine-fix-thinking-parameters-missing-if-not-set
...
fix volcengine thinking parameters missing when it set disable
2025-09-15 21:33:31 -07:00
Krish Dholakia
3f90fe938d
Merge pull request #14514 from iabhi4/fix-13581
...
feat(proxy): Assign default budget to auto-generated JWT teams
2025-09-15 21:22:25 -07:00
Krish Dholakia
f4e2870490
Merge pull request #14532 from timelfrink/feat/issue-14476-compactifai-provider
...
Add CompactifAI provider support
2025-09-15 21:15:00 -07:00
mubashir1osmani
f5f00043ce
fixed tests
2025-09-16 00:14:48 -04:00
LingXuanYin
f8c9009fe5
add more test
2025-09-16 12:11:10 +08:00
Krish Dholakia
784eb18366
Merge pull request #14558 from iabhi4/fix-14536
...
fix(proxy): Correctly parse multi-part MCP server aliases from URL paths
2025-09-15 21:05:22 -07:00
Ishaan Jaff
8e22cf5d65
[Fix] /responses API - add cancel endpoint + allow non-admins to use this as an llm api endpoint ( #14594 )
...
* fix: ensure /responses/cancel works for non admins
* test: cancel endpoint
* fix responses API cancel endpoint
* test fix
* TestGoogleAIStudioResponsesAPITest
2025-09-15 18:49:54 -07:00
Krish Dholakia
e1a6b9f858
Merge pull request #14582 from timelfrink/fix/issue-14573-aws-external-id-support
...
Add AWS external ID parameter support for Bedrock authentication
2025-09-15 17:32:00 -07:00
