Commit graph

6832 commits

Author SHA1 Message Date
Sameer Kankute
d213a2e066
correct the gaurdcontent name (#14684)
* correct the gaurdcontent name

* correct the gaurdcontent name

* fix model required error in test

* Add correct model
2025-09-18 11:00:19 -07:00
Fabrício Ceschin
cb3bed1d96
Merge branch 'BerriAI:main' into LangfuseUsageDetails 2025-09-18 09:38:52 -04:00
Adam Tao
9b0eaf0989 test: add a testcase for cached_tokens
Signed-off-by: Adam Tao <tcx4c70@gmail.com>
2025-09-18 18:40:11 +08:00
Michael Tan
6721189b7c
feat: add Asia/Bangkok timezone support for budget resets
- Add Asia/Bangkok (UTC+7) to timezone_map in duration_parser.py
- Update documentation to include Bangkok in common timezone values
- Add test case to verify Bangkok timezone functionality
2025-09-18 14:56:12 +07:00
Tim Elfrink
27a37ce57e Use requestMetadata parameter name for consistency with other Bedrock params
- Change parameter from request_metadata to requestMetadata to match camelCase convention
- Consistent with guardrailConfig and performanceConfig naming pattern
- Update all references in transformation code and error messages
- Update tests and documentation to use correct parameter name
- Fix type checking for parameter validation
2025-09-18 09:21:45 +02:00
Tim Elfrink
1d85623b66 Add comprehensive tests for Bedrock Converse requestMetadata support
- Test requestMetadata parameter support in get_supported_openai_params
- Test transformation to top-level field in Bedrock API request
- Test validation of AWS constraints: max 16 items, key/value length limits
- Test character set validation for keys and values
- Cover edge cases including empty values and special characters
- Ensure compatibility with existing test patterns
2025-09-18 09:21:45 +02:00
Krish Dholakia
bfaab8ad7e
Merge pull request #14557 from timelfrink/fix/issue-14478-bedrock-count-tokens-endpoint
Implement AWS Bedrock CountTokens API support
2025-09-17 23:51:06 -07:00
Krish Dholakia
e7bc700b20
Merge pull request #14640 from Sameerlite/litellm_gardrail_default_latest_message
[Feat]Add last message as default in gaurdrail
2025-09-17 23:42:40 -07:00
Tim Elfrink
c234b13275 Apply code formatting and linting fixes
- Apply Black formatting to all Bedrock CountTokens files
- Clean up imports and remove unused variables in tests
- Fix indentation and simplify test structure
- Fix pyright type error with type ignore annotation
- All tests continue to pass after cleanup
2025-09-18 08:28:17 +02:00
Tim Elfrink
e74ac35b5d Add comprehensive tests for Bedrock CountTokens functionality
- Add endpoint integration test in test_proxy_token_counter.py
- Add unit tests for transformation logic in bedrock/count_tokens/
- Test model extraction from request body vs endpoint path
- Test input format detection (converse vs invokeModel)
- Test request transformation from Anthropic to Bedrock format
- All tests follow existing codebase patterns and pass successfully
2025-09-18 08:16:56 +02:00
Sameerlite
edf95966c9 Handle consecutive user messages 2025-09-18 10:35:25 +05:30
Alexsander Hamir
8d96626044
fix: iscoroutine removed from hot path +50 RPS (#14649)
* fix: iscoroutine removed from hot path

* fix: replace all instances & separate concerns

1. Replaced all instances of iscoroutine with is_async_callable
2. Place the coroutine checker in its own file

* fix: PR comment changes

* fix: missing config setting declaration

* fix: revert non-performance related changes

* fix: revert to initial implementation

* fix: remove dead const
2025-09-17 19:01:43 -07:00
Krrish Dholakia
7a1f723751 test: update test 2025-09-17 18:00:59 -07:00
=
40fa415c89 added test for updating in memory guardrails 2025-09-17 18:00:59 -07:00
Krrish Dholakia
fc18f4decf test: add unit testing 2025-09-17 18:00:59 -07:00
Krrish Dholakia
4276d78169 test: fix test on ci/cd 2025-09-17 18:00:58 -07:00
Krrish Dholakia
00d8dedd16 test: update test 2025-09-17 17:44:14 -07:00
Krish Dholakia
44e0c730b9
Merge pull request #14653 from ARajan1084/in-memory-guardrail-fix
fix: In Memory Guardrail fails to update
2025-09-17 17:42:23 -07:00
Krrish Dholakia
adc71ad2e7 fix(langsmith.py): add langsmith_sampling_rate as a dynamic param
Closes LIT-879
2025-09-17 17:32:26 -07:00
=
268b984cb6 added test for updating in memory guardrails 2025-09-17 17:16:49 -07:00
Krish Dholakia
f51003538a
Merge pull request #14650 from BerriAI/litellm_dev_09_17_2025_p1
Bedrock Guardrails - support setting bedrock runtime endpoint + Protect `/health/test_connect` to prevent users without model creation permissions from calling it
2025-09-17 16:55:53 -07:00
Krrish Dholakia
1954bbffc0 test: fix test on ci/cd 2025-09-17 16:53:07 -07:00
Krrish Dholakia
2f45c7ffd3 feat(anthropic/chat/transformation.py): account for 1h vs. 5m cache creation token cost difference
Closes LIT-907
2025-09-17 16:16:20 -07:00
Krish Dholakia
895c41efa3
Merge pull request #14619 from BerriAI/litellm_dev_09_16_2025_p1
UI - allow team member to view service account keys they create + Anthropic - include cache creation tokens in prompt token total (separate out during cost tracking)
2025-09-17 15:43:04 -07:00
Krrish Dholakia
530662d735 test: add unit testing 2025-09-17 14:54:44 -07:00
Sameer Kankute
69c01488bd
remove not needed names (#14641) 2025-09-17 14:26:48 -07:00
Krrish Dholakia
08ba38adeb test: update unit test 2025-09-17 14:23:13 -07:00
Sameerlite
1371abf880 add last message as default in gaurdrail 2025-09-17 21:08:19 +05:30
Fabrício Ceschin
9fdbdaf846
Merge branch 'BerriAI:main' into LangfuseUsageDetails 2025-09-17 09:36:08 -04:00
Krish Dholakia
bf0dd4a284
Merge pull request #14418 from iabhi4/deep-copy-issue
fix: avoid deepcopy crash with non-pickleables in Gemini/Vertex
2025-09-16 22:55:31 -07:00
Krish Dholakia
2563c0ab2d
Merge pull request #14534 from hula-la/fix/response-api-bugfixes
fix: improve response api handling and cold storage configuration
2025-09-16 22:46:03 -07:00
Krish Dholakia
fcf84027e8
Merge pull request #14555 from mubashir1osmani/dd_spend_metric
DataDog shows spend metrics
2025-09-16 22:44:47 -07:00
Krish Dholakia
635dc72211
Merge pull request #14604 from Sameerlite/litellm_gemini_api_base_update
Litellm gemini api base update
2025-09-16 22:38:44 -07:00
Krish Dholakia
6bccdcd530
Merge pull request #14620 from BerriAI/litellm_dev_09_16_2025_p2
(feat) Anthropic - document pricing for cache creation tokens above 1hr
2025-09-16 22:33:11 -07:00
Krrish Dholakia
8273630e08 test: update tests 2025-09-16 22:30:40 -07:00
Mubashir Osmani
0fde408e35
added langfuse logging for responses api (#14597)
* added langfuse logging for responses api

* tests added
2025-09-16 21:43:19 -07:00
mubashir1osmani
e56b11cb54 added tests 2025-09-17 00:19:03 -04:00
Krrish Dholakia
22d3e492f9 fix(test_utils.py): fix test 2025-09-16 19:17:44 -07:00
Krrish Dholakia
e735808614 test(test_utils.py): fix cache creation tokens 2025-09-16 19:03:16 -07:00
Krrish Dholakia
e488312873 fix(utils.py): log cache_creation_tokens in prompt token details
Closes LIT-907
2025-09-16 18:24:10 -07:00
Krrish Dholakia
32f634e305 test: add unit testing 2025-09-16 16:54:03 -07:00
Sameer Kankute
ab1fb2b2e7
Add Support for Bedrock Guardrails to supportive selective Guarding (#14575)
* Add Support for Bedrock Guardrails to supportive selective Guarding

* Add method for better handling

* Add guarded_text content type

* Add guarded_text content type

* Update Dockerfile

* Update Dockerfile
2025-09-16 16:43:34 -07:00
Fabrício Ceschin
2a3d84e4be
Merge branch 'BerriAI:main' into LangfuseUsageDetails 2025-09-16 09:26:52 -04:00
Anubhav Singh
a9667e5930
Merge branch 'BerriAI:main' into wandb-inference 2025-09-16 16:46:58 +05:30
Sameerlite
f08fc45a0f add base url support for gemini 2025-09-16 15:15:24 +05:30
Krish Dholakia
f34bbd1f35
Merge pull request #14569 from talesofai/volcengine-fix-thinking-parameters-missing-if-not-set
fix volcengine thinking parameters missing when it set disable
2025-09-15 21:33:31 -07:00
Krish Dholakia
3f90fe938d
Merge pull request #14514 from iabhi4/fix-13581
feat(proxy): Assign default budget to auto-generated JWT teams
2025-09-15 21:22:25 -07:00
Krish Dholakia
f4e2870490
Merge pull request #14532 from timelfrink/feat/issue-14476-compactifai-provider
Add CompactifAI provider support
2025-09-15 21:15:00 -07:00
mubashir1osmani
f5f00043ce fixed tests 2025-09-16 00:14:48 -04:00
LingXuanYin
f8c9009fe5 add more test 2025-09-16 12:11:10 +08:00