Commit graph

14483 commits

Author SHA1 Message Date
Ishaan Jaffer
8296bfb866 fix: test metrics 2025-09-18 10:25:22 -07:00
Ishaan Jaffer
c6afa904bb fix: test_completion_with_no_model 2025-09-18 10:17:09 -07:00
Ishaan Jaffer
1e1d174733 fix: test_completion_with_no_model 2025-09-18 10:13:32 -07:00
Ishaan Jaffer
706b9214c0 fix: test_init_kwargs_for_pass_through_endpoint_basic 2025-09-18 07:59:05 -07:00
Ishaan Jaffer
3018c66867 fix: test_awesome_otel_with_message_logging_off 2025-09-18 07:49:39 -07:00
Fabrício Ceschin
cb3bed1d96
Merge branch 'BerriAI:main' into LangfuseUsageDetails 2025-09-18 09:38:52 -04:00
Adam Tao
9b0eaf0989 test: add a testcase for cached_tokens
Signed-off-by: Adam Tao <tcx4c70@gmail.com>
2025-09-18 18:40:11 +08:00
Michael Tan
6721189b7c
feat: add Asia/Bangkok timezone support for budget resets
- Add Asia/Bangkok (UTC+7) to timezone_map in duration_parser.py
- Update documentation to include Bangkok in common timezone values
- Add test case to verify Bangkok timezone functionality
2025-09-18 14:56:12 +07:00
Tim Elfrink
27a37ce57e Use requestMetadata parameter name for consistency with other Bedrock params
- Change parameter from request_metadata to requestMetadata to match camelCase convention
- Consistent with guardrailConfig and performanceConfig naming pattern
- Update all references in transformation code and error messages
- Update tests and documentation to use correct parameter name
- Fix type checking for parameter validation
2025-09-18 09:21:45 +02:00
Tim Elfrink
1d85623b66 Add comprehensive tests for Bedrock Converse requestMetadata support
- Test requestMetadata parameter support in get_supported_openai_params
- Test transformation to top-level field in Bedrock API request
- Test validation of AWS constraints: max 16 items, key/value length limits
- Test character set validation for keys and values
- Cover edge cases including empty values and special characters
- Ensure compatibility with existing test patterns
2025-09-18 09:21:45 +02:00
Krish Dholakia
bfaab8ad7e
Merge pull request #14557 from timelfrink/fix/issue-14478-bedrock-count-tokens-endpoint
Implement AWS Bedrock CountTokens API support
2025-09-17 23:51:06 -07:00
Krish Dholakia
ff36dfdc76
Merge pull request #14637 from akraines/feature/middle-truncate-spend-logs
feat: implement middle-truncation for spend log payloads
2025-09-17 23:47:04 -07:00
Krish Dholakia
e7bc700b20
Merge pull request #14640 from Sameerlite/litellm_gardrail_default_latest_message
[Feat]Add last message as default in gaurdrail
2025-09-17 23:42:40 -07:00
Tim Elfrink
c234b13275 Apply code formatting and linting fixes
- Apply Black formatting to all Bedrock CountTokens files
- Clean up imports and remove unused variables in tests
- Fix indentation and simplify test structure
- Fix pyright type error with type ignore annotation
- All tests continue to pass after cleanup
2025-09-18 08:28:17 +02:00
Tim Elfrink
e74ac35b5d Add comprehensive tests for Bedrock CountTokens functionality
- Add endpoint integration test in test_proxy_token_counter.py
- Add unit tests for transformation logic in bedrock/count_tokens/
- Test model extraction from request body vs endpoint path
- Test input format detection (converse vs invokeModel)
- Test request transformation from Anthropic to Bedrock format
- All tests follow existing codebase patterns and pass successfully
2025-09-18 08:16:56 +02:00
Sameerlite
edf95966c9 Handle consecutive user messages 2025-09-18 10:35:25 +05:30
Alexsander Hamir
8d96626044
fix: iscoroutine removed from hot path +50 RPS (#14649)
* fix: iscoroutine removed from hot path

* fix: replace all instances & separate concerns

1. Replaced all instances of iscoroutine with is_async_callable
2. Place the coroutine checker in its own file

* fix: PR comment changes

* fix: missing config setting declaration

* fix: revert non-performance related changes

* fix: revert to initial implementation

* fix: remove dead const
2025-09-17 19:01:43 -07:00
Krrish Dholakia
7a1f723751 test: update test 2025-09-17 18:00:59 -07:00
=
40fa415c89 added test for updating in memory guardrails 2025-09-17 18:00:59 -07:00
Mubashir Osmani
dc267e9032 fix: ci/cd tests + lint errors (#14646)
* fix: lint errors + tests

* fixed ci tests

* fixed tests

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2025-09-17 18:00:59 -07:00
Krrish Dholakia
fc18f4decf test: add unit testing 2025-09-17 18:00:59 -07:00
Krrish Dholakia
4276d78169 test: fix test on ci/cd 2025-09-17 18:00:58 -07:00
Krrish Dholakia
00d8dedd16 test: update test 2025-09-17 17:44:14 -07:00
Krish Dholakia
44e0c730b9
Merge pull request #14653 from ARajan1084/in-memory-guardrail-fix
fix: In Memory Guardrail fails to update
2025-09-17 17:42:23 -07:00
Krrish Dholakia
adc71ad2e7 fix(langsmith.py): add langsmith_sampling_rate as a dynamic param
Closes LIT-879
2025-09-17 17:32:26 -07:00
=
268b984cb6 added test for updating in memory guardrails 2025-09-17 17:16:49 -07:00
Mubashir Osmani
8b804303ed
fix: ci/cd tests + lint errors (#14646)
* fix: lint errors + tests

* fixed ci tests

* fixed tests

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2025-09-17 17:06:43 -07:00
Krish Dholakia
f51003538a
Merge pull request #14650 from BerriAI/litellm_dev_09_17_2025_p1
Bedrock Guardrails - support setting bedrock runtime endpoint + Protect `/health/test_connect` to prevent users without model creation permissions from calling it
2025-09-17 16:55:53 -07:00
Krrish Dholakia
1954bbffc0 test: fix test on ci/cd 2025-09-17 16:53:07 -07:00
Krrish Dholakia
2f45c7ffd3 feat(anthropic/chat/transformation.py): account for 1h vs. 5m cache creation token cost difference
Closes LIT-907
2025-09-17 16:16:20 -07:00
Krrish Dholakia
e32ce6b053 feat(anthropic/chat/transformation.py): separate 5m vs. 1h cache creation token details for anthropic cost tracking 2025-09-17 15:51:07 -07:00
Krish Dholakia
895c41efa3
Merge pull request #14619 from BerriAI/litellm_dev_09_16_2025_p1
UI - allow team member to view service account keys they create + Anthropic - include cache creation tokens in prompt token total (separate out during cost tracking)
2025-09-17 15:43:04 -07:00
Krrish Dholakia
530662d735 test: add unit testing 2025-09-17 14:54:44 -07:00
Sameer Kankute
69c01488bd
remove not needed names (#14641) 2025-09-17 14:26:48 -07:00
Krrish Dholakia
08ba38adeb test: update unit test 2025-09-17 14:23:13 -07:00
Fabrício Ceschin
6591550fe8
Merge branch 'BerriAI:main' into LangfuseUsageDetails 2025-09-17 13:33:02 -04:00
Sameerlite
1371abf880 add last message as default in gaurdrail 2025-09-17 21:08:19 +05:30
Carlos Marchal
e168161e64
Feat/add posthog observability (#14610)
* feat: add posthog observability

* docs: add posthog logging docs

* docs: posthog integration in proxy mode
2025-09-17 08:24:04 -07:00
Fabrício Ceschin
9fdbdaf846
Merge branch 'BerriAI:main' into LangfuseUsageDetails 2025-09-17 09:36:08 -04:00
Akiva Kraines
115a3e9ded feat: implement middle-truncation for spend log payloads
- Change truncation strategy from head-only to middle-truncation (35% start, 65% end)
- Preserve both beginning and end of long strings for better debugging context
- Apply same sanitization to response payloads when store_prompts_in_spend_logs is enabled
- Increase default MAX_STRING_LENGTH_PROMPT_IN_DB from 1000 to 2048 characters
- Update tests to verify new truncation behavior with 35%-65% split

This provides better diagnostic value by keeping the more important end context
while still maintaining storage limits.
2025-09-17 16:30:25 +03:00
Krish Dholakia
bf0dd4a284
Merge pull request #14418 from iabhi4/deep-copy-issue
fix: avoid deepcopy crash with non-pickleables in Gemini/Vertex
2025-09-16 22:55:31 -07:00
Krish Dholakia
2563c0ab2d
Merge pull request #14534 from hula-la/fix/response-api-bugfixes
fix: improve response api handling and cold storage configuration
2025-09-16 22:46:03 -07:00
Krish Dholakia
fcf84027e8
Merge pull request #14555 from mubashir1osmani/dd_spend_metric
DataDog shows spend metrics
2025-09-16 22:44:47 -07:00
Krish Dholakia
635dc72211
Merge pull request #14604 from Sameerlite/litellm_gemini_api_base_update
Litellm gemini api base update
2025-09-16 22:38:44 -07:00
Krish Dholakia
6bccdcd530
Merge pull request #14620 from BerriAI/litellm_dev_09_16_2025_p2
(feat) Anthropic - document pricing for cache creation tokens above 1hr
2025-09-16 22:33:11 -07:00
Krrish Dholakia
8273630e08 test: update tests 2025-09-16 22:30:40 -07:00
Mubashir Osmani
0fde408e35
added langfuse logging for responses api (#14597)
* added langfuse logging for responses api

* tests added
2025-09-16 21:43:19 -07:00
mubashir1osmani
e56b11cb54 added tests 2025-09-17 00:19:03 -04:00
Krrish Dholakia
0e747aaaf1 test: fix test 2025-09-16 19:20:12 -07:00
Ishaan Jaff
93524cf19b
[Feat] Batches - Add bedrock retrieve endpoint support (#14618)
* feat: add bedrock retrieve endpoint

* feat: feat: add bedrock retrieve endpoint

* test: batches mocked transform

* ruff fix

* refactor

* fix transform

* fix: parse_timestamp
2025-09-16 19:19:02 -07:00