Commit graph

894 commits

Author SHA1 Message Date
Ishaan Jaffer
0b1a9f6623 litellm_mapped_tests_proxy_part2 2026-02-14 15:37:18 -08:00
Ishaan Jaffer
0d697e6c3e litellm_mapped_tests_mcps 2026-02-14 15:20:25 -08:00
Ishaan Jaffer
bd766706c5 VertexAILlama3StreamingHandler 2026-02-14 13:18:26 -08:00
Ishaan Jaffer
ec7eb79634 ANTHROPIC_API_KEY 2026-02-14 12:20:55 -08:00
Sameer Kankute
bece16069e Add e2e tests for checking if all beta headers in the mapping works 2026-02-13 15:49:50 +05:30
Alexsander Hamir
b7993b14cf
Add semgrep & Fix OOMs (#20912) 2026-02-10 17:50:14 -08:00
yuneng-jiang
c68383068a
Merge pull request #20803 from BerriAI/litellm_ui_e2e_02
[Infra] UI - E2E Tests: Key Delete, Regenerate, and Update TPM/RPM Limits
2026-02-10 17:02:37 -08:00
yuneng-jiang
cfd261d679 Split e2e ui testing for browser 2026-02-10 16:30:31 -08:00
Alexsander Hamir
ebce0e5f8c
[Release - 02/10/2026] v1.81.10-nightly 2026-02-10 16:26:30 -08:00
Krish Dholakia
10d891a365
Guardrails - add logging to all unified_guardrails + link to custom code guardrail templates (#20900)
* feat(guardrail_hooks/): add guardrail logging to all unified guardrails

ensures unified guardrails use the 'log_guardrail_information' decorator for logging

* fix(custom_guardrail.py): don't log inputs on guardrail response - just emit state

* refactor: don't double log bedrock guardrail information

* feat: add in-product nudges for contributing + trying community custom code guardrails

allows users to contribute / share custom code guardrails
2026-02-10 15:13:54 -08:00
Ishaan Jaff
b78f4c924c
[Fix] A2a Agent Gateway Fixes - A2A agents deployed with localhost/internal URLs in their agent cards (e.g., http://0.0.0.0:8001/) (#20604)
* v1 card resolver fix

* fix: is_localhost_or_internal_url

* fix code

* test_fix_agent_card_url_replaces_localhost

* test restruct

* test_a2a_non_streaming

* test agnts

* add exception handling

* init errors

* add localhost retry

* add agent_testing

* test_a2a_non_streaming

* _build_streaming_logging_obj

* code qa fixes

* test_card_resolver_fallback_from_new_to_old_path

* fix linting
2026-02-06 15:02:34 -08:00
Ishaan Jaff
da4cf4942f
[Feat] Add xAI /realtime API Support - works with LiveKitSDK (#20381)
* init: _realtime_health_check + routing

* refactor: OpenAIRealtime

* refactor: XAI_API_BASE

* feat: XAIRealtime

* init feat: XAIRealtime

* OpenAIRealtime

* TestXAIRealtime

* test fixes

* test OAI

* TEST xAI, OAI

* clean realtime jobs

* refactor

* test XAI

* docs xAI

* fix xAI

* fix lint errors

* test_async_realtime_url_contains_model

* test fix

* document test changes

* _realtime_health_check

* docs xai realtime

* fix handlers

* add additional_headers

* fix
2026-02-03 19:58:28 -08:00
yuneng-jiang
33343bc1ef remove node_modules 2026-01-31 19:57:54 -08:00
yuneng-jiang
9926a576e7 clean install for build ui 2026-01-31 19:14:27 -08:00
Ishaan Jaff
476f0b29d2
[Feat] LiteLLM x Claude Agent SDK Integration (#20035)
* fix: bedrock invoke - does not support prompt-caching-scope

* fix: UNSUPPORTED_BEDROCK_INVOKE_BETA_PATTERNS

* init requirements.txt

* init README for claude Agent SDK

* fix: using converse models with UNSUPPORTED_BEDROCK_CONVERSE_BETA_PATTERNS

* fix main.py

* init: proxy_e2e_anthropic_messages_tests
2026-01-29 17:48:38 -08:00
Alexsander Hamir
69bd4426e8
[Release Day] - Fixed CI/CD issues & changed processes (#19902) 2026-01-28 17:57:24 -08:00
Alexsander Hamir
c442fcd922
CI/CD: Increase retries and stabilize litellm_mapped_tests_core (#19826)
* Fix PLR0915: Extract system message handling to reduce statement count

* fix mypy

* fix: add host_progress_callback parameter to mock_call_tool in test

The test_call_tool_without_broken_pipe_error was failing because the mock function did not accept the host_progress_callback keyword argument that the actual implementation passes to client.call_tool(). Updated the mock to accept this parameter to match the real implementation signature.

* fixing flaky tests around oidc and email

* Add documentation comment to test file

* add retry

* add dependency

* increase retry

---------

Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
2026-01-26 17:00:18 -08:00
Ishaan Jaffer
489d587332 CI/CD fixes - split local testing 2026-01-24 09:23:03 -08:00
yuneng-jiang
1c9731527f fixing circleci config 2026-01-24 08:54:22 -08:00
yuneng-jiang
acd8f21f68 fixing circleci config 2026-01-23 23:30:54 -08:00
yuneng-jiang
00ec939ea6 cache tests serial 2026-01-23 23:13:45 -08:00
yuneng-jiang
e1bb4ae280 deactivating non root tests 2026-01-23 22:55:36 -08:00
yuneng-jiang
fbe5ae9e17 fixing flaky tests 2026-01-23 12:20:27 -08:00
yuneng-jiang
89bf7e50c4 skipping flaky tests 2026-01-23 11:43:39 -08:00
yuneng-jiang
a28911845f fixing config 2026-01-23 11:11:11 -08:00
yuneng-jiang
2d4c0f4e65 ci cd fixes - linting security 2026-01-23 10:37:16 -08:00
Yuta Saito
7d7b78a53d test: Pydantic version up
mcp 1.25.0 depends on pydantic<3.0.0 and >=2.11.0
2026-01-20 10:49:33 +09:00
Yuta Saito
b229410fb5 test: ci requirements mcp version up 2026-01-20 10:42:58 +09:00
Yuta Saito
738ffa7eec test: ci mcp version up except mcp_testing 2026-01-20 10:35:46 +09:00
Yuta Saito
97c09743c0 test: ci mcp version up 2026-01-20 10:30:32 +09:00
Yuta Saito
44a166a792 fix: ci mcp version up 2026-01-19 14:27:00 +09:00
YutaSaito
7aba0f738a
Revert "Litellm staging 01 15 2026" 2026-01-17 06:31:34 +09:00
Sameer Kankute
84974d5745 Fix boto3 conflicting dependency 2026-01-16 16:55:12 +05:30
burnerburnerburnerman
5676c6c135
Chore: bump boto3 version (#19090) 2026-01-16 02:39:30 +05:30
Alexsander Hamir
15c3bc219b
[Refactor] Add CI enforcement for O(1) operations in _get_model_cost_key to prevent performance regressions (#19052)
* Optimize _get_model_cost_key to avoid expensive scans

- Remove expensive O(n) scan fallback that was causing 42.87% CPU overhead
- Only scan when size mismatch detected (O(1) check)
- Add warning in docstring: Only O(1) lookup operations are acceptable
- Clean up comments to be more concise
- Keep stale entry rebuild for pop() case (only triggers when stale entry found)

This fixes the performance issue where the scan was being triggered on every
failed lookup, causing severe CPU overhead during router operations.

* Add code quality check to enforce O(1) operations in _get_model_cost_key

- Add check_get_model_cost_key_performance.py to statically analyze _get_model_cost_key
- Detects O(n) operations (loops, comprehensions, problematic function calls)
- Recursively checks called functions to find nested O(n) operations
- Allows conditional O(n) rebuilds in helper functions (_rebuild_model_cost_lowercase_map, _handle_stale_map_entry_rebuild, _handle_new_key_with_scan)

* Integrate _get_model_cost_key performance check into CI pipeline

- Add check_get_model_cost_key_performance.py to check_code_and_doc_quality job
- Ensures O(1) requirement is enforced in CI to prevent performance regressions

* Remove unused performance test and clean up utils.py

- Remove test_get_model_info_performance.py (no longer needed)
- Remove extra blank line in utils.py

* Document allowed helper functions and exception process in _get_model_cost_key

- Add documentation listing allowed helper functions with O(n) operations
- Explain why these are acceptable (conditionally called)
- Add instructions for adding new exceptions to check_get_model_cost_key_performance.py

* Fix docstring detection and type checker error in performance check

- Add proper docstring tracking to skip docstring content (fixes false positive for 'map' in docstring)
- Add None check for docstring_quote to fix type checker error
- Restore _handle_new_key_with_scan to allowed_helpers list

* Remove check_get_model_cost_key_performance from CI pipeline

- Temporarily remove the performance check from CI to avoid blocking builds

* Restore performance check and remove memory leak tests from CI

- Add back check_get_model_cost_key_performance.py to CI pipeline
- Remove memory_leak_tests job that was causing port conflicts

* Remove extra blank line in CI config
2026-01-13 17:08:03 -08:00
Yuta Saito
b6bb90f252 test: helm chart testing
load local Docker image for helm chart tests
2026-01-13 06:40:57 +09:00
YutaSaito
a0b7e28fbc
Merge pull request #18882 from BerriAI/litellm_fix_db-migration-log-grep-flaky
[test] stabilize db_migration_disable_update_check test log check
2026-01-12 05:41:46 +09:00
yuneng-jiang
3a2de85e7b
Merge pull request #18885 from BerriAI/litellm_ui_fallbacks_refactor
[Infra] UI - E2E Test: New DB Branch Per Test Run
2026-01-09 23:37:44 -08:00
yuneng-jiang
bdab97c520 Neon Branch on every commit + skip flaky tests 2026-01-09 23:25:42 -08:00
yuneng-jiang
b2c7a59d4e directly set db url 2026-01-09 22:57:28 -08:00
yuneng-jiang
ff04a23656 db url 2026-01-09 22:54:13 -08:00
yuneng-jiang
e2d61a6131 final 2026-01-09 22:43:01 -08:00
yuneng-jiang
78313d79ba adding expiry 2026-01-09 22:37:30 -08:00
yuneng-jiang
b0bbdc2fad neon delete 2026-01-09 22:26:32 -08:00
yuneng-jiang
70e904102e fixing export 2026-01-09 22:22:55 -08:00
yuneng-jiang
554fc0d39d fixing parent pt2 2026-01-09 22:14:31 -08:00
yuneng-jiang
bb015ab172 fixing parent 2026-01-09 22:09:00 -08:00
yuneng-jiang
773a1f434b parent id 2026-01-09 22:06:16 -08:00
yuneng-jiang
7d5716a3e0 create branch from e2e 2026-01-09 22:00:00 -08:00
yuneng-jiang
83157b83f8 testing neon 2026-01-09 21:56:22 -08:00