Commit graph

33401 commits

Author SHA1 Message Date
Yuneng Jiang
c43d536ffb
chore: fixes 2026-04-04 23:43:03 -07:00
Ishaan Jaffer
ca9b449a04 feat(content-filter): add employment discrimination topic blockers for 5 protected classes
Adds YAML topic category files for military_status, disability, age_discrimination,
religion, and gender_sexual_orientation to block employment discrimination prompts
like "Do not hire veterans because they may have mental health issues."

Previously these were not blocked because:
- The prebuilt regex patterns used strict \b word boundaries that didn't match
  plurals (veterans, disabilities, Muslims)
- gender_sexual_orientation pattern was LGBTQ+-focused and missed women/female
- age_discrimination pattern missed "over 50" phrasing
- No conditional (identifier + discriminatory intent) detection existed for these
  protected classes

Each new YAML file uses the bias_racial.yaml pattern: identifier_words (protected
class terms) + additional_block_words (discriminatory employment actions), plus
always_block_keywords for explicit discriminatory phrases. Exceptions prevent false
positives for legitimate diversity programs, accommodation discussions, etc.

Also fixes regex plurals in patterns.json: veterans?, disabilit(y|ies), muslims?,
adds wom[ae]n?/females? to gender pattern, and over\s+\d+ to age pattern.

Evals: 100% precision/recall/F1/accuracy on all 5 new categories (89 total cases,
0 FP, 0 FN). Existing insults and investment evals unaffected.
2026-02-23 15:53:55 -08:00
ryan-crabbe
3faf5e5912
Merge pull request #21961 from BerriAI/litellm_revert_duplicate_workflows
Revert duplicate issue checker to text-based matching
2026-02-23 15:30:02 -08:00
Ryan Crabbe
079ff24d78 Revert duplicate issue checker to text-based matching, remove duplicate PR workflow
Remove the Claude Code-powered duplicate PR detection workflow and revert
the duplicate issue checker back to wow-actions/potential-duplicates with
text similarity matching.
2026-02-23 15:28:13 -08:00
Lei Nie
4ff2cfd8d6
feat(videos): add variant parameter to video content download (#21955)
openai videos models support the features to download variants.
See more details here: https://developers.openai.com/api/docs/guides/video-generation#use-image-references.
Plumb variant (e.g. "thumbnail", "spritesheet") through the full
video content download chain: avideo_content → video_content →
video_content_handler → transform_video_content_request. OpenAI
appends ?variant=<value> to the GET URL; other providers accept
the parameter in their signature but ignore it.
2026-02-23 14:45:19 -08:00
ryan-crabbe
0ca9869b99
Merge pull request #21950 from ryan-crabbe/docs/v1-81-14-perf-section
docs: add performance & reliability section to v1.81.14 release notes
2026-02-23 13:13:21 -08:00
Julio Quinteros Pro
6574d8d395
Merge pull request #21937 from jquinter/fix/flaky-rpm-limit-test
fix(tests): make RPM limit test sequential to fix race condition
2026-02-23 17:27:58 -03:00
Arindam Majumder
71b4bd12a7
Add OpenAI Agents SDK tutorial with LiteLLM Proxy to docs (#21221)
* Add OpenAI Agents SDK tutorial to docs

* Update OpenAI Agents SDK tutorial to use LiteLLM environment variables

* Enhance OpenAI Agents SDK tutorial with built-in LiteLLM extension details and updated configuration steps. Adjust section headings for clarity and improve the flow of information regarding model setup and usage.
2026-02-23 12:10:01 -08:00
Ron Zhong
73fd5a41e4
feat: Singapore guardrail policies (PDPA + MAS AI Risk Management) (#21948)
* feat: Singapore PDPA PII protection guardrail policy template

Add Singapore Personal Data Protection Act (PDPA) guardrail support:

Regex patterns (patterns.json):
- sg_nric: NRIC/FIN detection ([STFGM] + 7 digits + checksum letter)
- sg_phone: Singapore phone numbers (+65/0065/65 prefix)
- sg_postal_code: 6-digit postal codes (contextual)
- passport_singapore: Passport numbers (E/K + 7 digits, contextual)
- sg_uen: Unique Entity Numbers (3 formats)
- sg_bank_account: Bank account numbers (dash format, contextual)

YAML policy templates (5 sub-guardrails):
- sg_pdpa_personal_identifiers: s.13 Consent
- sg_pdpa_sensitive_data: Advisory Guidelines
- sg_pdpa_do_not_call: Part IX DNC Registry
- sg_pdpa_data_transfer: s.26 overseas transfers
- sg_pdpa_profiling_automated_decisions: Model AI Governance Framework

Policy template entry in policy_templates.json with 9 guardrail definitions
(4 regex-based + 5 YAML conditional keyword matching).

Tests:
- test_sg_patterns.py: regex pattern unit tests
- test_sg_pdpa_guardrails.py: conditional keyword matching tests (100+ cases)

* feat: MAS AI Risk Management Guidelines guardrail policy template

Add Monetary Authority of Singapore (MAS) AI Risk Management Guidelines
guardrail support for financial institutions:

YAML policy templates (5 sub-guardrails):
- sg_mas_fairness_bias: Blocks discriminatory financial AI (credit/loans/insurance by protected attributes)
- sg_mas_transparency_explainability: Blocks opaque/unexplainable AI for consequential financial decisions
- sg_mas_human_oversight: Blocks fully automated financial decisions without human-in-the-loop
- sg_mas_data_governance: Blocks unauthorized sharing/mishandling of financial customer data
- sg_mas_model_security: Blocks adversarial attacks, model poisoning, inversion on financial AI

Policy template entry in policy_templates.json with 5 guardrail definitions.
Aligned with MAS FEAT Principles, Project MindForge, and NIST AI RMF.

Tests:
- test_sg_mas_ai_guardrails.py: conditional keyword matching tests (100+ cases)

* fix: address SG pattern review feedback

- Update NRIC lowercase test for IGNORECASE runtime behavior
- Add keyword context guard to sg_uen pattern to reduce false positives

* docs: clarify MAS AIRM timeline references

- Explicitly mark MAS AIRM as Nov 2025 consultation draft
- Add 2018 qualifier for FEAT principles in MAS policy descriptions
- Update MAS guardrail wording to avoid release-year ambiguity

* chore: commit resolved MAS policy conflicts

* test:

* chore:
2026-02-23 12:08:22 -08:00
Julio Quinteros Pro
bb63de2f82 fix(tests): make RPM limit test sequential to avoid race condition
Concurrent requests via run_in_executor + asyncio.gather caused a race
condition where more requests slipped through the rate limiter than
expected, leading to flaky test failures (e.g. 3 successes instead of 2
with rpm_limit=2).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 16:34:52 -03:00
Julio Quinteros Pro
36813199b6
Merge pull request #21943 from jquinter/fix/interactions-incomplete-status
fix(tests): add INCOMPLETE to interactions status enum expected values
2026-02-23 16:31:41 -03:00
Ryan Crabbe
67ceade162 docs: add performance & reliability section to v1.81.14 release notes 2026-02-23 11:23:29 -08:00
Krish Dholakia
4dfec4661b
Guardrail Monitor - measure guardrail reliability in prod (#21944)
* fix: fix log viewer for guardrail monitoring

* feat(ui/): fix rendering logs per guardrail

* fix: fix viewing logs on overview tab of guardrail

* fix: log viewer

* fix: fix naming to align with metric
2026-02-23 10:23:14 -08:00
Julio Quinteros Pro
f94d0fe0b6 fix: add INCOMPLETE status to Interactions API enum and test
Google added INCOMPLETE to the Interactions API OpenAPI spec status enum.
Update both the Status3 enum in the SDK types and the test's expected
values to match.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 15:07:41 -03:00
ryan-crabbe
c4c48fe977
Merge pull request #21942 from BerriAI/litellm_network_mock
feat: Litellm network mock
2026-02-23 10:07:11 -08:00
Ryan Crabbe
e0ddb2a525 fix: guard print_aggregate against empty latencies 2026-02-23 10:06:10 -08:00
Ryan Crabbe
fb123ec52b refactor: use method-level import for MockOpenAITransport 2026-02-23 09:58:46 -08:00
Ryan Crabbe
95d9514054 fix: add auth headers and empty latencies guard to benchmark script 2026-02-23 09:58:40 -08:00
yuneng-jiang
bba98c2f15
Merge pull request #21886 from Chesars/docs/ui-contributing-pre-pr-checklist
docs(ui): add pre-PR checklist to UI contributing guide
2026-02-23 09:50:26 -08:00
Ryan Crabbe
8244ad1f0e docs: tweak benchmarks wording 2026-02-23 09:50:02 -08:00
Ryan Crabbe
5b41b009f6 docs: add network mock benchmarking section 2026-02-23 09:44:02 -08:00
Cesar Garcia
64d1de0552
docs: add Google GenAI SDK tutorial (JS & Python) (#21885)
* docs: add Google GenAI SDK tutorial for JS and Python

Add tutorial for using Google's official GenAI SDK (@google/genai for JS,
google-genai for Python) with LiteLLM proxy. Covers pass-through and
native router endpoints, streaming, multi-turn chat, and multi-provider
routing via model_group_alias. Also updates pass-through docs to use the
new SDK replacing the deprecated @google/generative-ai.

* fix(docs): correct Python SDK env var name in GenAI tutorial

GOOGLE_GENAI_API_KEY does not exist in the google-genai SDK.
The correct env var is GEMINI_API_KEY (or GOOGLE_API_KEY).
Also note that the Python SDK has no base URL env var.

* fix(docs): replace non-existent GOOGLE_GENAI_BASE_URL env var in interactions.md

The Python google-genai SDK does not read GOOGLE_GENAI_BASE_URL.
Use http_options={"base_url": "..."} in code instead.
2026-02-23 09:20:46 -08:00
Ryan Crabbe
d99d87f614 clean up mock transport: remove streaming, add defensive parsing 2026-02-23 09:16:47 -08:00
Julio Quinteros Pro
17d97f26f6
Merge pull request #21939 from jquinter/fix/model-prices-test-relative-path
fix(tests): use absolute path for model_prices JSON validation test
2026-02-23 14:10:47 -03:00
Julio Quinteros Pro
bf8c219860 fix(tests): use os.path instead of Path to avoid NameError
Path is not imported at module level. Use os.path.join which is already
available.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 13:56:11 -03:00
Julio Quinteros Pro
a74b6eee23 Update tests/test_litellm/test_utils.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-02-23 13:55:49 -03:00
Julio Quinteros Pro
11a774e110 fix(tests): use absolute path for model_prices JSON in validation test
The test used a relative path 'litellm/model_prices_and_context_window.json'
which only works when pytest runs from a specific working directory.
Use os.path based on __file__ to resolve the path reliably.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 13:55:49 -03:00
Julio Quinteros Pro
ac6f2f3c52
Merge pull request #21938 from jquinter/fix/flaky-prisma-self-heal-test
fix(tests): fix StopIteration in prisma self-heal cooldown test
2026-02-23 13:29:03 -03:00
Julio Quinteros Pro
fb8b11cc0a fix(tests): use counter-based mock for time.time in prisma self-heal test
The test used a fixed side_effect list for time.time(), but the number
of calls varies by Python version, causing StopIteration on 3.12 and
AssertionError on 3.14. Replace with an infinite counter-based callable
and assert the timestamp was updated rather than checking for an exact
value.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 13:25:02 -03:00
Krrish Dholakia
a26f83fd3c fix: update calendly on repo 2026-02-23 06:13:59 -08:00
Sameer Kankute
f97ee62fb0
Merge pull request #21909 from BerriAI/litellm_cost_tracking_gemini
Add  Priority PayGo cost tracking gemini/vertex ai
2026-02-23 18:58:57 +05:30
Sameer Kankute
61e63b6553
Merge pull request #21904 from BerriAI/litellm_fix_model_cost_map
fix model cost map for anthropic fast and inference_geo
2026-02-23 18:57:15 +05:30
Sameer Kankute
2f8d36be1b Fix test_aaamodel_prices_and_context_window_json_is_valid 2026-02-23 18:56:12 +05:30
Sameer Kankute
9b5bbee906
Merge pull request #21786 from BerriAI/litellm_oss_staging_02_21_2026
Litellm oss staging 02 21 2026
2026-02-23 18:51:55 +05:30
Sameer Kankute
8decf04d8a
Merge pull request #21877 from BerriAI/litellm_oss_staging_02_22_2026
Litellm oss staging 02 22 2026
2026-02-23 18:50:47 +05:30
Sameer Kankute
f6f46ae5ae Fix mypy issues 2026-02-23 18:46:25 +05:30
Sameer Kankute
37d45139f2
Merge pull request #21917 from BerriAI/litellm_fix_model_cost_map_wildcard
Fix: Anthropic model wildcard access issue
2026-02-23 18:45:49 +05:30
Sameer Kankute
16b25bd457
Merge pull request #21921 from Point72/ephrimstanley/batch-fixes-feb23
State management fixes for CheckBatchCost
2026-02-23 18:44:05 +05:30
Sameer Kankute
351921d5db Fix mypy issues 2026-02-23 18:41:28 +05:30
Sameer Kankute
a3c01b7259 Fix code qa issues 2026-02-23 18:39:46 +05:30
Sameer Kankute
4934d89cc7
Merge pull request #21872 from BerriAI/litellm_dev_02_21_2026_p4
Litellm dev 02 19 2026 p2 (#21871)
2026-02-23 18:35:57 +05:30
TomAlon
99184c48d9
Add Noma guardrails v2 based on custom guardrails (#21400) 2026-02-23 05:05:27 -08:00
Sameer Kankute
88cd0d31ed fix mypy issues: 2026-02-23 18:28:42 +05:30
Ephrim Stanley
27a33565e7 State management fixes for CheckBatchCost - Address greptile comments 2026-02-23 07:52:58 -05:00
Sameer Kankute
c7aafdf794
Merge pull request #21926 from BerriAI/main
merge main in oss 21 02
2026-02-23 18:17:30 +05:30
Sameer Kankute
57af8e6a93
Merge pull request #21924 from BerriAI/main
merge main in oss 22 02
2026-02-23 18:11:36 +05:30
Sameer Kankute
f23e549cce Fix PR review comments 2026-02-23 18:04:32 +05:30
Ephrim Stanley
7b5dc3fb9c State management fixes for CheckBatchCost 2026-02-23 07:16:25 -05:00
Harshit Jain
4c487acdf0
Merge pull request #21918 from Harshit28j/litellm_arrival_time_fix
fix: ensure arrival_time is set before calculating queue time
2026-02-23 17:41:05 +05:30
Sameer Kankute
164cde9bf6 Readd tpm limit 2026-02-23 17:23:12 +05:30