Commit graph

17 commits

Author SHA1 Message Date
user
7514bb4740
fix(guardrails): close mixed-list gap, drop dead code, rename helper
Greptile P2 follow-ups on _content_utils.py:

- Drop unreachable ``_resolve_messages``. The new
  ``_iter_inspection_messages`` walks ``messages`` AND ``input``
  independently; leaving the old fallback-only variant around invited a
  future maintainer to wire it back up and silently narrow coverage.
- Rename ``iter_user_text`` → ``iter_message_text``. The helper walks
  every role (user, assistant, system); the old name implied user-turn
  content only. Callers and tests updated.
- Close mixed-list coverage gap. When ``data["input"]`` was a list
  mixing content-part dicts and bare strings, ``iter_message_text`` and
  ``build_inspection_messages`` only saw the dict parts while
  ``walk_user_text`` already inspected both. ``_iter_text_parts_in_content``
  now treats bare strings inside a content list as text fragments, so
  read and write helpers agree on coverage.

Adds two regression tests for the mixed-list shape.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 04:10:04 +00:00
user
b1b00e4bdc
chore(guardrails): cover multimodal + Responses-API content shapes
Several guardrail hooks short-circuit when ``message.content`` is a list
or when the request uses the Responses-API ``input`` field instead of
``messages``. Centralise the content-walking logic in a shared helper and
update the affected hooks so list-format and Responses-API payloads no
longer skip inspection.

Also: Aim's ``async_post_call_success_hook`` now inspects every choice
(via ``asyncio.gather``) instead of only ``choices[0]`` — the prior
behaviour let ``n>1`` callers hide content in subsequent completions.

Hooks updated to use the new helper:
- aim, lakera_ai_v2, lasso (post a synthesised messages list to a remote
  guardrail service)
- azure_content_safety, ibm_detector, banned_keywords, openai_moderation,
  google_text_moderation (iterate text fragments locally)
- secret_detection (walk-and-rewrite to redact in place)

Drive-by fix: the legacy ``data["prompt"]`` list-handling path in
secret_detection rebound the loop variable instead of mutating the list,
leaving secrets unredacted on text-completion calls; corrected to index
back into the list.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 03:50:15 +00:00
Krish Dholakia
06906534b3
feat(audio_transcriptions/): calculate duration of audio file for cost calculation + feat (image_generations): cost tracking accuracy improved with output_format, quality, size values fixed per openai model
* feat(audio_transcriptions/): calculate duration of audio file for cost calculation

Fixes https://github.com/BerriAI/litellm/issues/11846

Closes https://github.com/BerriAI/litellm/issues/14605

* fix(cost_calculator.py): correctly use base model, when set

Fixes issue where azure base model was being ignored

* feat(cost_calculator.py): fix default cost tracking quality param for image generation

* feat(image_generations/): return output_format, quality, size

aligns response to openai spec and improves cost tracking accuracy

* fix(cost_calculator.py): refactor cost calculation for image generation to use image response instead of hidden params

* build: update build

* fix: fix cost calculation

* build: update poetry lock

* fix: fix ruff checks

* fix: fix aembedding

* fix: fix ruff errors

* fix: modify to catch errors

* fix: test

* fix: loosen test to handle openai lib out of sync

* fix: fix base models

* fix: fix usage object
2025-11-08 16:24:31 -08:00
Krish Dholakia
202eaeb1a2
Revert "(feat) Audio transcription - cost tracking + (feat) image generation …" (#16409)
This reverts commit c96da44265.
2025-11-08 15:38:16 -08:00
Krish Dholakia
c96da44265
(feat) Audio transcription - cost tracking + (feat) image generation - accurate cost tracking based on output_format/quality/size
* feat(audio_transcriptions/): calculate duration of audio file for cost calculation

Fixes https://github.com/BerriAI/litellm/issues/11846

Closes https://github.com/BerriAI/litellm/issues/14605

* fix(cost_calculator.py): correctly use base model, when set

Fixes issue where azure base model was being ignored

* feat(cost_calculator.py): fix default cost tracking quality param for image generation

* feat(image_generations/): return output_format, quality, size

aligns response to openai spec and improves cost tracking accuracy

* fix(cost_calculator.py): refactor cost calculation for image generation to use image response instead of hidden params

* build: update build

* fix: fix cost calculation

* build: update poetry lock

* fix: fix ruff checks

* fix: fix aembedding

* fix: fix ruff errors

* fix: modify to catch errors

* fix: test

* fix: loosen test to handle openai lib out of sync
2025-11-08 15:30:46 -08:00
Ishaan Jaffer
0a1fc0eeb2 Revert "fix: fix ruff errors"
This reverts commit eef864360e.
2025-11-08 14:33:51 -08:00
Krrish Dholakia
eef864360e fix: fix ruff errors 2025-11-08 14:15:12 -08:00
Ishaan Jaff
cea318330e
[Feat] Add Guardrails for /v1/messages and /v1/responses API (#15686)
* add get_guardrails_messages_for_call_type

* fix call type for /messages

* add anthropic endpoints

* fix bedrock guardrails

* fix config.yaml

* fix types

* fix async_pre_call_hook

* ruff fix

* fix guard

* fix test bedrock guardrail

* fix linting

* fix linting

* docs guardrails

* fix mypy linting
2025-10-17 18:09:00 -07:00
Jugal D. Bhatt
900c7f45c0
[MCP Gateway] Litellm mcp pre and during guardrails (#13188)
* add guardrail support

* add guardrail support

* guardrails for MCP

* added changes

* add mcp guardrails

* added test

* add ui

* fix guardrail form

* working with cursor

* remvoe print

* fix mcp servertests

* fix mypy and remove console logs

* fix mypy and remove console logs

* fix mypy tests
2025-08-01 20:02:25 -07:00
Ishaan Jaff
57b0b4edf3
[Bug fix] [Bug]: Verbose log is enabled by default (#12596)
* test find_set_verbose_assignments

* fix set verbose

* test set verbose

* fix unused import
2025-07-14 20:06:16 -07:00
Ishaan Jaff
a28ac57233 fix linting error 2025-03-12 18:42:42 -07:00
Ishaan Jaff
c7f14e936a
(code quality) run ruff rule to ban unused imports (#7313)
* remove unused imports

* fix AmazonConverseConfig

* fix test

* fix import

* ruff check fixes

* test fixes

* fix testing

* fix imports
2024-12-19 12:33:42 -08:00
Ishaan Jaff
4d1b4beb3d
(refactor) caching use LLMCachingHandler for async_get_cache and set_cache (#6208)
* use folder for caching

* fix importing caching

* fix clickhouse pyright

* fix linting

* fix correctly pass kwargs and args

* fix test case for embedding

* fix linting

* fix embedding caching logic

* fix refactor handle utils.py

* fix test_embedding_caching_azure_individual_items_reordered
2024-10-14 16:34:01 +05:30
Krish Dholakia
fac3b2ee42
Add pyright to ci/cd + Fix remaining type-checking errors (#6082)
* fix: fix type-checking errors

* fix: fix additional type-checking errors

* fix: additional type-checking error fixes

* fix: fix additional type-checking errors

* fix: additional type-check fixes

* fix: fix all type-checking errors + add pyright to ci/cd

* fix: fix incorrect import

* ci(config.yml): use mypy on ci/cd

* fix: fix type-checking errors in utils.py

* fix: fix all type-checking errors on main.py

* fix: fix mypy linting errors

* fix(anthropic/cost_calculator.py): fix linting errors

* fix: fix mypy linting errors

* fix: fix linting errors
2024-10-05 17:04:00 -04:00
Ishaan Jaff
b6ae2204a8
[Feat-Proxy] Slack Alerting - allow using os.environ/ vars for alert to webhook url (#5726)
* allow using os.environ for slack urls

* use env vars for webhook urls

* fix types for get_secret

* fix linting

* fix linting

* fix linting

* linting fixes

* linting fix

* docs alerting slack

* fix get data
2024-09-16 18:03:37 -07:00
Ishaan Jaff
6aad4e38d3 feat - add debugging for moderation response 2024-05-23 13:24:09 -07:00
Ishaan Jaff
1fe035c6dd feat - add open ai moderations check 2024-05-23 13:08:06 -07:00