mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-14 23:21:35 +00:00
8a9faa81b2
* feat(guardrails): add LLM_AS_A_JUDGE to SupportedGuardrailIntegrations * feat(types): add EvalVerdict, StandardLoggingEvalInformation; wire eval_information into SpendLogsMetadata * feat(guardrails): add self-contained llm_as_a_judge guardrail hook * fix(a2a): filter agent-only litellm_params from acompletion kwargs; pass agent_id into body * feat(ui): add LLMJudgeFields criteria builder component * feat(ui): wire LLM-as-a-Judge into add guardrail form * feat(ui): update EvalViewer — title 'LLM Judge Results', weighted score column, summary row * fix(ui): wire EvalViewer into LogDetailContent to show LLM judge results on logs page * fix(guardrails-ui): route llm_as_a_judge to criteria builder step; rename to LiteLLM LLM as a Judge; add litellm logo * fix(guardrail-viewer): stack lifecycle + eval details vertically to avoid badge overflow in narrow drawer * fix(guardrail-create): surface config validation errors on create instead of silently orphaning guardrail in DB * fix(guardrail-registry): hardcode llm_as_a_judge in initializer registry so it loads regardless of package install path * fix(llm-as-a-judge): fix P1 code quality issues - validate weights/on_failure, guard pre_call, handle multimodal, move imports to module level, fix spurious finally logging * fix(guardrail_endpoints): use correct PK field in rollback delete and log rollback failure * fix(llm_as_a_judge): support Pydantic object in _get_litellm_param fallback chain * fix(LLMJudgeFields): replace @tremor/react Button with antd Button * fix(llm_as_a_judge): remove dead registry dicts, fix KeyError in prompt builder, set correct status on judge failure * test(llm_as_a_judge): add unit tests for guardrail hook * fix(llm_as_a_judge): remove @log_guardrail_information decorator to fix duplicate guardrail_information entries The decorator and the manual finally block both called add_standard_logging_guardrail_information_to_request_data, producing two entries per request. The decorator also misclassified HTTPException(422) blocks as guardrail_failed_to_respond (it checks for 400). The finally block correctly tracks status throughout, so removing the decorator is sufficient. * fix(test_gcs_pub_sub): ignore metadata.eval_information in comparison * fix(test_spend_management): ignore metadata.eval_information in payload comparison * fix(types/guardrails): add input_type and messages to ApplyGuardrailRequest * fix(guardrail_endpoints): pass input_type and messages through apply_guardrail endpoint * fix(guardrail_endpoints): auto-detect post_call guardrails and use input_type=response * fix(a2a_endpoints): merge agent litellm_params guardrails into data before post_call hooks * fix(llm_as_a_judge): use float sum with tolerance for weight validation * fix(guardrail_registry): split long import line for black formatting * fix(llm_as_a_judge): guard guardrail_name Optional for mypy * fix(llm_as_a_judge): set guardrail_status=guardrail_intervened when score fails, regardless of on_failure mode * fix(a2a_endpoints): use try/finally so deferred spend log fires even when guardrail blocks with 422 * fix(litellm_logging): declare _defer_async_logging and _enqueue_deferred_logging on Logging class for mypy * fix(logging_worker): restore queue.join() in flush() to wait for in-flight callbacks |
||
|---|---|---|
| .. | ||
| _experimental/mcp_server | ||
| agent_endpoints | ||
| anthropic_endpoints | ||
| auth | ||
| client | ||
| common_utils | ||
| db | ||
| discovery_endpoints | ||
| experimental/mcp_server | ||
| google_endpoints | ||
| guardrails | ||
| health_endpoints | ||
| hooks | ||
| image_endpoints | ||
| management_endpoints | ||
| management_helpers | ||
| middleware | ||
| openai_files_endpoint | ||
| pass_through_endpoints | ||
| policy_engine | ||
| prompts | ||
| public_endpoints | ||
| rag_endpoints | ||
| realtime_endpoints | ||
| response_api_endpoints | ||
| spend_tracking | ||
| test_configs | ||
| ui_crud_endpoints | ||
| vector_store_endpoints | ||
| __init__.py | ||
| conftest.py | ||
| test_aiohttp_cleanup_closed.py | ||
| test_aiohttp_session_recovery.py | ||
| test_api_key_masking_in_errors.py | ||
| test_audio_speech_prometheus_hooks.py | ||
| test_batch_expiry.py | ||
| test_batch_metadata_none_fix.py | ||
| test_caching_routes.py | ||
| test_chat_completion_metadata.py | ||
| test_common_request_processing.py | ||
| test_cors_config.py | ||
| test_custom_proxy.py | ||
| test_empty_model_list.py | ||
| test_enforce_user_param.py | ||
| test_fallback_management_endpoints.py | ||
| test_fastapi_offline_routes.py | ||
| test_health_check_functions.py | ||
| test_health_check_max_tokens.py | ||
| test_litellm_pre_call_utils.py | ||
| test_max_budget_env_var.py | ||
| test_model_dump_with_preserved_fields.py | ||
| test_model_id_header_propagation.py | ||
| test_model_info_default_limits.py | ||
| test_model_level_guardrails.py | ||
| test_openapi_schema_validation.py | ||
| test_prometheus_cleanup.py | ||
| test_proxy_cli.py | ||
| test_proxy_server.py | ||
| test_proxy_types.py | ||
| test_proxy_utils.py | ||
| test_pyroscope.py | ||
| test_response_model_sanitization.py | ||
| test_route_a2a_models.py | ||
| test_route_llm_request.py | ||
| test_shared_health_check.py | ||
| test_spend_log_cleanup.py | ||
| test_swagger_chat_completions.py | ||
| test_team_member_update.py | ||
| test_team_org_move.py | ||
| test_tools_allowlist_enforcement.py | ||
| test_update_llm_router_resilience.py | ||
| test_utils.py | ||