litellm/tests
Shivam Rawat fca672f190
added support for metadata (#24261)
* added support for metadata

* fix: PR review - meta truthiness, BlobResourceContents mimeType, add Blob+empty meta tests

Made-with: Cursor

* pyproject to .25

* feat(teams): resolve access group models/MCPs/agents in team endpoints

Add access_group_models, access_group_mcp_server_ids, and
access_group_agent_ids to /team/info and /v2/team/list responses.
These fields contain resources inherited from access groups, kept
separate from direct assignments so the UI can distinguish the source.

Backend: _resolve_access_group_resources() helper resolves access
group resources via existing _get_*_from_access_groups() functions.

UI: Teams table and detail view show direct models as blue badges
and access-group-sourced models as green badges.

* perf(teams): single-pass access group resolution + asyncio.gather in list endpoint

- Fetch each access group object once and extract all 3 resource fields
  in a single pass instead of 3 separate calls (3N → N lookups)
- Use asyncio.gather to resolve access groups across teams concurrently
  in list_team_v2 instead of sequential awaits
- Add 5 unit tests for _resolve_access_group_resources

* docs: add default_team_params to config reference and update examples

- Add default_team_params to litellm_settings reference table in
  config_settings.md with all sub-fields documented
- Update self_serve.md and msft_sso.md examples to include
  team_member_permissions, tpm_limit, and rpm_limit
- Fix misleading comment that implied default_team_params only applies
  to SSO auto-created teams — it applies to all /team/new calls

* docs: clarify that models sub-field only applies to SSO auto-created teams

* fix: lazy import get_access_object to break cyclic import + short-circuit all-proxy-models display

- Remove get_access_object from module-level import in team_endpoints.py
  and use a lazy _get_access_object wrapper to avoid cyclic dependency
- Add _prisma_client is None early-exit guard in _resolve_access_group_resources
- Short-circuit UI to show "All Proxy Models" when team.models is empty
  or contains "all-proxy-models", skipping access group model resolution

* add: making organizations a select instead of read only badges

* fix(ui): only send organization_id when changed and use raw initial value

* fix(ui): add paginated team search to usage page filter

Replace the static team dropdown on the usage page with a new
TeamMultiSelect component that uses the paginated v2/team/list
endpoint with debounced server-side search and infinite scroll.

* fix(ui): fix imports and update placeholder for team multi select

* fix(ui): wire team_id filter to key alias dropdown on Virtual Keys tab

The Key Alias dropdown on the Virtual Keys page was showing aliases from
all teams regardless of which team was selected. The team_id was never
passed through the frontend chain to the backend /key/aliases endpoint.

- Backend: add optional team_id query param to /key/aliases endpoint
- networking.tsx: add team_id param to keyAliasesCall
- useKeyAliases: accept and forward team_id to API call and query key
- filter.tsx: pass allFilters context to custom filter components
- PaginatedKeyAliasSelect: read Team ID from allFilters and pass to hook

* fix(tests): correct mock targets in TestResolveAccessGroupResources

Three tests were patching the non-existent `get_access_object` instead
of `_get_access_object` (the lazy-import wrapper), causing AttributeError.
Also added missing `prisma_client` mock so tests get past the early-exit
guard and actually exercise the resolution logic.

* fix: use direct attribute access with or [] fallback in _resolve_access_group_resources

Replace getattr(ag, "field", []) with ag.field or [] for cleaner
access and safe handling if a field is None.

* fix(ui): remove model source legend from team detail view

The blue/green color distinction is self-explanatory; the legend added
visual clutter without providing enough value.

* fix(ui): add missing access_group fields to TeamData.team_info type

The TeamData interface was missing access_group_models,
access_group_mcp_server_ids, and access_group_agent_ids fields,
causing a TypeScript build failure.

* perf(teams): batch-fetch access groups in single DB query

Replace per-ID _resolve_access_group_resources loop with a single
find_many call that deduplicates IDs across all teams. Removes the
N+1 query pattern on cold cache for the team list endpoint.

* refactor(proxy): extract helpers to fix PLR0915 violations

Extract `_apply_non_admin_alias_scope` from `key_aliases`,
`_resolve_team_access_group_resources` from `team_info`, and
`_enforce_list_team_v2_access` from `list_team_v2` to bring each
function under ruff's 50-statement limit. No behavior changes.

* test(ui): update tests to match new team_id / access-group signatures

- useKeyAliases, PaginatedKeyAliasSelect: add trailing `undefined` to
  spy matchers for the new `team_id` param on `useInfiniteKeyAliases`
  and `keyAliasesCall`.
- EntityUsage: mock new `TeamMultiSelect` child so QueryClientProvider
  is not required for team-entity tests.
- ModelsCell: replace the overflow-accordion test with one that
  verifies the new collapse-on-`all-proxy-models` behavior (no
  accordion, single badge).

* fix(ui): send null (not '') for cleared organization_id on team update

AntD <Select allowClear> returns undefined when the user clears the
selection. Coalescing to "" caused the team-update payload to carry
organization_id: "" instead of null, relying on the backend to coerce
it. Send null directly so the intent is explicit at the source.

* poetry

* chore: regen poetry.lock for litellm-proxy-extras 0.4.64 bump

* chore: update Next.js build artifacts (2026-04-04 17:55 UTC, node v22.16.0)

---------

Co-authored-by: shivam <shivam@uni.minerva.edu>
Co-authored-by: Ryan Crabbe <ryan@berri.ai>
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
2026-04-04 12:22:58 -07:00
..
agent_tests refactor: make unit test 2026-03-28 21:44:46 -07:00
audio_tests test: cleanup test impacted by deployment name change 2026-03-30 18:07:50 -07:00
basic_proxy_startup_tests CircleCI test stability (#23055) 2026-03-07 15:19:39 -08:00
batches_tests test fix us.anthropic.claude-haiku-4-5-20251001-v1:0 (#24931) 2026-04-01 11:01:03 -07:00
benchmarks Add CodSpeed performance benchmarks (#23676) 2026-03-14 18:44:36 -07:00
code_coverage_tests [Fix] Remove unused aioboto3 dependency and botocore conflict workarounds 2026-04-01 14:25:44 -07:00
documentation_tests [Fix] Scope documentation workflow to match CircleCI and add missing router settings 2026-03-28 11:23:53 -07:00
enterprise test fix 2026-03-30 18:08:57 -07:00
guardrails_tests Add Akto Guardrails to LiteLLM (#23250) 2026-03-17 14:38:04 -07:00
image_gen_tests fix: cleanup tests 2026-03-30 16:24:35 -07:00
litellm fix(gemini-embeddings): convert task_type to camelCase taskType for Gemini API (#24191) 2026-03-20 22:32:22 +05:30
litellm-proxy-extras [Infra] Mirror litellm_table_patch source changes (no binaries) 2026-03-27 16:45:12 -07:00
litellm_core_utils
litellm_utils_tests Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00
llm_responses_api_testing test: remove bad e2e tests - e2e failing due to low rate limits on ci/cd 2026-03-30 19:18:10 -07:00
llm_translation [Infra] Merge internal dev branch with main (#25036) 2026-04-02 16:38:01 -07:00
load_tests test: update to new vertex ai keys 2026-03-28 20:19:05 -07:00
local_testing Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00
logging_callback_tests [Test Fix] fix gov pricing tests (#25022) 2026-04-02 15:55:55 -07:00
mcp_tests [Infra] Merging RC Branch with Main (#23786) 2026-03-16 15:32:20 -07:00
multi_instance_e2e_tests
ocr_tests test: update to new vertex ai keys 2026-03-28 20:19:05 -07:00
old_proxy_tests/tests fix: cleanup tests 2026-03-30 16:24:35 -07:00
openai_endpoints_tests Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00
otel_tests fix(tests): increase MAX_CALLS and reduce sleep in flaky e2e budget test 2026-03-13 00:04:31 -07:00
pass_through_tests Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00
pass_through_unit_tests Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00
proxy_admin_ui_tests [Fix] Remove neon CLI dependency and pin all JS dependencies 2026-04-01 16:15:32 -07:00
proxy_e2e_anthropic_messages_tests test fix us.anthropic.claude-haiku-4-5-20251001-v1:0 (#24931) 2026-04-01 11:01:03 -07:00
proxy_e2e_azure_batches_tests Increase file deletion retry budget to 50s for batch_processed race 2026-03-15 15:39:35 -07:00
proxy_security_tests
proxy_unit_tests Tag query fix (#25094) 2026-04-04 09:51:46 -07:00
router_unit_tests fix(router): address remaining Greptile review comments 2026-03-27 20:11:28 +05:30
scim_tests
search_tests Merge remote main, resolve conflict keeping new unit tests 2026-03-09 15:20:20 -07:00
spend_tracking_tests [Staging] - Ishaan March 17th (#23903) 2026-03-18 15:09:01 -07:00
store_model_in_db_tests test: fix assemblyai passthrough transcription request format 2026-03-30 18:49:04 -07:00
test_litellm added support for metadata (#24261) 2026-04-04 12:22:58 -07:00
unified_google_tests test: update to new vertex ai keys 2026-03-28 20:19:05 -07:00
vector_store_tests test: update test apis 2026-03-28 18:57:27 -07:00
windows_tests
__init__.py
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
README.MD
test_budget_management.py
test_callbacks_on_proxy.py
test_config.py
test_debug_warning.py
test_default_encoding_non_root.py fix: tiktoken cache nonroot offline (#23498) 2026-03-14 10:48:36 -07:00
test_end_users.py
test_entrypoint.py
test_fallbacks.py
test_gpt5_azure_temperature_support.py [Fix] Remove deprecated o1-preview from O-series test and deduplicate is_o_series check 2026-03-12 13:22:00 -07:00
test_health.py
test_keys.py
test_litellm_proxy_responses_config.py
test_logging.conf
test_models.py test: replace test_add_and_delete_models integration test with mock 2026-03-30 21:30:57 -07:00
test_new_vector_store_endpoints.py fix(ci): stabilize CI tests - conditional import, mock fixes, timing adjustments 2026-03-13 00:01:25 +00:00
test_openai_endpoints.py
test_organizations.py
test_otel_thread_leak.py
test_passthrough_endpoints.py
test_presidio_latency.py
test_proxy_server_non_root.py
test_ratelimit.py
test_resource_cleanup.py
test_service_logger_otel.py
test_spend_logs.py
test_team.py [Fix] Fail fast when team member spend not flushed in time 2026-03-13 12:40:17 -07:00
test_team_logging.py test: cleanup dead tests 2026-03-28 20:49:02 -07:00
test_team_members.py
test_users.py Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/test_litellm

This folder can only run mock tests.