litellm/tests
Richard Tweed 0ac093b59e
fix: role chaining and session name with webauthentication for aws bedrock (#13205)
* fix(bedrock): prevent duplicate role assumption in EKS/IRSA environments

Fixes issue where AWS role assumption would fail in EKS/IRSA environments
when trying to assume the same role that's already being used.

The problem occurred when:
1. EKS/IRSA automatically assumes a role (e.g., LitellmRole)
2. LiteLLM tries to assume the same role again, causing AccessDenied errors
3. Different models with different roles would fail due to incorrect role context

Changes:
- Added check in _auth_with_aws_role() to detect if already using target role
- Skip role assumption if current identity matches target role
- Return current credentials instead of attempting duplicate assumption
- Added comprehensive test coverage for the fix

This ensures proper role chaining works in EKS/IRSA environments where:
- Service Account can assume Role A
- Role A can assume Role B for different models/accounts

Resolves the AccessDenied errors reported in bedrock usage scenarios.

* fix(bedrock): simplify role assumption for EKS/IRSA environments

Fixes AWS Bedrock role assumption in EKS/IRSA environments by properly
handling ambient credentials when no explicit credentials are provided.

The issue occurred because commit 197e7efa8f
introduced changes that broke role assumption in EKS/IRSA environments.

Changes:
- Simplified _auth_with_aws_role() to use ambient credentials when no
  explicit AWS credentials are provided (aws_access_key_id and
  aws_secret_access_key are both None)
- This allows web identity tokens in EKS/IRSA to work automatically
  through boto3's credential chain
- Maintains backward compatibility for explicit credential scenarios

Added comprehensive test coverage:
- test_eks_irsa_ambient_credentials_used: Verifies ambient credentials work
- test_explicit_credentials_used_when_provided: Ensures explicit creds still work
- test_partial_credentials_still_use_ambient: Edge case handling
- test_cross_account_role_assumption: Multi-account scenarios
- test_role_assumption_with_custom_session_name: Custom session names
- test_role_assumption_ttl_calculation: TTL calculation verification
- test_role_assumption_error_handling: Error propagation
- test_multiple_role_assumptions_in_sequence: Sequential role assumptions

This fix ensures that in EKS/IRSA environments:
1. Service accounts can assume their initial role via web identity
2. That role can then assume other roles across accounts as configured
3. Different models can use different roles without conflicts

* fix(bedrock): add automatic IRSA detection for EKS environments

- Detect AWS_WEB_IDENTITY_TOKEN_FILE and AWS_ROLE_ARN environment variables
- Automatically use web identity token flow when IRSA is detected
- Read web identity token from file and pass to existing auth method
- Add test coverage for IRSA environment detection
- Fixes authentication errors in EKS with IRSA when no explicit credentials provided

* fix(bedrock): skip role assumption when IRSA role matches requested role

- Detect when AWS_ROLE_ARN environment variable matches the requested role
- Skip unnecessary role assumption when already running as the target role
- Use existing env vars authentication method for IRSA credentials
- Add test coverage for same-role IRSA scenario
- Fixes 'not authorized to perform: sts:AssumeRole' errors when trying to assume the same role

* fix(bedrock): use boto3's native IRSA support for cross-account role assumption

- Replace custom web identity token handling with boto3's built-in IRSA support
- boto3 automatically reads AWS_WEB_IDENTITY_TOKEN_FILE and assumes initial role
- Then use standard assume_role for cross-account access
- Update test to mock boto3 STS client instead of internal methods
- Fixes 'OIDC token could not be retrieved from secret manager' error

* fix(bedrock): improve IRSA error handling and add debug logging

- Add debug logging to show current identity and role assumption attempts
- Provide clearer error messages for trust policy issues
- Fix region handling in IRSA flow
- Re-raise exceptions instead of silently falling through
- This helps diagnose cross-account role assumption permission issues

* fix(bedrock): manually assume IRSA role with correct session name for cross-account scenarios

- When doing cross-account role assumption, manually assume the IRSA role first with the desired session name
- This ensures the session name in the assumed role ARN matches what's expected in trust policies
- For same-account scenarios, continue using boto3's automatic IRSA support
- Updated tests to handle the new flow
- This fixes the issue where cross-account trust policies require specific session names

* fix: Fix linting issues in base_aws_llm.py

- Fix f-string without placeholders (F541)
- Refactor _auth_with_aws_role to reduce statements count (PLR0915)
  - Extract _handle_irsa_cross_account helper method
  - Extract _handle_irsa_same_account helper method
  - Extract _extract_credentials_and_ttl helper method

---------

Co-authored-by: openhands <openhands@all-hands.dev>
2025-08-02 08:55:35 -07:00
..
basic_proxy_startup_tests (fix) don't block proxy startup if license check fails & using prometheus (#6839) 2024-11-20 17:55:39 -08:00
batches_tests openai.ConflictError 2025-07-12 17:07:21 -07:00
code_coverage_tests Fix - using managed files w/ OTEL + UI - add model group alias on UI (#13171) 2025-07-31 21:22:04 -07:00
documentation_tests Litellm dev 12 28 2024 p1 (#7463) 2024-12-28 20:26:00 -08:00
enterprise [Prometheus] Move Prometheus to enterprise folder (#12659) 2025-07-18 11:54:47 -07:00
guardrails_tests feat: Add Pillar Security guardrail integration (#12791) 2025-07-22 23:17:33 -07:00
image_gen_tests [Feat] Add Google AI Studio Imagen4 model family (#13065) 2025-07-28 21:25:40 -07:00
litellm-proxy-extras Prisma Migrate - support setting custom migration dir (#10336) 2025-04-26 12:05:06 -07:00
litellm_utils_tests test: update test 2025-07-29 21:08:36 -07:00
llm_responses_api_testing [MCP Guardrails] move pre and during hooks to ProxyLoggin (#13109) 2025-07-30 13:58:41 -07:00
llm_translation test fix xai - it goes through base llm tests already 2025-07-30 18:18:49 -07:00
load_tests test: test_embedding_performance 2025-05-14 21:31:07 -07:00
local_testing testing fixes - vertex ai deprecated claude 3 sonnet models 2025-08-01 21:23:52 -07:00
logging_callback_tests [Feat] Allow redacting message / response content for specific logging integrations - DD LLM Observability (#13158) 2025-07-31 16:44:16 -07:00
mcp_tests [MCP Gateway] Litellm mcp pre and during guardrails (#13188) 2025-08-01 20:02:25 -07:00
multi_instance_e2e_tests (e2e testing) - add tests for using litellm /team/ updates in multi-instance deployments with Redis (#8440) 2025-02-10 19:33:27 -08:00
old_proxy_tests/tests test: update tests to new deployment model (#10142) 2025-04-18 14:22:12 -07:00
openai_endpoints_tests Batches - support batch retrieve with target model Query Param + Anthropic - completion bridge, yield content_block_stop chunk (#12228) 2025-07-01 22:13:48 -07:00
otel_tests test_bedrock_guardrail_triggered 2025-07-09 17:05:06 -07:00
pass_through_tests test_anthropic_basic_completion_with_headers ci/cd fix 2025-07-12 11:08:03 -07:00
pass_through_unit_tests Azure api_version="preview" support + Bedrock cost tracking via Anthropic /v1/messages (#13072) 2025-07-29 08:13:55 -07:00
proxy_admin_ui_tests Fix e2e test (#12549) 2025-07-12 10:42:57 -07:00
proxy_security_tests (Security fix) - remove code block that inserts master key hash into DB (#8268) 2025-02-05 17:25:42 -08:00
proxy_unit_tests test: update test 2025-08-01 09:07:53 -07:00
router_unit_tests [Feat] Edit Auto Router Settings on UI (#12966) 2025-07-24 21:25:48 -07:00
scim_tests [Feat SSO] Add LiteLLM SCIM Integration for Team and User management (#10072) 2025-04-16 19:21:47 -07:00
spend_tracking_tests test_long_term_spend_accuracy_with_bursts 2025-03-31 21:09:29 -07:00
store_model_in_db_tests [MCP Gateway] add Litellm mcp alias for prefixing (#12994) 2025-07-25 17:57:52 -07:00
test_litellm fix: role chaining and session name with webauthentication for aws bedrock (#13205) 2025-08-02 08:55:35 -07:00
unified_google_tests [Bug Fix] Gemini-CLI Integration - ensure tool calling works as expected on generateContent (#13189) 2025-07-31 16:42:57 -07:00
vector_store_tests [Feat] UI Vector Stores - Allow adding Vertex RAG Engine, OpenAI, Azure (#12752) 2025-07-18 18:25:26 -07:00
windows_tests [Bug Fix] UnicodeDecodeError: 'charmap' on Windows during litellm import (#10542) 2025-05-03 21:31:05 -07:00
__init__.py [Feat] Add github co-pilot as a new LLM API provider (#12325) 2025-07-04 13:12:16 -07:00
gettysburg.wav feat(main.py): support openai transcription endpoints 2024-03-08 10:25:19 -08:00
large_text.py fix(router.py): check for context window error when handling 400 status code errors 2024-03-26 08:08:15 -07:00
openai_batch_completions.jsonl feat(router.py): Support Loadbalancing batch azure api endpoints (#5469) 2024-09-02 21:32:55 -07:00
README.MD add bedrock llama vision support + cohere / infinity rerank - 'return_documents' support (#8684) 2025-02-20 21:23:54 -08:00
test_budget_management.py Update enduser spend and budget reset date based on budget duration (#8460) 2025-06-08 08:39:14 -07:00
test_callbacks_on_proxy.py fix - test num callbacks 2024-05-17 22:06:51 -07:00
test_config.py fix testing - langfuse apis are flaky, we unit test team / key based logging in test_langfuse_unit_tests.py 2024-12-03 11:24:36 -08:00
test_debug_warning.py fix(utils.py): fix togetherai streaming cost calculation 2024-08-01 15:03:08 -07:00
test_end_users.py test: run test earlier to catch error 2025-03-27 23:08:52 -07:00
test_entrypoint.py (fix) clean up root repo - move entrypoint.sh and build_admin_ui to /docker (#6110) 2024-10-08 11:34:43 +05:30
test_fallbacks.py Ollama Chat - parse tool calls on streaming (#11171) 2025-05-27 16:14:49 -07:00
test_health.py (test) /health/readiness 2024-01-29 15:27:25 -08:00
test_keys.py test: temporarily skip test due to change testing model change - need to update test for new model 2025-05-09 09:02:08 -07:00
test_logging.conf feat(proxy_cli.py): add new 'log_config' cli param (#6352) 2024-10-21 21:25:58 -07:00
test_models.py Add model access groups on UI (#11719) 2025-06-13 21:20:25 -07:00
test_openai_endpoints.py test_chat_completion_ratelimit 2025-07-30 23:12:11 -07:00
test_organizations.py UI - fix adding vertex models with reusable credentials + fix pagination on keys table + fix showing org budgets on table (#10528) 2025-05-03 08:16:53 -07:00
test_passthrough_endpoints.py test test_basic_passthrough 2024-08-06 21:17:07 -07:00
test_ratelimit.py (Refactor / QA) - Use LoggingCallbackManager to append callbacks and ensure no duplicate callbacks are added (#8112) 2025-01-30 19:35:50 -08:00
test_resource_cleanup.py Fix: Properly close aiohttp client sessions to prevent resource leaks (#12251) 2025-07-09 09:25:17 -07:00
test_spend_logs.py (feat) - track org_id in SpendLogs (#8253) 2025-02-04 21:08:05 -08:00
test_team.py build: publish new litellm-proxy-extras file 2025-05-27 17:44:23 -07:00
test_team_logging.py test: skip flaky test 2024-11-22 19:23:36 +05:30
test_team_members.py test: add more unit testing for team member endpoints (#8170) 2025-02-01 11:23:00 -08:00
test_users.py test: fix imports 2025-05-26 22:06:53 -07:00

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/litellm

This folder can only run mock tests.