mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-06 02:48:13 +00:00
14dd98cd5f
The explicit AssumeRole branch of BaseAWSLLM.get_credentials returned without touching the process-wide IAM cache, so every model request issued a fresh sts:AssumeRole, and on ECS/EC2 an uncached sts:GetCallerIdentity ahead of it. Route the whole role branch through _get_or_set_cached_credentials with the TTL _auth_with_aws_role already computed and discarded. The cache key is the same aws_* argument snapshot the other flows use, taken before the session-name default is filled in, so each aws_session_name keeps its own STS session and no attributed identity can be served another's credentials. Credential fetches now single-flight behind striped locks. Without that, a burst of concurrent misses on one key each issued their own STS call, which is the same thundering herd the cache exists to prevent, moved to the miss window. |
||
|---|---|---|
| .. | ||
| batches | ||
| chat | ||
| count_tokens | ||
| embed | ||
| files | ||
| image | ||
| image_edit | ||
| invoke_agent | ||
| messages/invoke_transformations | ||
| passthrough | ||
| realtime | ||
| rerank | ||
| vector_stores | ||
| __init__.py | ||
| test_anthropic_beta_support.py | ||
| test_base_aws_llm.py | ||
| test_bedrock_common_utils.py | ||
| test_bedrock_ssl_verify.py | ||
| test_claude_platform_provider.py | ||
| test_converse_context_management.py | ||
| test_cross_region_inference_profile_mapping.py | ||
| test_mantle.py | ||
| test_web_identity_session_policy.py | ||