Update Bedrock documentation for Titan V2 encoding_format support + Anthropic - account for 1h vs. 5m cache creation token cost difference + UI - add langsmith_sampling_rate as a dynamic param
* add: generic object pool & tests
Introduced a reusable object pool that can be applied across the codebase.
Note: memory growth is managed via eviction settings—using a hard cap could
reduce performance, so eviction is the preferred safeguard.
* fix: simpler tests
* fix: avoid redundant __init__ calls on hot path
Previously, imports on the request hot path caused __init__ to run
excessively for every request. This change ensures initialization
happens once, reducing cpu overhead.
* fix: remove redundant __init__ import
The current implementation no longer requires an import at the top of the function.
* fix: placed on core utils for future reuse
* test: add coverage & remove inline import
A general import-checking tool across all endpoints would be a large PR.
This commit focuses on a smaller, targeted fix for the discussed case.
* added import check to CI
- Test encoding_format='float' parameter mapping and response handling
- Test encoding_format='base64' parameter mapping to binary format
- Verify parameter transformation and response processing
- Mock AWS API responses for both float and binary formats
- Ensure OpenAI compatibility with new encoding_format support
- Add Asia/Bangkok (UTC+7) to timezone_map in duration_parser.py
- Update documentation to include Bangkok in common timezone values
- Add test case to verify Bangkok timezone functionality
- Change parameter from request_metadata to requestMetadata to match camelCase convention
- Consistent with guardrailConfig and performanceConfig naming pattern
- Update all references in transformation code and error messages
- Update tests and documentation to use correct parameter name
- Fix type checking for parameter validation
- Test requestMetadata parameter support in get_supported_openai_params
- Test transformation to top-level field in Bedrock API request
- Test validation of AWS constraints: max 16 items, key/value length limits
- Test character set validation for keys and values
- Cover edge cases including empty values and special characters
- Ensure compatibility with existing test patterns
- Apply Black formatting to all Bedrock CountTokens files
- Clean up imports and remove unused variables in tests
- Fix indentation and simplify test structure
- Fix pyright type error with type ignore annotation
- All tests continue to pass after cleanup
- Add endpoint integration test in test_proxy_token_counter.py
- Add unit tests for transformation logic in bedrock/count_tokens/
- Test model extraction from request body vs endpoint path
- Test input format detection (converse vs invokeModel)
- Test request transformation from Anthropic to Bedrock format
- All tests follow existing codebase patterns and pass successfully
* fix: iscoroutine removed from hot path
* fix: replace all instances & separate concerns
1. Replaced all instances of iscoroutine with is_async_callable
2. Place the coroutine checker in its own file
* fix: PR comment changes
* fix: missing config setting declaration
* fix: revert non-performance related changes
* fix: revert to initial implementation
* fix: remove dead const
