Commit graph

37665 commits

Author SHA1 Message Date
Josh Bonczkowski
bf32afb0f2 Configuration for the logging callback definition. 2026-04-24 15:00:05 -04:00
Josh Bonczkowski
ddf74b1b84 Updating linting, adding tests, defining data type for UI. 2026-04-24 15:00:05 -04:00
Josh Bonczkowski
ccaa8c5556 Validating testing / formatting for our changes. 2026-04-24 15:00:05 -04:00
Josh Bonczkowski
615a060d73 Addressing feedback. 2026-04-24 15:00:05 -04:00
Josh Bonczkowski
f0d9b96f7d Addressing some feedback. 2026-04-24 15:00:05 -04:00
Josh Bonczkowski
6dc714506a Addressed feedback where the newrelic options might not apply correctly. 2026-04-24 15:00:04 -04:00
Josh Bonczkowski
d2fd373e1c Addressing feedback.
- A few more cases were found where the dictionary access might not return the correct value.
- Handling cases where `traceparent` is not lower cased
2026-04-24 15:00:04 -04:00
Josh Bonczkowski
511fca4d9c Addressing minor feedback on length of generated, fallback traceId. 2026-04-24 15:00:04 -04:00
Josh Bonczkowski
ad00671fbb Addressing feedback about a potential issue around a timestamp for the
supportability metric.
2026-04-24 15:00:04 -04:00
Josh Bonczkowski
8aeaaa3808 Adding a new catch exception block based on feedback. 2026-04-24 15:00:04 -04:00
Josh Bonczkowski
6101890893 Addressing feedback on accessing default values. Added tests for most of
these cases.
2026-04-24 15:00:04 -04:00
Josh Bonczkowski
37d8be3779 Addressing feedback for potential better practice. 2026-04-24 15:00:04 -04:00
Josh Bonczkowski
807b8ed72d Updating the NR event timestamps to more accurately reflect when
the messages were generated.
2026-04-24 15:00:04 -04:00
Josh Bonczkowski
855dd592c8 Addressing minor feedback
- Put a cap on newrelic version on v13. v12 just released and is still compatible.
- Ensured the shell script ends in a newline
2026-04-24 15:00:04 -04:00
Josh Bonczkowski
aba60d19ae Addressing feedback.
- When an error occurs, still check if the periodic supportability metric should be emitted
- Added a check to ensure the extension is ready in the error handler to match _process_success
2026-04-24 15:00:04 -04:00
Josh Bonczkowski
4b177e23ff Addressing new feedback.
- Proper handling of time to floats. Created a util method and updated code to use it.
- added the missing guard to ensure the app is enabled
2026-04-24 15:00:04 -04:00
Josh Bonczkowski
6fb7a586eb Addressing additional automated feedback.
- Removed a legacy comment about the New Relic header
- Reordered imports in one file
- Switched another file to use the import at the top of the file instead of inline when used
- Added unit tests for untested methods that were identified
2026-04-24 15:00:04 -04:00
Josh Bonczkowski
9ff663e8bb Reformating for the linter. 2026-04-24 15:00:04 -04:00
Josh Bonczkowski
e9651e2180 Address reviewer feedback on New Relic integration.
- Fix _record_error_metric to use app.record_custom_metric() instead of
  module-level newrelic.agent.record_custom_metric() so the call works
  outside of an active transaction context
- Remove unreachable except ImportError block in _get_trace_context
- Update stale "23 hours" comment to "27 hours" (matches 97200s threshold)
- Remove commented-out debug code from _process_success
- Fix docs typo: NEW_RELIC_CUSTOM_INSIGHTS_EVENTS_MAX_SAMPLES_STOREDA ->
  NEW_RELIC_CUSTOM_INSIGHTS_EVENTS_MAX_SAMPLES_STORED
- Update TestRecordErrorMetric to verify app.record_custom_metric call

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 15:00:04 -04:00
Josh Bonczkowski
3db9fbc24a Updating linting and the unit tests that are not running in the CI environment. 2026-04-24 15:00:04 -04:00
Josh Bonczkowski
183b95d46f Adding unit tests for the New Relic extension. 2026-04-24 15:00:04 -04:00
Josh Bonczkowski
522d3f43c7 Moving New Relic imports into the methods to support unit tests. 2026-04-24 15:00:04 -04:00
Josh Bonczkowski
283e381a0b Updating LiteLLM documentation for New Relic extension. 2026-04-24 15:00:04 -04:00
Josh Bonczkowski
3e6ab1d28b Updating newrelic.md document. 2026-04-24 15:00:04 -04:00
Josh Bonczkowski
bc27cb17ad Use a proper name for the doc file. 2026-04-24 15:00:04 -04:00
Josh Bonczkowski
2eaae4ee66 Initial version of public doc for New Relic. 2026-04-24 15:00:04 -04:00
Josh Bonczkowski
35f06ac245 Removing debugging.
Fixed logic / comments around how often to send supportability metric.
2026-04-24 15:00:04 -04:00
Josh Bonczkowski
34e6866ca9 Cleaning up logic to turn off recording content via either the
LiteLLM configuration or an env var.
2026-04-24 15:00:04 -04:00
Josh Bonczkowski
1dd2a4f007 Mised file. 2026-04-24 15:00:04 -04:00
Josh Bonczkowski
994a506a05 Implementing a configuration so that users can use litellm configuration
to disable sending LLM messages to New Relic. There is a second method
to do this via New Relic env var.
2026-04-24 15:00:04 -04:00
Josh Bonczkowski
5100a4b6c7 Use a fake trace_id if we cannot find one. 2026-04-24 15:00:04 -04:00
Josh Bonczkowski
db149980c4 If we cannot find trace information, send the AI events without the
trace ID attached.
2026-04-24 15:00:04 -04:00
Josh Bonczkowski
129d2ba505 Enabling New Relic agent to be wired when docker container starts if an environment variable
is set.
2026-04-24 15:00:04 -04:00
Josh Bonczkowski
6247823ac2 Make use of turn_off_message_logging configuration that is available
by default from CustomLogger.
2026-04-24 15:00:04 -04:00
Josh Bonczkowski
f86ccdf4b5 Add the start/end times to the messages before sending them so that the
start time and end time reflect the correct time and both are not set
to 'now'.
2026-04-24 15:00:04 -04:00
Josh Bonczkowski
c6ea6b883c Emit supportability metric every hour instead of once a day. 2026-04-24 15:00:04 -04:00
Josh Bonczkowski
478d8d1d52 Supportability metric is sent on first request. 2026-04-24 15:00:04 -04:00
Josh Bonczkowski
d73498f77d Implemented basic support for the success path. Generates New Relic
custom events needed by the AI Monitorin interface.
2026-04-24 15:00:04 -04:00
Josh Bonczkowski
efbcd04e03 Minor fixes for basic observability. 2026-04-24 15:00:04 -04:00
Josh Bonczkowski
085520a901 initial New Relic integration. 2026-04-24 15:00:04 -04:00
shubham-arora-clear
e5786c6c35
fix(bedrock): preserve cache_control TTL on tools for Claude 4.5+ (#25855)
Bedrock enforces non-increasing TTL ordering across cache_control blocks
(tools → system → messages). The tool cache_control TTL was being
unconditionally dropped to the default 5m, while system blocks preserved
the user-specified TTL for Claude 4.5+ models. This mismatch caused
"a ttl='1h' block must not come after a ttl='5m' block" errors when
users set ttl='1h' on both tools and system.

Converse path: add_cache_point_tool_block() now accepts a model param
and preserves TTL for Claude 4.5+, matching _get_cache_point_block().

Invoke path: _remove_ttl_from_cache_control() now also processes tools
(was only processing system and messages).

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-23 21:54:26 -07:00
yuneng-jiang
e9e86ed956
Merge pull request #26298 from BerriAI/litellm_internal_staging
[Infra] Promote interal staging to main
2026-04-22 19:16:23 -07:00
yuneng-jiang
6a25866f51
Merge pull request #26295 from BerriAI/yj_bump_apr22
Some checks are pending
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (key-generation, tests/proxy_unit_tests/test_key_generate_prisma.py, 30, 0) (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (proxy-utils, tests/proxy_unit_tests/test_proxy_utils.py, 20, 8) (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py --ignore=tests/proxy_unit_tests/test_p… (push) Waiting to run
Unit Tests: Security / security (push) Waiting to run
[Infra] bump versions
2026-04-22 18:33:03 -07:00
yuneng-jiang
8bc6948a5e
Merge pull request #26290 from BerriAI/litellm_/bold-goldstine-b86b67
[Fix] Stabilize spend accuracy test transport flakes
2026-04-22 18:30:10 -07:00
Yuneng Jiang
95fa7678af
uv lock 2026-04-22 18:25:37 -07:00
Yuneng Jiang
9f46d838fd
bump: version 1.83.11 → 1.83.12 2026-04-22 18:21:47 -07:00
Yuneng Jiang
3ddb3cbdf6
bump: version 0.4.67 → 0.4.68 2026-04-22 18:20:21 -07:00
ryan-crabbe-berri
c4c1861389
Merge pull request #26195 from BerriAI/litellm_team_member_total_spend
Track per-member total spend on team memberships
2026-04-22 18:20:16 -07:00
michelligabriele
c67d193400
fix(docker.non_root): use numeric UID 65534 for K8s runAsNonRoot (#26268) 2026-04-22 18:00:04 -07:00
Yuneng Jiang
a292845dcf
[Fix] Harden spend accuracy test against transient aiohttp connection errors
Two changes, both test-only:

- Configure the aiohttp session with TCPConnector(force_close=True) and an
  explicit ClientTimeout(total=30, connect=10). Prevents reuse of idle TCP
  connections that the proxy/kernel may have closed during the long window
  between setup POSTs and the later poll loop, and surfaces a blocked proxy
  event loop quickly instead of hanging on aiohttp's 5-minute default.

- In poll_key_spend_until, catch aiohttp.ClientError and asyncio.TimeoutError
  around the single /key/info call. A transient transport hiccup now logs and
  retries on the next tick instead of failing the entire polling loop.

Addresses the ConnectionTimeoutError observed on the first /key/info call
after the 20 chat completions.
2026-04-22 17:40:42 -07:00