mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-04 07:27:51 +00:00
250af8f027
* fix(responses): log the encrypted response id on streamed /v1/responses spend rows
With response id security on, the stream hook encrypts the resp_ id on every
frame, but logging had already received the SDK's own copy of the terminal
response, so the spend row, the standard logging payload and callbacks got the
managed id (or the chatcmpl id on a bridged stream) the client never saw. The
hook now records the exact id it advertises on the request's logging object,
and the logging layer stamps that id on the logged streamed response
* fix(responses): return the id the client sent on GET and cancel, and keep read rows off the create row's id
Every GET /v1/responses/{id} re-encrypted the response id, so a client that continued a bridged conversation with the
id a GET returned referenced a ciphertext no spend row was filed under. The follow-up lost turn 1 and opened a new
session. GET and cancel now answer with the exact id the client sent when it is an id this proxy issued and it decodes
to the response the provider returned. A create with previous_response_id still gets a fresh id for its own response,
and a raw provider id still gets a managed id
Because a read now carries the create's id, a GET or cancel spend row is filed as <id>_retrieve_<call id> or
<id>_cancel_<call id>. That keeps one row per read, as before, and a read row can never take the create row's
request_id, so the create row survives whichever row is written first and a follow-up replays turn 1 once. The
background settlement row keeps <id>_background_cost, and the cost poll's own status checks keep the response id
* fix(responses): log a fresh id for each read and cancel, echo the held id only to its owner
A GET or cancel of a stored response now logs a new encrypted id of its own,
so callbacks and spend rows never reuse the create's id, while the owner of
the id still gets back the exact id they sent. An admin reading someone
else's response gets an id bound to the admin, as before. The spend row id
suffix for reads and cancels is gone, so with response id security off the
rows and session history match the default branch. A streamed GET follows
the same rules.
* fix(responses): echo the sent id to every reader that passes the response access check
A GET, streamed GET or cancel of a response now answers with the id the caller sent whenever the caller passed
the access check, not only when the caller is the user or team the id was issued to. An admin, the master key
or a teammate who read someone else's streamed response got a fresh id back, and the session lookup could not
reach the first turn from it, so a bridged follow-up through that id lost the conversation history
Every read and cancel still logs a fresh id bound to the reader, never the create's id
* fix(responses): keep a read or cancel row in session history only when it brings new output
A GET, streamed GET or cancel sent with a session header joins the
create's session, and its stored response repeats the create's output.
The replay then sent the same assistant turn once per read.
Session history now keeps an aget_responses or acancel_responses row
only when its stored Responses output has an item id that no earlier
kept row carried. A polled background response keeps the first poll
that returns the finished output, since its queued create row has none,
and later polls of the same output are dropped. Every other row keeps
today's rule
* fix(responses): replay each history item once, echo the sent id on delete, keep background cost files apart
A create row now owns its output items in session history, and a read or cancel row adds only items no
create and no earlier read carried, in one linear pass, so a background response polled while it grows,
or a read that ends before its streamed create, replays the answer once and after its prompt
DELETE /v1/responses/{id} answers with the id the caller sent, on a copy, and logs a fresh encrypted id
like a read does
Azure Data Lake Storage per-event files and unbatched GCS objects are named after the event's spend row
id, so a streamed background create and its cost event, which now share the client's id, keep separate
files
The integration tests that decoded the managed id now match the exact id the client received
* fix(responses): keep main's answer for a raw-id delete and main's storage names outside background cost events
A DELETE by a raw provider id returns the provider's id as main did, and logging still records a fresh
encrypted id. A streamed GET matches the held id by upstream response id, like a plain GET
Only a background response's cost event gets the _background_cost suffix on its Azure Data Lake Storage
file and unbatched GCS object, so batch retrieves and events with no id keep main's names
An item a session read replays takes the newest read's copy at the first read's place, so an item a
poll caught half written replays finished
Tests pin the client's id over the router's continued response id, and the storage test no longer reads
the clock after the upload
* fix(spend): give background cost events their own OpenMeter and DynamoDB ids, replay id-less read items
A streamed background create and its later cost event share the client's
id, so OpenMeter dropped the second event and DynamoDB overwrote the
create's item. Both now key the cost event as <id>_background_cost, like
its spend row and storage file.
Session history again replays a read's output items that carry no id,
as main did.
* fix(integrations): keep OpenMeter and DynamoDB logging working without the proxy extras
* fix(integrations): give background cost events their own ClickHouse request id and DeepEval span id
A streamed background create and its later cost event now share the id the
client received, so ClickHouse and DeepEval saw both events under one id.
Route both through logged_event_key, the way the Azure storage, GCS,
OpenMeter and DynamoDB loggers already do, so the cost event's ClickHouse
request_id matches its spend row's `<id>_background_cost` and its DeepEval
span gets its own uuid. ClickHouse response_id stays the raw provider id.
* test(deepeval): annotate the background cost span test's locals with Final
* test(responses): run the DELETE wire test that pins the echoed client id
DELETE /v1/responses/{id} now answers with the id the caller sent, so the
skip that marked it as a known bug no longer applies. The test passes on
the /v1, /openai/v1 and bare routes.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
531 lines
19 KiB
Python
531 lines
19 KiB
Python
import json
|
|
import os
|
|
from collections.abc import Mapping
|
|
from typing import Final
|
|
from unittest.mock import AsyncMock, MagicMock, patch
|
|
|
|
import pytest
|
|
|
|
import litellm
|
|
from litellm.constants import INTERNAL_CALL_ORIGIN_METADATA_KEY
|
|
from litellm.integrations.openmeter import OpenMeterLogger
|
|
from litellm.litellm_core_utils.internal_call_metadata import BACKGROUND_RESPONSE_COST_REQUEST_ID_SUFFIX
|
|
from litellm.types.passthrough_endpoints.pass_through_endpoints import (
|
|
SETTLES_BACKGROUND_RESPONSE_KWARG,
|
|
BackgroundResponseSettlement,
|
|
)
|
|
from litellm.types.utils import BACKGROUND_RESPONSE_COST_POLL_CALL_ORIGIN
|
|
|
|
BACKGROUND_COST_POLL_KWARGS: Final[Mapping[str, object]] = {
|
|
"call_type": "aget_responses",
|
|
"litellm_params": {"metadata": {INTERNAL_CALL_ORIGIN_METADATA_KEY: BACKGROUND_RESPONSE_COST_POLL_CALL_ORIGIN}},
|
|
}
|
|
PASSTHROUGH_SETTLEMENT_KWARGS: Final[Mapping[str, object]] = {
|
|
"call_type": "pass_through_endpoint",
|
|
"litellm_params": {"metadata": {}},
|
|
SETTLES_BACKGROUND_RESPONSE_KWARG: BackgroundResponseSettlement.SETTLES,
|
|
}
|
|
|
|
|
|
class TestOpenMeterIntegration:
|
|
"""Test suite for OpenMeter integration"""
|
|
|
|
def setup_method(self):
|
|
"""Set up test environment"""
|
|
# Set required environment variables
|
|
os.environ["OPENMETER_API_KEY"] = "test-api-key"
|
|
os.environ["OPENMETER_API_ENDPOINT"] = "https://test.openmeter.com"
|
|
|
|
def teardown_method(self):
|
|
"""Clean up test environment"""
|
|
# Clean up environment variables
|
|
os.environ.pop("OPENMETER_API_KEY", None)
|
|
os.environ.pop("OPENMETER_API_ENDPOINT", None)
|
|
os.environ.pop("OPENMETER_EVENT_TYPE", None)
|
|
os.environ.pop("OPENMETER_TRUST_REQUEST_USER", None)
|
|
|
|
def test_openmeter_logger_initialization(self):
|
|
"""Test that OpenMeterLogger initializes correctly with required env vars"""
|
|
logger = OpenMeterLogger()
|
|
assert logger is not None
|
|
|
|
def test_openmeter_logger_missing_api_key(self):
|
|
"""Test that OpenMeterLogger raises exception when API key is missing"""
|
|
os.environ.pop("OPENMETER_API_KEY", None)
|
|
with pytest.raises(Exception, match=r"Missing keys.*OPENMETER_API_KEY"):
|
|
OpenMeterLogger()
|
|
|
|
@pytest.mark.parametrize(
|
|
"cost_kwargs",
|
|
(
|
|
pytest.param(BACKGROUND_COST_POLL_KWARGS, id="cost-poll"),
|
|
pytest.param(PASSTHROUGH_SETTLEMENT_KWARGS, id="passthrough-settlement"),
|
|
),
|
|
)
|
|
@pytest.mark.usefixtures("sdk_only_install")
|
|
def test_background_cost_event_gets_its_own_event_id_beside_its_create(
|
|
self, cost_kwargs: Mapping[str, object]
|
|
) -> None:
|
|
logger: Final = OpenMeterLogger()
|
|
response_obj: Final = {"id": "resp_YWJjZA"}
|
|
create_kwargs: Final[Mapping[str, object]] = {
|
|
"user": "user-1",
|
|
"call_type": "aresponses",
|
|
"litellm_params": {"metadata": {}},
|
|
}
|
|
|
|
create_event: Final = logger._common_logic(create_kwargs, response_obj)
|
|
cost_event: Final = logger._common_logic({**cost_kwargs, "user": "user-1", "response_cost": 0.42}, response_obj)
|
|
|
|
assert create_event["id"] == "resp_YWJjZA"
|
|
assert cost_event["id"] == f"resp_YWJjZA{BACKGROUND_RESPONSE_COST_REQUEST_ID_SUFFIX}"
|
|
|
|
def test_common_logic_with_string_user(self):
|
|
"""Test that _common_logic correctly handles string user parameter"""
|
|
logger = OpenMeterLogger()
|
|
|
|
kwargs = {
|
|
"user": "test-user-123",
|
|
"model": "gpt-3.5-turbo",
|
|
"response_cost": 0.001,
|
|
"litellm_call_id": "test-call-id",
|
|
}
|
|
|
|
# Mock response object
|
|
response_obj = {
|
|
"id": "test-response-id",
|
|
"usage": {"prompt_tokens": 10, "completion_tokens": 5, "total_tokens": 15},
|
|
}
|
|
|
|
result = logger._common_logic(kwargs, response_obj)
|
|
|
|
# Verify subject is a string, not a tuple
|
|
assert isinstance(result["subject"], str)
|
|
assert result["subject"] == "test-user-123"
|
|
assert result["data"]["model"] == "gpt-3.5-turbo"
|
|
assert result["data"]["cost"] == 0.001
|
|
|
|
def test_common_logic_with_integer_user(self):
|
|
"""Test that _common_logic correctly converts integer user to string"""
|
|
logger = OpenMeterLogger()
|
|
|
|
kwargs = {
|
|
"user": 12345, # Integer user ID
|
|
"model": "gpt-4",
|
|
"response_cost": 0.002,
|
|
"litellm_call_id": "test-call-id-2",
|
|
}
|
|
|
|
response_obj = {
|
|
"id": "test-response-id-2",
|
|
"usage": {"prompt_tokens": 20, "completion_tokens": 10, "total_tokens": 30},
|
|
}
|
|
|
|
result = logger._common_logic(kwargs, response_obj)
|
|
|
|
# Verify subject is converted to string
|
|
assert isinstance(result["subject"], str)
|
|
assert result["subject"] == "12345"
|
|
|
|
def test_common_logic_missing_user(self):
|
|
"""Test that _common_logic raises exception when user is missing"""
|
|
logger = OpenMeterLogger()
|
|
|
|
kwargs = {
|
|
"model": "gpt-3.5-turbo",
|
|
"response_cost": 0.001,
|
|
"litellm_call_id": "test-call-id",
|
|
}
|
|
|
|
response_obj = {"id": "test-response-id"}
|
|
|
|
with pytest.raises(Exception, match="OpenMeter: user is required"):
|
|
logger._common_logic(kwargs, response_obj)
|
|
|
|
def test_common_logic_none_user(self):
|
|
"""Test that _common_logic raises exception when user is None"""
|
|
logger = OpenMeterLogger()
|
|
|
|
kwargs = {
|
|
"user": None,
|
|
"model": "gpt-3.5-turbo",
|
|
"response_cost": 0.001,
|
|
"litellm_call_id": "test-call-id",
|
|
}
|
|
|
|
response_obj = {"id": "test-response-id"}
|
|
|
|
with pytest.raises(Exception, match="OpenMeter: user is required"):
|
|
logger._common_logic(kwargs, response_obj)
|
|
|
|
def test_common_logic_empty_string_user(self):
|
|
"""Test that _common_logic correctly handles an empty string user"""
|
|
logger = OpenMeterLogger()
|
|
|
|
kwargs = {
|
|
"user": "",
|
|
"model": "gpt-3.5-turbo",
|
|
"response_cost": 0.001,
|
|
"litellm_call_id": "test-call-id",
|
|
}
|
|
|
|
response_obj = {"id": "test-response-id"}
|
|
|
|
result = logger._common_logic(kwargs, response_obj)
|
|
assert isinstance(result["subject"], str)
|
|
assert result["subject"] == ""
|
|
|
|
@patch("litellm.integrations.openmeter.HTTPHandler")
|
|
def test_log_success_event(self, mock_http_handler):
|
|
"""Test synchronous log_success_event method"""
|
|
mock_post = MagicMock()
|
|
mock_http_handler.return_value.post = mock_post
|
|
|
|
logger = OpenMeterLogger()
|
|
|
|
kwargs = {
|
|
"user": "test-user",
|
|
"model": "gpt-3.5-turbo",
|
|
"response_cost": 0.001,
|
|
"litellm_call_id": "test-call-id",
|
|
}
|
|
|
|
response_obj = {
|
|
"id": "test-response-id",
|
|
"usage": {"prompt_tokens": 10, "completion_tokens": 5, "total_tokens": 15},
|
|
}
|
|
|
|
logger.log_success_event(kwargs, response_obj, None, None)
|
|
|
|
# Verify HTTP call was made
|
|
mock_post.assert_called_once()
|
|
|
|
# Verify the data structure
|
|
call_args = mock_post.call_args
|
|
data = json.loads(call_args[1]["data"])
|
|
|
|
assert data["subject"] == "test-user"
|
|
assert isinstance(data["subject"], str)
|
|
assert data["data"]["model"] == "gpt-3.5-turbo"
|
|
|
|
@patch("litellm.integrations.openmeter.get_async_httpx_client")
|
|
@pytest.mark.asyncio
|
|
async def test_async_log_success_event(self, mock_get_client):
|
|
"""Test asynchronous log_success_event method"""
|
|
mock_post = AsyncMock()
|
|
mock_client = MagicMock()
|
|
mock_client.post = mock_post
|
|
mock_get_client.return_value = mock_client
|
|
|
|
logger = OpenMeterLogger()
|
|
|
|
kwargs = {
|
|
"user": "async-test-user",
|
|
"model": "gpt-4",
|
|
"response_cost": 0.002,
|
|
"litellm_call_id": "async-test-call-id",
|
|
}
|
|
|
|
response_obj = {
|
|
"id": "async-test-response-id",
|
|
"usage": {"prompt_tokens": 20, "completion_tokens": 10, "total_tokens": 30},
|
|
}
|
|
|
|
await logger.async_log_success_event(kwargs, response_obj, None, None)
|
|
|
|
# Verify async HTTP call was made
|
|
mock_post.assert_called_once()
|
|
|
|
# Verify the data structure
|
|
call_args = mock_post.call_args
|
|
data = json.loads(call_args[1]["data"])
|
|
|
|
assert data["subject"] == "async-test-user"
|
|
assert isinstance(data["subject"], str)
|
|
assert data["data"]["model"] == "gpt-4"
|
|
|
|
def test_cloudevents_structure(self):
|
|
"""Test that the CloudEvents structure is correct"""
|
|
logger = OpenMeterLogger()
|
|
|
|
kwargs = {
|
|
"user": "cloudevents-test-user",
|
|
"model": "gpt-3.5-turbo",
|
|
"response_cost": 0.001,
|
|
"litellm_call_id": "cloudevents-test-call-id",
|
|
}
|
|
|
|
response_data = {
|
|
"id": "cloudevents-test-response-id",
|
|
"usage": {"prompt_tokens": 15, "completion_tokens": 8, "total_tokens": 23},
|
|
}
|
|
response_obj = litellm.ModelResponse(**response_data)
|
|
|
|
result = logger._common_logic(kwargs, response_obj)
|
|
|
|
# Verify CloudEvents required fields
|
|
assert result["specversion"] == "1.0"
|
|
assert result["type"] == "litellm_tokens" # default value
|
|
assert result["id"] == "cloudevents-test-response-id"
|
|
assert result["source"] == "litellm-proxy"
|
|
assert "time" in result
|
|
assert isinstance(result["subject"], str)
|
|
assert result["subject"] == "cloudevents-test-user"
|
|
|
|
# Verify data structure
|
|
assert "data" in result
|
|
assert result["data"]["model"] == "gpt-3.5-turbo"
|
|
assert result["data"]["cost"] == 0.001
|
|
assert result["data"]["prompt_tokens"] == 15
|
|
assert result["data"]["completion_tokens"] == 8
|
|
assert result["data"]["total_tokens"] == 23
|
|
|
|
def test_custom_event_type(self, monkeypatch):
|
|
"""Test that custom event type is used when set"""
|
|
monkeypatch.setenv("OPENMETER_EVENT_TYPE", "custom_event_type")
|
|
|
|
logger = OpenMeterLogger()
|
|
|
|
kwargs = {
|
|
"user": "custom-event-user",
|
|
"model": "gpt-4",
|
|
"response_cost": 0.003,
|
|
"litellm_call_id": "custom-event-call-id",
|
|
}
|
|
|
|
response_obj = {
|
|
"id": "custom-event-response-id",
|
|
"usage": {"prompt_tokens": 25, "completion_tokens": 12, "total_tokens": 37},
|
|
}
|
|
|
|
result = logger._common_logic(kwargs, response_obj)
|
|
|
|
assert result["type"] == "custom_event_type"
|
|
|
|
def test_common_logic_user_from_token_user_id(self):
|
|
"""Test that _common_logic uses user_api_key_user_id when no user provided"""
|
|
logger = OpenMeterLogger()
|
|
|
|
kwargs = {
|
|
"model": "gpt-3.5-turbo",
|
|
"response_cost": 0.001,
|
|
"litellm_call_id": "test-call-id",
|
|
"litellm_params": {"metadata": {"user_api_key_user_id": "token-user-123"}},
|
|
# No "user" parameter - should use token user_id
|
|
}
|
|
|
|
response_obj = {
|
|
"id": "test-response-id",
|
|
"usage": {"prompt_tokens": 10, "completion_tokens": 5, "total_tokens": 15},
|
|
}
|
|
|
|
result = logger._common_logic(kwargs, response_obj)
|
|
|
|
# Verify user was set from token user_id
|
|
assert isinstance(result["subject"], str)
|
|
assert result["subject"] == "token-user-123"
|
|
assert result["data"]["model"] == "gpt-3.5-turbo"
|
|
|
|
def test_common_logic_direct_user_takes_priority_over_token(self):
|
|
"""Test that direct user parameter takes priority over token user_id"""
|
|
logger = OpenMeterLogger()
|
|
|
|
kwargs = {
|
|
"user": "direct-user-456", # Direct user should take priority
|
|
"model": "gpt-4",
|
|
"response_cost": 0.002,
|
|
"litellm_call_id": "test-call-id",
|
|
"litellm_params": {
|
|
"metadata": {
|
|
"user_api_key_user_id": "token-user-123" # This should be ignored
|
|
}
|
|
},
|
|
}
|
|
|
|
response_obj = {
|
|
"id": "test-response-id",
|
|
"usage": {"prompt_tokens": 20, "completion_tokens": 10, "total_tokens": 30},
|
|
}
|
|
|
|
result = logger._common_logic(kwargs, response_obj)
|
|
|
|
# Verify direct user takes priority
|
|
assert isinstance(result["subject"], str)
|
|
assert result["subject"] == "direct-user-456"
|
|
assert result["subject"] != "token-user-123"
|
|
|
|
def test_common_logic_missing_user_and_token_user_id(self):
|
|
"""Test that exception is raised when neither user nor token user_id available"""
|
|
logger = OpenMeterLogger()
|
|
|
|
kwargs = {
|
|
"model": "gpt-3.5-turbo",
|
|
"response_cost": 0.001,
|
|
"litellm_call_id": "test-call-id",
|
|
"litellm_params": {
|
|
"metadata": {
|
|
# No user_api_key_user_id
|
|
}
|
|
},
|
|
# No "user" parameter
|
|
}
|
|
|
|
response_obj = {"id": "test-response-id"}
|
|
|
|
with pytest.raises(Exception, match="OpenMeter: user is required"):
|
|
logger._common_logic(kwargs, response_obj)
|
|
|
|
def test_common_logic_token_user_id_none(self):
|
|
"""Test that exception is raised when token user_id is None"""
|
|
logger = OpenMeterLogger()
|
|
|
|
kwargs = {
|
|
"model": "gpt-3.5-turbo",
|
|
"response_cost": 0.001,
|
|
"litellm_call_id": "test-call-id",
|
|
"litellm_params": {
|
|
"metadata": {"user_api_key_user_id": None} # Explicitly None
|
|
},
|
|
}
|
|
|
|
response_obj = {"id": "test-response-id"}
|
|
|
|
with pytest.raises(Exception, match="OpenMeter: user is required"):
|
|
logger._common_logic(kwargs, response_obj)
|
|
|
|
|
|
def test_common_logic_integer_token_user_id(self):
|
|
"""Test that integer token user_id is converted to string"""
|
|
logger = OpenMeterLogger()
|
|
|
|
kwargs = {
|
|
"model": "gpt-4",
|
|
"response_cost": 0.003,
|
|
"litellm_call_id": "test-call-id",
|
|
"litellm_params": {
|
|
"metadata": {"user_api_key_user_id": 12345} # Integer user_id
|
|
},
|
|
}
|
|
|
|
response_obj = {
|
|
"id": "test-response-id",
|
|
"usage": {"prompt_tokens": 25, "completion_tokens": 12, "total_tokens": 37},
|
|
}
|
|
|
|
result = logger._common_logic(kwargs, response_obj)
|
|
|
|
# Verify integer user_id is converted to string
|
|
assert isinstance(result["subject"], str)
|
|
assert result["subject"] == "12345"
|
|
|
|
def test_common_logic_trust_request_user_false_ignores_request_user(self, monkeypatch):
|
|
"""OPENMETER_TRUST_REQUEST_USER=false makes the key-bound user_id win
|
|
over a request-supplied `user` (forge-attribution mitigation)."""
|
|
monkeypatch.setenv("OPENMETER_TRUST_REQUEST_USER", "false")
|
|
logger = OpenMeterLogger()
|
|
|
|
kwargs = {
|
|
"user": "forged-by-client",
|
|
"model": "gpt-4",
|
|
"response_cost": 0.002,
|
|
"litellm_call_id": "test-call-id",
|
|
"litellm_params": {
|
|
"metadata": {"user_api_key_user_id": "real-tenant-id"}
|
|
},
|
|
}
|
|
|
|
response_obj = {
|
|
"id": "test-response-id",
|
|
"usage": {"prompt_tokens": 20, "completion_tokens": 10, "total_tokens": 30},
|
|
}
|
|
|
|
result = logger._common_logic(kwargs, response_obj)
|
|
|
|
assert result["subject"] == "real-tenant-id"
|
|
assert result["subject"] != "forged-by-client"
|
|
|
|
def test_common_logic_trust_request_user_false_still_raises_without_key_user(self, monkeypatch):
|
|
"""OPENMETER_TRUST_REQUEST_USER=false still raises when no
|
|
user_api_key_user_id is available — the request `user` is not a
|
|
fallback in this mode."""
|
|
monkeypatch.setenv("OPENMETER_TRUST_REQUEST_USER", "false")
|
|
logger = OpenMeterLogger()
|
|
|
|
kwargs = {
|
|
"user": "would-have-worked-without-the-flag",
|
|
"model": "gpt-3.5-turbo",
|
|
"response_cost": 0.001,
|
|
"litellm_call_id": "test-call-id",
|
|
}
|
|
|
|
response_obj = {"id": "test-response-id"}
|
|
|
|
with pytest.raises(Exception, match="OpenMeter: user is required"):
|
|
logger._common_logic(kwargs, response_obj)
|
|
|
|
def test_common_logic_trust_request_user_default_preserves_behavior(self):
|
|
"""Default (unset OPENMETER_TRUST_REQUEST_USER) keeps request `user`
|
|
taking priority — backward compatibility."""
|
|
# OPENMETER_TRUST_REQUEST_USER intentionally unset
|
|
logger = OpenMeterLogger()
|
|
|
|
kwargs = {
|
|
"user": "request-user",
|
|
"model": "gpt-4",
|
|
"response_cost": 0.002,
|
|
"litellm_call_id": "test-call-id",
|
|
"litellm_params": {
|
|
"metadata": {"user_api_key_user_id": "key-user"}
|
|
},
|
|
}
|
|
|
|
response_obj = {
|
|
"id": "test-response-id",
|
|
"usage": {"prompt_tokens": 20, "completion_tokens": 10, "total_tokens": 30},
|
|
}
|
|
|
|
result = logger._common_logic(kwargs, response_obj)
|
|
assert result["subject"] == "request-user"
|
|
|
|
@patch("litellm.integrations.openmeter.HTTPHandler")
|
|
def test_integration_token_user_id_scenario(self, mock_http_handler):
|
|
"""Integration test simulating the exact scenario that was failing"""
|
|
mock_post = MagicMock()
|
|
mock_http_handler.return_value.post = mock_post
|
|
|
|
logger = OpenMeterLogger()
|
|
|
|
# Simulate the exact scenario: request with token that has user_id but no direct user param
|
|
kwargs = {
|
|
"model": "gpt-3.5-turbo",
|
|
"messages": [{"role": "user", "content": "Hello"}],
|
|
"response_cost": 0.001,
|
|
"litellm_call_id": "test-integration-call-id",
|
|
"litellm_params": {
|
|
"metadata": {
|
|
"user_api_key_user_id": "user123-from-token",
|
|
"user_api_key": "hashed-key-abc",
|
|
"user_api_key_metadata": {},
|
|
}
|
|
},
|
|
# No "user" parameter - this was causing "OpenMeter: user is required" error
|
|
}
|
|
|
|
response_obj = {
|
|
"id": "chatcmpl-test123",
|
|
"usage": {"prompt_tokens": 15, "completion_tokens": 10, "total_tokens": 25},
|
|
}
|
|
|
|
# This should NOT raise "OpenMeter: user is required" anymore
|
|
logger.log_success_event(kwargs, response_obj, None, None)
|
|
|
|
# Verify HTTP call was made
|
|
mock_post.assert_called_once()
|
|
|
|
# Verify the data structure contains user from token
|
|
call_args = mock_post.call_args
|
|
data = json.loads(call_args[1]["data"])
|
|
|
|
assert data["subject"] == "user123-from-token"
|
|
assert isinstance(data["subject"], str)
|
|
assert data["data"]["model"] == "gpt-3.5-turbo"
|