5 commits

Author	SHA1	Message	Date
Ishaan Jaffer	e8461b5b97	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
Sameer Kankute	6e6f5be3e4	feat(triton): add embedding usage estimation for self-hosted responses (#25345) ... * feat(triton): add embedding usage estimation for self-hosted responses Populate Triton embedding usage from request input using token counting with a safe fallback so cost/observability flows work even when provider usage is missing. Made-with: Cursor * fix(triton): sum per-input embedding token counts for batches Joining batch strings with newlines before token_counter added spurious tokens. Count each input separately and sum, matching OpenAI-style usage. Made-with: Cursor	2026-04-08 21:14:27 -07:00
Krish Dholakia	5c929317cd	fix(triton/completion/transformation.py): remove bad_words / stop wor… (#10163) ... * fix(triton/completion/transformation.py): remove bad_words / stop words from triton call parameter 'bad_words' has invalid type. It should be either 'int', 'bool', or 'string'. * fix(proxy_track_cost_callback.py): add debug logging for track cost callback error	2025-04-19 11:23:37 -07:00
Minwoo Lee	c1f2ae97c5	Add streaming test	2025-02-13 15:43:42 +09:00
Ishaan Jaff	6107f9f3f3	[Bug fix ]: Triton /infer handler incompatible with batch responses (#7337) ... * migrate triton to base llm http handler * clean up triton handler.py * use transform functions for triton * add TritonConfig * get openai params for triton * use triton embedding config * test_completion_triton_generate_api * test_completion_triton_infer_api * fix TritonConfig doc string * use TritonResponseIterator * fix triton embeddings * docs triton chat usage	2024-12-20 20:59:40 -08:00