fix(router): price baseline candidates by base_model, not the deployment name

`litellm_params.model` is not always a model. On Azure it is the deployment
name, which is absent from the cost map, so pricing it directly returned nothing
and the candidate dropped out of the pool with only a debug line. If that
candidate was the priciest, the baseline quietly became the second priciest and
every saving was understated; if the pool was all Azure, nothing priced, the
driver was disabled and the card read $0.00 with nothing at default log level
saying why. Wildcard and aliased deployments drop the same way.

`model_info.base_model` is what names the real model, and router.py already
resolves pricing through the same base_model, base_model, model chain in
`_get_model_from_deployment` and its cooldown counterpart. Candidate resolution
now follows it.
This commit is contained in:
Tin Chi Lo 2026-08-01 13:18:58 -07:00
parent 68c56fe1f6
commit 11c66887b7
145 changed files with 2179 additions and 0 deletions

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View file

@ -0,0 +1,35 @@
(globalThis.TURBOPACK||(globalThis.TURBOPACK=[])).push(["object"==typeof document?document.currentScript:void 0,624687,e=>{"use strict";var r=e.i(843476),a=e.i(271645),t=e.i(115504);let s=a.forwardRef(({className:e,...a},s)=>(0,r.jsx)("textarea",{ref:s,"data-slot":"textarea",className:(0,t.cn)("flex field-sizing-content min-h-16 w-full rounded-md border border-input bg-transparent px-2.5 py-2 text-base shadow-xs transition-[color,box-shadow] outline-none placeholder:text-muted-foreground focus-visible:border-ring focus-visible:ring-3 focus-visible:ring-ring/50 disabled:cursor-not-allowed disabled:opacity-50 aria-invalid:border-destructive aria-invalid:ring-3 aria-invalid:ring-destructive/20 md:text-sm dark:bg-input/30 dark:aria-invalid:border-destructive/50 dark:aria-invalid:ring-destructive/40",e),...a}));s.displayName="Textarea",e.s(["Textarea",0,s])},515288,e=>{"use strict";var r=e.i(843476),a=e.i(271645),t=e.i(115504);let s=a.forwardRef(({className:e,size:a="default",...s},i)=>(0,r.jsx)("div",{ref:i,"data-slot":"card","data-size":a,className:(0,t.cn)("group/card flex flex-col gap-(--card-spacing) rounded-xl bg-card py-(--card-spacing) text-sm text-card-foreground shadow-xs ring-1 ring-foreground/10 [--card-spacing:--spacing(6)] has-[>img:first-child]:pt-0 data-[size=sm]:[--card-spacing:--spacing(4)] *:[img:first-child]:rounded-t-xl *:[img:last-child]:rounded-b-xl",e),...s}));s.displayName="Card";let i=a.forwardRef(({className:e,...a},s)=>(0,r.jsx)("div",{ref:s,"data-slot":"card-header",className:(0,t.cn)("group/card-header @container/card-header grid auto-rows-min items-start gap-1 rounded-t-xl px-(--card-spacing) has-data-[slot=card-action]:grid-cols-[1fr_auto] has-data-[slot=card-description]:grid-rows-[auto_auto] [.border-b]:pb-(--card-spacing)",e),...a}));i.displayName="CardHeader";let d=a.forwardRef(({className:e,...a},s)=>(0,r.jsx)("div",{ref:s,"data-slot":"card-title",className:(0,t.cn)("text-base leading-normal font-medium group-data-[size=sm]/card:text-sm",e),...a}));d.displayName="CardTitle";let o=a.forwardRef(({className:e,...a},s)=>(0,r.jsx)("div",{ref:s,"data-slot":"card-description",className:(0,t.cn)("text-sm text-muted-foreground",e),...a}));o.displayName="CardDescription";let n=a.forwardRef(({className:e,...a},s)=>(0,r.jsx)("div",{ref:s,"data-slot":"card-action",className:(0,t.cn)("col-start-2 row-span-2 row-start-1 self-start justify-self-end",e),...a}));n.displayName="CardAction";let l=a.forwardRef(({className:e,...a},s)=>(0,r.jsx)("div",{ref:s,"data-slot":"card-content",className:(0,t.cn)("px-(--card-spacing)",e),...a}));l.displayName="CardContent";let c=a.forwardRef(({className:e,...a},s)=>(0,r.jsx)("div",{ref:s,"data-slot":"card-footer",className:(0,t.cn)("flex items-center rounded-b-xl px-(--card-spacing) [.border-t]:pt-(--card-spacing)",e),...a}));c.displayName="CardFooter",e.s(["Card",0,s,"CardAction",0,n,"CardContent",0,l,"CardDescription",0,o,"CardFooter",0,c,"CardHeader",0,i,"CardTitle",0,d])},972520,e=>{"use strict";let r=(0,e.i(475254).default)("arrow-right",[["path",{d:"M5 12h14",key:"1ays0h"}],["path",{d:"m12 5 7 7-7 7",key:"xquz4c"}]]);e.s(["ArrowRight",0,r],972520)},411929,e=>{"use strict";var r=e.i(843476),a=e.i(271645),t=e.i(972520),s=e.i(174886),i=e.i(519455),d=e.i(515288),o=e.i(624687),n=e.i(571303),l=e.i(602869),c=e.i(727749);let u=({accessToken:e})=>{let[u,m]=(0,a.useState)(`{
"model": "openai/gpt-4o",
"messages": [
{
"role": "system",
"content": "You are a helpful assistant."
},
{
"role": "user",
"content": "Explain quantum computing in simple terms"
}
],
"temperature": 0.7,
"max_tokens": 500,
"stream": true
}`),[p,f]=(0,a.useState)(""),[x,g]=(0,a.useState)(!1),h=async()=>{g(!0);try{let s;try{s=JSON.parse(u)}catch(e){c.default.fromBackend("Invalid JSON in request body"),g(!1);return}let i={call_type:"completion",request_body:s};if(!e){c.default.fromBackend("No access token found"),g(!1);return}let d=await (0,l.transformRequestCall)(e,i);if(d.raw_request_api_base&&d.raw_request_body){var r,a,t;let e,s,i=(r=d.raw_request_api_base,a=d.raw_request_body,t=d.raw_request_headers||{},e=JSON.stringify(a,null,2).split("\n").map(e=>` ${e}`).join("\n"),s=Object.entries(t).map(([e,r])=>`-H '${e}: ${r}'`).join(" \\\n "),`curl -X POST \\
${r} \\
${s?`${s} \\
`:""}-H 'Content-Type: application/json' \\
-d '{
${e}
}'`);f(i),c.default.success("Request transformed successfully")}else{let e="string"==typeof d?d:JSON.stringify(d);f(e),c.default.info("Transformed request received in unexpected format")}}catch(e){console.error("Error transforming request:",e),c.default.fromBackend("Failed to transform request")}finally{g(!1)}};return(0,r.jsxs)("div",{className:"p-2",children:[(0,r.jsx)("h1",{className:"text-lg font-medium text-foreground",children:"Playground"}),(0,r.jsx)("p",{className:"text-sm text-muted-foreground",children:"See how LiteLLM transforms your request for the specified provider."}),(0,r.jsxs)("div",{className:"mt-4 grid grid-cols-1 gap-4 lg:grid-cols-2",children:[(0,r.jsxs)(d.Card,{children:[(0,r.jsxs)(d.CardHeader,{children:[(0,r.jsx)(d.CardTitle,{className:"text-2xl font-bold",children:"Original Request"}),(0,r.jsx)(d.CardDescription,{children:"The request you would send to LiteLLM /chat/completions endpoint."})]}),(0,r.jsx)(d.CardContent,{children:(0,r.jsx)(o.Textarea,{className:"h-72 resize-none p-4 font-mono text-sm field-sizing-fixed",value:u,onChange:e=>m(e.target.value),onKeyDown:e=>{(e.metaKey||e.ctrlKey)&&"Enter"===e.key&&(e.preventDefault(),h())},placeholder:"Press Cmd/Ctrl + Enter to transform"})}),(0,r.jsx)(d.CardFooter,{className:"justify-end",children:(0,r.jsxs)(i.Button,{onClick:h,disabled:x,children:[(0,r.jsx)("span",{children:"Transform"}),x?(0,r.jsx)(n.UiLoadingSpinner,{className:"size-4"}):(0,r.jsx)(t.ArrowRight,{})]})})]}),(0,r.jsxs)(d.Card,{children:[(0,r.jsxs)(d.CardHeader,{children:[(0,r.jsx)(d.CardTitle,{className:"text-2xl font-bold",children:"Transformed Request"}),(0,r.jsx)(d.CardDescription,{children:"How LiteLLM transforms your request for the specified provider."}),(0,r.jsx)("p",{className:"mt-2 text-xs text-muted-foreground",children:"Note: Sensitive headers are not shown."})]}),(0,r.jsx)(d.CardContent,{children:(0,r.jsxs)("div",{className:"relative rounded-md bg-muted",children:[(0,r.jsx)("pre",{className:"h-72 overflow-auto p-4 font-mono text-sm",children:p||`curl -X POST \\
https://api.openai.com/v1/chat/completions \\
-H 'Authorization: Bearer sk-xxx' \\
-H 'Content-Type: application/json' \\
-d '{
"model": "gpt-4",
"messages": [
{
"role": "system",
"content": "You are a helpful assistant."
}
],
"temperature": 0.7
}'`}),(0,r.jsx)(i.Button,{variant:"ghost",size:"icon-sm","aria-label":"Copy to clipboard",className:"absolute top-2 right-2",onClick:()=>{navigator.clipboard.writeText(p||""),c.default.success("Copied to clipboard")},children:(0,r.jsx)(s.Copy,{})})]})})]})]}),(0,r.jsx)("div",{className:"mt-4 text-right",children:(0,r.jsxs)("p",{className:"text-sm text-muted-foreground",children:["Found an error? File an issue"," ",(0,r.jsx)("a",{className:"underline underline-offset-4",href:"https://github.com/BerriAI/litellm/issues",target:"_blank",rel:"noopener noreferrer",children:"here"}),"."]})})]})};var m=e.i(135214);e.s(["default",0,function(){let{accessToken:e}=(0,m.default)();return(0,r.jsx)(u,{accessToken:e})}],411929)}]);

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

Some files were not shown because too many files have changed in this diff Show more