mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-14 23:21:35 +00:00
fix(router): price baseline candidates by base_model, not the deployment name
`litellm_params.model` is not always a model. On Azure it is the deployment name, which is absent from the cost map, so pricing it directly returned nothing and the candidate dropped out of the pool with only a debug line. If that candidate was the priciest, the baseline quietly became the second priciest and every saving was understated; if the pool was all Azure, nothing priced, the driver was disabled and the card read $0.00 with nothing at default log level saying why. Wildcard and aliased deployments drop the same way. `model_info.base_model` is what names the real model, and router.py already resolves pricing through the same base_model, base_model, model chain in `_get_model_from_deployment` and its cooldown counterpart. Candidate resolution now follows it.
This commit is contained in:
parent
68c56fe1f6
commit
11c66887b7
145 changed files with 2179 additions and 0 deletions
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
|
|
@ -0,0 +1,35 @@
|
|||
(globalThis.TURBOPACK||(globalThis.TURBOPACK=[])).push(["object"==typeof document?document.currentScript:void 0,624687,e=>{"use strict";var r=e.i(843476),a=e.i(271645),t=e.i(115504);let s=a.forwardRef(({className:e,...a},s)=>(0,r.jsx)("textarea",{ref:s,"data-slot":"textarea",className:(0,t.cn)("flex field-sizing-content min-h-16 w-full rounded-md border border-input bg-transparent px-2.5 py-2 text-base shadow-xs transition-[color,box-shadow] outline-none placeholder:text-muted-foreground focus-visible:border-ring focus-visible:ring-3 focus-visible:ring-ring/50 disabled:cursor-not-allowed disabled:opacity-50 aria-invalid:border-destructive aria-invalid:ring-3 aria-invalid:ring-destructive/20 md:text-sm dark:bg-input/30 dark:aria-invalid:border-destructive/50 dark:aria-invalid:ring-destructive/40",e),...a}));s.displayName="Textarea",e.s(["Textarea",0,s])},515288,e=>{"use strict";var r=e.i(843476),a=e.i(271645),t=e.i(115504);let s=a.forwardRef(({className:e,size:a="default",...s},i)=>(0,r.jsx)("div",{ref:i,"data-slot":"card","data-size":a,className:(0,t.cn)("group/card flex flex-col gap-(--card-spacing) rounded-xl bg-card py-(--card-spacing) text-sm text-card-foreground shadow-xs ring-1 ring-foreground/10 [--card-spacing:--spacing(6)] has-[>img:first-child]:pt-0 data-[size=sm]:[--card-spacing:--spacing(4)] *:[img:first-child]:rounded-t-xl *:[img:last-child]:rounded-b-xl",e),...s}));s.displayName="Card";let i=a.forwardRef(({className:e,...a},s)=>(0,r.jsx)("div",{ref:s,"data-slot":"card-header",className:(0,t.cn)("group/card-header @container/card-header grid auto-rows-min items-start gap-1 rounded-t-xl px-(--card-spacing) has-data-[slot=card-action]:grid-cols-[1fr_auto] has-data-[slot=card-description]:grid-rows-[auto_auto] [.border-b]:pb-(--card-spacing)",e),...a}));i.displayName="CardHeader";let d=a.forwardRef(({className:e,...a},s)=>(0,r.jsx)("div",{ref:s,"data-slot":"card-title",className:(0,t.cn)("text-base leading-normal font-medium group-data-[size=sm]/card:text-sm",e),...a}));d.displayName="CardTitle";let o=a.forwardRef(({className:e,...a},s)=>(0,r.jsx)("div",{ref:s,"data-slot":"card-description",className:(0,t.cn)("text-sm text-muted-foreground",e),...a}));o.displayName="CardDescription";let n=a.forwardRef(({className:e,...a},s)=>(0,r.jsx)("div",{ref:s,"data-slot":"card-action",className:(0,t.cn)("col-start-2 row-span-2 row-start-1 self-start justify-self-end",e),...a}));n.displayName="CardAction";let l=a.forwardRef(({className:e,...a},s)=>(0,r.jsx)("div",{ref:s,"data-slot":"card-content",className:(0,t.cn)("px-(--card-spacing)",e),...a}));l.displayName="CardContent";let c=a.forwardRef(({className:e,...a},s)=>(0,r.jsx)("div",{ref:s,"data-slot":"card-footer",className:(0,t.cn)("flex items-center rounded-b-xl px-(--card-spacing) [.border-t]:pt-(--card-spacing)",e),...a}));c.displayName="CardFooter",e.s(["Card",0,s,"CardAction",0,n,"CardContent",0,l,"CardDescription",0,o,"CardFooter",0,c,"CardHeader",0,i,"CardTitle",0,d])},972520,e=>{"use strict";let r=(0,e.i(475254).default)("arrow-right",[["path",{d:"M5 12h14",key:"1ays0h"}],["path",{d:"m12 5 7 7-7 7",key:"xquz4c"}]]);e.s(["ArrowRight",0,r],972520)},411929,e=>{"use strict";var r=e.i(843476),a=e.i(271645),t=e.i(972520),s=e.i(174886),i=e.i(519455),d=e.i(515288),o=e.i(624687),n=e.i(571303),l=e.i(602869),c=e.i(727749);let u=({accessToken:e})=>{let[u,m]=(0,a.useState)(`{
|
||||
"model": "openai/gpt-4o",
|
||||
"messages": [
|
||||
{
|
||||
"role": "system",
|
||||
"content": "You are a helpful assistant."
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Explain quantum computing in simple terms"
|
||||
}
|
||||
],
|
||||
"temperature": 0.7,
|
||||
"max_tokens": 500,
|
||||
"stream": true
|
||||
}`),[p,f]=(0,a.useState)(""),[x,g]=(0,a.useState)(!1),h=async()=>{g(!0);try{let s;try{s=JSON.parse(u)}catch(e){c.default.fromBackend("Invalid JSON in request body"),g(!1);return}let i={call_type:"completion",request_body:s};if(!e){c.default.fromBackend("No access token found"),g(!1);return}let d=await (0,l.transformRequestCall)(e,i);if(d.raw_request_api_base&&d.raw_request_body){var r,a,t;let e,s,i=(r=d.raw_request_api_base,a=d.raw_request_body,t=d.raw_request_headers||{},e=JSON.stringify(a,null,2).split("\n").map(e=>` ${e}`).join("\n"),s=Object.entries(t).map(([e,r])=>`-H '${e}: ${r}'`).join(" \\\n "),`curl -X POST \\
|
||||
${r} \\
|
||||
${s?`${s} \\
|
||||
`:""}-H 'Content-Type: application/json' \\
|
||||
-d '{
|
||||
${e}
|
||||
}'`);f(i),c.default.success("Request transformed successfully")}else{let e="string"==typeof d?d:JSON.stringify(d);f(e),c.default.info("Transformed request received in unexpected format")}}catch(e){console.error("Error transforming request:",e),c.default.fromBackend("Failed to transform request")}finally{g(!1)}};return(0,r.jsxs)("div",{className:"p-2",children:[(0,r.jsx)("h1",{className:"text-lg font-medium text-foreground",children:"Playground"}),(0,r.jsx)("p",{className:"text-sm text-muted-foreground",children:"See how LiteLLM transforms your request for the specified provider."}),(0,r.jsxs)("div",{className:"mt-4 grid grid-cols-1 gap-4 lg:grid-cols-2",children:[(0,r.jsxs)(d.Card,{children:[(0,r.jsxs)(d.CardHeader,{children:[(0,r.jsx)(d.CardTitle,{className:"text-2xl font-bold",children:"Original Request"}),(0,r.jsx)(d.CardDescription,{children:"The request you would send to LiteLLM /chat/completions endpoint."})]}),(0,r.jsx)(d.CardContent,{children:(0,r.jsx)(o.Textarea,{className:"h-72 resize-none p-4 font-mono text-sm field-sizing-fixed",value:u,onChange:e=>m(e.target.value),onKeyDown:e=>{(e.metaKey||e.ctrlKey)&&"Enter"===e.key&&(e.preventDefault(),h())},placeholder:"Press Cmd/Ctrl + Enter to transform"})}),(0,r.jsx)(d.CardFooter,{className:"justify-end",children:(0,r.jsxs)(i.Button,{onClick:h,disabled:x,children:[(0,r.jsx)("span",{children:"Transform"}),x?(0,r.jsx)(n.UiLoadingSpinner,{className:"size-4"}):(0,r.jsx)(t.ArrowRight,{})]})})]}),(0,r.jsxs)(d.Card,{children:[(0,r.jsxs)(d.CardHeader,{children:[(0,r.jsx)(d.CardTitle,{className:"text-2xl font-bold",children:"Transformed Request"}),(0,r.jsx)(d.CardDescription,{children:"How LiteLLM transforms your request for the specified provider."}),(0,r.jsx)("p",{className:"mt-2 text-xs text-muted-foreground",children:"Note: Sensitive headers are not shown."})]}),(0,r.jsx)(d.CardContent,{children:(0,r.jsxs)("div",{className:"relative rounded-md bg-muted",children:[(0,r.jsx)("pre",{className:"h-72 overflow-auto p-4 font-mono text-sm",children:p||`curl -X POST \\
|
||||
https://api.openai.com/v1/chat/completions \\
|
||||
-H 'Authorization: Bearer sk-xxx' \\
|
||||
-H 'Content-Type: application/json' \\
|
||||
-d '{
|
||||
"model": "gpt-4",
|
||||
"messages": [
|
||||
{
|
||||
"role": "system",
|
||||
"content": "You are a helpful assistant."
|
||||
}
|
||||
],
|
||||
"temperature": 0.7
|
||||
}'`}),(0,r.jsx)(i.Button,{variant:"ghost",size:"icon-sm","aria-label":"Copy to clipboard",className:"absolute top-2 right-2",onClick:()=>{navigator.clipboard.writeText(p||""),c.default.success("Copied to clipboard")},children:(0,r.jsx)(s.Copy,{})})]})})]})]}),(0,r.jsx)("div",{className:"mt-4 text-right",children:(0,r.jsxs)("p",{className:"text-sm text-muted-foreground",children:["Found an error? File an issue"," ",(0,r.jsx)("a",{className:"underline underline-offset-4",href:"https://github.com/BerriAI/litellm/issues",target:"_blank",rel:"noopener noreferrer",children:"here"}),"."]})})]})};var m=e.i(135214);e.s(["default",0,function(){let{accessToken:e}=(0,m.default)();return(0,r.jsx)(u,{accessToken:e})}],411929)}]);
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
Some files were not shown because too many files have changed in this diff Show more
Loading…
Add table
Reference in a new issue