mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-06 08:16:43 +00:00
de80e3afe4
* fix(helm): scale the classic chart's HPA out at the documented 60 percent CPU The litellm-helm chart shipped targetCPUUtilizationPercentage: 80, which is unexamined helm create scaffold rather than a chosen number. It arrived packaged with the stock minReplicas: 1, maxReplicas: 100, a commented-out targetMemoryUtilizationPercentage: 80, and the boilerplate "such as Minikube" comment, the same provenance as the 128Mi resource example this file just corrected. 60 is the documented recommendation. The mechanism behind it is scale-up lag: the chart's own startupProbe is failureThreshold: 30 times periodSeconds: 10, so a replica can take up to 300 seconds to become ready, and a pod added at 80 percent utilization arrives minutes after saturation. The memory target stays commented out on purpose. The prisma query engine's resident memory is a high-water mark that ratchets to the pod's worst-ever write and is never returned, so a memory-target HPA reads the largest write a pod ever did rather than what it is doing now, and replicas ratchet up without scaling back in. hpa_tests.yaml carried its second suite after a YAML document separator, and helm-unittest loads only the first document per file, so that suite never ran; an assertion planted in it still passed. Fold it into the one live suite and add coverage pinning the rendered CPU target, the absence of a memory metric by default, and that overrides still take effect. Bump the chart to 1.1.2, since rendered output changes for anyone running with autoscaling enabled. * fix(helm): bump litellm-helm to 1.1.3 after rebase onto 1.1.2 Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
41 lines
1.7 KiB
YAML
41 lines
1.7 KiB
YAML
apiVersion: v2
|
|
|
|
# We can't call ourselves just "litellm" because then we couldn't publish to the
|
|
# same OCI repository as the "litellm" OCI image
|
|
name: litellm-helm
|
|
description: Call all LLM APIs using the OpenAI format
|
|
|
|
# A chart can be either an 'application' or a 'library' chart.
|
|
#
|
|
# Application charts are a collection of templates that can be packaged into versioned archives
|
|
# to be deployed.
|
|
#
|
|
# Library charts provide useful utilities or functions for the chart developer. They're included as
|
|
# a dependency of application charts to inject those utilities and functions into the rendering
|
|
# pipeline. Library charts do not define any templates and therefore cannot be deployed.
|
|
type: application
|
|
|
|
# This is the chart version. This version number should be incremented each time you make changes
|
|
# to the chart and its templates, including the app version.
|
|
# Versions are expected to follow Semantic Versioning (https://semver.org/)
|
|
version: 1.1.3
|
|
|
|
# This is the version number of the application being deployed. This version number should be
|
|
# incremented each time you make changes to the application. Versions are not expected to
|
|
# follow Semantic Versioning. They should reflect the version the application is using.
|
|
# It is recommended to use it with quotes.
|
|
appVersion: v1.85.1
|
|
|
|
annotations:
|
|
org.opencontainers.image.source: "https://github.com/BerriAI/litellm"
|
|
org.opencontainers.image.url: "https://docs.litellm.ai/"
|
|
|
|
dependencies:
|
|
- name: "postgresql"
|
|
version: "14.3.1"
|
|
repository: oci://registry-1.docker.io/bitnamicharts
|
|
condition: db.deployStandalone
|
|
- name: redis
|
|
version: "18.19.1"
|
|
repository: oci://registry-1.docker.io/bitnamicharts
|
|
condition: redis.enabled
|