scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-07 02:59:05 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 103

repro steps for health endpoint issue

Browse source
This commit is contained in:
Alexsander Hamir 2026-01-12 14:39:39 -08:00
parent 834b0207ed
commit a40104e7ec
2 changed files with 28 additions and 11 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

22
docker-compose.yml
View file

@ -4,21 +4,21 @@ services:
context: . context: .
args: args:
target: runtime target: runtime
image: docker.litellm.ai/berriai/litellm:main-stable # image: docker.litellm.ai/berriai/litellm:main-stable
######################################### volumes:
## Uncomment these lines to start proxy with a config.yaml file ## - ./config.yaml:/app/config.yaml # Mount your config file
# volumes: command:
# - ./config.yaml:/app/config.yaml - "--config=/app/config.yaml"
# command:
# - "--config=/app/config.yaml"
##############################################
ports: ports:
- "4000:4000" # Map the container port to the host, change the host port if necessary - "4000:4000" # Main proxy port
- "4001:4001" # Separate health app port
environment: environment:
DATABASE_URL: "postgresql://llmproxy:dbpassword9090@db:5432/litellm" DATABASE_URL: "postgresql://llmproxy:dbpassword9090@db:5432/litellm"
STORE_MODEL_IN_DB: "True" # allows adding models to proxy via UI STORE_MODEL_IN_DB: "True" # allows adding models to proxy via UI
env_file: SEPARATE_HEALTH_APP: "1" # Enable separate health check app
- .env # Load local .env file SEPARATE_HEALTH_PORT: "4001" # Port for health app (default is 4001)
# env_file:
# - .env # Optional: Load local .env file if it exists
depends_on: depends_on:
- db # Indicates that this service depends on the 'db' service, ensuring 'db' starts first - db # Indicates that this service depends on the 'db' service, ensuring 'db' starts first
healthcheck: # Defines the health check configuration for the container healthcheck: # Defines the health check configuration for the container

17
steps_to_repro.md Normal file
View file

@ -0,0 +1,17 @@
1. docker compose up
2. Start the fake LLM provider and make sure the delay is set to 10sec
3. Make the request, and wait 3 seconds:
``` bash
# Create the JSON file first
@'
{"model":"db-openai-endpoint","messages":[{"role":"user","content":"Say hello"}],"max_tokens":2000}
'@ | Out-File -FilePath request.json -Encoding utf8
# Then use it with curl
curl.exe -X POST http://localhost:4000/v1/chat/completions -H "Content-Type: application/json" -d "@request.json"
```
4. On a different terminal run:
```bash
docker kill --signal=SIGTERM (docker ps --filter "name=litellm" --format "{{.Names}}" | Select-Object -First 1)
```
5. Check if the call gets terminated or if LiteLLM waits the 10 seconds.