- Add GCS_DEFAULT_USE_BATCHED_LOGGING constant (default: True)
- Add GCS_USE_BATCHED_LOGGING environment variable to enable/disable batching
- When disabled, sends logs individually as separate GCS objects (legacy behavior)
- When enabled (default), uses new batched NDJSON format for improved throughput
- Document the new environment variable in config_settings.md
- Implement sequential log sending for legacy mode (matching original behavior)
- Remove redundant comments that don't add value
- Remove maxsize queue limit (queue is now unbounded)
- Restore original verbose logging patterns from before branch
- Simplify code while maintaining functionality
- Replace List with asyncio.Queue for thread-safe concurrent access
- Implement true payload batching: combine multiple logs into NDJSON format, upload as single GCS object
- Add bounded queue with configurable GCS_QUEUE_MAXSIZE for backpressure management
- Add config grouping to handle mixed GCS configurations (bucket/credentials) within batches
- Add observability: queue size monitoring via print statements
- Sanitize sensitive config keys in logs using hash-based identifiers
- Dramatically reduce API calls (e.g., 100 logs = 1 API call instead of 100)
- Prevents OOM errors when background job falls behind during high-traffic periods
- Implement true payload batching: combine multiple logs into single NDJSON object
- Reduces API calls from N per batch to 1 per batch
- Dramatically improves throughput and reduces latency
- Uses newline-delimited JSON (NDJSON) format for standard log processing
- Add bounded queue to prevent OOM
- Queue maxsize defaults to 10x batch_size (configurable via GCS_QUEUE_MAXSIZE)
- Provides backpressure when queue is full
- Warns when queue exceeds 80% capacity
- Update object naming for batched logs
- Format: {date}/batch-{timestamp}-{uuid}.ndjson
- Maintains backward compatibility with existing structure
- Add observability improvements
- Queue size warnings when approaching capacity
- Enhanced print statements for batch processing
- Better error handling for batch uploads
This addresses performance issues where queue size grew to 2500+ items
by reducing upload overhead from N sequential requests to 1 batched request.
- Add print statement on logger initialization with config details
- Add print statements when success/failure events are queued with queue size
- Add print statements for batch processing start/completion with metrics
- Add print statements for individual log sends to GCS bucket
- Add print statements for error handling during queueing and sending
- Use print() instead of verbose_logger for direct visibility
- Add health_check_client.py for monitoring model availability
- Add health_check_client_README.md with usage documentation
- Add health_check_requirements.txt for dependencies
- Add run_parallel_health_checks.ps1 (PowerShell version)
- Add run_parallel_health_checks.sh (Bash version)
- Organize all scripts under scripts/health_check/ directory
* docs: update UI contributing guide with correct commands
- Replace outdated proxy_cli.py command with poetry run litellm
- Add config.yaml example with required settings
- Clarify that UI comes pre-built in the repo
- Add two development options: Build Mode and Dev Mode (hot reload)
- Note about redirect issues in Dev Mode
* docs: add hot reload login flow and PR submission section
- Document the 3000 -> 4000 -> 3000 login flow for hot reload
- Reorder: Hot Reload as Option A, Build Mode as Option B
- Add section 4 on submitting PRs
- Add note that UI changes don't require tests
* Update login flow navigation URL in contributing.md
Replace copy.deepcopy with model_dump + model_validate in streaming
iterator logging to handle Pydantic ValidatorIterator objects that
cannot be pickled when tool_choice uses allowed_tools mode.
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
