mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-15 09:12:18 +00:00
9ba216d521
* fix(vertex_ai): reuse one Google token across vector store searches * fix(vertex_ai): load Vertex credentials once when a cold worker gets a burst * test(vertex_ai): count Google token fetches for RAG Engine searches through the proxy * test(vertex_ai): type the fixtures and token callback in the concurrent search test * fix(vertex_ai): lock Google token loads per credential and reload a key Google rejects A shared RAG Engine config made one store's slow token fetch hold up every other store, and kept a rotated key file's deleted key until a restart. First loads and expired refreshes now take a lock per credentials and project, refreshes take a lock per credentials object, and any RefreshError reloads the credentials once. * fix(vertex_ai): share one Google token load or failure per credential and reload rejected keys on async calls * test(vertex_ai): check async callers sharing an expired credential refresh it once * fix(vertex_ai): reload Google credentials only when a refresh is rejected for good A retryable refresh error such as a 503 now raises without reloading the key, so an outage does not double google-auth's retries. Credentials that cannot be weak-referenced share the config's refresh lock again, as on main. The burst test for a rejected key moves to VertexBase and holds the token response until the other callers have started * fix(vertex_ai): reload Google credentials only when Google answers invalid_grant google-auth leaves its retryable flag False on metadata server, impersonation, and workload identity outages, and on a 502, so the previous check reloaded the key file on those outages and doubled the token tries. Now only an invalid_grant answer or "Reauthentication is needed" reloads The sync reload also refills the cache under the key the caller asked with, so a call without vertex_project keeps one cached copy, as the async reload already did * fix(vertex_ai): pick up a swapped Google key file on the next call The credential cache key now carries the key file's inode, mtime and size, for a vertex_credentials path and for GOOGLE_APPLICATION_CREDENTIALS. A key file that is replaced or rewritten, such as a rotated Kubernetes secret, is loaded on the next call instead of after Google rejects the old key or the proxy restarts * fix(vertex_ai): keep the earlier key while a replaced key file does not load A key file caught halfway through a non-atomic write, or replaced by one that does not load, now keeps the credentials loaded from its earlier version in use and logs a warning, as the cache did before key files were re-read. The next complete file is loaded on the next call. The pub/sub test no longer writes a key file it only reaches through the seeded cache * fix(vertex_ai): try a replaced key file again when its first token fetch fails The earlier key now stands in only when the replaced file does not load. A token fetch that fails on the new key raises like any first load, so the next call tries the new key again instead of keeping the old one * fix(vertex): raise when an async reload keeps an earlier key Google rejected --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| spend_logs_payload.json | ||
| test_pub_sub.py | ||