Krish Dholakia
4351c77253
Support Gemini audio token cost tracking + fix openai audio input token cost tracking ( #9535 )
...
* fix(vertex_and_google_ai_studio_gemini.py): log gemini audio tokens in usage object
enables accurate cost tracking
* refactor(vertex_ai/cost_calculator.py): refactor 128k+ token cost calculation to only run if model info has it
Google has moved away from this for gemini-2.0 models
* refactor(vertex_ai/cost_calculator.py): migrate to usage object for more flexible data passthrough
* fix(llm_cost_calc/utils.py): support audio token cost tracking in generic cost per token
enables vertex ai cost tracking to work with audio tokens
* fix(llm_cost_calc/utils.py): default to total prompt tokens if text tokens field not set
* refactor(llm_cost_calc/utils.py): move openai cost tracking to generic cost per token
more consistent behaviour across providers
* test: add unit test for gemini audio token cost calculation
* ci: bump ci config
* test: fix test
2025-03-26 17:26:25 -07:00
Krrish Dholakia
04490c99d7
test: fix test
2025-03-26 17:12:09 -07:00
Krrish Dholakia
72c0ad419f
build(migration.sql): add migration file for new dailyusertable
...
documents prisma db changes
2025-03-26 17:11:25 -07:00
Krrish Dholakia
d4adc9764b
test(test_db_schema_migration.py): ci/cd test to enforce schema migrations are documented in .sql files
2025-03-26 16:59:50 -07:00
Krish Dholakia
37265c8b1c
Add Daily User Spend Aggregate view - allows UI Usage tab to work > 1m rows ( #9538 )
...
* ci: update github action
* build(schema.prisma): enable daily user spend table
allows storing aggregate view of user's daily spend
* build(schema.prisma): add new daily user spend table
* feat: working daily user spend tracking
maintains an aggregate view for easier querying in high traffic
* setup_google_dns
* ci: update ci yaml
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2025-03-26 16:36:36 -07:00
Krrish Dholakia
fb9abd40f3
ci(config.yml): add pytest-postgres to ci/cd
2025-03-26 16:25:08 -07:00
Krish Dholakia
132d3f7baa
feat(prisma-migrations): add baseline db migration file ( #9565 )
...
adds initial baseline db migration file
enables future schema changes to be documented via .sql files
2025-03-26 16:22:56 -07:00
Ishaan Jaff
b7f4abd13a
Merge pull request #9542 from BerriAI/litellm_fix_vertex_ai_ft_models
...
[Feature]: Support for Fine-Tuned Vertex AI LLMs
2025-03-26 16:19:41 -07:00
Ishaan Jaff
0aae9aa24a
rename _is_model_gemini_spec_model
2025-03-26 14:28:26 -07:00
Ishaan Jaff
c38b41f65b
test_get_supports_system_message
2025-03-26 14:26:08 -07:00
Ishaan Jaff
72f08bc6ea
unit tests for VertexGeminiConfig
2025-03-26 14:21:35 -07:00
Ishaan Jaff
8eaf4c55c0
test_gemini_fine_tuned_model_request_consistency
2025-03-26 14:18:11 -07:00
Ishaan Jaff
da9d849348
test_gemini_fine_tuned_model_request_consistency
2025-03-26 14:10:32 -07:00
Ishaan Jaff
82faa49668
doc fix Fine-tuned Models
2025-03-26 14:09:27 -07:00
Ishaan Jaff
28ab8fdccc
docs vertex ft model
2025-03-26 14:08:19 -07:00
Ishaan Jaff
e24a601897
docs verte ft models
2025-03-26 14:03:42 -07:00
Ishaan Jaff
82016eba0a
docs vertex ft models
2025-03-26 13:55:38 -07:00
Ishaan Jaff
12eb77d02d
docs litellm vertex ai ft models
2025-03-26 12:24:49 -07:00
Ishaan Jaff
93daf5cbac
_get_model_name_from_gemini_spec_model
2025-03-26 12:16:18 -07:00
Krish Dholakia
801ecb6517
Nova Canvas complete image generation tasks ( #9177 ) ( #9525 )
...
* Nova Canvas complete image generation tasks (#9177 )
* add initial support for Amazon Nova Canvas model
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* adjust name to AmazonNovaCanvas and map function variables to config
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* tighten model name check
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* fix quality mapping
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* add premium quality in config
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* support all Amazon Nova Canvas tasks
* remove unused import
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* add tests for image generation tasks and fix payload
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* add missing util file
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* update model prices backup file
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* remove image tasks other than text->image
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* add color guided generation task for Nova Canvas
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* fix merge
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* add nova canvas image generation documentation
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
* add nova canvas unit tests
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
---------
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
* ci(config.yml): bump ci config
* test: fix test
---------
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
Co-authored-by: omrishiv <327609+omrishiv@users.noreply.github.com>
2025-03-26 11:28:20 -07:00
Ishaan Jaff
793a920caa
rename _is_model_gemini_spec_model
2025-03-26 11:14:51 -07:00
Ishaan Jaff
1822c45c0e
test fix test_pick_cheapest_chat_model_from_llm_provider
2025-03-26 11:13:21 -07:00
Ishaan Jaff
baa9b34950
Merge branch 'main' into litellm_fix_vertex_ai_ft_models
2025-03-26 11:11:54 -07:00
Ishaan Jaff
a2ae9be6a2
undo changes to utils
2025-03-26 11:09:29 -07:00
Ishaan Jaff
fee20250a6
pick_cheapest_chat_models_from_llm_provider
2025-03-26 10:59:02 -07:00
Ishaan Jaff
8a72b67b18
undo code changes
2025-03-26 10:57:08 -07:00
Ishaan Jaff
3ee7962f9c
fix llm request utils
2025-03-26 10:55:56 -07:00
Ishaan Jaff
bbe69a47a9
_is_model_gemini_gemini_spec_model
2025-03-26 10:53:23 -07:00
Ishaan Jaff
8c845847cd
fix vertex embedding perf test
2025-03-26 10:25:50 -07:00
Ishaan Jaff
5f51228b72
DNS lookup for Redis host
2025-03-26 09:28:59 -07:00
Ishaan Jaff
d7884114d0
setup_google_dns
2025-03-26 09:26:33 -07:00
Ishaan Jaff
68665ea028
DNS lookup for Redis host
2025-03-26 09:25:14 -07:00
Krrish Dholakia
a97d0563b4
build(model_prices_and_context_window.json): add commercial rate limits for gemini 2.0 flash lite
2025-03-26 08:58:35 -07:00
Ishaan Jaff
c3dc4a2ecd
fix DNS resolution
2025-03-26 08:55:18 -07:00
Ishaan Jaff
e5a5dca777
fix load_testing
2025-03-26 08:46:23 -07:00
Ishaan Jaff
7123b69cca
fix caching unit tests
2025-03-26 08:42:55 -07:00
Ishaan Jaff
22dfc4dea9
fix user_api_key_auth example config
2025-03-26 08:36:11 -07:00
Ishaan Jaff
e7a8b5a809
run ci/cd again
2025-03-26 08:12:51 -07:00
Ishaan Jaff
fb31006cd8
select_model_for_request_transformation
2025-03-26 07:12:25 -07:00
Ishaan Jaff
27c085cc56
fix util vertex
2025-03-26 00:08:16 -07:00
Ishaan Jaff
2bef0481af
_transform_request_body
2025-03-26 00:05:45 -07:00
Ishaan Jaff
830ecbdb8c
fix get_optional_params
2025-03-25 23:54:57 -07:00
Ishaan Jaff
aa8ba9b8f2
fix base_model in param mapping
2025-03-25 23:54:29 -07:00
Ishaan Jaff
efce84815a
test_gemini_fine_tuned_model_request_consistency
2025-03-25 23:54:06 -07:00
Krish Dholakia
6fd18651d1
Support litellm.api_base for vertex_ai + gemini/ across completion, embedding, image_generation ( #9516 )
...
* test(tests): add unit testing for litellm_proxy integration
* fix(cost_calculator.py): fix tracking cost in sdk when calling proxy
* fix(main.py): respect litellm.api_base on `vertex_ai/` and `gemini/` routes
* fix(main.py): consistently support custom api base across gemini + vertexai on embedding + completion
* feat(vertex_ai/): test
* fix: fix linting error
* test: set api base as None before starting loadtest
2025-03-25 23:46:20 -07:00
Ishaan Jaff
8657816477
fix gemini/gemini-2.0-flash-lite on model cost map
2025-03-25 23:20:43 -07:00
Krrish Dholakia
e0880734d9
docs(config_settings.md): cleanup docs
2025-03-25 23:11:45 -07:00
Krrish Dholakia
24b3e80eba
ci: update github action
2025-03-25 23:11:45 -07:00
Ishaan Jaff
c61214dcf1
Merge pull request #9523 from BerriAI/litellm_add_gemini_flash_lite
...
[Feat - New Model] Add VertexAI `gemini-2.0-flash-lite` and Google AI Studio `gemini-2.0-flash-lite`
2025-03-25 23:11:44 -07:00
Ishaan Jaff
c473e2b1c2
setup_google_dns
2025-03-25 23:02:02 -07:00
