Classic298
9e51fe421c
fix: read S3 files with long non-ASCII names ( #30418 )
...
With S3 storage, a file whose stored name is close to the 255-byte filename limit and contains non-ASCII characters (for example a Cyrillic name of about 210-218 bytes) uploads fine, but every later read fails with "File name too long". Processing never gets the content, so the file shows as attached while the model receives no text.
The read path used boto3's download_file, which first writes to a temporary name with 9 extra characters. boto3 caps that temporary name by characters, not bytes, so multibyte names end up over the limit even though the final name fits.
The download now streams straight into the local path with download_fileobj, the same way the Azure provider already writes its local copy. That path is the one the upload just wrote successfully, so it always fits. ASCII names, key prefixes and multipart downloads behave as before.
Fixes #30409
2026-09-23 23:30:55 -04:00
Timothy Jaeryang Baek
d27aa72ab4
refac
2026-09-06 17:13:32 -04:00
Timothy Jaeryang Baek
bb0f898b43
refac
2026-07-31 17:41:14 -04:00
Timothy Jaeryang Baek
6d0295588e
refac: modernize type annotations (PEP 604 / PEP 585)
2026-05-12 17:10:15 +09:00
Timothy Jaeryang Baek
8172c7e3d5
refac
2026-04-12 19:08:30 -05:00
Timothy Jaeryang Baek
15f9a8f3f1
refac
2026-04-12 12:36:21 -05:00
Timothy Jaeryang Baek
de3317e26b
refac
2026-03-17 17:58:01 -05:00
Timothy Jaeryang Baek
f376d4f378
chore: format
2026-02-11 16:24:11 -06:00
Classic298
823b9a6dd9
chore/perf: Remove old SRC level log env vars with no impact ( #20045 )
...
* Update openai.py
* Update env.py
* Merge pull request open-webui#19030 from open-webui/dev (#119 )
Co-authored-by: Tim Baek <tim@openwebui.com>
Co-authored-by: Claude <noreply@anthropic.com>
---------
Co-authored-by: Tim Baek <tim@openwebui.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-12-20 08:16:14 -05:00
Timothy Jaeryang Baek
9726c0b7ee
refac/fix: s3 checksum validation
2025-08-21 12:44:16 +04:00
Suleiman Elkhoury
e10d6ad79c
Fix S3 allowed characters in Tags.
...
In Amazon S3 storage, only the following charaters allowed in Tagging "letters (a-z, A-Z), numbers (0-9), and spaces representable in UTF-8, and the following characters: + - = . _ : / @". Added a sanitizer function to clear tags before the put request.
2025-05-23 11:09:40 +02:00
Suleiman Elkhoury
ccc64ac6b7
Update provider.py
...
Add S3_ENABLE_TAGGING to add tags optionally based on the env file.
2025-05-07 14:48:49 +02:00
Timothy Jaeryang Baek
63fe426a52
chore: format
2025-04-28 17:46:05 +04:00
Jarrod Lowe
141810039f
Add tags to S3 uploaded files
2025-04-28 16:56:56 +12:00
Sara Angel-Murphy
5e873bc643
feat: add AWS workload identity support
2025-02-27 13:12:54 -05:00
Timothy Jaeryang Baek
3be5e3129b
Merge pull request #10752 from NovoNordisk-OpenSource/yvedeng/standardize-logging
...
refactor: replace print statements with logging
2025-02-25 10:53:02 -08:00
Yifang Deng
0e5d5ecb81
refactor: replace print statements with logging for better error tracking
2025-02-25 15:53:55 +01:00
orenzhang
4c3dd84b32
feat(s3): support for s3 virtual path style
2025-02-23 20:31:08 +08:00
Chris Pietschmann
7404494772
formatting
2025-02-18 15:19:35 -05:00
Chris Pietschmann
a29f83c4e7
updates to formatting
2025-02-18 15:17:49 -05:00
Chris Pietschmann
fd3c24af4e
Add AzureStorageProvider
2025-02-18 13:25:31 -05:00
Timothy Jaeryang Baek
60095598ec
chore: format
2025-02-09 22:20:47 -08:00
Patrick Deniso
f8a8218149
fix bug where '/' was not properly inserted in s3 key strings
2025-02-07 14:42:16 -05:00
Patrick Deniso
7f82476926
use key_prefix in rest of S3StorageProvider
2025-02-07 13:56:57 -05:00
Patrick Deniso
5ca6afc0fc
add s3 key prefix support
2025-02-07 12:15:54 -05:00
Timothy Jaeryang Baek
8d3c73aed5
chore: format backend
2025-01-21 19:33:33 -08:00
kahghi
4aa9b8d5e7
removed unused pytest-env and imports
2025-01-21 15:38:41 +08:00
kahghi
b1887fef1b
added suggestions
2025-01-21 14:47:56 +08:00
kahghi
55cc127b03
gcs tests pass, updated get_file and delete_file to retrieve blob instead of instantiating blob
2025-01-21 12:53:49 +08:00
kahghi
4b56c15a3f
test_upload_file working, added gcp-storage-emulator, updated gcs client instantiation
2025-01-20 22:35:43 +08:00
kahghi
44574c434d
removed unnecessary GCS_PROJECT_ID variable
2025-01-20 15:27:55 +08:00
kahghi
49f31ddcd8
added alternative storage client instantiation method, corrected filepaths, added missing type hinting
2025-01-19 22:42:29 +08:00
kahghi
1764de41f3
updated to use credentials json, tested with local built image and upload to gcs works
2025-01-17 21:53:41 +08:00
kahghi
8efc543f49
added google-cloud-storage package, added gcs related functions
2025-01-17 16:16:25 +08:00
kahghi
42b7beb8a7
add GCSStorageProvider
2025-01-17 09:39:59 +08:00
Rodrigo Agundez
dd16c3d5c1
add function to retrieve the storage provider so it can be easily tested
2025-01-16 08:09:56 +08:00
Rodrigo Agundez
a3f737c0c6
Split the storage providers into separate classes in preparation for other storage providers like GCS
2025-01-16 07:50:12 +08:00
Timothy Jaeryang Baek
76decdba2a
fix: delete file
2024-12-28 17:40:00 -08:00
Timothy J. Baek
6027c13227
fix: s3 file path
2024-11-03 01:07:43 -08:00
Timothy J. Baek
5d2714bc92
fix: s3 file upload
2024-10-25 21:50:31 -07:00
Timothy J. Baek
cd8271eb95
fix
2024-10-20 23:48:28 -07:00
Timothy J. Baek
7984980619
feat: s3 support
2024-10-20 23:38:26 -07:00
Timothy J. Baek
cb86e09005
feat: s3 support (cherry-picked from #6117 )
...
Co-Authored-By: ssnnow <ssnnow@gmail.com>
2024-10-20 22:53:45 -07:00
