Releases: s3tools/s3cmd
Releases · s3tools/s3cmd
v2.4.0
- Added "setversioning" command for versioning configuration (Kuan-Chun Wang)
- Added "settagging", "gettagging", and "deltagging" commands for bucket/object tagging (Kuan-Chun Wang)
- Added "setobjectretention" and "setobjectlegalhold" commands (Etienne Adam/Withings SAS)
- Added "setownership" and "setblockpublicaccess" commands
- Added "cfinval" command to request Cloudfront to invalidate paths (#1256)
- Added "--keep-dirs" option to have the folder structure preserved on remote side
- Added --skip-destination-validation option for "setnotification" command (Kuan-Chun Wang)
- Added "--max-retries" flag and "max_retries" config option (#914)
- Added FIPS support (Michael Roth)
- Added "object ownership" and block public access" values to "info" command output for buckets
- Added to "ls" command a "DIROBJ" tag for directory objects in S3 remote
- Added server profiles to run-tests.py to skip tests depending on the server type
- Fixed "TypeError: sequence item 1: expected str instance, bytes found" error with Python 3.12 (#1343)
- Fixed a missing return for "object_batch_delete" of S3.py (James Hewitt)
- Fixed "object is not callable" error because of md5 FIPS test (#1005)
- Fixed "compute_content_md5 is not defined" error for "setversioning" (#1312) (Gavin John)
- Fixed list objects to use NextMarker when only prefixes are returned (Albin Parou)
- Fixed upload to not retry when an S3 compatible server is full
- Fixed recursive delete of objects named with whitespace (#976)
- Fixed the mime type when uploading directories to be "application/x-directory"
- Fixed "string indices must be integers" error for sync when in dry-run mode (#1313) (Shohei Tanaka)
- Fixed SignatureDoesNotMatch error when modifying an object on Cloudflare R2 (#1332) (Philip C Nilsson)
- Fixed Cloudfront invalidation issue for paths with wildcard or special characters
- Fixed Cloudfront crash because of error reporting for retries
- Fixed Cloudfront "unable to parse URL" error (#1292)
- Improved the handling of "empty" files on the remote side to sync with local folders
- Improved "abortmp" command by requiring an object to avoid bad accidents when using Ceph (Joshua Haas)
- Improved file download by retrying when encountering SlowDown or TooManyRequests errors (Robin Geiger)
- Improved error messages in case of connection error or host unreachable
- Improved error messages to be more explicit for upload errors after retries
- Improved remote2local attributes setting code
- Improved remote2local with more explicit error messages when setting attributes (#1288)
- Improved remote2local output messages by using the "mkdir" prefix instead of "make dir"
- Improved the SortedDict class
- Improved run-test.py by using "--include" when calling Curl instead of "-include" (Matthew James Kraai)
- Improved GitHub CI by enabling pip cache in actions/setup-python (Anton Yakutovich)
- Improved GitHub CI by adding a "codespell" check on push and PRs (Yaroslav Halchenko)
- Updated GitHub CI tests to use more recent versions of Minio and Python
- Upgraded GitHub actions (Anton Yakutovich)
- Cleanup and update of copyright headers, docs, comments and setup.py
- Cleanup to fix "invalid escape sequence" syntax warnings
- Many other bug fixes and cleanups
New Contributors
- @maroth96 made their first contribution in #1233
- @jeffwang0516 made their first contribution in #1277
- @Fumesover made their first contribution in #1278
- @drakulavich made their first contribution in #1281
- @Jamstah made their first contribution in #1034
- @jahschwa made their first contribution in #1289
- @xcir made their first contribution in #1314
- @Pandapip1 made their first contribution in #1340
- @yarikoptic made their first contribution in #1324
- @PhilipNilsson7 made their first contribution in #1333
- @kraai made their first contribution in #1296
- @withings-sas made their first contribution in #1297
Full Changelog: v2.3.0...v2.4.0
v2.3.0
- Added "getnotification", "setnotification", and "delnotification" commands for notification policies (hrchu)
- Added support for AWS_STS_REGIONAL_ENDPOINTS (#1218, #1228) (Johan Lanzrein)
- Added ConnectionRefused [111] exit code to handle connection errors (Salar Nosrati-Ershad)
- Added support for IMDSv2. Should work automatically on ec2 (Anthony Foiani)
- Added --list-allow-unordered to list objects unordered. Only supported by Ceph based s3-compatible services (#1269) (Salar Nosrati-Ershad)
- Fixed --exclude dir behavior for python >= 3.6 (Daniil Tararukhin)
- Fixed Cloudfront invalidate retry issue (Yuan-Hsiang Lee)
- Fixed 0 byte cache files crashing s3cmd (#1234) (Carlos Laviola)
- Fixed --continue behavior for the "get" command (#1009) (Anton Ustyugov)
- Fixed unicode issue with fixbucket (#1259)
- Fixed CannotSendRequest and ConnectionRefusedError errors at startup (#1261)
- Fixed error reporting for object info when the object does not exist
- Fixed "setup.py test" to do nothing to avoid failure that could be problematic for distribution packaging (#996)
- Improved expire command to use Rule/Filter/Prefix for LifecycleConfiguration (#1247)
- Improved PASS/CHECK/INCLUDE/EXCLUDE debug log messages
- Improved setup.py with python 3.9 and 3.10 support info(Ori Avtalion)
- Many other bug fixes
v2.2.0
- Added support for metadata modification of files bigger than 5 GiB
- Added support for remote copy of files bigger than 5 GiB using MultiPart copy (Damian Martinez, Florent Viard)
- Added progress info output for multipart copy and current-total info in output for cp, mv and modify
- Added support for all special/foreign character names in object names to cp/mv/modify
- Added support for SSL authentication (Aleksandr Chazov)
- Added the http error 429 to the list of retryable errors (#1096)
- Added support for listing and resuming of multipart uploads of more than 1000 parts (#346)
- Added time based expiration for idle pool connections in order to avoid random broken pipe errors (#1114)
- Added support for STS webidentity authentication (ie AssumeRole and AssumeRoleWithWebIdentity) (Samskeyti, Florent Viard)
- Added support for custom headers to the mb command (#1197) (Sébastien Vajda)
- Improved MultiPart copy to preserve acl and metadata of objects
- Improved the server errors catching and reporting for cp/mv/modify commands
- Improved resiliency against servers sending garbage responses (#1088, #1090, #1093)
- Improved remote copy to have consistent copy of metadata in all cases: multipart or not, aws or not
- Improved security by revoking public-write acl when private acl is set (#1151) (ruanzitao)
- Improved speed when running on an EC2 instance (#1117) (Patrick Allain)
- Reduced connection_max_age to 5s to avoid broken pipes as AWS closes https conns after around 6s (#1114)
- Ensure that KeyboardInterrupt are always properly raised (#1089)
- Changed sized of multipart copy chunks to 1 GiB
- Fixed ValueError when using more than one ":" inside add_header in config file (#1087)
- Fixed extra label issue when stdin used as source of a MultiPart upload
- Fixed remote copy to allow changing the mime-type (ie content-type) of the new object
- Fixed remote_copy to ensure that meta-s3cmd-attrs will be set based on the real source and not on the copy source
- Fixed deprecation warnings due to invalid escape sequences (Karthikeyan Singaravelan)
- Fixed getbucketinfo that was broken when the bucket lifecycle uses the filter element (Liu Lan)
- Fixed RestoreRequest XML namespace URL (#1203) (Akete)
- Fixed PARTIAL exit code that was not properly set when needed for object_get (#1190)
- Fixed a possible infinite loop when a file is truncated during hashsum or upload (#1125) (Matthew Krokosz, Florent Viard)
- Fixed report_exception wrong error when LANG env var was not set (#1113)
- Fixed wrong wiki url in error messages (Alec Barrett)
- Py3: Fixed an AttributeError when using the "files-from" option
- Py3: Fixed compatibility issues due to the removal of getchildren() from ElementTree in python 3.9 (#1146, #1157, #1162, # 1182, #1210) (Ondřej Budai)
- Py3: Fixed compatibility issues due to the removal of encodestring() in python 3.9 (#1161, #1174) (Kentaro Kaneki)
- Fixed a crash when the AWS_ACCESS_KEY env var is set but not AWS_SECRET_KEY (#1201)
- Cleanup of check_md5 (Riccardo Magliocchetti)
- Removed legacy code for dreamhost that should be necessary anymore
- Migrated CI tests to use github actions (Arnaud J)
- Improved README with a link to INSTALL.md (Sia Karamalegos)
- Improved help content (Dmitrii Korostelev, Roland Van Laar)
- Improvements for setup and build configurations
- Many other bug fixes
v2.1.0
- Changed size reporting using k instead of K as it a multiple of 1024 (#956)
- Added "public_url_use_https" config to generate public url using https (#551, #666) (Jukka Nousiainen)
- Added option to make connection pooling configurable and improvements (Arto Jantunen)
- Added support for path-style bucket access to signurl (Zac Medico)
- Added docker configuration and help to run test cases with multiple Python versions (Doug Crozier)
- Relaxed limitation on special chars for --add-header key names (#1054)
- Fixed all regions that were automatically converted to lower case (Harshavardhana)
- Fixed size and alignment of DU and LS output reporting (#956)
- Fixes for SignatureDoesNotMatch error when host port 80 or 443 is specified, due to stupid servers (#1059)
- Fixed the useless retries of requests that fail because of ssl cert checks
- Fixed a possible crash when a file disappears during cache generation (#377)
- Fixed unicode issues with IAM (#987)
- Fixed unicode errors with bucked Policy/CORS requests (#847) (Alex Offshore)
- Fixed unicode issues when loading aws_credential_file (#989)
- Fixed an issue with the tenant feature of CephRGW. Url encode bucket_name for path-style requests (#1080)
- Fixed signature v2 always used when bucket_name had special chars (#1081)
- Allow to use signature v4 only, even for commands without buckets specified (#1082)
- Fixed small open file descriptor leaks.
- Py3: Fixed hash-bang in headers to not force using python2 when setup/s3cmd/run-test scripts are executed directly.
- Py3: Fixed unicode issues with Cloudfront (#1006)
- Py3: Fixed http.client.RemoteDisconnected errors (#1014) (Ryan Huddleston)
- Py3: Fixed 'dictionary changed size during iteration' error when using a cache-file (#945) (Doug Crozier)
- Py3: Fixed the display of file sizes (Vlad Presnyak)
- Py3: Python 3.8 compatibility fixes (Konstantin Shalygin)
- Py2: Fixed unicode errors sometimes crashing remote2remote sync (#847)
- Added s3cmd.egg-info to .gitignore (Philip Dubé)
- Improved run-test script to not use hard-coded bucket names(#1066) (Doug Crozier)
- Renamed INSTALL to INSTALL.md and improvements (Nitro, Prabhakar Gupta)
- Improved the restore command help (Hrchu)
- Updated the storage-class command help with the recent aws s3 classes (#1020)
- Fixed typo in the --continue-put help message (Pengyu Chen)
- Fixed typo (#1062) (Tim Gates)
- Improvements for setup and build configurations
- Many other bug fixes
v2.0.2
- Fixed unexpected timeouts encountered during requests or transfers due to AWS strange connection short timeouts (#941)
- Fixed a throttle issue slowing down too much transfers in some cases (#913)
- Added support for $AWS_PROFILE (#966) (Taras Postument)
- Added clarification comment for the socket_timeout configuration value OS limit
- Avoid distutils usage at runtime (Matthias Klose)
- Python 2 compatibility: Fixed import error of which with fallback code (Gianfranco Costamagna)
- Fixed Python 3 bytes string encoding when getting IAM credentials (Alexander Allakhverdiyev)
- Fixed handling of config tri-state bool values (like acl_public) (Brian C. Lane)
- Fixed V2 signature when restore command is used (Jan Kasiak)
- Fixed setting full_control on objects with public read access (Matthew Vernon)
- Fixed a bug when only one path is supplied with Cloudfront. (Mikael Svensson)
- Fixed signature errors with 'modify' requests (Radek Simko)
- Fixes #936 - Fix setacl command exception (Robert Moucha)
- Fixes error reporting if deleting a source object failed after a move (#929)
- Many other bug fixes (#525, #933, #940, #947, #957, #958, #960, #967)
Important info:
AWS S3 doesn't allow anymore uppercases and underscores in bucket names since march 1, 2018.
v2.0.1
- Support for Python 3 is now stable
- Fixed signature issues due to upper cases in hostname (#920)
- Improved support for Minio Azure gateway (Julien Maitrehenry, Harshavardhana)
- Added signurl_use_https option to use https prefix for signurl (Julien Recurt)
- Fixed a lot of remaining issues and regressions for Python 3 (#922, #921, #908)
- Fixed --configure option with Python 3
- Fixed non string cmdline parameters being ignored
- Windows support fixes (#922)
- Don't force anymore to have a / on last parameter for the "modify" command (#886)
- Removed the python3 support warning
- Detect and report error 403 in getpolicy for info command (#894)
- Added a specific error message when getting policy by non owner (#885)
- Many other bug fixes (#905, #892, #890, #888, #889, #887)
v2.0.0
- Added support for Python 3 (Shaform, Florent Viard)
- Added getlifecycle command (Daniel Gryniewicz)
- Added --cf-inval for invalidating multiple CF distributions (Joe Mifsud)
- Added --limit to "ls" and "la" commands to return the specified number of objects (Masashi Ozawa)
- Added --token-refresh and --no-token-refresh and get the access token from the environment (Marco Jakob)
- Added --restore-priority and --restore-days for S3 Glacier (Robert Palmer)
- Fixed requester pays header with HEAD requests (Christian Rodriguez)
- Don't allow mv/cp of multiple files to single file (Guy Gur-Ari)
- Generalize wildcard certificate forgiveness (Mark Titorenko)
- Multiple fixes for SSL connections and proxies
- Added support for HTTP 100-CONTINUE
- Fixes for s3-like servers
- Big cleanup and many unicode fixes
- Many other bug fixes
V1.6.1
v1.6.0
s3cmd-1.6.0 - 2015-09-18
- Support signed URL content disposition type
- Added 'ls -l' long listing including storage class
- Added --limit-rate=RATE
- Added --server-side-encryption-kms-id=KEY_ID
- Added --storage-class=CLASS
- Added --requester-pays, [payer] command
- Added --[no-]check-hostname
- Added --stop-on-error, removed --ignore-failed-copy
- Added [setcors], [delcors] commands
- Added support for cn-north-1 region hostname checks
- Output strings may have changed. Scripts calling s3cmd expecting
specific text may need to be updated. - HTTPS is now the default
- Many unicode fixes
- Many other bug fixes