/usr/lib/rads/venv/lib/python3.13/site-packages/s3transfer/__pycache__
Edit: /usr/lib/rads/venv/lib/python3.13/site-packages/s3transfer/__pycache__/upload.cpython-313.pyc (31474B)
j
| 0 S SK r S SKJr S SKJrJr S SKJr S SKJ r S SK
J r J r J
r
Jr S SKJrJrJrJr " S S 5 r " S
S 5 r " S S
5 r " S S\5 r " S S\5 r " S S\5 r " S S\
5 r " S S\5 r " S S\5 r g) N)BytesIOreadableseekable)FULL_OBJECT_CHECKSUM_ARGS)IN_MEMORY_UPLOAD_TAG)CompleteMultipartUploadTaskCreateMultipartUploadTaskSubmissionTaskTask)ChunksizeAdjusterDeferredOpenFile
get_callbacksget_filtered_dictc 0 \ rS rSrSS jrS rS rS rSrg) AggregatedProgressCallback! c * Xl X l SU l g)a Aggregates progress updates for every provided progress callback
:type callbacks: A list of functions that accepts bytes_transferred
as a single argument
:param callbacks: The callbacks to invoke when threshold is reached
:type threshold: int
:param threshold: The progress threshold in which to take the
aggregated progress and invoke the progress callback with that
aggregated progress total
r N)
_callbacks
_threshold _bytes_seen)self callbacks thresholds D/usr/lib/rads/venv/lib/python3.13/site-packages/s3transfer/upload.py__init__#AggregatedProgressCallback.__init__" s $# c U =R U-
sl U R U R : a U R 5 g g N)r r _trigger_callbacks)r bytes_transferreds r __call__#AggregatedProgressCallback.__call__2 s7 -- t . # # % /r c F U R S: a U R 5 gg)z@Flushes out any progress that has not been sent to its callbacksr N)r r! r s r flush AggregatedProgressCallback.flush7 s! a # # % r c X U R H nU" U R S9 M SU l g )N)r" r )r r )r callbacks r r! -AggregatedProgressCallback._trigger_callbacks< s& H t'7'7 8 ( r )r r r N)i ) __name__
__module__ __qualname____firstlineno__r r# r' r! __static_attributes__ r r r r ! s &
&
r r c J \ rS rSrSrS rS S jrS
S jrS rS r S r
S
r S r g)InterruptReaderB a Wrapper that can interrupt reading using an error
It uses a transfer coordinator to propagate an error if it notices
that a read is being made while the file is being read from.
:type fileobj: file-like obj
:param fileobj: The file-like object to read from
:type transfer_coordinator: s3transfer.futures.TransferCoordinator
:param transfer_coordinator: The transfer coordinator to use if the
reader needs to be interrupted.
c Xl X l g r _fileobj_transfer_coordinator)r fileobjtransfer_coordinators r r InterruptReader.__init__P s
%9"r Nc U R R ( a U R R eU R R U5 $ r )r8 exceptionr7 read)r amounts r r> InterruptReader.readT s: % % / /,,66 6}}!!&))r c : U R R X5 g r )r7 seek)r wherewhences r rB InterruptReader.seek^ s
5)r c 6 U R R 5 $ r )r7 tellr& s r rG InterruptReader.tella s }}!!##r c 8 U R R 5 g r )r7 closer& s r rJ InterruptReader.closed s
r c U $ r r1 r& s r __enter__InterruptReader.__enter__g s r c $ U R 5 g r )rJ )r argskwargss r __exit__InterruptReader.__exit__j s
r r6 r )r )
r, r- r. r/ __doc__r r> rB rG rJ rM rR r0 r1 r r r3 r3 B s* :**$ r r3 c b \ rS rSrSrSS jr\S 5 rS rS r S r
S r S
r S r
S rS
rSrg)UploadInputManagern a" Base manager class for handling various types of files for uploads
This class is typically used for the UploadSubmissionTask class to help
determine the following:
* How to determine the size of the file
* How to determine if a multipart upload is required
* How to retrieve the body for a PutObject
* How to retrieve the bodies for a set of UploadParts
The answers/implementations differ for the various types of file inputs
that may be accepted. All implementations must subclass and override
public methods from this class.
Nc ( Xl X l X0l g r )_osutilr8 _bandwidth_limiter)r osutilr: bandwidth_limiters r r UploadInputManager.__init__~ s %9""3 r c [ S5 e)zDetermines if the source for the upload is compatible with manager
:param upload_source: The source for which the upload will pull data
from.
:returns: True if the manager can handle the type of source specified
otherwise returns False.
z must implement _is_compatible()NotImplementedErrorcls
upload_sources r
is_compatible UploadInputManager.is_compatible s ""CDDr c [ S5 e)a Whether the body it provides are stored in-memory
:type operation_name: str
:param operation_name: The name of the client operation that the body
is being used for. Valid operation_names are ``put_object`` and
``upload_part``.
:rtype: boolean
:returns: True if the body returned by the manager will be stored in
memory. False if the manager will not directly store the body in
memory.
z%must implement store_body_in_memory()r_ r operation_names r stores_body_in_memory(UploadInputManager.stores_body_in_memory s ""IJJr c [ S5 e)zProvides the transfer size of an upload
:type transfer_future: s3transfer.futures.TransferFuture
:param transfer_future: The future associated with upload request
z&must implement provide_transfer_size()r_ r transfer_futures r provide_transfer_size(UploadInputManager.provide_transfer_size s ""JKKr c [ S5 e)a Determines where a multipart upload is required
:type transfer_future: s3transfer.futures.TransferFuture
:param transfer_future: The future associated with upload request
:type config: s3transfer.manager.TransferConfig
:param config: The config associated to the transfer manager
:rtype: boolean
:returns: True, if the upload should be multipart based on
configuration and size. False, otherwise.
z*must implement requires_multipart_upload()r_ r rm configs r requires_multipart_upload,UploadInputManager.requires_multipart_upload s ""NOOr c [ S5 e)a Returns the body to use for PutObject
:type transfer_future: s3transfer.futures.TransferFuture
:param transfer_future: The future associated with upload request
:type config: s3transfer.manager.TransferConfig
:param config: The config associated to the transfer manager
:rtype: s3transfer.utils.ReadFileChunk
:returns: A ReadFileChunk including all progress callbacks
associated with the transfer future.
z$must implement get_put_object_body()r_ rl s r get_put_object_body&UploadInputManager.get_put_object_body s ""HIIr c [ S5 e)a Yields the part number and body to use for each UploadPart
:type transfer_future: s3transfer.futures.TransferFuture
:param transfer_future: The future associated with upload request
:type chunksize: int
:param chunksize: The chunksize to use for this upload.
:rtype: int, s3transfer.utils.ReadFileChunk
:returns: Yields the part number and the ReadFileChunk including all
progress callbacks associated with the transfer future for that
specific yielded part.
z)must implement yield_upload_part_bodies()r_ )r rm chunksizes r yield_upload_part_bodies+UploadInputManager.yield_upload_part_bodies s ""MNNr c [ XR 5 nU R ( a$ U R R XR SS9nU$ )NF)enabled)r3 r8 rZ get_bandwith_limited_stream)r r9 s r
_wrap_fileobj UploadInputManager._wrap_fileobj sI !'+E+EF " "--II33U J G r c D [ US5 nU( a [ U5 /$ / $ )Nprogress)r r )r rm r s r _get_progress_callbacks*UploadInputManager._get_progress_callbacks s& !/:> .y9: : r c D U Vs/ s H o"R PM sn$ s snf r )r' )r aggregated_progress_callbacksr* s r _get_close_callbacks'UploadInputManager._get_close_callbacks s /LM/L8/LMMMs )rZ rY r8 r )r, r- r. r/ rT r classmethodrd ri rn rs rv rz r r r r0 r1 r r rV rV n sP
4
E E
K L
P
J O Nr rV c ^ \ rS rSrSr\S 5 rS rS rS r S r
S r S r S
r
S rS rS
rg)UploadFilenameInputManager z Upload utility for filenamesc " [ U[ 5 $ r )
isinstancestrra s r rd (UploadFilenameInputManager.is_compatible s ---r c g)NFr1 rg s r ri 0UploadFilenameInputManager.stores_body_in_memory s r c UR R U R R UR R R
5 5 g r )metarn rY
get_file_size call_argsr9 rl s r rn 0UploadFilenameInputManager.provide_transfer_size s: 22 LL & &';';'E'E'M'M N
r c H UR R UR : $ r )r sizemultipart_thresholdrq s r rs 4UploadFilenameInputManager.requires_multipart_upload s ##((F,F,FFFr c U R U5 u p#U R U5 nU R U5 nU R U5 nUR R
nU R R UUUUUS9$ )Nr9
chunk_sizefull_file_sizer close_callbacks)&_get_put_object_fileobj_with_full_sizer r r r r rY #open_file_chunk_reader_from_fileobj)r rm r9 full_sizer r r s r rv .UploadFilenameInputManager.get_put_object_body s ! H H
$$W-00 A 33I> ##(( ||??$ + @
r c # # UR R nU R X5 n[ SUS- 5 H nU R U5 nU R U5 nX%S-
- nU R
UR R R UUUS9u pU R U 5 n U R R U UU
UUS9n X[4v M g 7f)N )
start_byte part_sizer r ) r r _get_num_partsranger r '_get_upload_part_fileobj_with_full_sizer r9 r rY r ) r rm ry r num_parts part_numberr r r r9 r read_file_chunks r rz 3UploadFilenameInputManager.yield_upload_part_bodies s (--22''C IM2K 44_EI " 7 7 BO"Ao6J "&!M!M $$..66%# - "N " G ((1G #ll N N $ (# / O O . .5 3s C
Cc B [ XU R R S9nU$ )N)
open_function)r rY open)r r9 r s r _get_deferred_open_file2UploadFilenameInputManager._get_deferred_open_file2 s" " t||/@/@
r c UR R R nUR R nU R US5 U4$ )Nr )r r r9 r r r rm r9 r s r r AUploadFilenameInputManager._get_put_object_fileobj_with_full_size8 sB !&&0088 ##((++GQ7==r c < US nUS nU R X5 U4$ )Nr r )r )r r9 rQ r r s r r BUploadFilenameInputManager._get_upload_part_fileobj_with_full_size= s- L)
+, ++G@)KKr c [ [ R " UR R [ U5 - 5 5 $ r )intmathceilr r float)r rm r s r r )UploadFilenameInputManager._get_num_partsB s, 499_ 1 1 6 6y9I IJKKr r1 N)r, r- r. r/ rT r rd ri rn rs rv rz r r r r r0 r1 r r r r sG &. .
G
0 /> >
L
Lr r c @ \ rS rSrSr\S 5 rS rS rS r S r
Sr g )
UploadSeekableInputManageriF z&Upload utility for an open file objectc < [ U5 =( a [ U5 $ r r ra s r rd (UploadSeekableInputManager.is_compatibleI s
&B8M+BBr c US:X a gg)N
put_objectFTr1 rg s r ri 0UploadSeekableInputManager.stores_body_in_memoryM s \ )r c UR R R nUR 5 nUR SS5 UR 5 nUR U5 UR R XC-
5 g )Nr )r r r9 rG rB rn )r rm r9 start_position end_positions r rn 0UploadSeekableInputManager.provide_transfer_sizeS se !&&0088 ! Q ||~ ^$ 22 )
r c V UR US 5 n[ U5 [ U5 4$ )Nr )r> r len)r r9 rQ datas r r BUploadSeekableInputManager._get_upload_part_fileobj_with_full_size` s, ||F; /0 t}c$i''r c UR R R nUR 5 UR R - nX#4$ r )r r r9 rG r r s r r AUploadSeekableInputManager._get_put_object_fileobj_with_full_sizeo s? !&&0088 ||~ 4 4 9 99} r r1 N) r, r- r. r/ rT r rd ri rn r r r0 r1 r r r r F s- 0C C
( r r c n ^ \ rS rSrSrS
U 4S jjr\S 5 rS rS r S r
S r S r SS
jr
S rS rU =r$ ) UploadNonSeekableInputManageriw z7Upload utility for a file-like object that cannot seek.c 4 > [ TU ] XU5 SU l g )Nr )superr
_initial_data)r r[ r: r\ __class__s r r &UploadNonSeekableInputManager.__init__z s
7HI r c [ U5 $ r )r ra s r rd +UploadNonSeekableInputManager.is_compatible~ s
&&r c g)NTr1 rg s r ri 3UploadNonSeekableInputManager.stores_body_in_memory s r c g r r1 rl s r rn 3UploadNonSeekableInputManager.provide_transfer_size s r c 2 UR R b# UR R UR : $ UR R R nUR nU R X4S5 U l [ U R 5 U: a gg)NFT)r r r r r9 _readr r )r rm rr r9 r s r rs 7UploadNonSeekableInputManager.requires_multipart_upload s $ $ 0"'',,0J0JJ J "&&0088.. !ZZE B t!! "Y .r c U R U5 nU R U5 nUR R R nU R U R UR 5 - X#5 nS U l U$ r )r r r r r9
_wrap_datar r> )r rm r r r9 bodys r rv 1UploadNonSeekableInputManager.get_put_object_body sl 00 A 33I>!&&0088 /
" r c # # UR R R nSn U R U5 nU R U5 nUS-
nU R X25 nU( d g U R
XuU5 nS nXH4v M[ 7f)Nr r )r r r9 r r r r ) r rm ry file_objectr r r part_content part_objects r rz 6UploadNonSeekableInputManager.yield_upload_part_bodies s %**44<< 44_EI " 7 7 BO 1 K ::k=L // K
L * * s A?Bc l [ U R 5 S:X a UR U5 $ U[ U R 5 :: a, U R SU nU( a U R US U l U$ U[ U R 5 -
nU R UR U5 - nU( a SU l U$ )a
Reads a specific amount of data from a stream and returns it. If there
is any data in initial_data, that will be popped out first.
:type fileobj: A file-like object that implements read
:param fileobj: The stream to read from.
:type amount: int
:param amount: The number of bytes to read from the stream.
:type truncate: bool
:param truncate: Whether or not to truncate initial_data after
reading from it.
:return: Generator which generates part bodies from the initial data.
r Nr )r r r> )r r9 r? truncater amount_to_reads r r #UploadNonSeekableInputManager._read s $ t!! "a '<<' ' S++, ,%%gv.D %)%7%7%@"K
#d&8&8"99!!GLL $@@ !$D r c U R [ U5 5 nU R R U[ U5 [ U5 UUS9$ )ah
Wraps data with the interrupt reader and the file chunk reader.
:type data: bytes
:param data: The data to wrap.
:type callbacks: list
:param callbacks: The callbacks associated with the transfer future.
:type close_callbacks: list
:param close_callbacks: The callbacks to be called when closing the
wrapper for the data.
:return: Fully wrapped data.
r )r r rY r r )r r r r r9 s r r (UploadNonSeekableInputManager._wrap_data sL $$WT]3||??4y t9 + @
r )r r )T)r, r- r. r/ rT r r rd ri rn rs rv rz r r r0
__classcell__)r s @r r r w sH A! ' '
" +((T
r r c ~ \ rS rSrSrSS/r\S/- r/ SQr/ SQ\- r S r
SS jr S
r S r
S rS
rS rS rS rSrg)UploadSubmissionTaski z.Task for submitting tasks to execute an upload ChecksumType
MpuObjectSize)ChecksumAlgorithmSSECustomerKeySSECustomerAlgorithmSSECustomerKeyMD5 RequestPayerExpectedBucketOwner)r r r r r r r c [ [ [ /nUR R R
nU H nUR
U5 ( d M Us $ [ SU S[ U5 S35 e)a? Retrieves a class for managing input for an upload based on file type
:type transfer_future: s3transfer.futures.TransferFuture
:param transfer_future: The transfer future for the request
:rtype: class of UploadInputManager
:returns: The appropriate class to use for managing a specific type of
input for uploads.
zInput z
of type: z is not supported.) r r r r r r9 rd RuntimeErrortype)r rm upload_manager_resolver_chainr9 upload_manager_clss r _get_upload_input_manager_cls2UploadSubmissionTask._get_upload_input_manager_cls sw
' & ))
% "&&0088"? !//88)) #@ WIZW
6H I
r Nc U R U5 " X0R U5 nUR R c UR U5 UR XR5 ( d U R
UUUUUU5 gU R UUUUUU5 g)aj
:param client: The client associated with the transfer manager
:type config: s3transfer.manager.TransferConfig
:param config: The transfer config associated with the transfer
manager
:type osutil: s3transfer.utils.OSUtil
:param osutil: The os utility associated to the transfer manager
:type request_executor: s3transfer.futures.BoundedExecutor
:param request_executor: The request executor associated with the
transfer manager
:type transfer_future: s3transfer.futures.TransferFuture
:param transfer_future: The transfer future associated with the
transfer request that tasks are being submitted for
N)r r8 r r rn rs _submit_upload_request_submit_multipart_request)r clientrr r[ request_executorrm r\ upload_input_managers r _submit UploadSubmissionTask._submit3 s 6 $ A A
,,.? A
$ $ , 6 6 G $==
' ' $
* * $
r c 8 UR R nU R UR 5 nU R US5 n U R
R
U[ U R
UUR U5 UR UR US.SS9U S9 g )Nr )r r9 bucketkey
extra_argsT)r: main_kwargsis_finaltag) r r _extra_put_object_argsr _get_upload_task_tagr8 submit
PutObjectTaskrv r r
)
r r rr r[ r rm r r put_object_extra_argsput_object_tags
r r +UploadSubmissionTask._submit_upload_requestl s $((22 $ ; ; !
22 ,
"")) %)%?%? $ 3 G G' ( . .$=="7
*
r c UR R n[ HB nXR ; d M SUR S' UR SS5 UR S' MD U R UR 5 n U R R U[ U R UUR UR U S.S95 n
/ n U R UR 5 n U R US5 n
UR R n[ 5 nUR UR U5 nUR# UU5 nU H` u nnU R% U R R U[' U R UUUR UR UU S .S
U
0S 9U
S 95 Mb U R) UR 5 nU R R U[+ U R UUR UR US.U
U S
.SS95 g )N FULL_OBJECTr Checksum r )r r r
r )r: r upload_part)r r9 r r
r r upload_id)r: r pending_main_kwargsr )r partsT)r: r r r )r r r r replace _extra_create_multipart_argsr8 r r
r r
_extra_upload_part_argsr r r
adjust_chunksizemultipart_chunksizerz appendUploadPartTask _extra_complete_multipart_argsr )r r rr r[ r rm r r checksumcreate_multipart_extra_argscreate_multipart_future part_futuresextra_part_argsupload_part_tagr adjusterry
part_iteratorr r9 complete_multipart_extra_argss r r .UploadSubmissionTask._submit_multipart_request s0 $((22 2H///7D $$^4
r5 r9 r r r r r$ r) r# r r r0 r1 r r r r sp 8 *O<!:o=N!N " "
> 7r%
Na
FD
K
r r c \ rS rSrSrS rSrg)r i z Task to do a nonmultipart uploadc f U nUR " SX4US.UD6 SSS5 g! , ( d f g= f)a
:param client: The client to use when calling PutObject
:param fileobj: The file to upload.
:param bucket: The name of the bucket to upload to
:param key: The name of the key to upload to
:param extra_args: A dictionary of any extra arguments that may be
used in the upload.
)BucketKeyBodyNr1 )r )r r r9 r r
r r s r _mainPutObjectTask._main s+ NV4 N: N WWs "
0r1 Nr, r- r. r/ rT rJ r0 r1 r r r r s *
Or r c \ rS rSrSrS rSrg)r( i z+Task to upload a part in a multipart uploadc
U nUR " SUUUUUS.UD6n SSS5 W S n
XS.n SU; a# US R 5 n SU 3n
X; a X X' U $ ! , ( d f NB= f)a[
:param client: The client to use when calling PutObject
:param fileobj: The file to upload.
:param bucket: The name of the bucket to upload to
:param key: The name of the key to upload to
:param upload_id: The id of the upload
:param part_number: The number representing the part of the multipart
upload
:param extra_args: A dictionary of any extra arguments that may be
used in the upload.
:rtype: dict
:returns: A dictionary representing a part::
{'Etag': etag_value, 'PartNumber': part_number}
This value can be appended to a list to be used to complete
the multipart upload.
)rG rH UploadId
PartNumberrI NETag)rQ rP r r r1 )r upper)r r r9 r r
r r r r responseetag
part_metadataalgorithm_namechecksum_members r rJ UploadPartTask._main" s , )) "&
H !%A
* , '(; < B B DN ( (8 9O *191J
. ! Ws A
A&r1 NrL r1 r r r( r( s
5& r r( ) r ior s3transfer.compatr r s3transfer.constantsr s3transfer.futuresr s3transfer.tasksr r
r r s3transfer.utilsr
r r r r r3 rV r r r r r r( r1 r r r_ s 0 : 3 B) )XxN xNvZL!3 ZLz. !; . bG
$6 G
TK> K\
OD
O ) T ) r