+
$Lj8 d ^ RI t ^ RIHt ^ RIHt ^ RIt^ RIt^ RIH t ^RI
H t H t H
t
HtHtHtHtHtHtHtHtHtHtHtHtHtHtH t H t H t H t ]! RR4 ' d ^ RI H!t! M^ RI"H!t! ] ! 4 ' d
^ RI#H$u H%t& ^R I'H(t( ^R
I)H*t* ](! ]+4 t,RR R llt-RR
lt.RR R llt/RR R llt0R# ) N)Path)Optional)
load_model)
MODEL_NAMEOPTIMIZER_NAMERNG_STATE_NAMESAFE_MODEL_NAMESAFE_WEIGHTS_NAME SAMPLER_NAME SCALER_NAMESCHEDULER_NAME WEIGHTS_NAMEget_pretty_nameis_cuda_availableis_hpu_availableis_mlu_availableis_musa_availableis_neuron_availableis_sdaa_availableis_torch_versionis_torch_xla_availableis_xpu_availableloadsavez>=z2.4.0)
GradScaler)
get_logger) PartialStatec V ^8 d QhR\ R\ \ , R\ R\ R\ R\ R\ R\ \
, R \ R
\ /
# )
output_dir model_states
optimizers
schedulers dataloaders
process_indexstepscalersave_on_each_nodesafe_serialization)strlistdictintr r bool)formats "D/app/.local/lib/python3.14/site-packages/accelerate/checkpointing.py __annotate__r1 ? s u uut*u u u
u
u u
Z u u u c
\ V 4 p \ V4 Fl w rV ' g \ M\ p V
^ 8 d V P RRV
R24 p V P V 4 p
\
WWR7 \ P RV
24 Kn \ V4 Fi w rVP 4 p V
^ 8X d
\ R2M \ RV
R2pV P V4 p\
V VVRR7 \ P RV 24 Kk \ V4 Fj w p
pVP 4 p V
^ 8X d
\ R2M \ RV
R2pV P V4 p\
V VVRR7 \ P RV 24 Kl \ V4 F w p
pV
^ 8X d
\ R2M \ RV
R2pV P V4 p^R I
HpHp \! VP" V4 ' d2 VP% 4 p\! VV4 ' d \
VVVRR7 \' VR
R4 ' dH V
^ 8X d R MR V
R2pV P V4 pVP 4 p \( P ! V V4 \ P R
V
RV 24 EK VeU VP 4 p V P \* 4 p \( P ! V V 4 \ P RV 24 / p \, RV R2p VV R& \. P0 ! 4 V R& \2 P. P5 4 V R& \( P6 ! 4 V R& \9 4 ' d" \( P: P= 4 V R& \? 4 ' d# \( P@ P= 4 V R& Mc\C 4 ' d# \( PD P= 4 V R& M1\G 4 ' d" \( PH P= 4 V R& \K 4 ' d" \( PL P= 4 V R& \O 4 ' d" \( PP P= 4 V R& \S 4 ' d" \( PT P= 4 V R& \W 4 ' d \X P6 ! 4 V R & V P V 4 p \( P ! V V 4 \ P R V 24 V # ) a
Saves the current states of the models, optimizers, scaler, and RNG generators to a given directory.
If `safe_serialization` is `True`, models will be saved with `safetensors` while the rest are saved using native
`pickle`.
Args:
output_dir (`str` or `os.PathLike`):
The name of the folder to save all relevant weights and states.
model_states (`List[torch.nn.Module]`):
A list of model states
optimizers (`List[torch.optim.Optimizer]`):
A list of optimizer instances
schedulers (`List[torch.optim.lr_scheduler._LRScheduler]`):
A list of learning rate schedulers
dataloaders (`List[torch.utils.data.DataLoader]`):
A list of dataloader instances to save their sampler states
process_index (`int`):
The current process index in the Accelerator state
step (`int`):
The current step in the internal step tracker
scaler (`torch.amp.GradScaler`, *optional*):
An optional gradient scaler instance to save;
save_on_each_node (`bool`, *optional*):
Whether to save on every node, or only the main node.
safe_serialization (`bool`, *optional*, defaults to `True`):
Whether to save the model using `safetensors` or the traditional PyTorch way (that uses `pickle`).
._)r( r) zModel weights saved in .binFzOptimizer state saved in zScheduler state saved in IterableDatasetShardSeedableRandomSampleruse_stateful_dataloaderdl_state_dict.bindl_state_dict_z Sampler state for dataloader z
saved in z Gradient scaler state saved in .pklr& random_statenumpy_random_seedtorch_manual_seedtorch_xpu_manual_seedtorch_mlu_manual_seedtorch_sdaa_manual_seedtorch_musa_manual_seedtorch_hpu_manual_seedtorch_neuron_manual_seedtorch_cuda_manual_seedxm_seedzRandom states saved in )-r enumerater r
replacejoinpathr loggerinfo
state_dictr r
r data_loaderr8 r9
isinstancedataset get_samplergetattrtorchr r randomgetstatenp get_state
get_rng_stater xpuget_rng_state_allr mlur sdaar musar hpur neuronr cudar xm)!r r! r" r# r$ r% r& r' r( r) istate weights_nameoutput_model_fileoptoptimizer_nameoutput_optimizer_file schedulerscheduler_nameoutput_scheduler_file
dataloader sampler_nameoutput_sampler_filer8 r9 samplerdataloader_state_dict_name!output_dataloader_state_dict_filerN output_scaler_filestates states_nameoutput_states_files! &&&&&&&&&& r0 save_accelerator_staterw ? sh X j!J l++=|CT q5'//q1X>L & / / = U9Jr -.?-@AB
, J' 45FN+40>BRRSTUSVVZ@[ * 3 3N C U)=Nchi /0E/FGH ( "*- 9$$&45FN+40>BRRSTUSVVZ@[ * 3 3N C U)=Nchi /0E/FGH . #;/
:01Q, t,|nAaSPT ? ? ,,.G'#899W 1EVkpq : 8% @ @@AQ)"$))"5"5"7F "'"5"5"7F */))*E*E*G&'*/))*E*E*G&' +0::+G+G+I'( +0::+G+G+I'(*/))*E*E*G&' -2\\-K-K-M)*+0::+G+G+I'( , , .y#,,[9 JJv)*
KK)*<)=>? r2 c \ 4 p
VR$9 d \ R4 hVf RpMVR8X d \ 4 P pVf / p\ V 4 p \ V4 F w rV ^ 8 d RV 2MRp
V P
\ V
R24 pVP 4 ' d \ W3R\ V4 /V B K\ V P
\ V
R24 p\ WR 7 pV P ! V3/ V B K \ P R
4 \ V4 FX w p pV ^ 8X d
\ R2M \ RV R2pV P
V4 p\ V3R V/VB pW+, P V4 KZ \ P R 4 \ V4 FP w p pV ^ 8X d
\" R2M \" RV R2pV P
V4 p\ V3/ VB pVP V4 KR \ P R
4 \ V4 F w p pV ^ 8X d
\$ R2M \$ RV R2pV P
V4 p^RIHpHp \- VP. V4 ' d= VP1 4 p \- V V 4 ' d VP3 \ V4 4 p \5 VRR4 ' g K V ^ 8X d RMRV R2p V P
V 4 p V P 4 ' g K \ V 3/ VB pVP V4 EK \ P R4 VeG V P
\6 4 p \ V 4 p!VP V!4 \ P R4 \ V P
\8 RV R24 4 p"RV"9 d
V"R, V
R& \: P< ! V"R, 4 \> P: PA V"R, 4 \B PD ! V"R, 4 \G 4 ' d' \B PH PK V"R, 4 \M 4 ' d) \B PN PK V"R, 4 EM\Q 4 ' d( \B PR PK V"R , 4 M\U 4 ' d( \B PV PK V"R , 4 M\Y 4 ' d( \B PZ PK V"R , 4 M]\] 4 ' d( \B P^ PK V"R , 4 M&\B P` PK V"R , 4 \c 4 ' d \d PD ! V"R!, 4 \ P R"4 V
# \f d \ P R#4 T
# i ; i)%a
Loads states of the models, optimizers, scaler, and RNG generators from a given directory.
Args:
input_dir (`str` or `os.PathLike`):
The name of the folder to load all relevant weights and states.
models (`List[torch.nn.Module]`):
A list of model instances
optimizers (`List[torch.optim.Optimizer]`):
A list of optimizer instances
schedulers (`List[torch.optim.lr_scheduler._LRScheduler]`):
A list of learning rate schedulers
dataloaders (`List[torch.utils.data.DataLoader]`):
A list of dataloader instances used in your program
process_index (`int`):
The current process index in the Accelerator state
scaler (`torch.amp.GradScaler`, *optional*):
An optional *GradScaler* instance to load
map_location (`str`, *optional*):
What device to load the optimizer state onto. Should be one of either "cpu" or "on_device".
load_kwargs (`dict`, *optional*):
Additional arguments that can be passed to the `load` function.
load_model_func_kwargs (`dict`, *optional*):
Additional arguments that can be passed to the model's `load_state_dict` method.
Returns:
`dict`: Contains the `Accelerator` attributes to override while loading the state.
cpu on_devicezaUnsupported optimizer map location passed, please choose one of `None`, `'cpu'`, or `'on_device'`r5 z .safetensorsdevicer6 ) map_locationz%All model weights loaded successfullyr} z(All optimizer states loaded successfullyz(All scheduler states loaded successfullyr7 r: Fr; r< z1All dataloader sampler states loaded successfullyz$GradScaler state loaded successfullyr= r& r> r? r@ rA rB rC rD rE rF rG rH z%All random states loaded successfullyz Could not load random states)Nry rz )4r, TypeErrorr r| r rI rK r existsr r* r r load_state_dictrL rM r r
r rO r8 r9 rP rQ rR set_samplerrS r r rU setstaterW set_staterT
set_rng_stater rZ set_rng_state_allr r\ r r] r r^ r r_ r r` ra r rb Exception)# input_dirmodelsr" r# r$ r% r' r} load_kwargsload_model_func_kwargsoverride_attributesrc modelendinginput_model_filerN rg rh input_optimizer_fileoptimizer_staterj rk input_scheduler_filescheduler_staterm rn input_sampler_filer8 r9 rp rq input_dataloader_state_dict_fileinput_scaler_file scaler_statert s# &&&&&&&&&, r0 load_accelerator_stater s R &55 o
$#~,, YI f% E1QCr$-- 0A& .VW " " $ $ u cs<7H cLb c ) 1 1ZL2M N .JJ ! !* G0F G & KK78 J'345FN+40>BRRSTUSVVZ@[ ( 1 1. A 3^,^R]^
%%o6 (
KK:; "*- 945FN+40>BRRSTUSVVZ@[ ( 1 1. A 3C{C!!/2 .
KK:;";/
:01Q, t,|nAaSPT ? ? ,,.G'#899$006H1IJ : 8% @ @@AQ)r s ( 2 D'""$)))
H upI XOU Ur2