Models

PeftModel is the base model class for specifying the base Transformer model and configuration to apply a PEFT method to. The base PeftModel contains methods for loading and saving models from the Hub.

PeftModel

class peft.PeftModel

< source >

( model: PreTrainedModel peft_config: PeftConfig adapter_name: str = 'default' autocast_adapter_dtype: bool = True low_cpu_mem_usage: bool = False )

Parameters

model (PreTrainedModel) — The base transformer model used for Peft.
peft_config (PeftConfig) — The configuration of the Peft model.
adapter_name (str, optional) — The name of the adapter, defaults to "default".
autocast_adapter_dtype (bool, optional) — Whether to autocast the adapter dtype. Defaults to True. Right now, this will only cast adapter weights using float16 and bfloat16 to float32, as this is typically required for stable training, and only affect select PEFT tuners.
low_cpu_mem_usage (bool, optional, defaults to False) — Create empty adapter weights on meta device. Useful to speed up the loading loading process.

Don’t use low_cpu_mem_usage=True when creating a new PEFT adapter for training.

Base model encompassing various Peft methods.

Attributes:

base_model (torch.nn.Module) — The base transformer model used for Peft.
peft_config (PeftConfig) — The configuration of the Peft model.
modules_to_save (list of str) — The list of sub-module names to save when saving the model.
prompt_encoder (PromptEncoder) — The prompt encoder used for Peft if using PromptLearningConfig.
prompt_tokens (torch.Tensor) — The virtual prompt tokens used for Peft if using PromptLearningConfig.
transformer_backbone_name (str) — The name of the transformer backbone in the base model if using