How to apply on the FacebookAI/roberta-large model?

[LonglongaaaGo]

Hi @fxmeng ,

I encounted a problem for applying on the FacebookAI/roberta-large model, using the hyper-parameters shown below:

--base_model_path
FacebookAI/roberta-large
--output_dir
pissa_out
--init_weights
pissa_niter_16
--lora_r
128
--lora_alpha
128
--lora_dropout
0
--target_modules
key
query
value

It shows:

Traceback (most recent call last):
  File "xxxx/init_pissa.py", line 58, in <module>
    peft_model.save_pretrained(script_args.output_dir)
  File "/xxxx/site-packages/transformers/modeling_utils.py", line 2830, in save_pretrained
    safe_save_file(shard, os.path.join(save_directory, shard_file), metadata={"format": "pt"})
  File "/xxxx/site-packages/safetensors/torch.py", line 286, in save_file
    serialize_file(_flatten(tensors), filename, metadata=metadata)
                   ^^^^^^^^^^^^^^^^^
  File "/xxxxx/site-packages/safetensors/torch.py", line 488, in _flatten
    raise RuntimeError(
RuntimeError: 
            Some tensors share memory, this will lead to duplicate memory on disk and potential differences when loading them again: [{'roberta.embeddings.word_embeddings.weight', 'lm_head.decoder.weight'}].
            A potential way to correctly save your model is to use `save_model`.
            More information at https://huggingface.co/docs/safetensors/torch_shared_tensors
            
ERROR conda.cli.main_run:execute(49): `conda run python /xxxx/init_pissa.py --base_model_path FacebookAI/roberta-large --output_dir pissa_out --init_weights pissa_niter_16 --lora_r 128 --lora_alpha 128 --lora_dropout 0 --target_modules key query value` failed. (See above for error)

How can I solve this problem?

Thank you so much