Skip to content

Commit

Permalink
Update torchtune/models/phi4/_model_builders.py
Browse files Browse the repository at this point in the history
Co-authored-by: Felipe Mello <[email protected]>
  • Loading branch information
krammnic and felipemello1 authored Jan 14, 2025
1 parent 1d03294 commit bdf478f
Showing 1 changed file with 1 addition and 3 deletions.
4 changes: 1 addition & 3 deletions torchtune/models/phi4/_model_builders.py
Original file line number Diff line number Diff line change
Expand Up @@ -75,9 +75,7 @@ def lora_phi4_mini(
"""
Builder for creating a Phi4 (14b) model with LoRA enabled.
The Phi4 defaults are the same as in :func:`~torchtune.models.phi4.phi4_mini`,
while LoRA default params are based on
https://github.com/tloen/alpaca-lora/blob/8bb8579e403dc78e37fe81ffbb253c413007323f/finetune.py#L41-L43.
The Phi4 defaults are the same as in :func:`~torchtune.models.phi4.phi4_mini`.
Args:
lora_attn_modules (List[LORA_ATTN_MODULES]): list of which linear layers
Expand Down

0 comments on commit bdf478f

Please sign in to comment.