(Experimental) Add support to NTK RoPE scaling#118

Merged

turboderp merged 6 commits intoturboderp:masterfrom Panchovix:master

Jul 1, 2023

+9-1

Commits on Jun 29, 2023

(Experimental) Add support to NTK RoPE scaling
Panchovix
authored
(Experimental) Add support to NTK RoPE scaling
Panchovix
authored

Commits on Jun 30, 2023

Revert alpha value to be set manually
Panchovix
authored
Apply correctly the value when using -a
Panchovix
authored
Add calculate rotary embedding base correctly on model_init.py
Panchovix
authored
Remove obsolete code
Panchovix
authored