54 commits

Author	SHA1	Message	Date
oobabooga	d8fb506aff	Add RoPE scaling support for transformers (including dynamic NTK) ... https://github.com/huggingface/transformers/pull/24653	2023-08-08 21:25:48 -07:00
Gennadij	0e78f3b4d4	Fixed a typo in "rms_norm_eps", incorrectly set as n_gqa (#3494)	2023-08-08 00:31:11 -03:00
oobabooga	412f6ff9d3	Change alpha_value maximum and step	2023-08-07 06:08:51 -07:00
oobabooga	65aa11890f	Refactor everything (#3481)	2023-08-06 21:49:27 -03:00