Sultan Alrashed

@srashedll

Pretraining multilingual language models

Saudi Arabia شامل ہوئے Şubat 2024

16 فالونگ18 فالوورز

Sultan Alrashed ری ٹویٹ کیا

Larry Dial@classiclarryd·1 Şub

New NanoGPT Speedrun WR at 97.8 (-1.2s) from @srashedll , with an update to the attention initialization. Motivated by mimetic initialization techniques, experiments uncovered that small random init outperformed zero init on attention out projection. github.com/KellerJordan/m…

English

8.7K

Sultan Alrashed ری ٹویٹ کیا

Tyler Chang@tylerachang·29 Eki

Very very excited that Global PIQA is out! This was an incredible effort by 300+ researchers from 65 countries. The resulting dataset is a high-quality, participatory, and culturally-specific benchmark for over 100 languages.

Multilingual Representation Workshop @ EMNLP 2025@mrl_workshop

Introducing Global PIQA, a new multilingual benchmark for 100+ languages. This benchmark is the outcome of this year’s MRL shared task, in collaboration with 300+ researchers from 65 countries. This dataset evaluates physical commonsense reasoning in culturally relevant contexts.

English

1.6K

Sultan Alrashed ری ٹویٹ کیا