TypNLP

Excited for my first presentation at @sig_typ @eaclmeeting on 6th May! My PhD project "Gradual Language Model Adaptation Using Fine-Grained Typology" may increase model adaptation efficiency for resource-poor languages. See you there! #eacl2023 #sigtyp

This has been one of the most eventful weeks I've had at @CompSciAAU @aautech @dkw_aau! A huge thanks to the tutors who got our SW BSc students off to a flying start, and to @Carlsbergfondet for getting @TypNLP going with a dream-team working on #SDGs with #NLProc @johannesbjerva

Thus the goals of @heather_nlp's work will be to 1) engage with communities to work toward needed NLP technologies, 2) leveraging Creoles as a lens to research & improve methods in transfer learning. #SDGs #creoles 3/3 @johannesbjerva

At the same time, the linguistic makeup of #creoles, featuring multilingual lexicon and syntax, provide an ideal (yet under studied!) environment to investigate the strengths 🦾 and limitations 🛑 of transfer learning, & to develop and test new methods. 2/3 @johannesbjerva

Finally, @heather_nlp brings a whole new set of languages to the table! #creoles are spoken across the globe, but lack of resources & pervasive stigma hinder the development of technologies for these languages. #NLProc 1/3 @johannesbjerva

While MWE's have achieved state-of-the-art perf. on cross-lingual transfer for low-resource lang's, there is much potential left unused in linguistic resources for various linguistic features. #LingTyp @YiyiChen will investigate how to best integrate these! 2/2 @johannesbjerva

It was a busy and exciting project start yesterday! The project's final PhD student, @YiyiChen, will work on a framework to combine a resource-efficient fine-tuning strategy and linguistic features for better transferability of Multilingual Word Embeddings. 1/2 @johannesbjerva

This has been shown to be facilitated by typological (structural) similarity between lang's. Can cross-lingual transfer be enhanced by model adaptation in a step-by-step fashion, maximizing typological similarity between the lang's in each step? 🤔 #NLProc 2/2 @johannesbjerva

.@v4rmer will look at multilingual language models which can generalize linguistic info between different languages. Via cross-lingual transfer, these models can leverage knowledge from resource-rich languages to the benefit of resource-poor ones #NLProc ✨ 1/2 @johannesbjerva

However, there is explicit typological information available (e.g. word ordering, morphological features) from #LingTyp! How can we best leverage these explicit typological features for improving, e.g., machine translation to and from low-resource languages? 2/2 @johannesbjerva

.@EstherPloeger notes that Large Language Models have improved perf for #NLProc in English, but also enlarged the perf gap between high-resource and low-resource languages: for the latter, large quantities of raw text (required by LLMs) are often unavailable. 1/2 @johannesbjerva