CFM: Language-aligned Concept Foundation Model for Vision

Mon, 01 Jun 2026 00:00:00 +0100

Multilingual Language Model Fine-Tuning

Fri, 01 Mar 2024 00:00:00 +0100

We analyse how different fine-tuning methods affect language models on under-represented languages such as Quechua (quy_Latn). The methods we compare are full fine-tuning, BitFit, LoRA and IA3, applied to XGLM-564M and GPT-2 on multilingual corpora from NLLB.

Beyond the usual performance numbers, we look at what fine-tuning does to the models internally. Token and sentence representations are projected with PCA and t-SNE to see how the multilingual space is reorganised, which we then weigh against the performance trade-offs of each parameter-efficient method.

Code

Kai Wittenmayer

CFM: Language-aligned Concept Foundation Model for Vision

Multilingual Language Model Fine-Tuning