While Large Language Models (LLMs) have shown remarkable performance in various Natural Language Processing (NLP) tasks, their effectiveness seems to be heavily biased toward high-resource languages.This proposal aims to address this gap by developing efficient training strategies for low-resource languages.We propose various techniques for efficient learning in simulated low-resource settings for English.We then plan to adapt these methods for lowresource languages.We plan to experiment with both natural language generation and understanding models.We evaluate the models on similar benchmarks as the BabyLM challenge for English.For other languages, we plan to use treebanks and translation techniques to create our own silver test set to evaluate the low-resource LMs.