I've seen some domain applications where a BERT model of 10Million parameters with knowledge about a domain (over an LLM with 1 billion parameters) is good enough. This is a BERT model already trained on Material Science: https://huggingface.co/m3rg-iitd/matscibert

If the researcher is set on utilizing an LLM, it might be easier to simply fine-tune a pre-trained model on the MatScie dataset
https://github.com/M3RG-IITD/MatSciBERT

LoRA Fine tuning : https://www.databricks.com/blog/efficient-fine-tuning-lora-guide-llms