Small language model
Small language model are artificial intelligence language model that designed for human natural language processing including language and text generation. Unlike large language model, small language model are much more smaller in scale and scope.
Normally, a large language model training parameter has more than hundreds of billions, some are even more than trillion parameter. The scale large language model is so large make the model has huge amount of information and able to generate better content, however, this requires huge amount of computation power, make individual impossible to train a large language model using a single or few computer and GPU.
Small language model, however, just use much fewer parameters, normally from a few million to a few billion, make it more feasible to train and host in a resource constrained environments such as single computer or even a mobile device. [1][2][3][4]
References
- ^ Rina Diane Caballar. "What are small language models?". IBM.
- ^ John JOhnson. "Small Language Models (SLM): A Comprehensive Overview". Huggingface.
- ^ Kate Whiting. "What is a small language model and how can businesses leverage this AI tool?". The World Economic Forum.
- ^ "SLM (Small Language Model) with your Data". Microsoft.