The language is a transformer-based large language model that incorporates many cutting-edge features, including ALiBi position embeddings