Stability AI, the generative AI company behind Stable Diffusion, has launched its first Japanese Language Model (LM) called Japanese StableLM Alpha. This model is considered to be the most proficient publicly available model for Japanese speakers, as confirmed by a benchmark evaluation against four other Japanese LMs. With an architecture of 7 billion parameters, Japanese StableLM Alpha is a versatile and high-performing tool for various linguistic tasks, positioning itself as an industry leader.
The commercial iteration of the model, Japanese StableLM Base Alpha 7B, will be released under the Apache License 2.0. This specialized model has been trained on a massive dataset of 750 billion tokens from both Japanese and English text, sourced from online repositories. Stability AI collaborated with the EleutherAI Polyglot project’s Japanese team and the Japanese community to create these datasets. The development process also involves the use of EleutherAI’s GPT-NeoX software.
In addition to the Japanese StableLM Alpha, Stability AI has also introduced the Japanese StableLM Instruct Alpha 7B for research purposes. This model is designed to adhere to user instructions using a methodical approach known as Supervised Fine-tuning (SFT) with multiple open datasets.
Both models underwent rigorous evaluations using EleutherAI’s Language Model Evaluation Harness, and they outperformed their contemporaries in various domains such as sentence classification, sentence pair classification, question answering, and sentence summarization.
The launch of Stability AI’s Japanese LM is notable considering SoftBank’s recent announcement about its venture into homegrown Large Language Models (LLM) for the Japanese market. The competition between these models will determine the supremacy in the field of generative AI.
Overall, Stability AI’s Japanese Language Model represents a significant step forward in enhancing the Japanese generative AI landscape.