Bilibili Open-Sources the Lightweight Index-1.9B Series Models, Leading a New Era of AI Conversation

Introduction

In the field of artificial intelligence, conversational systems have always been one of the hot topics of research. Recently, Bilibili has open-sourced its lightweight Index-1.9B series models, an action that undoubtedly injects new vitality into the development of AI conversation technology. This article will delve into the characteristics, application prospects, and future impact on humanity of this model.

Overview of Bilibili's Index-1.9B Series Models

The Index-1.9B series models open-sourced by Bilibili include multiple versions, each with its own features:

• Index-1.9B base: As the base model, it has 190 million non-subword embedding parameters, trained on 2.8T of Chinese and English corpora, mainly, and shows leading performance on multiple evaluation benchmarks.
• Index-1.9B pure: A control model with the same parameters and training strategy as the base version, but all instruction-related data is filtered to verify the impact of instructions on benchmarks.
• Index-1.9B chat: A conversational model, based on the base version, aligned through SFT (Supervised Fine-Tuning) and DPO (Dialogue Policy Optimization), incorporating internet community corpora to enhance the fun of chatting.
• Index-1.9B character: A role-playing model, introducing RAG (Retrieval-Augmented Generation) on the basis of SFT and DPO to achieve fewshots customization for role-playing.

Model Features and Advantages

1. Large-scale data training: The model used 2.8T of data during the pre-training phase, with a Chinese to English ratio of 4:5 and 6% code, ensuring the model's generalization ability and depth of language understanding.
2. Diverse application scenarios: From basic conversation to role-playing, the Index-1.9B series models can adapt to different conversational needs, providing users with a richer and more personalized interactive experience.
3. Entertainment and interactivity: Especially in the Index-1.9B chat and Index-1.9B character versions, the model's entertainment and interactivity have been significantly improved, making the conversation more vivid and attractive.

Project Address and Community Contribution

The project address for Bilibili's Index-1.9B series models is: https://github.com/bilibili/Index-1.9B/blob/main/README.md. The open-sourced model code provides the community with opportunities for learning and contribution, promoting the sharing and advancement of technology.

Impact on the Future of Humanity

With the continuous advancement of AI technology, the open-sourcing of the Index-1.9B series models will have a profound impact on human society:

• Enhancing human-computer interaction experience: Smarter and more interesting conversational systems will greatly enhance the experience of human-computer interaction, making machines more closely meet human needs.
• Promoting the development of AI technology: The open-sourced model provides valuable resources for researchers and developers, helping to promote the development and innovation of AI technology.
• Facilitating cross-cultural exchange: The Chinese and English-based corpus training makes the model better serve users from different language backgrounds, promoting cross-cultural exchange and understanding.

Conclusion

The open-sourcing of Bilibili's Index-1.9B series models is not only a leap in technology but also a strong push for the future development of AI conversation systems. We look forward to the application of this model in a wider range of fields, bringing more convenience and fun to human society.