Frost & Sullivan releases the '2023 China AI Large Model Industry White Paper'

AI large models are short for pre-trained large language models, which encompass both 'pre-training' and 'large model'.The combination of the two has produced a new artificial intelligence model. That is, after the model is pre-trained on large-scale datasets, it can directly support various applications with only a small amount of data or even no fine-tuning required. These models typically have multi-layer neural network structures and are trained using advanced optimization algorithms and computing resources. They possess strong generalization, versatility, and practicality, and can achieve breakthrough performance improvements in multiple fields such as natural language processing, computer vision, and intelligent voice.

AI large models are milestone technologies that lead artificial intelligence towards general artificial intelligence.With the current popularChatGPTAs an example, ChatGPT's greatest contribution lies in its basic realization of the idealLLMThe interface layer enables the LLM to adapt autonomously to human habituated command expressions, thereby enhancing the usability of the LLM and improving user experience. InstructGPT/ChatGPT first recognized this issue and provided corresponding solutions, which are more in line with human expression habits than the previous few-shot prompting approach.

There are two major milestones in the development of artificial intelligence: First, in 2012, CNN won first place on ImageNet.It marks the beginning of machine vision recognition capabilities gradually surpassing human eye recognition accuracy, ushering in an artificial intelligence revolution;Secondly, the emergence of ChatGPT in 2022 has sparked another wave of artificial intelligence development, with large models + RLHFThe core technology implementation signifies the beginning of a new AI paradigm.Artificial intelligence-related industries have begun to develop based on powerful foundational models, continuously unlocking the capabilities of these models through human feedback and reinforcement learning to solve massive open-ended tasks, bringing about a new research paradigm.

Large models are the best technical path towards the AGI era and are beginning to be demonstrated in scenarios represented by autonomous driving.Taking autonomous driving scenarios as an example, at the input layer, large models can cover the perception environment across the entire chain and generate a large number of real-world images. At the output layer, decoders are responsible for reconstructing the 3D environment, predicting path planning, interpreting the motivations behind autonomous driving, etc. Large models can achieve integrated perception and decision-making in autonomous driving, closer to human driving behavior prediction, which helps improve the safety, reliability, and explainability of autonomous driving.

The scaling laws of large models are closely related to the emergence of AGI.The scaling law refers to the phenomenon where task performance improves as the model size gradually increases; emergent capabilities mean that as the model grows beyond a certain threshold, there is a sudden improvement in performance for certain types of tasks, emerging new abilities. When all human knowledge is stored in large models and these knowledge are dynamically connected, their intelligence far exceeds people's expectations.

AGI will evolve from the 'data wheel' to the 'intelligent wheel', ultimately moving towards human-machine co-intelligence.The existing AI system is mainly based on the data wheel, and AGI has given rise to a new research paradigm - the smart wheel. By continuously unlocking new capabilities of the base model through reinforcement learning and human feedback, it can solve massive open-ended tasks more efficiently.

The emergence of large models will reconstruct the production paradigm of artificial intelligence.The traditional software development model involves forming dedicated models through task/business datasets, with small models continuously iterating. Developers use clear code to express the logic of program execution. However, as business scenarios evolve from general to long-tail and fragmented scenarios, this model gradually faces challenges such as high development costs and poor accuracy.

With the support of large models, a new software development paradigm centered on large models combined with human feedback reinforcement learning is gradually taking shape.Through model fine-tuning, domain-specific or industry-specific large models can be created based on ultra-large-scale pretrained models, thereby covering more industry-specific scenarios. At the same time, through prompt engineering, one only needs to express the expected goal to the computer with examples, and the computer will automatically find methods to achieve the goal through neural networks.

During the traditional software development era, deep learning methods for solving single problems and industrialized small model production tools gradually matured. At present, they are still applied in some vertical fields such as medical imaging and industrial inspection.The new paradigm for future software development will be the foundation of business models and product designs driven by large AI models.

In the era of small models for artificial intelligence, deep learning methods for solving single problems and industrialized small model production tools have gradually matured.In the era of large models, on the foundation of AI-native infrastructure, Model as a Service combined with data feedback loops is the basis for future business models and product designs driven by large AI models. In the current scenario, new paradigms will pay more attention to infrastructure costs, computing power and data scale, as well as real-time user big data feedback and iteration.

The new '80/20 Rule' is taking shape, and AI large models will unleash developers' productivity.Entering the era of large models, in the future, 80% of software value will be provided by AI large models, with the remaining 20% consisting of prompt engineering and traditional business development. This has led to the formation of a new '80/20 rule.' Large models train code through machine learning and directly generate program code that meets requirements. Not only can large models generate code and complete necessary code blocks, but they can also ensure a certain level of accuracy. High-precision code generation based on large models can improve software development efficiency and mark a further advancement of artificial intelligence towards AGI.

With the upgrade of AI technology and the maturity of large models, the successful cross-border integration of AI painting and ChatGPT has brought about a development inflection point for generative AI technology, significantly increasing industry attention.Generative AI applications represented by ChatGPT, Midjourney, Wenxin Yige, SenseTime, and Codex possess text language understanding capabilities, emergent abilities, and reasoning capabilities based on thought chains. Currently, industries in China such as e-commerce, gaming, culture and entertainment, design, etc., are actively using related generative AI applications to improve their work efficiency, especially text-to-image applications.

Generative AI can not only enhance and accelerate design in downstream domains but also has the potential to 'invent' new designs and objects that humans may miss.Generative AI has the advantage of generating large-scale, high-quality, and low-cost content. With the support of computing power and algorithms, it can generate massive amounts of content, and the quality of generated content will continue to surpass that of UGC (User Generated Content) and PGC (Professional Generated Content). In the future, it is expected to provide content support for various industries and promote their content prosperity, maximizing the release of content productivity.

Text generation is a mature field that is easy to cross-border transform into other formats, with the multimodal generation track showing the highest development potential.In generative AI applications, technologies such as speech synthesis, text generation, and image attribute editing are currently quite mature. Cross-modal generation and policy generation are application scenarios with high growth potential and have extremely high application value in fields such as autonomous driving and robot control. With the continuous development and maturation of future technologies, it is expected that stable implementation can be achieved in 3-5 years.

From the 12th Five-Year Plan to the 14th Five-Year Plan, the state has provided significant support for new artificial intelligence technologies and industries at the macro level. At the same time, the state attaches importance to the security, credibility, and ethical order of the artificial intelligence industry and has begun to introduce corresponding regulatory recommendations for generative AI, such as the 'Administrative Measures for Generative Artificial Intelligence Services (Draft for Soliciting Opinions)', to further support the orderly development of the large model ecosystem.

The neural network architecture of large models and the AI infrastructure for training large models,They are gradually developing and maturing, driving the production of large models towards greater systematization and engineering.The AI deployment needs of downstream enterprise users are further developing on a larger scale.There is an urgent need to obtain basic value such as reduced development thresholds for AI applications and improved deployment accuracy, supported by large upstream models, so as to reduce the cost of AI large-scale deployment.

Looking ahead,The development of large models is advancing towards both generalization and specialization, as well as platformization and simplification. The development roadmap for large models—'training infrastructure—basic technology—basic applications—vertical applications' is gradually becoming clear. Relying on Model as a Service, large models have established differentiated business models for government, enterprises, consumers, and other groups, and are gradually forming a commercial architecture that integrates basic models, domain-specific models, and industry-specific models.Large models will accelerate the empowerment of various industries and fields such as transportation, healthcare, and finance, triggering a new wave of intelligent revolution represented by strong AI and general AI. This will significantly improve production and living efficiency, bringing about profound economic, social, and industrial changes.

Multiple challenges at the levels of technology, security ethics, and other aspects have become obstacles on the development and application path of large models, testing the technical and AI governance capabilities of large model manufacturers.

Large model vendors are inAccumulation of capabilities for full-stack large model training and research and development, including data management, AI infrastructure construction and operation, model system and algorithm designIndispensable for the development and implementation of large models. Based onProsperous open-source ecosystem, manufacturers in recent yearsAccumulation of business scenario implementation experience, capable of incubating iterative superior technology products. In advanced and continuous AI security governance initiativesWith the support of this, AI large model vendors can avoid the disruption of AI technology to ethical order and promote the commercialization and implementation of large models. By mastering key success factors, large model vendors will build competitive advantages and compete in the market.