Aixin Tongyuan NPU completes the adaptation of Llama 3 and Phi-3 large models, promoting the popularization of AI large model technology
Time:2024-05-17
Views:141
Source: Aixin Yuanzhi Author: Aixin Yuanzhi
China, April 28, 2024 - AI chip research and development and basic computing platform company Aixinyuan Zhi announced that Meta and Microsoft have recently released the landmark Llama 3 series and Phi-3 series models. In order to further provide developers with more experiences, the NPU toolchain team of Aixin has responded quickly and has completed the adaptation of Llama 38B and Phi-3-mini models based on the AX650N platform.
Llama 3
On Friday, Meta released the Meta Llama 3 series of language models (LLM), including an 8B model and a 70B model. In the test benchmarks, the Llama 3 model performed remarkably well, and in utility and safety assessments, it was comparable to the popular closed-source models.
The official website: https://llama.meta.com/llama3
At the architectural level, Llama 3 opted for the standard decoder-only Transformer architecture, using a word classifier with a 128K token vocabulary. Llama 3 uses more than 15 terabytes of publicly available data for training, 5% of which is non-English, covering more than 30 languages and seven times more training data than the previous generation Llama 2.
According to Meta‘s test results, the Llama 3 8B model outperformed the Gemma 7B and Mistral 7B Instruct on a number of performance benchmarks including MMLU, GPQA, HumanEval, etc. The 70B model outperforms Sonnet, an intermediate version of the well-known closed-source Claude 3 model, and is two wins out of three compared to Google‘s Gemini Pro 1.5.
Actual on-board results: At present, AX650N has completed the adaptation of Llama 38B Int8 version, and if Int4 quantization is adopted, the number of tokens per second can be doubled again, which can meet the normal man-machine communication.
Phi-3
Not long after the launch of Llama 3, competitors arrived, and they were small models that could run on mobile phones.
On Tuesday, Microsoft unveiled its own miniature model, the Phi-3. Although the Phi-3-mini is optimized to be deployed on mobile phones, its performance is comparable to models such as the Mixtral 8x7B and GPT-3.5. According to Microsoft, this innovation is mainly due to the use of better quality training data sets.
Online DEMO:https://ai.azure.com/explore/models/Phi-3-mini-4k-instruct/version/2/registry/azureml
Actual board results: At present, AX650N has completed the adaptation of Phi-3-mini Int8 version, which can meet the normal man-machine communication.
Other results
AX650N can complete the adaptation of Llama 3 and Phi-3 in the first time, because relevant teams have quietly completed the support and tuning of the existing NPU tool chain for large language models since this year. At present, in addition to Llama 3, it has completed the adaptation of Llama 2, TinyLlama, Phi-2, Qwen1.5, ChatGLM3 and other mainstream open source large language models at home and abroad.
The results have been released to the developer community (https://github.com/AXERA-TECH/ax-llm).
Follow-up plan
This year is the first year of AIPC, AIxin Yuanzhi will provide more solutions for common applications of AIPC. Give full play to the technical advantages of Aixin Tongyuan NPU‘s high energy efficiency ratio, let a variety of interesting large models achieve high price localization deployment, so that "everyone can afford the large model", and deeply practice "inclusive AI, create a better life".
About Aixin Yuanzhi:
Founded in May 2019, Aixin Semiconductor Co., Ltd. is committed to creating the world‘s leading artificial intelligence sensing and edge computing chips to serve the huge edge and end-side device markets such as smart city, intelligent driving, robotics, and AR/VR. The company has developed two core technologies - AIxin Zhimou AI-ISP and Aixin Tongyuan Mixed precision NPU. In the past four years, it has completed the research and development and mass production of more than four generations of smart chip products. Adhering to the mission of "Puhui AI, creating a better life" and the vision of "building a world-class perception and computing platform", Aixin is committed to becoming the digital entrance to the physical world, providing the basic computing power platform of artificial intelligence for different industries, so that intelligence is truly within reach.
|
Disclaimer: This article is transferred from other platforms and does not represent the views and positions of this site. If there is any infringement or objection, please contact us to delete it. thank you! |
Business consulting
13823761625
Mail me