Author: Yang Liping
On December 7, 2019, the "DevRun Summit:Login 2020" hosted by Huawei Cloud officially landed at 751D.
This activity mainly consists of two parts: "technology sharing" and "interactive experience". Among them, technology tycoons from Huawei, bilibilibili and poor rating enterprises have joined the eight exchange and sharing meetings. Whether it's practical dry goods sharing such as "collaborative AI practice of edge cloud based on kubeedge", or "the wonderful flower of learning hegemony" debate in the north of Qing Dynasty, or profound discussion links such as "women in technology have power", we can see that the whole scene continues to be wrapped by passion, and warm applause and cheers are coming out again and again.
Below, Lei Feng network (public address: Lei Feng net) AI developers collate the main contents of the activity technology and other highlights.
Technical session I:
New computing architecture and open source practice
In the past two years, with the gradual deepening of social awareness of computing architecture in the technical field, multi architecture is also the mainstream consensus of the industry advocated by Huawei. But all we know is that Kunpeng and shengteng have been released, and their mysterious veil seems to have never been uncovered. For this reason, this activity specially set up a special field for Kunpeng processor, shengteng development tool chain and their open source practice, providing the most intuitive and comprehensive display for curious developers.
First of all, Tan Tao, an engineer of shengteng AI, brought "from neural network to hardware, the whole process application practice of shengteng development tool chain" to the theme report, explained the development tool chain of shengteng in detail, and demonstrated the new version of development tools and the molding project based on atlas products, so that participants can intuitively feel the unique charm of the special development tools of shengteng processor.
"If you want to be good at something, you must first make use of the tools," Tan Tao said, which is the core idea in the design of shengteng AI software stack. Shengteng AI software stack, a "sharp tool", has developed all kinds of full-featured tools, forming a "generalist" tool chain. No matter from the construction of neural network, to offline model generation, or related hardware implementation, each link can provide corresponding tools for assistance.
Tan Tao also pointed out that the Da Vinci architecture of shengteng AI processor adopts the customized design of computing resources in the hardware design, highly adapts in the function realization, and provides a strong hardware foundation for the improvement of convolutional neural network computing performance.
Next, teacher Gu Gang, senior engineer of Kunpeng big data, brought to you a report entitled "Building a high performance open source big data computing platform based on Kunpeng processor ". She talks about how while AI5G IoT is now bringing applications and business model innovations, the huge amount of data behind it is also hidden by the lack of computing performance, long analysis cycles, and poor resource scalability.
Kunpeng's big data infrastructure solution is tailored for this purpose. It combines Huawei cloud's rich experience in big data platforms and basic cloud services, including: open source big data platforms such as Apache, HDP / CDH, mature and complete big data components, as well as rich ecology such as acceleration features, compilers, operating systems, hardware foundation, etc. It can effectively carry out high-performance, high-reliability big data business, and help enterprises quickly realize data and intelligent transformation.
At present, the high-performance big data computing platform based on Kunpeng processor has been open-source; at the same time, more dry goods maps about big data solutions can be viewed in Kunpeng Forum on Huawei's official website.
Address of Kunpeng Forum:
If Kunpeng ecology is added, many developers may face changes in the development process and specific transplantation methods. In the special session of the activity, sun gengze, product manager of Huawei cloud ECs, launched a report with the theme of "transplantation and development practice based on Kunpeng architecture".
In the report, sun gengze compared the Kunpeng processor with the common data center processor in instruction set, architecture, technology, ecology and openness. From the perspective of products, he reinterpreted Kunpeng for participants. The data also shows Kunpeng 920's comprehensive upgrade super computing power. For example, its computing core number has increased by one time, up to 64 times; the performance of SPECT has increased by more than two times, and exceeded the original record of the industry.
For Kunpeng transplantation, sun gengze also showed the elastic cloud server and bare metal server corresponding to the Kunpeng series chips independently developed by Huawei cloud, and the details of the corresponding general computing enhanced KC1 products are shown in the figure below.
Installation and migration related content:
Technical session II:
In recent years, thanks to distributed technology, big data technology and computing power have been improved exponentially; AI technology and application have been developing vigorously, and many technology bulls have laid out AI industry.
However, in order to realize the ubiquitous vision of AI, in addition to solving the three challenges of computing power, algorithm and data, it also faces multiple challenges such as cost, complexity, scalability and data privacy. The large-scale implementation of AI applications requires the integration of full stack AI capabilities, best practices of full scenarios and the support of full stack AI development experience.
Huawei's full stack and full scene AI covers these problems in all directions, including: Ascend (Shengteng) series chips, chip operator libraries and highly automated operator development tools.
First of all, Wang Jun, a senior engineer of mindspore, introduced this AI computing framework to us in detail. The framework not only supports terminal, edge computing, cloud full scene requirements and collaborative unified training and reasoning, but also excels in use experience.
Compared with tensorflow, mindspore framework can effectively lower the threshold of AI development, make in-depth learning faster and cheaper, and provide developers with powerful full scene AI model development, model operation and model end-to-end deployment capabilities.
For the AI development work, Du Qi, senior engineer of modelarts, put forward ten major changes needed by AI, including the transformation of "a work to be completed by senior engineer" to "one-stop platform to provide basic skills", which is one of the significance of developing the AI development platform.
Therefore, modelarts is also known as a one-stop AI development platform for "zero code or low code". Duqi also said that AI application has become a new force in various industries, and the development of such a high-performance AI application as modelarts can not only greatly reduce the difficulty of AI development, but also meet the needs of the current industry in industrial practice.
After that, Wei Zhenqiang, senior solution architect at AI, brought the title < />
When the professional AI development platform is matched with the last one-stop AI application development, distribution and deployment of the end cloud collaborative management platform, more extended functions are developed, such as: mass device management, dynamic application deployment, online training, end-side model optimization and other end cloud collaborative capabilities. These functions can be well applied to the development of "end side cloud" scenarios, such as UAV, unmanned driving, face recognition gate, AR / VR, logistics parcel sorting, etc.
Technical session 3:
High efficiency Development practice of Enterprise Application
With the increasing volume of data, some new technical fields are emerging. Cloud computing, as a platform that carries most of the business, can rely on the cloud to complete a large number of data tasks, whether it is new business online or technical practice.
At this stage, most enterprises are moving closer to the cloud. The rapid iteration and agile delivery of business have become the norm. Rapid response to market demand has also become a decisive factor in enterprise competition. Therefore, Devops, micro services, service grid and other technical fields are increasingly being sought after by the market. Therefore, in this technology field, developers mainly discuss how to ensure the efficient development of applications in the current context.
The first report of the special session, "hierarchical testing and quality assurance of cloud native development", was shared by Xia dongran, an engineer of Huawei cloud devcloud. He pointed out that for innovation and entrepreneurship, new business models often need small steps and quick trial and error, while the usual waterfall development not only takes a long time, but also the products may not work normally; at this time, agile development can solve this problem well.
However, the emergence of Devops has further built a bridge between development and operation and maintenance organizations, and reconciled the contradiction between iteration and stability. At the same time, under the agile and Devops development mode, CI / CD continuous automated testing can achieve efficient test feedback and ensure the quality of product release at any time. Therefore, the report shows that Devops has become the mainstream of enterprise software research and development, and has been adopted by many enterprises.
As for the more detailed content of quality assurance, Guo dongshuo, an engineer of Huawei cloud software analysis lab, made a detailed introduction in the second report "continuous construction of trusted code inspection service". During the speech, Guo dongshuo repeatedly stressed that under the background of cloud native as the development trend of cloud computing, application testing and quality assurance become more important.
On the one hand, the code content in the era of big data is complex, so the errors are more diverse; on the other hand, after these errors are often generated, it is difficult to use "restart" and other operations to recover, or even bring incalculable losses. Therefore, in the daily development of Huawei, they adopt an intelligent code detection and repair framework, which adopts a defect detection and repair mechanism based on double insurance of deterministic rules and opportunity machine learning. It can guarantee the security and reliability of its code in any situation.
According to data statistics, more than 170 countries and regions, more than 3 million enterprise users and developers, and more than 500 million end cloud users have started to use this tool. At present, this multi language code detection framework has been partially open to the public, and can be directly used on the platform, thus improving the efficiency of developers' code writing.
In addition to common development security and quality assurance, Huawei has long had insights into the hot blockchain field in the near future. Xue Tengfei, the blockchain architect, also showed us the application practice of using technology fission to seize the opportunity to build a credible, shared and secure blockchain with high efficiency. The technologies involved include: Huawei cloud security, Huawei cloud big data analysis, automatic operation and maintenance, etc.
Technical session 4:
Cloud native accelerates AI application innovation and landing
At this stage, with the development of deep learning technology slowing down, the artificial intelligence based on big data seems to be deeply affected by it, falling into a certain bottleneck period. With the "new generation" technology represented by cloud Nativity growing up, the "cloud Nativity" movement with kubernetes as the core is expanding, and has been widely recognized as the future trend of cloud computing in the industry. In this context, how can developers better integrate cloud native and AI?
Xu Fei, senior cloud engineer of Huawei / core member of kubeedge project, gave his answer through the sharing of "collaborative AI practice of edge cloud based on kubeedge". Xu Fei said that in the cloud native domain, edge computing is a very important part, while cloud native edge computing can make the edge have the same elasticity as the cloud, so that the application can be deployed to the edge "smoothly" and keep the consistency between the application on the edge and the cloud.
Kubeedge can help engineers free themselves from the management of underlying technical facilities through better architecture and technology implementation, so as to focus on application development at a higher level of abstraction. In this way, "cloud edge end" is like a perfect whole, and end users do not need to perceive the complex distribution of edge devices.
Later, Li Mingzhe, senior engineer of Huawei cloud / Volvo init maintainer / committer, shared his theme report "Volvo: running high performance AI jobs in kubernetes". He said that with the popularization of containerization and container arrangement technology, more and more upper businesses are embracing k8s ecology. As one of the leaders in cloud native field, Huawei officially opened the cloud native task batch computing processing platform volacno for high-performance computing in June this year.
Volcano provides a set of mechanisms that kubernetes currently lacks in batch and elastic workload processing, including machine learning / deep learning, bioinformatics / genomics, and other "big data" applications. These types of applications usually run on the tensorflow, spark, pytorch, MPI and other general domain frameworks integrated by volcano.
Volcano open source address:
Interesting activities in the exhibition hall
In addition to the speech and report full of dry goods, another interesting part of the whole activity is the debris collection activity. Participants are randomly divided into different groups, and the corresponding fragments are obtained through the live experience of "technology and the future" exhibition area. Finally, the champion team will be rewarded with a mate305g mobile phone for each person.
There are many kinds of activities with different characteristics, such as DIY type practice, such as: dialogue with artificial intelligence, animal and plant identification of modelarts, invention and creation display, assembly of dodge car, etc.; knowledge harvesting type, such as: mechanical class, driving class, natural class and other characteristic experience courses; lecture type activities, such as: roadshows of developers such as individual entrepreneurs, open source community, Huawei developer sand area, etc Classroom; as well as recreational activities, such as: Unicorn grab chair, programmer's screen machine, lottery twist machine and other small games.
At the activity site, under the leadership of the group leader, all members of the team also made full efforts to rush around in various exhibition areas and lecture halls; at the same time of leisure and entertainment, they also had harmonious and natural technical exchanges with all developers. In the end, Shahe team won the championship.
In this era of explosive growth of computing power, the future computing speed is no longer the bottleneck; and the combination of high-speed 5g and cloud technology makes the massive data and information resources within reach; all these mean that the development difficulty and threshold are constantly reducing.
For developers, the ability to master algorithms will become more and more important in the future competition. In the field of high-performance computing, developers will face greater challenges. Program apes who only stick to the program and are unwilling to innovate may soon face the pressure and challenges from the new generation after 2000.
But this kind of challenge is not an opportunity for developers? Kunpeng ecology, shengteng chip, super powerful computing processor, multi architecture, cloud native, one-stop development platform modelarts, AI framework mindsport and other tools provide developers with the most cutting-edge technology, as long as you want to start, everyone can develop!
Of course, in addition to mastering the technology itself, inspiration and imagination are also essential elements for the development of science and technology. Just as Huawei cloud's "login 2020" activity itself leads us to imagine, it can seamlessly integrate science and technology with life, meet various scenarios and diversified complex needs, and constantly innovate business models, so as to promote the prosperity of life and create an intelligent world of interconnection of all things Bound.