Microsoft unveiled a new deep learning acceleration platform designed for real-time artificial intelligence, codenamed “Project Brainwave,” at Hot Chips 2017.
The platform has three main layers:
- a high-performance, distributed system architecture;
- a hardware DNN (deep neural network) engine synthesized onto FPGAs (field programmable gate arrays); and
- a compiler and runtime for low-friction deployment of trained models.
Project Brainwave leverages the massive FPGA infrastructure from Project Catapult that Microsoft has been deploying in Azure and Bing over the past few years.
AI in Real Time
“FPGA is a way to add and apply dedicated task-specific computing power geared to deep neural nets to conventional cloud infrastructure,” said Doug Henschen, principal analyst at Constellation Research.
“This makes it easier to develop for conventional server capacity and let the FPGAs provide the computing power necessary for AI workloads,” he told TechNewsWorld.
Attaching high-performance FPGAs directly to Microsoft’s data center network lets DNNs be served as hardware microservices, calling them by a server with no software in the loop. This reduces latency and allows very high throughput.
“Real-time AI is the eventual goal for the vast majority of projects,” said Rob Enderle, principal analyst at the Enderle Group.
“AI should be able to move at the speed of thought, or it’ll just be an advanced script,” he told TechNewsWorld.
Project Brainwave’s Guts
Project Brainwave uses a soft DNN processing unit, or DPU, synthesized onto commercially available programmable gate arrays. This lets it scale across a range of data types, with the desired data type being a synthesis-time decision.
Microsoft’s soft DPUs combine the ASIC digital signal processing blocks on the FPGAs with the synthesizable logic to provide a greater and more optimized number of functional units.
The DPUs use highly customized, narrow-precision data types defined by Microsoft, which increase performance without real losses in model accuracy. Research innovations can be incorporated into the hardware platform rapidly, typically in weeks.
Project Brainwave incorporates a software stack supporting the Microsoft Cognitive Toolkit (MCTK) and Google’s Tensorflow. Support for other frameworks will be added later.
Tensorflow is “the currently dominating machine learning technique,” said Holger Mueller, principal analyst at Constellation Research.
“That buys Microsoft time to strengthen MCTK,” he told TechNewsWorld.
Read more: http://www.technewsworld.com/story/Microsoft-Unveils-Real-Time-AI-for-Azure-84762.html