Chinese Journal of Computers   Full Text
Title	Design of Barrier Network of Dawning 5000 High Performance Computer
Authors	CAO Zheng1),2),3) WANG Da-Wei1),2),3) LIU Xin-Chun1),2) SUN Ning-Hui1),2)
Address	1)(Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190) 2)(Key Laboratory of Computer System and Architecture， Chinese Academy of Sciences, Beijing 100190) 3)(Graduate University of Chinese Academy of Sciences, Beijing 100039)
Year	2008
Issue	No.10(1727—1736)
Abstract &   Background	Abstract To lower barrier operation’s latency and improve large-scale parallel applications’ efficiency in Dawning 5000 system, this paper proposes a hardware-based accelerating solution to barrier. The design, which implements tree-based barrier by enhancing Dawning 5000 switch chip, has features of low latency, high scalability, high reliability, and high serviceability. Dawning 5000 barrier network supports 16 concurrent barrier operations. Compared with related works in Fat-tree topology, it is a more proper solution. In ideal situation, the barrier operation of 1024 nodes can be finished within 1.7 microseconds. Based on characteristics of barrier reducing and distributing, two different mechanisms are used to guarantee reliability. The prototype system of proposed design has been verified on FPGA platform. Keywords high performance computer; MIN; fat-tree; Barrier; synchronization; combine; distribute; reliability Background This paper is supported by the National High Technology Research and Development Program (863 Program) of China project "Dawning 5000 High Productivity Computer(Project No.2006AA01A102)". Dawning 5000 System uses HPP (Hyper Parallel Processing) architecture, which is proposed to meet challenges of peraflops computing. HPP architecture implements PGAS (Physical Globally Address Space) and supports both Message Passing and Share Memory programming models. Barrier operation, which is a kind of global synchronization operations, is widely used in both programming models. Barrier operation has global blocking semantic, as a result, it affects system performance directly. The larger system scale is, the more time is spent in barrier operation. Optimizing barrier operations can not only lower execution time, but also benefits scalability of applications. In this case, Dawning 5000 System uses a hardware-based barrier network, which is proposed in this paper, to accelerate barrier operations. IBM, Cray and NEC implemented dedicated barrier networks in their supercomputers and had gained good performance. However, take cost-effective into account, embedded barrier network is suitable to Dawning 5000 System. Compared with other embedded barrier networks, Dawning 5000 Barrier Network can gain much better performance for fat-tree topology. In ideal situation, the barrier operation of 1024 nodes can be finished within 1.7 microseconds.