"Distributed Ranges": Model for Building Distributed Data Structures, Algorithms & Views - Ben Brock

#cpp #cppcon

"Distributed Ranges": A Model for Building Distributed Data Structures, Algorithms, and Views - Benjamin Brock - CppCon 2023
https://github.com/CppCon/CppCon2023
https://www.youtube.com/watch?v=X_dlJcV21YI

Writing programs that run on clusters of computers is notoriously difficult. Distributed data structures can simplify this problem by automatically splitting up data across multiple memory spaces or nodes in a distributed memory program. Algorithms in most distributed data structures libraries, however, are not generic, but operate only on a specific data structure. In this work, we discuss how modern C++ features like concepts, customization points, and the ranges library can be used to support generic algorithms and views that can interoperate with multiple distributed data structures. We first discuss background in distributed computing and distributed data structures, including common techniques used to create high-level distributed data structures. We then show how customization point objects and concepts can be used to allow the implementation of generic algorithms that can be used with any distributed data structure as long as it fulfills a "distributed range" concept. We then show how to implement views that satisfy this concept, allowing algorithms to operate on lazily evaluated views of distributed data structures. Finally, we discuss two implementations of this model, for multi-GPU and multi-node programs, as well as performance considerations when running on state-of-the-art HPC systems.
---

Benjamin Brock

Benjamin Brock is a research scientist at Intel Labs, where he works on building libraries, tools, and algorithms for high-performance computing. In 2022, he completed a PhD in distributed data structures at UC Berkeley. His work focuses primarily on building cross-platform data structures for distributed applications, with applications in sparse linear algebra and graph algorithms. Ben is a member of the GraphBLAS Languages Committee and is involved in drafting the GraphBLAS C++ Specification.