IARPA has initiated a new program, the Advanced Graphic Intelligence Logical computing Environment (AGILE), to develop innovative, efficient, scalable computer architecture designs capable of executing next-generation large-scale data-analytic applications. This effort will require flexible, scalable, and detailed simulation of the architecture designs to assess the performance, efficiency, and validity. Data-intensive applications transform massive, unstructured, heterogeneous data streams into actionable knowledge.
Today, data is increasing exponentially in volume, velocity, variety, and complexity. It is often sparse and stored in structures with poor data locality. The processing needed to ingest, transform, and store data can no longer be ignored. In many scenarios, they account for a significant fraction of the overall execution time and consume a substantial portion of machine resources.
Given the nature of data analytic applications, simplistic benchmarks that measure individual computations in insolation, without consideration of the processing required to manage the data – the reorganization and movement of data between computational components, and the reporting of results – cannot measure the actual performance and scalability of real-world data-intensive applications.
Bill Harrod is a program manager at IARPA and he is responsible for the SuperCables, RAVEN, and AGILE programs. Previously he was the Division Director for the Department of Energy/Office of Science Advanced Scientific Computing Research program’s Computational Science Research and Partnerships Division. Before joining DOE, Bill was a Program Manager in the Information Processing Technology Office (IPTO) in the Defense Advanced Research Projects Agency (DARPA).
Previously, Bill, who received his PhD in Mathematics from the University of Tennessee at Knoxville, was a Technology Fellow and then a Principal Engineer at the National Reconnaissance Office and an Engineering Manager of Computational and Applied Mathematics at SGI and the Section leader of the Mathematical Software Group at Cray. While at SGI, he was the developer of the SHMEM (shared memory) architecture which provided the foundation for one-way communication libraries used on today’s high performance computers.