High performance, minor impact:
the future of computing

Interview with Marco Aldinucci, coordinator of the High-Performance Center for Artificial Intelligence (HPC4AI) at the University of Turin and co-leader of Spoke 1 of ICSC, dedicated to High Performance Computing (HPC) and Big Data.

What are we talking about when we refer to HPC?

High Performance Computing means using extremely high computing power to solve a problem faster or to solve a bigger problem than the initial one in the same amount of time. In the first case, HPC comes into play for solving scientific or industrial problems where the value of the information degrades over time. Examples include weather forecasting or simulations of natural phenomena that trigger operational scenarios, pharmaceutical chemistry, and materials science, where the computational complexity is enormous and the analysis needs to be completed in a reasonable time.

In the second case, HPC allows to solve problems where the computational grid becomes denser, increasing the number of calculations and the size of the problem. Staying with the example of weather forecasting, I might need to move from a 1 km by 1 km grid to a 100-meter by 100-meter or even a 1-meter by 1-meter grid, for instance to predict a flash flood or to determine whether a self-driving car will encounter ice at a specific intersection. Or, looking at Artificial Intelligence (AI), I might want to increase the number of parameters in my Large Language Model (LLM) to make it capable of addressing more complex tasks, moving from translating a text to reasoning, or understanding irony. To do this, I will need a larger model: I’ll have to go from 7 to 70, or even 700 billion parameters, which means a matrix with 700 billion cells, a nonlinear increase in computation time (the complexity of the basic operation, multiplying dense matrices, is cubic), and a multiplication of the space occupied. While in fact for a standard computer, a laptop, the power is achieved through miniaturisation, in the case of a computer that is a million times more powerful than a laptop, we do not have the technology to miniaturise a million times further. So, to solve this problem, we place many systems side by side, covering an area as large as a soccer field. Sometimes, jokingly, when someone asks me what HPC is, I say: “If you can see it from a satellite, it’s HPC,” but I could also say, “If it consumes more than 1 megawatt, it’s HPC.” In short, it’s a big deal, and an energy-hungry one too.