Most people don’t really think about data centers, but we all use internet-connected apps, streaming services, and communication tools that rely on processing and storing vast amounts of information. As the world becomes more connected and it becomes easier to create and distribute vast amounts of data, the systems and processes required to handle it all continue to evolve. Sandra Rivera, Intel’s Executive Vice President and General Manager of the Data Center and AI Group was recently in Bangalore, and Gadgets 360 had the opportunity to hear her thoughts on current trends and her vision for the future. Many things have changed due to the pandemic, and of course artificial intelligence is a big part of the story going forward.
We first brought you Sandra Rivera’s commentary on what Intel is doing in India and everything the company is doing here. For now, here are some more excerpts from that conversation about hardware and software innovation, the evolving nature of the data center, and competition with Nvidia.
How data centers have become more important, and how things have changed recently:
Sandra Rivera: All of our innovations and products are clearly driven by our customers. We are in a large and growing TAM (Total Addressable Market) and this is most evident in India as we move forward with digital transformation and the digitization of every aspect of our lives. We need more computation; we are creating more data. It needs to be compressed, secured, sent over the network and stored. It needs to be provided, and you need to derive valuable insights from it, which of course is where artificial intelligence comes in.
One of the interesting things that’s happened during COVID is that as we’re all trying to overcome supply chain constraints, we’re seeing customers tend to leverage more of the infrastructure they have. AI, networking, and security are very hungry for the latest innovations and solutions, but more of the web layer; office applications running in cloud infrastructure; enterprise resource planning systems; accounting systems; etc., are actually very focused on utilization rate.
The greatest growth occurs at what we call the edge or on-premises of the network. Computing has come to the point of data creation and data consumption. A lot of the challenges we face are working with our OEMs to simplify on-premises applications to process data; run machine learning, artificial intelligence, data analytics, network functions, security. A lot of work needs to be done on both the hardware side and the software side.
The same is true for India. (Some of it) is driven by power limitation, so if they can have power dedicated to those leading-edge applications and infrastructure and then limit power to more mainstream applications, then that’s a smart use of the power budget, which is A big problem deal.
India is very important to us from an R&D perspective; I mean we’ve been here for decades. We also see that with all the investments the government is making in digital transformation and infrastructure, India will also be a huge consumer market for us. The opportunity to build more infrastructure here, more data centers, more enterprise solutions, software ecosystem solutions and services is very exciting. We continue to invest not only in our workforce, but in the market opportunity here.
Even with strong GPU demand, the continued importance of the CPU, and how this disrupts data center design:
Sandra Rivera: The continued adoption of 5G drives high-growth workloads such as artificial intelligence and networking, as well as security and storage. One of the dynamics we’re seeing in the market is that, in the short term, there’s a lot of interest in accelerated computing, namely GPUs and AI accelerators.
Customers want to shift some of their capex to GPUs. CPUs are part of that, but in the short term, more capex will go to GPUs. We don’t think this is a permanent market condition. From a price/performance and programmability standpoint, this CPU is quite good for many AI workloads. In many cases, the fact that customers already have a Xeon CPU, so they can do AI machine learning[with it]is a tailwind for our business.
Intel AI Continuum
Everyone is talking about generative AI and large language models these days, but AI is so much more than that, right? AI is all the data preparation that goes on before training a model; it’s data curation, filtering, and cleaning. So if you’re trying to build an application to recognize cats, (for example) you don’t want any dogs in those pictures. All of this was done up front with the CPU, and in fact it’s almost entirely done with the Xeon these days. This is part of the AI workflow. Then you enter the actual model training phase. This CPU is ideal for small to medium models (10 billion parameters or less) or mixed workloads where machine learning or data analysis is part of a broader application. CPUs are very flexible, highly programmable, and you probably already have one.
When you’re talking about the largest models with 100, 200, 300 billion parameters, you need a much more parallel architecture, and that’s what GPUs provide, and you also benefit from dedicated deep learning acceleration like we’re doing in Same as in Gaudi. After training the model, you move into what we call the inference or deployment phase. Usually, you are local. If you’re in a retail organization or a fast food restaurant, you’ll usually run it on the CPU or some accelerator that uses little power and is cheap. In the inference phase, we can compete very effectively with our CPU and with some smaller GPUs and accelerators.
Currently, there is a lot of interest in those largest language models and generative AI. We’re seeing more and more customers saying they want to make sure they have some GPU capabilities. We do see complexities in this dynamic but long-term market. It is growing. We are in the early stages of artificial intelligence. We think we have a good opportunity to leverage the broad capabilities we have across our portfolio. So I don’t think generative AI is small; it’s small. But it cannot be solved by large-scale GPU alone.
How Intel sees Nvidia and how it plans to compete
Sandra Rivera: Everyone knows that Nvidia does a great job of bringing GPUs to the market. This is a giant player. Let me put this in perspective. The Gaudi 2 performs better than the Nvidia A100, which is the most common GPU today. It doesn’t offer more raw performance than the H100 at the moment, but it’s actually in a very good position from a price/performance standpoint. One of the data formats supported by the Gaudi 2 hardware is FP8, and software supporting this format will be released next quarter. We expect to see very good performance, but you’ll have to wait and see what we release in November. Next year we will have the Gaudi 3 on the market and it will compete very effectively with the H100 and even the next generation on Nvidia’s roadmap. Our forecast looks very good. Our pricing is very aggressive. Customers want alternatives, and we absolutely want to be an alternative to the biggest players in the market. It will be what we do, not what we say.
Intel’s roadmap for sustainable data centers.
Sandra Rivera: We use more than 90% and sometimes 100% renewable energy in all our manufacturing processes around the world. We are second to none in terms of renewable energy and the total carbon footprint of product manufacturing. These competitors, like most of their competitors around the world, are producing their products in foundries in Taiwan or South Korea. Taiwan is of course the largest, but their footprint in renewable energy is actually quite small. This is an island; everything is transported on diesel. When we look at the data centers we’ve built for our own fabs and IT infrastructure, we see that over 90% are renewable. We also work closely with OEMs and cloud service providers to help optimize green and renewable energy.
In the 4th generation of Xeons, we introduced Power Optimized Mode, where you can actually reduce power consumption by 20% by intelligently shutting down the cores and tuning the processor during idle time. We’re able to do this with a very small impact on performance, less than 5%, which customers like because they don’t always need their processors running at full capacity, and it saves a lot of energy.
The Current State and Future Potential of Neuromorphic and Quantum Computing in the Data Center
Sandra Rivera: Neuromorphic and quantum computing are frontier technologies. We have been investors in the quantum field for at least fifteen years. We’ve been investors in silicon photonics; optical networking and interconnects are becoming more and more interesting, especially in these very high-end massive computing platforms. We know memory technology is critical to our future. We’ve been investing in memory technology with our partners as well as ourselves. The commercial viability of these technologies sometimes takes 10-20 years, but innovation is the lifeblood of our business. Together with Intel Labs we have extraordinary capabilities. We have a lot of fellows, senior fellows, and industry luminaries. The process technology is one of the most complex and exquisite engineering in the world.
We will continue to stay ahead from an innovation standpoint. Commercial viability is entirely dependent on the speed of market change. We do think that AI is disruptive and some of these technologies may be accelerated (developed), especially networking and memory. There’s been a lot of innovation in power and thermals; these chips and systems are getting bigger and hotter. When the time comes, it’s not always easy to answer. Some of these techniques may not be commercially successful, but you can bring some of them to other fields. I think it’s a business of innovation, and we’re very proud of our history. These (teams) can do a lot of very interesting things, and they’re very dynamic.
Some responses have been condensed and slightly edited for clarity.
Disclosure: Intel sponsored the flight of journalists to the Bengaluru event.
Svlook
