Will ChatGPT change data centers as we know them?

ChatGPT has been the talk of the town for a while now, and it’s quickly spawned competitors from multiple companies. While ChatGPT is the name of a specific product, it’s also becoming a bit of an umbrella term similar to how we reference Alexa and Siri for voice assistants.

The actual term is Generative AI, which translates as algorithms which can generate content based on instructions the user gives them. The AI bases the generated content on the massive databases which were used to train it. The content could – and usually does – have at least parts of the information from those databases, but it also has been modified by the AI and its specific generative ideas.

The current big thing

So, what does Generative AI have to do with data centers? A lot. For a long time, it was thought Generative AI could only exist in massive data centers. There are alternative projects now which have been able to run the finished version of ChatGPT on smartphones without the need of additional resources. For the most part though, Generative AI will rely heavily on data centers and the cloud. ChatGPT, for example, has already been integrated with Microsoft’s Azure platform to offer services for third parties.

Generative AI has the potential to bring a lot of changes to our lives. The main goal of the current projects is to help people in their everyday work tasks, like writing answers for emails, generating basic reports, data charts, etc. Long term, the aim is to have Generative AI basically everywhere, but this will require a lot more resources.

Data centers therefore will become even more important and integral to our everyday lives. “AI will fundamentally change every software category, starting with the largest category of all – search,” said Satya Nadella, CEO of Microsoft, quoted by DataCenterFrontier.

His colleague and competitor agrees: “We’re now at a pivotal moment in our AI journey. Breakthroughs in generative AI are fundamentally changing how people interact with technology,” says Thomas Kurian, CEO of Google Cloud.

Recently Bill Gates also proclaimed that the era of AI has begun. Adobe, Nvidia, Meta and a lot of other companies, both big and small, have already entered the generative AI world. It’s clear that this technology is not “the next big thing” as it’s often said – in fact, it’s already the “current big thing”.

New world, more data

Generative AI will need a lot more data to learn from, and obviously, it will also generate a lot more data. For this to happen though, it will need serious computing power to process all of it, learn from it, generate it, etc.

These needs will grow exponentially. Unlike previous technologies which were introduced as ready-made, the IT industry has been much more careful this time. It has stated constantly that generative AI is not yet ready for prime time, and that it’s still in its infancy, but does have incredible potential to improve and grow.. However, this also means there will be exponential growth in the needed resources as generative AI becomes more and more powerful.

“AI workloads differ dramatically from traditional cloud applications built around web servers and databases. They thus require special configurations of hardware and software to reach optimal performance. Clouds that are purpose built for AI workloads, with specialized silicon, schedulers, and interconnect, represent another growing market where startups can play,” says Foundation Capital in an analysis.

“Public cloud providers are the driving force behind the current AI resurgence,” says InfoWorld. It notes that these companies will make AI both a tool they use for their own infrastructure, but also a service they offer for other companies to deploy in their cloud environments.

Hardware manufacturers are also eager to get on the fun. “Accelerated computing and AI have arrived,” says Jensen Huang, CEO of Nvidia, during the company’s yearly GTC Developer Conference. “The iPhone moment of AI has started. The impressive capabilities of generative AI create a sense of urgency for companies to re-imagine their products and business models,” said Huang.

Nvidia introduced the DGX Cloud to allow the quick build of AI supercomputers and cloud services, based on AI. The company will also offer H100 GPUs to be scaled in the cloud.

Generative AI may be in its infancy, but it’s already making big waves and demanding big changes in the IT world. Microsoft has announced it’s accelerating the build of its data center infrastructure to support the massive investment in ChatGPT; the company will build a new hyperscale data center in Quincy, WA to support AI.

The company said it has invested over $15 billion in building its global infrastructure. It has also spent an additional $9 billion on research and development to increase efficiency.

Hungry for energy

One thing that must be noted is that AI, in general, is very energy intensive. What’s not helping is that OpenAI hasn’t disclosed the energy consumption figures of ChatGPT. This is a key point to note, because OpenAI doesn’t have its own infrastructure; it’s using cloud services, presumably only Azure as Microsoft is an investor in the company and has provided credits as part of the deal.

What is known, however, is that ChatGPT used 1.287 GWh energy for its initial training, and according to TechHQ, that’s the equivalent to the energy that 120 U.S. homes use for an entire year. It’s not known what the energy consumption is of ChatGPT during actual use by users like you and me. Is it more? Is it less? OpenAI hasn’t disclosed any information and it’s unlikely it ever will. Why the secrecy? Well, the company also reduced the number of details it shared for the latest GPT-4 model. OpenAI said it won’t disclose full details anymore as it was clear that it’s the wrong approach and can be used by bad actors. So, OpenAI won’t be as Open as it once was.

Of course, one thing is for sure: Energy consumption isn’t likely to come down. The more AI is used, the more energy will be required to power it. There will be many optimizations made along the way which will make AI run far more efficiently, but the sheer scale will be more than enough to ensure the consumption will increase, even if it’s less than expected without those optimizations.

The uptake in AI use will for sure force data center operators to invest in even more energy projects in search for the best balance between energy costs and environmental impact. Companies are already chasing their carbon neutral – and even carbon negative – goals they have promised to achieve in the coming years.

Security will also change

ChatGPT and generative AI as a whole will bring even more challenges to the table, like in cybersecurity, for example. ChatGPT can also write code, and while it does have limitations and safeguards on how it can be used, hackers quickly started to find ways to bypass geographical restrictions and how to trick the algorithm into doing and saying things it shouldn’t.

Most of these “hacks” aren’t impressive. But one did highlight that there was another vulnerability in ChatGPT which could uncover the titles of other users’ previous chats. That could happen under specific circumstances, but it showed that even AI isn’t 100% safe from vulnerabilities.

And then there’s the case of hackers using generative AI for cyberattacks. The obvious one would be to ask the algorithm to create the code for various attacks. “We’re seeing coders – even non-coders – using ChatGPT to generate exploits that can be used effectively,” said Dion Hinchcliffe, VP and principal analyst at Constellation Research, to DataCenterKnowledge.

Happily, it turns out that ChatGPT can be quite the hacker itself, simply because of its massive knowledge base. “[It has read basically] every research vulnerability report. Every forum discussion by all the security experts. It’s like a super brain on all the ways you can compromise a system,” Hinchcliffe said.

Where there’s an issue, there’s a solution. ChatGPT could be the way to combat attacks created from… ChatGPT!

“You’re seeing it write a lot of code for security orchestration, automation and response tools, DevSecOps, and general cloud container hygiene. There are a tremendous amount of security and privacy policies being generated by ChatGPT. Perhaps, most noticeably, there are a lot of tests to create high quality phishing emails, to hopefully make our defenses more resilient in this regard,” Jim Reavis, CEO at Cloud Security Alliance says.

“I expect to see ChatGPT-interfaced commercial solutions quite soon, but I think the sweet spot right now is the systems integration of multiple cybersecurity tools with ChatGPT and DIY security automation in public clouds… Over time, almost any cybersecurity function will be augmented by machine learning. In addition, we know that malicious actors are using tools like ChatGPT, and it is assumed you are going to need to leverage AI to combat malicious AI,” Reavis said.

So, yes. Generative AI really will bring many changes to the world, and this will be reflected in data centers, too. That means a lot of our current ideas and expectations will have to be reformed and realigned. The era of AI will require a lot more computing power which will drive the progression and transformation of the entire IT industry. And with data centers as the backbone of AI, they will be the first ones to undergo all changes and continue to change as often as needed when the technology progresses.