The Evolution of the Data Center Industry: AI, Hyperscale, and Beyond

you're listening to the data center Hawk podcast where we demystify the data center Market data center Hawk is your online platform for data and commentary on the data center Market stay tuned and be sure to join the thousands of others who rely on data center Hawk to make decisions in the data center space welcome to the data center Hawk podcast I'm Mike ner uh today we're talking about AI so I am joined by an AI representation of David liot AI David liot how are you doing I'm just great thank you Mike okay just a real no different yes than the regular this is a really great AI uh before we get into that we have a couple of business items to talk about if Rhett has done his job properly you will all be aware of this keenly aware that we launched our new site so this is kind of data center ha 3.0 uh if you've been on the site for a very long time you probably remember Data Center Rock 1.0 that's right uh and as I would say fun is it is to look back at the site and go man it's nothing compared to what we have now it was awesome at the time 2.0 was what we we just sunsetted uh and then this is now 3.0 so why don't you tell us a little bit about David like why why now why did we make the changes we made yeah um well it's been really fun to get to see all the versions um you know a couple things I would say number one I mean our mission why our company exists is to help people make better decisions in the data center space that's what we are trying to do so all of our work from a technology perspective from a market data standpoint even like you know the work that R and team are doing from a marketing as as the company's grown one thing that we try to thread through everybody's work is that's what we're trying to do and so certainly our site is a big part of that um and as the it's actually really interesting when you think about how much the industry has changed there's some things that um you know you build a technology project or product around what are the main parts of the industry and it was as time changes you have to change a product to match that so couple things that I'm really excited about you know one is is uh we we've talked about this before we have a whole series on it this hyperscale part of the data center industry has grown significantly and so now to be able to see the market view from both the collocation standpoint this is multi-tenant Data Center collocation and the hypers skill growth so this is when these large companies go out and buy their own land they build their own data center they operate it themselves to see that picture in areas like Dallas and Chicago and London and Singapore and you know these different cities in the US and internationally is really fascinating and that's a lot of the maturity that we've seen on an international basis has been those companies growing so that's one thing you know two if I think about like the um some of the things that might seem minor but have become part of the major decision-making going on one one is um submarket distinction so if you're in a um you know one of these large data center markets we're seeing uh data center submarkets come up and almost these facilities cluster close to one another because of certain you know power reasons or connectivity um reasons land sight size those type of of things are making that happen so so really being able to break those down understand how big these submarkets are why they're growing the way they are why areas have not grown you know there's some submarkets in Phoenix there's one that the city council's basically just said hey we're we we're tapped out we're done with data center growth so this used to we don't like the internet right that's what they're saying this used to be a very vibrant area and there's large data centers there there's nothing coming online so those are the type of things that now on our product you can really understand um and then we've got some really really um this sounds nerdy but like very cool mapping solutions that you can do on our site now you know we are trying to almost make we're trying to make an invisible industry come to life and you to be able to see that quickly so that you can make a decision I mean these are not people know this on this podcast but we're not talking about like you know $10 million 20 Mill These are hundred million doll sometimes a billion dollar decisions that companies are making I'm not making it up billion dooll decisions and so to be able to see it and to understand it uh more granularly more clearly um that is why we've done what we've done so pardon the Soliloquy but no no and I would say that's what we've done you know this is basically the result of two years worth of effort you know from our Dev team our requirements team our research team to build all these new data sets and visualizations but it also came with you I say hundreds of hours of conversations with our customers to go hey does this help you do your job better does this help you make better decisions space and um so if you are one of those customers we talked to thank you very much for your feedback it it was extremely instrumental for us developing the new so if you haven't been in the site in the last two weeks go to Des center.com uh check it out it's our greatest super slick it's it's our great it's culmination of our life's work that's right well David's life's work it's our last two years uh okay so let's talk about AI it's the the buzzword of all buzzwords in the data center space really across all spaces yeah I still think back to I think it was the um Arizona I te they just like mentioned that they were using Ai and their quarter learnings went like shot up like oh wait a minute what is AI have to do with IC te back down um but you know there's some real meat on the bone now so I think let's just set the table a little bit for AI and actually as you're getting as you're warming up to that the one thing I would say industry-wise you know we've been been doing this now for a long time and all the buzzwords and technologies that have been mentioned to us about hey this is really going to change the data center industry this is really going to impact the market um I would tell you that in my experience this artificial intelligence and how the industry believes it will be used and how it's being used has had the biggest impact on our space uh versus all these other technologies that are out there and I and when I say that I mean like from a demand perspective from design some of the things we're going to talk about but this is having this is not us you know blowing smoke this is having a significant impact you are on the record is not blowing smoke that's right and it will it's illustrated by what you're about to share too yeah so if you look at you know again the biggest players in Space the hyper skill users that we all know they're all investing significant billions of dollars to your point into the infrastructure the software everything that's needed to support AI compute to productize it in a way that's going to benefit their customers but you look at like Microsoft you know last quarter 10 almost 11 billion which they've said much of that is on AI Cloud compute Google 7 billion and capex says the largest component for servers which represents a meaningful increase in investments in AI computing um Amazon for the year of 2023 about 50 billion capex historically about 40% of that capex has B towards AWS but they've also said the percentage of their capex towards AWS from Amazon as a whole towards AWS is increasing um you know meta is saying they're spending about $30 billion in 2023 again primarily focused on AI infrastructure you know Blackstone big fund that most people have heard of uh purchased qts a couple years ago said they're actually liquidating some office properties in in order to put that money into digital infrastructure they've called this a once in a generation opportunity and they're pretty smart guys over there and in addition to all those groups you've got a number of startups that are investing significant capital I think cor weave a company a lot of people have heard of you know raised you know 2.3 billion in in debt financing from a number of groups I black Zone included and there's a number of groups like them that are very quickly ramping up their AI capabilities specifically around AI cloud computing so what that as the backdrop um just kind of people understanding like hey this is a big deal the biggest most technologically advanced smartest companies in the world are dumping billions multiple tens of billions of dollars into this space so that as a backdrop it only uh goes to imply that that would have an impact on demand so people actually need data center power space and cooling uh which is what we have historically tracked sure uh so from your standpoint how has that impacted demand really over the last probably six months yeah and maybe even longer than that if you think like maybe 12 to 18 months um it's impacted demand in a number of ways number one the requirement sizes are growing so if on previous podcast we've talked about how 15 years ago data center demand the biggest deals probably 2 to six megawatts and then it started like growing and growing and growing and I would say when we went through the I'll call it the cloud craze that's probably not a good term for it but that's what I'm going to use uh from like 2017 to 2020 is 2021 you know deals were starting to get in like the 36 megawatt range 72 megawatt range we saw some that were like deployments over a period of time that were in the 100 plus megawatt range you know now we are starting off with projects that that like have that as their end goal 100 Megs 200 Megs you know there's groups that are asking about needing to deploy 500 megawatts over a 2-year period you know the word gigawatt has been used on different um requirements and so these are just you know I mean think about this from a construction ins side of things you know if you said hey it's $10 million a megawatt as a as a construction cost or whatever now take that over you know gig I mean we're talking so much money and and what Mike what you just mentioned you know if you if you do follow the money it's like like that just shows you the level of conviction that these companies have around chasing AI so number one the sizes have gotten larger it's pushed um the demand has also changed because it's it's making companies look at like the the actual sources of the energy because now we're not talking about a 30 megawatt project that is built in two years and that's it we're talking about hundreds of megawatts being delivered and so these the due diligence that's being done is done a lot further up the stack it's not just the site and the power and the power company is now trying to figure out can this region support this type of growth um and it's it's definitely impacted the way site Acquisitions have been happening over the last 12 to 24 months we're seeing companies spend a lot more money on you know uh buying sites that they can use in the next two three four five years uh to you know house some of this demand and so we're talking you know this like hundred1 million 150 million $200 million on a landsite yeah to to go do this work and um you know 10 years ago we would joke that the land cost was a rounding error when you think about the amount of money that's going on now still is a very is a much smaller part of the total Financial picture when you think about one of these data center projects but it's not a rounding area yeah the percentage has gotten larger so and and just the the trickiness the the effort that's required to make sure that you can have that Runway to the significant amount of power it reduces the number of lands sites that are viable uh or the amount of work that's required to make a landside viable yeah you know the other thing that demand has done that as it's gotten larger is it's created more um awareness of these projects in a good way and probably bad way for the industry you know one from a good way it's it's allowed people to see like what this technology is doing in the world how it's helping Industries you know at some point it's like saving lives I mean there's doing some incredible things if you flip it you go hey there's people that in some ways are against this type of growth because they don't want these type of buildings in their communities or they they're on the side of AI where they go hey I don't want to promote something that there's some fear around what AI can do down the road so um it has had a massive impact on demand I can't I can't state that enough I mean and and that's impacting excuse me the ways that a lot of these companies are having to bring the supply online you know thinking through just the projects as a whole and things like that yeah and that this kind of spike in demand has come on top of what is already an extremely Supply constrainted industry so it's a bit you know good news for some bad news for others but it's it's a little bit crazy how much additional demand we're seeing on top of the existing demand that's been there for two three years we were already in a supply constrained Market before these large projects start coming to the market so it's it's it's had a massive impact on the space yeah speaking of which anybody who is like concerned about the AI taking over the world I would recommend Mark andreon he's done like he did like a podcast tour almost he like five different podcast he kind of had the same message on each of them but I think he's got a very reasonable take on AI and where it will ultimately end up which spoiler alert not taking over the world good thing so we're saved there is this General demand component but the nature of the demand is different obviously I think a lot of people are aware that these AI workloads are very different in nature than the GPU demand is very different than the CPU uh demand but just to put a you know kind of quantify that historically data center density was you know 3 to 7 maybe 10 to at 15 Kow per cabinet per rack um and and that's slowly ticked up over time but not nearly to the level that we're seeing with AI where you know you talk about 100 plus KW ARA um you know these GPU servers are extremely dense could be six to 13 you know kilowatts in a in a foru space so if you build that out over a cabinet you can do the math um so that's changing the way these groups are having to adjust it's obviously more power and a smaller footprint um and so talk a little bit about how that is impacted these groups is it can you get smaller acreage and build a smaller site with the same amount of power or how has that changed how these groups are kind of handling those requirements uh I used to go on data center tours 10 years ago and every data center tour you went on you would hear about the density we can we're a data center we can do you know 150 watts of then 200 wats of foot 300 watts foot and so then if you just ask the question okay where's that being done in the and they said well we're not doing it yet because and it wasn't that they couldn't it was that the customers weren't mature enough with their infrastructure design to want to need that um now if you go into some of these facilities you can follow the noise or follow the heat because that is where these higher density Solutions are and so um so so so part of it is just the customers embracing um you know higher density and I would say like let's let's put AI to the side for a second these are like Financial companies technology companies insurance companies historically that have just done it the way they had done it I think now they're doing it differently because there are some there's value into the higher densities but but now a lot of the um design conversation is around you know the the cooling aspect of this if you are going to densify these racks significantly you know and we're not talking let's talk 30 to 50 KW per rack and then if you go go even higher there's some news recently that you know Cyrus one's talking about doing I think it's 300 KW per rack with with some liquid cooling solutions um you've got to think through hey how do we cool this in a different way because we're now allowing a much denser environment than we had in the past yeah I think this will push some of that you know higher density cooling solutions forward and there's a variety of them as you kind of go up the density you know chart I think this will probably push some of those Technologies forward yeah and and the AI stuff it is so it's so much more dense than the other solutions that we've seen you know data center operators are busy thinking through you know level one designs the phase or phase one designs phase two designs and I think we will see that take a similar path that we saw the cloud uh data centers get built over the last you know 5 to 10 years where historically we were seeing you know 20 megawatt building and then it switched to going hey can we deliver 49 megawatt Halls to make 36 Meg can we put a building right next to that to make 72 Meg you're starting I think you'll see the same thing happen on the the the AI side from a density perspective you know one of the challenges is going to be with AI and the growth of AI is is the hardware and how quickly the the components can be produced to grow the technology and that's going to be a challenge you know moving forward if the industry and companies adopt it the way I think people think they will yeah I think it's G to it's going to require a lot of flexibility there's some super smart folks working on this but like this concept of like almost like mixed density where you have that flexibility within your site to house some high density racks some lower density racks and all that that entails um but there's some super sharp people working on this and some of the conversations I've had recently it's like way over my pay grade technically um but again it's a it's an opportunity for some but it's going to require some work um you know then from like a location standpoint we've touched on this in the past but like how does this specific AI demand impact you know the location of where these you know where the demand is going yeah um the you know story in the data center industry over the last five years has been major markets getting bigger Northern Virginia Phoenix uh Dallas you know Atlanta is starting to grow more um so there's these areas Chicago that historically have been one size now they're growing you know if you take that to Europe you're looking at areas like London Frankfurt um and I think over the last 12 months you've seen AI gobble up what's remaining of the power in these markets as well as uh sites that are under construction or in the pipeline to be delivered the good news about uh AI deployments is they are not as latency sensitive as some of the cloud uh availability ility zones are so with that comes the ability for those projects to move into maybe some areas that Geographic areas that historically haven't been like data center hubs or um growth areas the so I think that if you can find the the land you can find the power I think you'll have some audiences that will listen to what you how quickly you can deliver it uh the the challenge when we see data center projects move from mature markets to let mature areas would be you trade you can trade one problem for the other so you solve one problem but in doing so now you've got another issue that you got to solve for so if you can solve the site and power problem in a tertiary Market it's great but you might have a connectivity issue you might have a tax incentive issue you might have a labor issue you know I hear all the time like you can build a can't you build a datos anywhere and it's like well yes but you have to solve all the problems and that costs money and people someone go hey I can solve this problem over here faster and cheaper than I can solve it over here so it doesn't make sense to build the data center anywhere um yeah it's like every data center project is is solving 15 problems at once and AI probably changes the equation for two or three of those problems so the other problems still persist and still need to be solved but it does change it slightly yeah the the calculus and to meet the large power requirements these These deployments are going to have to go to places that are either that that are creatively thought through on how they can do that and um and it and it's a long game approach um the good news about our space is I think there's a lot of people that would love to get into the development side and the site selection side but you got to have a lot of money to do it so it's in one way it's good because it keeps out new groups that would probably stumble and it would be a challenge to catch up to the way that these large companies are using this um you know space I would say the bad news is is it's you know it takes a lot of money and a lot of equipment and while there is a lot of money interested in the data C industry right now which is good um you know the interest rates have been a bit challenging for some of these companies to figure out how to deliver what they need to and hit the returns they need to but then also just the equipment being able to keep up and um and match what the power requirements are uh but the good news for our space is that we can locate these projects in areas typically that you wouldn't think are large data center markets um but the challenges just come in how you solve all the other problems yeah and so then kind of all those things added together the demand plus the design changes plus the location challenges that's going to have an impact on pricing and I think it may take some time for that to all work itself through the system but there's just a pure demand standpoint that's going to drive potentially pricing up yep um but then there's trying to figure out probably from the operator side how to price these comp newly complex Deals Deals that are complex in new ways than they've dealt with before but what are you seeing from like just a pricing impact across the board yeah from the ad demand I mean it's gone up considerably you know you have these new companies that will pay higher rates on a price per KW per month basis than these larger companies that have better credit and have been around a long time um these newer companies you know as you mentioned newly formed so they don't have the his the historical track record of performance so they've got to do something to number one get in line to get the power before anyone else does and number two um drisk the investment by the data center operator so that's driven you know that pricing up um you know the the the other thing that I I think is a really in my opinion positive to the industry you know 15 years when I got in the space there was like the all of the data center operator Community was pushing the users to do it one one way you know you have to lease in these increments of you know whether it's 2 megawatt blocks or 1.2 megawatt blocks or 4 and half megga blocks and this is how the pricing needs to be set up we don't do anything different than this this is how it is and that's changed considerably like you couldn't the industry couldn't grow the way it is growing if there wasn't more creativity put around deal structure so you know right now price per KW per month on like a gross Plus Electric basis people in the space would know what I'm talking about doesn't always makes sense if a company wants to lease you know 72 megawatts or 96 megawatt to take the whole building they might want to do a triple net deal where they actually control the expenses you know so you bump that deal down by 20% and then they just pay the base rent and the expenses on top of that we're also seeing some of these um you know yield on cost solutions that have allowed these big companies to get more comfortable with uh doing the deals that that are needed but when you look across each one of those deals de structures rates are up M and they have to be for the what you talked about related to just some of the issues with um lack of Supply but then also with the size and how U quickly things are growing so um the rates have just gone down since I got into the space you know back in 2007 and then in 2020 2021 things started going back up and they're escalating now for a different reason uh than they were three years ago you know this isn't a coid escalation necessarily this is a escalation related to you know lack of supply and we've talked about this before I mean I've said this probably 10 times on this podcast I don't see it changing anytime soon it just it just can't the projects too big it takes too long to bring power to a site building a substation you know costs 50 to100 millionar and it just doesn't happen fast so um anyway it's probably more than you want to but but the yeah the rates are increasing significantly and the deal structure deal structures are getting more creative yeah and and there is some uncertainty with the future as far as like some of the projections for AI demand are are are astronomical um some people think there's some sentiment that maybe this is a bubble there probably will be some companies that are unsuccessful in bringing their products to Market and so that could temporarily or in a certain localities bring down demand but I do think you know in general we believe this trend is going to persist yeah in addition to all the other Trends We Believe will persist across the space so this is just kind of one more layer um that is making it again there's plenty of opportunity there for groups but it's it's harder and harder to capture that opportunity yeah and one thing I would add as we wrap up here is I think with a demand everybody needs to make sure that you're that there's due diligence around what is really being committed to because it's very easy to see some of these uh announcements and some of even some things that I've seen in the industry that have been announced related to like how much demand has actually been committed to and typically it's far less than what you see yeah tap the brakes yeah from an absorption standpoint I mean these companies are like smart they're not going to you know uh fully commit to things they don't know that they need but they will set up an economic structure and a deal structure that allows them to have access to do that when they need to you know and and I think um so there's there's just some additional due diligence which is what you know we will spend our time focused on around really trying to figure out hey what has been committed to so when you hear oh my gosh this deal was a 600 you know megawatt deal or a 400 megawatt deal it's like okay well let's figure out what that really what is a capital commitment by the company what did they just sign up for did they just sign up for 600 megawatts of at 140 bucks of KW for 15 years probably not now could they get there over time maybe but there's a an approach that a data center operator has to take and a user has to take to make sure that deal can make sense financially over time and give them the ability to scale up um when they move forward but so I've just in summary you got to do due diligence on those numbers that come out and get thrown around on deals that are getting sure and it's almost to the point that no number would shock me but sure at the same time some numbers are too good to be true yeah it's got to beat it up a little bit you got think about and think about like what is a company actually committing to financially you know when we talk when we hear these groups are like yeah this group's building a 400 megawatt data center it's like you're telling me that a group is spending you know x amount of billions of dollars without a lease commitment from a customer they're not going to do that they're smarter than that so those are the type of things you got to beat up I think that's going to be on your gravestone here lives David look get quote you got to beat it up a little bit end quote well gosh it's a sad life if that's what it is that's just uh Mr due diligence here so there you go okay well hey that was great I think again we're going to continue keep an eye on this it's like it's front and center and a lot of the conversations we're having research we're doing so um as it continues to impact the space we'll report back here otherwise thanks for joining us we'll see you next [Music] time

Channel: datacenterHawk

Views: 8,470

Keywords: data center, data centers, AI, data center ai, artificial intelligence, hyperscale, ai hyperscale, google, aws, amazon, google data center, meta data center, aws data center, facebook data center, google ai, meta ai, aws ai

Id: GamEH72DkGo

Length: 27min 15sec (1635 seconds)

Published: Thu Nov 09 2023