AI Infrastructure Hardware and Software Accrue the Most Value in AI Stack

AI Infrastructure Hardware and Software Accrue the Most Value in AI Stack

by | Mar 12, 2024 | AI Digital Transformation, Technology Trends

AI share in the global VC funding keeps increasing, around $20B In February, more than a fifth of all venture funding in February went to AI companies, with $4.7 billion invested in the sector. That was up significantly from the $2.2 billion invested in AI companies in January 2024  and the $2.1 billion invested in February 2023, according to Crunchbase. GPU cloud, humanoid robots, and enterprise AI search got the bigger rounds that stand out.

The deal value in AI startups ends Q4’24 with $22.3 billion across 1,665 deals. Powering these startups are AI-specific datacenters, whose specialized infrastructure and operations are designed to meet the rigors of AI workloads. Some startups in this space are challenging Microsoft, Google, and AWS (The Q4 2023 AI & ML report from Pitchbook). Also, Pitchbook reports that startups in the infrastructure SaaS market are ready to capitalize on the sea change that AI is bringing. The sector saw an uptick in VC funding in Q4 2023—reaching $2.2 billion compared to Q3’s $1.9 billion. Dharmesh Thakker of Battery Ventures predicted the software spend of around $600 billion will increase twofold to threefold over the next five to seven years as AI delivers productivity gains and leads to GDP growth. Let’s borrow the drawing depicted by Elad Gil – the wave of AI product and application building is coming, after the current infrastructure building wave. All are speculations. The current AI wave is seen as a paradigm shift similar to mobile or the internet. Steve Sloane of Menlo Ventures gave a much longer review of tech waves and lessons (here). Because generative AI is riding on the readiness of software, digital transformation, data, and computing infrastructure already built previously, it has achieved an unprecedented adoption rate. But, that doesn’t equal the revenue generation rate. The most important model of AI is the business model. Menlo Ventures also surveyed more than 450 enterprise executives across the U.S. and Europe and summarized a report – The State of Generative AI in the Enterprise.

Half of the enterprises we polled implemented some form of AI, whether into customer-facing products or for internal automation, prior to 2023. Enterprise investment in generative AI—which we estimate to be $2.5 billion this year—is surprisingly small compared to the enterprise budgets for traditional AI ($70 billion) and cloud software ($400 billion). Enterprises already used AI before. Enterprise execs cite “unproven return on investment” as the most significant barrier to the adoption of generative AI. The median enterprise we surveyed spends more on AI for product and engineering (4.7% of all enterprise tech spend) than on all other departments combined (3.5%). 3 predictions from the report are:

  • Despite the hype, enterprise adoption of generative AI will be measured, like early cloud adoption (took years).
  • The market will continue to favor incumbents who embed AI into existing products.
  • Powerful context-aware, data-rich workflows will be the key to unlocking enterprise generative AI adoption.

We agree with what Anthony Schiavo of Lux Research wrote in The AI revolution will look like a failure. It’s unlikely that the AI use cases generating hype right now will be the ones that drive a lot of revenue. However, LLMs’ ability to make interfacing with digital systems easy for nonexperts might have a meaningful impact on improving the adoption rate. These digital systems could be software, codes, and all physical systems, for example, factory floors, robots, oil field operational systems, defense systems, etc. And, it could enable collaboration between industry experts and those systems. Where will the most value accrue throughout the AI landscape?

Application companies might grow revenues quickly but often struggle with retention, product differentiation, and gross margins from lessons learned previously unless there are moats of adoption such as industry expertise and relationships. All the AI hype and hopes fuel the demands for AI infrastructure – software and hardware. Infrastructure providers are likely the biggest winners in this market so far and in the short term, capturing the majority of dollars flowing through the AI stack. 

AI data centers are “picks-and-shovels” for all the above developments, desired explorations with generative AI, and even struggling efforts to find AI business models – all aren’t possible without AI servers. Big funding from VCs or tech titans going into the software infrastructure of generative AI stack in the past year only increased the devastated demand for AI servers and GPUs – hardware is more a bottleneck than software now. Nvidia and Supermicro are enjoying explosive revenue growth and have been breaking records for their stock market caps for quite some time (Supermicro stock goes up over 3X this year, 20X in 2 years, and Nvidia goes up over 2X this year, 4X in 2 years). But both depend on TSMC’s advanced node chip manufacturing and packaging technology. Only 2 months into 2024, TSMC’s revenue has grown around 10% despite the iPhone demand declining, TSMC market cap has grown 50% this year and became the 9th largest enterprise worldwide.

Taiwan has been manufacturing and shipping 90% of global computing servers even before the AI hype. Recently the Taiwan government needs to expand the capacity and area of air cargo operations to support the strong demand for global AI server shipments. TSMC has to keep building the most advanced fabs across Taiwan cities 24/7 (yes, the construction goes 24/7). There are tightly connected ecosystems for semiconductor and hardware manufacturing, that collaborate and respond to demands fast on the small island. Large companies invest in frontier innovations and build technology moats, their small key suppliers can grow fast riding on the wave with high capital efficiency and labor efficiency. These investment opportunities aren’t understood by or accessible to most US or global VCs. If you like to learn more, sign up here.

 

Thinking of eras — Cisco and HP in the 80’s, Apple and Microsoft in the 90’s, Amazon and Google in the early 2000s, and Facebook and Tesla in the 2010s. In the case of the dot-com bubble, Google and e-commerce allow people faster access to information and goods. Now generative AI is bringing us the next wave, what companies will become the representations of this era, and what opportunities are left for startups?

Anand at CB Insights notes: “The number of deals to AI startups backed by US big tech companies & their strategic venture arms jumped 50% year-over-year. This is the next big super trend, much like the internet and mobile phones, and big tech is committed to not just not missing it but to driving it. They are all in on genAI.” James Mawson of Global Corporate Venturing reminds us: Silicon Valley is the center of the AI world now – but for how long? Abu Dhabi has set up a new technology investment company, MGX, looking to invest up to $100bn in AI infrastructure (data centers and connectivity), semiconductors and core technology and applications such as AI models, software, data, life sciences, and robotics.

AI Stack, by Mayfield

Related:

Abu Dhabi sets up technology investment company MGX amid AI push

Saudi Arabia reportedly in talks with VC firms like Andreessen Horowitz to create mammoth $40 billion AI fund

Gen AI: The next S curve for the semiconductor industry?

Magnificent Seven to Watch for the Next AI Era

Magnificent Seven to Watch for the Next AI Era

by | Feb 13, 2024 | AI Digital Transformation

In the 2024 Tech Trend published by CBInsights, the first to watch is GPU shortage (it has been for most of 2023). Nvidia hit a $1T market cap in Q2’23, its rise signals the next platform shift. The new term “Magnificent 7” big tech giants, was created by Bank of America analyst Michael Hartnett in 2023.

Startups link up with big tech to access their AI chips and compute power and also look outside of big tech for cheaper cloud computing. Startups building software to help AI models run efficiently on available hardware gain traction, the GPU shortage also opens the door for experimenting with novel processor approaches. Big tech is the biggest buyer, hoarding H100s in the thousands, but Nvidia’s biggest customers are also pushing their own chips. And Sam Altman, OpenAI CEO, might raise trillions of dollars for a new semiconductor fab to manufacture advanced AI chips (not to promote him here, but to show the crucial position of semiconductors in AI wars).

As reported by CBInsights, in 2023, AI startups raised $42.5B across 2,500 equity rounds., although down 10% year-over-year (YoY), it fell far less than broader venture funding (-42% in 2023). Generative AI attracted 48% of all AI funding. The US saw AI funding jump 14% YoY in 2023, fueled by mega rounds. AI deals got bigger in 2023, with the average deal size climbing 21% YoY to $23.4M, and the median deal size also held steady YoY at $4.4M (the second-highest level on record). M&A exits rebounded in 2023 to 317, a record high. The acceleration in M&A points to a wave of consolidation, as incumbents look to quickly bring AI capabilities on board. Google was the most active investor in AI in Q4’23, backing 9 AI startups. It was followed by KB Securities, Nvidia, and Plug and Play Ventures with 8 companies apiece, and then Lightspeed Venture Partners with 7.

In the US, big tech companies’ R&D investment outpaces that of overall US venture funding by a wide margin. This underscores the scale these companies play in driving tech innovation. Tech giants are throwing their weight behind promising AI startups, offering computing power and funds for development, especially in the realm of generative AI. (source)

Not only big tech giants, companies are rushing to build AI data centers, and mentions of AI data centers skyrocket on earning calls.

On another advanced computing front, Quantum computing funding surged to a record high in 2023, bucking the broader venture downturn for 2 years in a row. Neutral atom quantum computers are having a breakout moment, hinting at accelerated timelines for commercialization. More enterprises are running pilots with quantum startups. Interest in the intersection between quantum computing and AI is heating up as quantum advances can enable more powerful AI models down the road.

In this AI era, access to power is a differentiator. The number one operating cost of data centers is power! Generative AI makes the data center demand from megawatt to gigawatt scale. That adds extra stress to grids or is even prohibitive without some change. And, regulations will require disclosure of carbon emissions, sustainability is more than a buzzword. There is evidence that LPs are aware of the scale of data center opportunities, and its power hunger issue (also water). Using a combination of grid-sourced power and a portfolio of other renewables, small modular nuclear reactors, wind turbines, and batteries are all possibilities. (more)

The other differentiator is semiconductor capability, this topic might take a series of posts, including advanced packaging techniques, advanced semiconductor manufacturing nodes, new circuit architectures, new materials, photonics, and more. The White House on Feb. 9, 2024, announced the U.S. government’s plan to spend $11 billion on semiconductor-related research and development and said it was launching the $5 billion National Semiconductor Technology Center. NSTC will establish an investment fund to help emerging semiconductor companies advance technologies toward commercialization. (source) Will this matter to the international semiconductor sovereignty race that started quite a while ago and now AI is added to the race?

Water Innovations and Investments are Growing Strong

Water Innovations and Investments are Growing Strong

by | Feb 13, 2024 | Climate Tech

The investor panel recording on the water innovations (Jan.26, 2024). Thanks to all the panelists for the great conversation and the chat record is here.

A suggestion from Paul Burgon, Exit Ventures – (he has a background in running the North American division of a public European water tech company and also being on their board)

My observations: I agree with your investment opportunities points. One clarification I would add: startups should not focus on customers in the municipal water sector, it is a complete mismatch. Municipal water operators are extremely risk averse and they won’t be interested in a startup’s new technology until it has seen years of use and testing, unless the municipality is almost desperate. They are very rarely desperate. So, startups should focus on industrial water testing and reclamation rather than municipal. The industrial segment will be much more open to trying newer technologies with a strong ROI and so startups can be much more successful in the industrial segment.

Context: In this downturn, VC investment amount in water tech is growing strong, since it’s underinvested in past years.

Highlights on water deals from Crunchbase: (published on Feb. 12, 2024)

Total funding to water industry categories in 2023 was higher than in 2021. And this year is also off to a strong start. The annual investment since 2019 to companies in the water, water purification, and water transportation categories is charted below.

10 Trends in Climate Tech in 2024 + AI and Climate

10 Trends in Climate Tech in 2024 + AI and Climate

by | Feb 5, 2024 | Climate Tech

Climate Notes

Saudi Arabia ordered state-owned Aramco (ARMCO) to maintain its oil production capacity, marking a major reversal from the energy giant’s plan to boost capacity. The Maximum Sustainable Capacity of the world’s largest crude producer will be maintained at 12M bbl/day, instead of a planned capacity expansion to 13M bbl/day by 2027.

The European Union pumped out 8% less carbon dioxide from the fossil fuels it burned in 2023 than it did in 2022, the Guardian reported, pushing these emissions down to their lowest level in 60 years. Yet, over this period, the economy has tripled – showing that climate change can be combated without foregoing economic growth. More than half of the drop in emissions came from the use of cleaner electricity, the report found. The EU built record levels of solar panels and wind turbines in 2023, according to industry data, and was able to make more electricity from dams and nuclear power plants. Analysts say emissions overall are still falling too slowly.

S&P Global Commodity Insights has issued its latest report on the Top 10 Trends in Clean Energy Technology in 2024.

  1. Clean Energy Technology Investment to reach nearly US$ $800 billion in 2024 (up 10% to 20% from 2023) and $1 trillion by 2030. Solar will be the largest share of the additional spend and account for some 55% of the total investment. Onshore wind will be the second largest segment. The fastest-growing areas for new investments will be battery energy storage and electrolysis. CCUS and Hydrogen are both getting policy tailwinds.
  2. The average capex of clean energy technologies to decline by another 15%-20% by 2030. The combination of oversupply and falling raw material prices is rapidly driving down the costs of solar and batteries.
  3. Clean energy technology manufacturers are making decarbonization core to both products & and strategies. Two keys are 1) the use of low-carbon electricity, 2) a progressive reduction in materials consumption, and the use of lower carbon footprint materials.
  4. Oversupply is driving solar and storage manufacturers into a price war – compressing margins and jeopardizing localization efforts. Smaller manufacturers are likely to face negative gross margins.
  5. Expect record-high offshore wind capacity auctions in 2024 despite rising capital costs. More than 60 GW of new capacity is set to be auctioned in at least 17 different markets, enough to cover Poland’s total power demand.
  6. Western wind turbine giants face growing competition from the East. China’s recently announced turbine production surpasses that of Western counterparts by at least 30% in rated capacity, while the price gap has grown to nearly 70% between both groups. Expect the technology arms race and pricing pressure to continue.
  7. Expect higher global interest for low-carbon hydrogen as feedstock for ammonia, synthetic methane, and synthetic liquids. Aided by subsidies and driven by mandates, investment in hydrogen as a feedstock is now flowing.
  8. 2024 will be a milestone year for technology-based carbon dioxide removal (CDR). The EU is expected to adopt a carbon removal certification framework in 2024. A significant increase in projects can be expected in 2024, especially those aiming to capture biogenic and/or atmospheric CO2.
  9. Efforts to alleviate grid congestion and permitting constraints, a global phenomenon, will continue to streamline renewable power development. Global markets are expected to focus on two key means: 1) Higher investment in transmission and distribution (T&D) and storage. The biggest factor behind grid-connection bottlenecks is that T&D investment lags generation capacity investment. 2) Facilitation of development of other renewable technologies (e.g., offshore, geothermal), which have suffered from cost hikes and challenges in big interconnection and permitting.
  10. Transmission system operators (TSOs) will be required to assess flexibility needs from 2025, which will drive additional large-scale energy storage procurements.

We are meant to be in a golden era of investing in climate technology, but lots of investors have yet to reap the benefits. For example, the Invesco Solar ETF (TAN) is down more than 40% in the last year, with one of its largest holdings down 75% during the same period. SolarEdge’s revenue plummeted off a cliff in Q3 2023, and EnerSys (the market-leading battery provider) has also been losing businesses and firing workers in recent years and putting IRA central to its future growth. Hydrogen ETFs have lost 22% to 57% on a one-year timeline or lost 53% to 80% on a three-year timeline. Is it about timing, about company selection, or about missing something from the whole picture? Why? (Share your thoughts: Partner@WiseOcean.Tech)

 

AI and Climate

Sam Altman, OpenAI CEO, is engaged in discussions with key Middle Eastern investors and the Taiwanese chip giant TSMC to launch a new chip venture to design and build semiconductors for accelerating AI workloads as well as solving its power hunger. This initiative, aimed at reducing OpenAI’s dependence on Nvidia, involves collaboration with influential figures like Sheikh Tahnoon bin Zayed al-Nahyan of the UAE. Sheikh Tahnoon, a key figure in this negotiation, oversees substantial investment funds and entities in Abu Dhabi and is involved with G42, which collaborates with Microsoft and OpenAI. (One of the references)

AI is currently supercharging the development of data centers. DigitalBridge pointed out: “With the speed of adoption of Generative AI being the fastest on record. We believe the Generative AI opportunity will be at least as big as the cloud market today over the next 5-10 years… Access to digital infra in size, at the lowest total cost, is a key success factor. The new specialized AI chips and GPUs consume 2-3x the power of prior generations. There is going to be higher power density on a per rack basis with each rack filled with power-hungry GPUs drawing 40 or more Kw compared to traditional data center racks drawing 10kw or less.”

The number one operating cost of data centers is power! Generative AI makes the data center demand from megawatt to gigawatt scale. That adds extra stress to grids or is even prohibitive without some change. And, regulations will require disclosure of carbon emissions, sustainability is more than a buzzword. There is evidence that LPs are aware of the scale of data center opportunities, and its power hunger issue (also water). Using a combination of grid-sourced power and a portfolio of other renewables, small modular nuclear reactors, and even LNG and batteries are all possibilities. (more)

Quoted from Robeco: “To illustrate, Nvidia’s currently supplied A100 AI chip has a constant power consumption of roughly 400W per chip, whereas the power intake of its latest microchip, the H100, nearly doubles that – consuming 700W, similar to a microwave. If a full hyperscaler data center with an average of one million servers replaced its current CPU servers with these types of GPUs, the power needed would increase 4-5 times (1500MW) – equivalent to a nuclear power station! Current data centers are ill-equipped and in need of a revamp, upgrades will benefit the smart energy themes.” (more)

We might review AI chip innovations with sustainability impact later.

ClimateTech Year 2023 Review (Water and Industry are Emerging)

ClimateTech Year 2023 Review (Water and Industry are Emerging)

by | Jan 23, 2024 | Climate Tech, Venture Investment

Here below we select first-order signals from the 2023 annual report created by Net Zero Insights – a data and research platform for Climate Tech.

ClimateTech Year 2023 Review

In 2023, the two-year funding boom concluded with a significant 30% YoY drop. Despite a challenging year, the funding landscape holds promise with growth surpassing 2018 (2.76X) and 2020 (2X). Funding slows globally, except for Asia (+19% YOY). Europe is more resilient than North America.

Net Zero Insights uses a product innovation framework: “Breakthrough” innovation entails extensive research and development, characterized by long-term efforts and innovations in the proof-of-concept phase. “Adoption” concentrates on implementing and deploying established solutions, with a focus on improving metrics like cost or efficiency. While “physical” and “digital” represent what the company builds.

80% of all venture funding raised in the last 5 years across equity, debt, and grants went to adoption. Of this, 70% went to physical solutions, with most of it being non-dilutive. Note: Breakthrough solutions in the US secure over twice the funding compared to Europe. ($4.7B vs $2.5B)

The path to a net-zero economy demands readily deployable physical solutions and later-stage investors and lenders seem increasingly prepared to support it. First-of-a-Kind (FOAK) projects gained prominence last year but still pose challenges in definition and present funding complexities. In 2023, funding flowed towards adoption-focused innovations and later stages, indicating the increasing maturity of the market.

Circular economy, water, and industry are the only challenge areas to see positive YOY growth.

Energy attracts the largest investor base, while interest in industry and water is on the rise.

Overall, energy and transport are the most mature sub-sectors with the most funding and investors in the past 5 years. Circular economy, water, and industry are active emerging sub-sectors with obvious growth in seed and early-stage investments.

Companies headquartered in the US, China, and UK raised the most funding in 2023.

The top 10 most funded cities show a signal about the strength of cities.

In 2023, acquisitions reached a historic high (250 deals), predominantly in the energy, transport, and food & agriculture sectors while IPOs and SPACs returned to pre-pandemic levels. Acquisitions are gaining movement with corporate buyers.

Corporations exhibit resilience and sustained activity in Climate Tech M&A in the downturn. Corporate investments are concentrated in energy, food and agriculture, and transport, despite a slight decrease in funding in some of these sectors. Energy, industrials, and IT emerge as the predominant sectors among corporate acquirers, showcasing their strategic focus and interest in climate tech organizations.

Venture investors have a clear preference on digital types of solutions, funding them regardless of breakthrough or innovation.

Non-equity investors are not only intensifying their funding efforts but are also gaining prominence, particularly in later-stage deals.

Governmental funding is assuming an ever-growing significance in shaping current climate tech dynamics, particularly fostering breakthrough innovations.

Look at the number of investors, most investors favor adoption in energy and transport, with a busier-than-expected scene in physical breakthroughs.

The above is just extracting a part of the annual climate tech report, dive in for deeper insights in the report here from Net Zero Insights.

 

Are Water Innovations On Your Radar?

Water innovation startups may be underinvested, as water remains one of the most undervalued and underinvested resources in agrifood innovation, despite its intrinsic role in the food system. However, there are signs that this trend is changing, with more than 18 ocean-focused funds being launched in the last 18 months. Public investments are also on the rise, such as the Inflation Reduction Act in the US and initiatives from the World Bank. The Valuing Water Finance Initiative aims to convince the world’s largest corporates to treat water as a valuable resource, representing 64 institutional investors with a total of $9.8 trillion in assets under management. (source)

Privacy Policy