• As AI workloads shift from training to inference, custom chips can cut costs and power use through fit-for-purpose design.
• For leaders in technology and industrial manufacturing, chip strategy now shapes AI economics, speed, resilience, and product design, not just performance.
• AI evolution opens opportunities for companies that can optimise AI semiconductors for customer requirements, combining efficiency, scalability, and innovation. 

Investment in artificial intelligence (AI) within the semiconductor industry is rapidly increasing, mainly driven by GPU-based AI chips. As custom AI semiconductors emerge, the question arises: Who will take the lead?

The answer is complex, as workloads shift from training to inference—that is, using AI models to get work done. With nearly half (49%) of technology leaders confirming AI is “fully integrated” into their core business strategies (PwC Pulse Survey, October 2024), the focus shifts to tailored semiconductor technologies. These chips, designed for efficiency, scalability, and specific customer needs, are driving the next wave of AI applications and redefining industry standards.

What is an AI chip, and why do we need it?

Unlike general-purpose chips, AI chips are designed to handle the unique demands of artificial intelligence workloads while being optimised for parallel processing. By leveraging low-precision arithmetic and faster memory access, they execute the repetitive computations required by AI algorithms rapidly and effectively. This makes them indispensable for real-time applications, from large-scale training in data centres to edge-based inference in robots, Internet of Things (IoT) systems, and smart devices.

AI chips can be categorised into training and inference types based on their main purpose. Training chips are used to train AI models by processing large datasets to optimise model parameters while balancing speed and accuracy. These chips prioritise raw computational power and scalability, enabling them to handle increasingly large and complex workloads required for developing AI models. Once a model is trained, an inference chip can be used to make predictions or decisions on new, unseen data in real time or in batch mode. Inference chips are designed for efficiency, focusing on low latency, low power consumption, and cost-effectiveness. Their compact design makes them ideal for deployment in edge and cloud environments.

The demand for such specialised chips is growing rapidly as AI services become increasingly complex and diverse. 

As AI services increase, a greater portion of AI workload in data centres will be dedicated to executing AI inference tasks.

Glenn Burm
Global Semiconductors Leader, PwC US

This transformation is fundamentally impacting AI chip development. The future lies in the ability to develop state-of-the-art AI chips for specific applications—be it by reducing costs, enabling new application possibilities, or efficiently leveraging supply chains. This development not only opens a competitive advantage but will become a decisive factor for companies that want to be successful in the dynamic world of AI technology in the long term.

Cost reduction strategy

A key challenge lies in the rising adoption costs of AI chips. Supply chain disruptions and surging demand have driven up production costs for GPUs, which have been the backbone of AI infrastructure. Escalating costs affect not only procurement but also exacerbate the scalability challenges of AI services.

A comparison highlights the difference: While conventional GPU-based AI chips remain indispensable for AI training due to their flexibility, they are often overpowered and energy-intensive for inference phases. Customised AI chips, designed for specific workloads, provide a cost-efficient alternative, with total cost of ownership around 40~60% less than for GPU-based AI chips.

Compounding the issue are the exponentially increasing operational costs associated with the growing use of AI services. Energy consumption is a particularly critical factor here: while high-performance GPU-based AI chips are renowned for computational capacity, they often exhibit higher energy consumption compared to dedicated inference-customised chips. AI inference chips, specifically optimised for certain AI tasks, achieve up to 50% greater power efficiency, offering the potential to save around 10–20% of the operational costs.

To manage costs in the long term, companies are increasingly turning to specialised inference solutions that significantly reduce both scaling costs and energy requirements. This trend is reflected in the growing demand for AI chips, which achieve a balance between performance and cost-effectiveness.

A fragmented and specialised supply chain emerges

The supply chain for semiconductor production has fundamentally changed and enables the rapid development of customised AI chips for specific applications. Previously, integrated device manufacturers (IDMs) such as Intel or Samsung dominated the entire production chain—from design to manufacturing. However, this vertically integrated model required very high levels of capital investment, resulting in high barriers to entry and relatively few options for customers looking to buy chips customised to their specific needs.

This model has been replaced by a fragmented and specialised supply chain. Today, specialised players such as design houses, foundries (for example, TSMC), and OSAT (Outsourced Semiconductor Assembly and Test) service providers each take over a part of the value chain.

Design houses play a central role in this new ecosystem, bridging the gap between design and production. They work closely with foundries and use advanced design tools to develop optimised chips for specific applications, such as AI inference at the edge. This close collaboration speeds up prototyping, reduces development costs, and enables chips to be customised to end-user requirements. The specialised supply chain opens new opportunities for innovation, reduces costs, and provides companies of all sizes with access to semiconductor development.

Enabling localised, efficient processing for edge computing

The rapid growth of artificial intelligence and the IoT has given rise to two main applications: Cloud AI, which processes data centrally in large data centres, and Edge AI, which analyses data directly at its source—on devices such as IoT systems, smartphones, or edge servers. Looking to differentiate in a competitive market, Edge AI offers a strategic advantage by enabling localised, efficient processing that complements centralised cloud solutions.

The number of IoT deployments will double from 16 billion in 2023 to 32 billion by 2030.

Kimihiko Uchimura
Partner, PwC Japan

Applications such as autonomous vehicles that process data from sensors and cameras are dependent on immediate reactions. Edge computing enables the ultra-fast processing needed for safety and efficiency in such applications.

Smart glasses, wearable devices that integrate technology into eyeglasses to provide augmented reality (AR) or other digital features, leverage edge computing to process audio, video, and command inputs directly on the device. This enables real-time functionalities such as hands-free photography, navigation. Another groundbreaking application is brain-computer interfaces (BCIs), systems that enable direct communication between the brain and external devices. Many companies are currently developing devices that interpret neural signals, enabling people with paralysis to control digital devices through their thoughts alone.

Edge computing complements the growing demands of AI and IoT by providing localised, efficient data processing that central systems cannot always achieve. Its ability to reduce latency and support real-time applications makes it a practical solution for scenarios requiring immediate responses and minimal power consumption. With continued advancements in customised AI chips, edge computing allows businesses to stay ahead in a rapidly evolving market by providing scalable, high-performance solutions for emerging applications.

Four priorities for a custom silicon strategy

AI is transforming the semiconductor industry and opening up opportunities for companies to develop customised AI chips to meet their specific needs. To succeed in this dynamic environment and secure competitive advantage, companies can set the following strategic priorities:

1. Customisation potential

Understanding how much of an AI chip can be customised is essential for companies aiming to optimise performance and align hardware with specific application needs. Evaluating the degree of adaptability within current AI chips allows organisations to identify opportunities for tailored functionality, whether through advanced design modifications or the integration of workload-specific features.

2. Capability internalisation

Building internal capabilities for AI chip development enables companies to reduce reliance on external resources and enhance strategic flexibility. This involves assessing existing expertise in areas like chip design and integration, identifying gaps, and establishing frameworks to internalise critical development processes that align with long-term innovation goals.

3. Strategic partnerships

Collaborating with design houses, foundries, and specialised providers such as OSAT services enables companies to access advanced design tools, state-of-the-art manufacturing technologies, and third-party assembly, testing, and packaging expertise. These partnerships accelerate the development of custom AI chips while fostering innovation through shared expertise. Integrating best-in-class components into chiplet designs and leveraging a flexible semiconductor network enhances supply chain efficiency and allows internal resources to focus on high-value innovation, resulting in highly optimised and scalable solutions.

4. Emerging technologies

Staying ahead in the competitive AI semiconductor market requires a continuous focus on investigating and adopting new technologies. From advanced materials and novel architectures to breakthrough fabrication techniques, identifying and assessing emerging innovations ensures that companies remain at the forefront of performance. By dedicating resources to exploring these technologies, organisations can position themselves to leverage new opportunities and maintain a competitive edge in the evolving market.

AI evolution opens massive opportunities for those who optimise AI semiconductors specifically to customer requirements, combining efficiency, scalability, and innovation. The next big player will not only deliver performance, but also close the gap between customised technology and cost efficiency. Whoever masters these challenges will dominate the market for the next generation of AI chips.