AI chips
Full-custom design for AI chips.
An AI chip spends its energy and area on a few circuits repeated very many times, which is where full-custom design does the most. As a TSMC direct account, we offer custom cells, library characterization, hand-crafted netlists, custom clock trees and manual place and route for AI inference, training, large-model, edge and vision chips.
- Process
- TSMC direct account; 7 nm FinFET design authorisation; 6 nm AI full-custom chip design
- Full-custom techniques
- Custom cells, library characterization and re-characterization, hand-crafted netlists, manual place and route, custom clock trees
- Compute blocks
- MAC arrays, multipliers, adder trees and pipeline registers
- Ways to work together
- Technical evaluation, optimisation plan, design consulting and full-custom design
AI chip types
The AI chips that full-custom techniques can serve.
The compute core of each of these chips has large, regular, repeating structures, which is where full-custom design does the most.
AI inference chips
These run trained models and answer requests, in the cloud or in devices. Buyers care most about cost, power and latency per request.
See manual place and route- Energy per inference
- Memory bandwidth
- Area efficiency of the MAC array
AI training chips
These fit model weights from data, usually in large clusters. Interconnect and memory matter as much as compute.
See full-custom design- Efficient floating-point tensor cores
- High-bandwidth memory and chip-to-chip links
- Power at rack level
Large language model inference chips
These generate tokens one by one for transformer models. The work splits into compute-heavy prefill and memory-bound decode.
See hand-crafted netlists- Energy and cost per token
- KV-cache memory bandwidth
- Time to first token
Edge and on-device AI chips
These run models inside devices or gateways, and increasingly run local large models.
See library characterization- A strict power budget
- LPDDR bandwidth
- Bill-of-materials cost
NPU chips
A neural processing unit is a block or chip built for neural-network arithmetic. It is now standard in phone and PC processors.
See manual place and route- Energy efficiency of the MAC array
- Sustained thermal limits
- Fit with the software stack
Vision AI chips
These combine image processing, video coding and an NPU for cameras and vision devices. Newer parts run transformers beside CNNs.
See library characterization- Power per video stream
- Heat in sealed housings
- Cost
Custom cloud AI ASICs (XPUs)
AI accelerators that cloud providers specify for their own workloads.
See full-custom design- Efficient tensor cores
- High-bandwidth memory
- Development schedule
AI PC and AI phone chips
The NPU, GPU and CPU share the model work, and these chips reach consumers in very large volumes.
See library characterization- Sustained performance and heat
- Battery life
- LPDDR bandwidth
Automotive and robotics chips
These handle perception and planning with real-time and safety needs. Humanoid robots and drones work on tighter power.
See full-custom design- Functional safety
- Heat
- Sensor interfaces
AI glasses and wearable chips
Very low-power chips for small batteries, often paired with an always-on co-processor.
See library characterization- Always-on power
- Heat in the frame
- Bill-of-materials cost
Speech AI chips
Always-on processors for wake words, voice commands and noise reduction.
See library characterization- Standby power
- Cost
- Model size
In-memory computing chips
These compute inside or beside memory arrays to reduce data movement, in digital and analog forms.
See custom cells- Adder trees and bit cells
- Capacity for large models
- Software support
How full-custom techniques apply to AI chips
Every technique in our methodology has a place in an AI chip.
An AI chip's compute core repeats register, combinational logic, register. It switches often and is regular in structure, which suits full-custom design.
Custom cells
Registers, latches, full adders and compressor cells designed for the MAC array and pipeline. The more registers, the greater the gain from custom sequential cells.
Multi-bit and dynamic sequential cells
Multi-bit registers share one clock pin across several bits to lower clock power, and the architecture decides between static or dynamic cells, and flip-flops or latches.
Library characterization and re-characterization
Custom cells are characterized and libraries extended to the real operating voltage, with models that sign off together with foundry cells.
Hand-crafted netlists
Multipliers, adder trees and pipeline stages in the RTL are mapped to custom cells, with regular naming for scripts and placement.
Manual place and route
A MAC array is placed in regular bit slices, shortening wires and reducing area. The larger the array, the greater the gain.
Custom clock trees
Clock distribution designed for a regular array, controlling skew and clock power.
Low-voltage design
Dynamic power follows the square of the voltage. Full-custom design lets a compute core run at lower voltage, raising compute per watt.
Mixed-threshold cells
With hand placement, one cell can mix threshold types, for deep optimisation of critical paths and power.
Power delivery and IR drop
The power network for dense compute is designed with the placement, with static and dynamic IR drop analysis.
Technical plans
Full-custom technical plans for NPUs.
From our full-custom NPU work for a top AI customer's AI processor, with customer details removed.
Two depths of customisation
Light customisation uses a custom cell library with synthesis and automated place and route. Deep customisation builds blocks from custom cells, for a greater gain and more independent sign-off.
MAC array customisation
Block-level custom design of fixed-point multipliers such as INT8, adder trees and pipeline registers. The larger the MAC array, the more customisation pays.
Evaluation against a baseline
With standard cells and automated place and route as the baseline, each of custom cells, hand-crafted netlists, manual clock trees and manual place and route is evaluated for area, power and speed.
Choice of sequential cells
Dynamic flip-flops, dynamic latches or static cells, chosen by architecture and clock scheme. A latch design can use three-phase non-overlapping clocks.
Block-level sign-off
Custom blocks sign off on their own, and top-level static timing analysis reads the block models, in step with the digital back-end flow.
Compute-core test
A compute core can include built-in functional test, with functional vectors that check its results.
Working with the front end
Clear RTL hierarchy and data flow, and accurate cell data for power simulation.
Low-voltage targets
Library characterization or re-characterization at the target voltage, with circuit simulation calibrating the timing and power models of custom cells.
Deliverables
A full-custom flow document, and reports on custom cells, gate-level netlists and place and route.
Ways to work together
Four stages, one step at a time.
Technical evaluation
You send your technical questions; we answer in detail and assess the case for full-custom design.
Optimisation plan
We analyse your design and propose directions, technical plans and the expected PPA improvement.
Design consulting
Flow guidance, design reviews and implementation support for your team.
Full-custom design
The full design service from optimised design to tape-out, with complete design data delivered.
Hot technologies in AI chips
- MAC array
- A compute array of many multiply-accumulate units, at the heart of NPUs and inference chips.
- HBM (high-bandwidth memory)
- Stacked DRAM packaged with the processor. Current products use HBM3E and HBM4, and HBM4E comes next.
- CoWoS and 2.5D packaging
- Advanced packaging that joins a processor and HBM on an interposer. Its capacity shapes the supply of high-end AI chips.
- 3D stacking and hybrid bonding
- Dies stacked vertically and joined by hybrid bonding, to shorten interconnect.
- Chiplets and UCIe
- A large chip divided into several dies. UCIe is an open standard for connecting them.
- Co-packaged optics (CPO)
- Optical engines packaged beside a switch or compute chip, to lower the power of long-reach links.
- In-memory computing
- Computing inside or beside memory arrays to reduce data movement.
- Low-precision quantisation
- Weights and activations held in fewer bits, such as INT8, INT4, FP8 or FP4, to reduce the cost of multipliers and memory.
- Transformer acceleration
- Datapaths optimised for attention and matrix arithmetic.
- KV cache and cost per token
- The KV cache of large model inference grows with context length, and cost per token has become a key measure of inference chips.
- Large SRAM
- Model weights or context held on the chip, to lower access energy and latency.
- Network on chip (NoC)
- The on-chip interconnect between compute units and memory.
- RISC-V for AI
- AI processors and accelerators based on the open instruction set.
- Low-voltage and near-threshold design
- A compute core run close to the transistor threshold voltage for energy efficiency. Low-voltage design relies on accurate library models.
- Backside power delivery
- The power network moved to the back of the wafer, to lower IR drop.
- GAA and 2 nm
- Gate-all-around transistors, which replace FinFET at the 2 nm generation.
Applications
AI chip applications
Questions about AI chips
What is the difference between inference chips and training chips?
Training chips fit model weights from data, so floating-point compute, memory bandwidth and cluster interconnect matter most. Inference chips run trained models, so cost, power and latency per request matter most.
How do LLM inference chips differ from AI inference chips?
An LLM inference chip is a kind of AI inference chip that generates tokens for transformer models. Cost per token and the memory bandwidth for the KV cache matter most.
What is an edge AI chip?
An edge or on-device AI chip runs models inside a device such as a phone, PC, pair of glasses or camera, without relying on the cloud. Its key limits are the power budget, LPDDR bandwidth and bill-of-materials cost.
What is an NPU chip?
An NPU is a processor built for neural-network arithmetic, with a MAC array and local memory at its core. It can be a separate chip or a block in an SoC.
What is an in-memory computing chip?
It computes inside or beside memory arrays, reducing the data moved between memory and compute units. It comes in digital and analog forms.
Which parts of an AI chip suit full-custom design?
Regular blocks that switch often suit it best, such as MAC arrays, multipliers, adder trees and pipeline registers. Control logic and interfaces stay on a standard tool flow.
How do library characterization and manual place and route help an AI chip?
Library characterization gives custom cells and low-voltage libraries accurate models, and manual place and route makes a MAC array more regular with shorter wires. Together they let a compute core run at lower voltage.
Which processes do you support?
We are a TSMC direct account, and our design work covers FinFET at 16 nm, 12 nm, 7 nm and 6 nm.
Related pages
Have an AI chip project?
Tell us the model, the compute target, the power budget and the target process, and we will reply.