Halloween DealsAmazon USHalloween week deals: useful picks before month-endAmazon US: current deals, useful picks and tech finds.Check DealsPC HealthRecommendedCrashes, freezes, slowdowns? Check your PC nowSpot repairable issues before they interrupt work.Check PCHalloween DealsAmazon USOctober picks are moving - compare before checkoutAmazon US: current deals, useful picks and tech finds.See Picks×
Skip to content
World desk9 min

How AI Chips Are Designed, Manufactured, and Used in Data Centers

AI accelerators are designed around workloads, manufactured as silicon dies, packaged with memory and links, and deployed in data-center systems where software, power, cooling, and networking shape their usefulness.
Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

AI chips are designed around the workloads they must run, fabricated as patterned layers on silicon wafers, packaged with memory and interconnect, then installed in systems that supply power, cooling, networking, and software. Their usefulness depends on that whole chain—not just how quickly a chip can perform arithmetic.

What makes an AI chip a data-center component?

An AI accelerator is a processor designed to perform the operations used by neural networks efficiently. It may be a GPU or a purpose-built application-specific integrated circuit (ASIC), such as Google’s Tensor Processing Unit (TPU). But a chip is only one part of the machine that runs an AI workload. The processor must receive data from memory, communicate with other processors when a job is distributed, and operate inside a server and data center with suitable power, cooling, networking, and software.

As an Amazon Associate I earn from qualifying purchases.

That is why chip design is a systems problem. A fast compute unit can sit idle if data cannot reach it quickly enough. A chip with useful memory bandwidth still may not fit a model if its memory capacity is insufficient. And a collection of capable chips may not work as one large system without suitable links, software, and deployment infrastructure.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

The basic path is: define the workload and system requirements; design and verify the chip; manufacture and test its dies; package compute, memory, and links; install the resulting accelerators in servers or racks; and run training or inference software across them.

#1 Best Overall
NVD RTX PRO 6000 Blackwell Professional Workstation Edition Graphics Card for AI, Design, Simulation, Engineering - 96GB DDR7 ECC Memory - 4th Gen RT/5th Gen Tensor Core GPU - OEM Packaging
  • ATTENTION BUYERS: EXPORTING this item outside the U.S. requires strict compliance with the Bureau of Industry and Security (BIS) U.S. Export Administration Regulations (EAR). Non-compliance carries fines up to $1 MILLION PER VIOLATION AND UP TO 20 YEARS IMPRISONMENT for both the buyer and seller. To uphold our regulatory obligations, we strictly follow BIS "Know Your Customer" guidance by screening each order and immediately aborting transactions showing unresolved regulatory red flags. ***
  • [Double-Flow-Through Design] The RTX PRO 6000 Blackwell features a double-flow-through cooling design, optimizing efficiency and airflow to sustain peak performance under 600W power loads. | [5th Gen Tensor Cores] Deliver up to 3X the performance of the previous generation and support for FP4 precision for faster AI model processing times with reduced memory usage, enabling local fine-tuning of LLMs and generative AI | [4th Gen Ray Tracing Cores] Double the ray-triangle intersection rate of the previous generation to create photoreal, physically accurate scenes and immersive 3D designs with RTX Mega Geometry, which enables up to 100X more ray-traced triangles.
  • [PCIe Gen 5] Support for PCIe Gen 5 provides double the bandwidth of PCIe Gen 4, improving data-transfer speeds from CPU memory and unlocking faster performance for data-intensive tasks like AI, data science, and 3D modeling. | [GDDR7 Memory] With 96 GB of GPU memory and 1.8 TB ps bandwidth, it can tackle massive 3D and AI projects, fine-tune AI models locally, explore large-scale VR environments, and drive larger multi-app workflows.
  • [DisplayPort 2.1] Achieve unparalleled visual clarity and performance, driving high resolution displays at up to 8K at 240 Hz and 16K at 60 Hz. Increased bandwidth enables seamless multi-monitor setups while HDR and higher color depth support ensures superior color accuracy for precision work, such as video editing, 3D design, and live broadcasting.
  • [NVIDIA Blackwell Streaming Multiprocessor] The new SM features increased processing throughput, and new neural shaders that integrate neural networks inside of programmable shaders | DLSS 4: Multi Frame Generation ensures ultra-smooth frame pacing for lifelike simulations. | [Universal MIG] Divide a single RTX PRO 6000 Blackwell into multiple isolated instances, each with dedicated resources, allowing for concurrent execution of multiple workloads, optimized GPU utilization, and secure isolation of different applications or users. [WARRANTY] 3 YR Manufacturer's Warranty. Bulk OEM Packaging. Retail Packaging is NOT included.

How does the target workload shape chip design?

Design begins with the work the system is expected to do. Training and inference both involve neural-network computation, but they place different demands on computing resources and data movement. Designers therefore make choices about compute units, supported numerical formats, memory hierarchy, and communication between parts of a chip and between chips. The intended software and deployment scale influence those decisions too.

Compute is only one design target

A chip’s arithmetic capability matters, but so do the amount of data it can keep close to its compute units and the rate at which it can move data. Designers have to balance compute, memory capacity, memory bandwidth, and communication. These factors determine whether the processors can be kept productively occupied for the target workload; a headline throughput figure alone does not establish how quickly a real model will run.

Interconnect affects more than speed

Links let different dies or processors exchange data. They can influence how a system is assembled and scaled, as well as performance, energy use, reliability, and manufacturing yield. NVIDIA describes NVLink-C2C as providing coherent, high-bandwidth transfers between processors. NVIDIA also claims that, for its chips, NVLink-C2C provides up to 6× the energy efficiency and 3.5× the area efficiency of a PCIe Gen 6 PHY. Those are NVIDIA’s comparisons, not independent measurements or a universal result for every system.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Design is verified before fabrication

A chip design has to be checked before it is turned into physical silicon. Verification helps establish that the design behaves as intended, while software tools also support the design and manufacturing workflow. NVIDIA’s semiconductor overview describes computational lithography as a way to optimize photomasks to counteract patterning issues. This is one example of how computation supports the process of making chips; it does not replace physical fabrication.

What happens when a chip design is manufactured?

Once a design has been completed and verified, a foundry uses it to make dies on silicon wafers. The general process involves forming repeated layers and patterns, processing those patterns to create device structures and wiring, testing the wafer, and cutting it into individual dies. The exact process varies by product and foundry; this is an overview, not a complete fabrication recipe.

  1. Prepare the design for manufacturing. The verified design and its manufacturing data guide the patterning process. Computational lithography can help optimize photomasks to address issues that arise when patterns are transferred to a wafer.
  2. Build and pattern layers. The wafer goes through repeated processing to create layers and patterns. NVIDIA’s account of Blackwell production names layering, patterning, and etching among the steps.
  3. Test and dice the wafer. The wafer is tested and separated into individual dies. NVIDIA’s Blackwell account also names dicing as part of production. A die is the piece of silicon that can proceed to packaging; it is not yet a complete data-center system.
  4. Package and integrate. Suitable dies are assembled with the components and connections required by the product. The packaged device is then integrated into a board, server, or larger system.

Not every die necessarily becomes a usable product. NVIDIA notes that interconnect layout can affect fabrication yield, illustrating why manufacturing concerns feed back into design choices. The available information does not establish a yield percentage for the examples discussed here.

Why do multiple dies, memory, and packaging matter?

Packaging brings components into a physical assembly. Depending on the product, it may connect multiple compute dies and memory closely enough to support the intended data flows. This is not one universal packaging recipe: different accelerator designs make different choices about dies, memory, and system links.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
Rank #2
NVIDIA RTX PRO 4000 Blackwell Graphics Card - 24GB GDDR7 ECC Memory, PCIe 5.0 x16, 4X DisplayPort 2.1b, Single Slot Full Height AI Workstation GPU, Retail Packaging
  • Professional GPU with Blackwell Architecture
  • Blackwell Architecture
  • 24GB GDDR7 with PCIe 5.0 & Ray Tracing
  • AI Workstation

NVIDIA says its Blackwell GPU combines two reticle-limited dies connected by a chip-to-chip interconnect. The connection allows the two dies to function as parts of one GPU design. Google’s Ironwood TPU, by contrast, is described with high-bandwidth memory (HBM) stacks associated with each chip. The examples show why packaging is an architectural issue: compute needs a practical route to data, and the way components are connected helps define the module and the larger system.

Memory capacity answers how much data can be held in the accelerator’s memory; bandwidth describes how quickly data can move between memory and compute. Neither number alone explains performance, but both help determine whether a workload’s data can be served effectively. Moving beyond one accelerator adds another requirement: chips must communicate with each other over the scale-up links inside a system and, for larger deployments, through the wider system network.

How do accelerators become servers and racks?

A packaged accelerator is installed in a larger platform. Servers provide host processors and other infrastructure; systems also need power delivery, cooling, network connections, and software to coordinate work. When many accelerators are combined, those resources and the links among them become part of the design problem, not optional surroundings.

NVIDIA describes GB200 NVL72 as a rack-scale system built from GB200 Grace Blackwell Superchips. Google describes Ironwood TPU pods and scaling multiple superpods through a data-center network. These illustrate different system approaches: a rack-scale GPU system on one hand and TPU pods and superpods on the other. They should be compared by architecture, scaling approach, software, availability, and workload fit—not by assuming that a product name or a single chip specification determines which is faster.

Free tools Windows power users keep installed

One-click scans. No signup required.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Power and cooling place practical limits on deployment. A buyer or operator needs to consider the complete system’s requirements, including how its power is delivered and heat is removed, as well as how it connects to other servers. The cited product descriptions establish rack- and pod-scale designs, but do not supply a common power, cooling, or cost comparison for those systems.

How are AI chips used for training and inference?

Training

Training runs use data to adjust a model’s parameters. Because the computation and data movement have to be coordinated across the chosen hardware, the software stack and interconnect matter alongside each accelerator’s local resources. Larger deployments can distribute work across multiple accelerators and servers, making the network and system design relevant to completing the job.

Inference

Inference runs a trained model to produce outputs from new inputs. It can be performed on general-purpose processors or accelerators, and purpose-built ASICs have been used for this role in data centers. The first-generation TPU is a historical example: Google researchers described it as a custom ASIC deployed in Google data centers for neural-network inference beginning in 2015. Its published performance figure belongs to that first-generation device and should not be treated as a current accelerator benchmark.

Rank #3
PNY NVIDIA RTX PRO 2000 Graphics Card | 16 GB, GDDR7
  • API Supported: DirectX 12 for advanced graphics rendering and gaming performance
  • API Supported: OpenGL 4.6 for professional graphics applications and cross-platform compatibility
  • API Supported: Vulkan 1.4 for high-efficiency graphics and compute operations
  • API Supported: OpenCL 3.0 for parallel computing and heterogeneous system programming
  • Number of Monitors Supported: 4 displays simultaneously for enhanced productivity and multitasking

Software completes the path from silicon to useful work

Hardware must be usable through software that can run the workload and coordinate the system. Google frames Ironwood as system-level co-design rather than an isolated chip: the accelerator, software, and deployment environment are considered together. In practice, choosing an accelerator therefore involves checking that the needed software supports it and that the workload can use its memory and scaling characteristics. The cited specifications do not establish identical software support across vendors.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

What do published AI-chip specifications tell you?

The figures below describe different devices and system levels, and come from different sources and dates. They are useful for understanding design choices, not for ranking products: they are not results from a controlled, head-to-head benchmark.

Example Published specification What it describes
NVIDIA Blackwell GPU NVIDIA reports 208 billion transistors, a custom TSMC 4NP process, two reticle-limited dies, and a 10 TB/s chip-to-chip interconnect. Manufacturer-reported architecture specifications for Blackwell; they are not independent performance measurements.
NVIDIA GB200 NVL72 NVIDIA describes 72 Blackwell GPUs and 36 Grace CPUs in a rack-scale system, built from 36 GB200 Grace Blackwell Superchips. A vendor-described system configuration, not a single-chip specification.
Google Ironwood TPU Google Cloud reports eight HBM3E stacks per chip, 192 GiB of memory capacity, and peak bandwidth of 7.4 TB/s. Google-published specifications for this TPU generation; not an independently measured comparison with Blackwell.
First-generation Google TPU The 2017 paper reports a 65,536-unit 8-bit MAC matrix multiply unit, peak throughput of 92 TeraOps/second (TOPS), and 28 MiB of software-managed on-chip memory. A historical inference ASIC specification with its own generation and metric; not comparable as a current-system benchmark.

The 2017 paper puts the TPU figure this way: “The heart of the TPU is a 65,536 8-bit MAC matrix multiply unit that offers a peak throughput of 92 TeraOps/second (TOPS) and a large (28 MiB) software-managed on-chip memory.” The quote describes that paper’s first-generation TPU, not the Ironwood chip. The distinction matters because product generations, numerical formats, workload conditions, and metrics differ.

How should you evaluate an AI accelerator for a real workload?

Start with the job and deployment constraints rather than selecting by peak arithmetic figures. A useful evaluation asks whether the accelerator can run the required software, hold or access the workload’s data, and communicate effectively at the scale the system needs. The available product descriptions do not provide a controlled current GPU-versus-TPU benchmark, so they cannot establish a general performance winner.

  • Workload: Is the priority training, inference, or both? Which numerical formats and operations does the workload use?
  • Software: Are the frameworks and deployment tools needed by the workload supported on the target platform?
  • Memory: Does the system offer enough capacity, and is its bandwidth appropriate for the workload’s data movement?
  • Interconnect and scale: How do accelerators communicate within a module or rack, and how does the deployment connect larger groups of systems?
  • Operations: Can the data center provide the required power, cooling, networking, and system management?
  • Access model: Will the organization operate its own infrastructure or use a cloud service? For cloud deployments, confirm the current region, configuration, and commercial terms with the provider; an architecture description alone does not establish availability or pricing.

These questions help avoid common comparison errors. A peak-throughput number does not predict application performance by itself; a memory bandwidth figure does not tell you capacity; and a rack or pod description does not provide a like-for-like benchmark. Compare platforms on the same workload and operating conditions when measured results are available, and keep vendor specifications distinct from independent measurements.

What’s actually slowing this PC down?

Pick the symptom - the matching free tool is one click away.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Where the evidence is strongest—and where it stops

NVIDIA’s Blackwell and GB200 descriptions and Google Cloud’s Ironwood page provide vendor-published, generation-specific specifications. They explain product architecture but are not independent evaluations. The 2017 TPU paper is a primary historical source for the first-generation device and its inference role; its figures should remain attached to that product and context.

Those sources establish how design, fabrication, packaging, memory, and system scaling fit together, but they do not provide a controlled comparison of current GPU and TPU systems, a shared measure of power or cooling, or current cloud pricing and regional availability. Any buying or deployment decision needs current platform details and workload-specific measurements beyond a list of headline specifications.

Quick Recap

Bestseller No. 2
NVIDIA RTX PRO 4000 Blackwell Graphics Card - 24GB GDDR7 ECC Memory, PCIe 5.0 x16, 4X DisplayPort 2.1b, Single Slot Full Height AI Workstation GPU, Retail Packaging
NVIDIA RTX PRO 4000 Blackwell Graphics Card - 24GB GDDR7 ECC Memory, PCIe 5.0 x16, 4X DisplayPort 2.1b, Single Slot Full Height AI Workstation GPU, Retail Packaging
Professional GPU with Blackwell Architecture; Blackwell Architecture; 24GB GDDR7 with PCIe 5.0 & Ray Tracing
$3,134.00
Bestseller No. 3
PNY NVIDIA RTX PRO 2000 Graphics Card | 16 GB, GDDR7
PNY NVIDIA RTX PRO 2000 Graphics Card | 16 GB, GDDR7
API Supported: DirectX 12 for advanced graphics rendering and gaming performance; API Supported: Vulkan 1.4 for high-efficiency graphics and compute operations
$1,600.64

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

Leave a Reply

Your email address will not be published. Required fields are marked *

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

More from the Wire

  1. World desk4 min
    How to Spot an AI Voice Scam Before Sending MoneyDon’t rely on how a caller sounds. Pause, call back through a known number, and verify the emergency with another trusted person before sending money.
  2. Mountain View desk4 min
    Google’s SynthID Detector: How to Check AI-Generated Images, Video and AudioGoogle’s SynthID Detector looks for an embedded watermark in supported images, video and audio. Here is what its results do—and do not—show.
  3. Redmond desk20 min
    How to create a link to File or Folder in Windows 11Windows 11 gives you several ways to point to a file or folder without moving or duplicating it. You can create a desktop shortcut,…
Recommended PC Tool
Recommended PC Tool
Outdated Drivers Are Slowing You DownFree scan - exact matches
PC Slower Than It Used to Be?Free scan - under a minute

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.