What is the dif­fer­ence between CUDA, Tensor and RT Cores?

Many modern NVIDIA GeForce and RTX GPUs combine three spe­cial­ized pro­cess­ing units, or hardware blocks, each designed for specific tasks. CUDA Cores serve as general-purpose computing units for tra­di­tion­al graphics pro­cess­ing and parallel cal­cu­la­tions. Tensor Cores are dedicated AI ac­cel­er­a­tors that perform matrix op­er­a­tions used in deep learning, while RT Cores are spe­cial­ized hardware for cal­cu­lat­ing light rays in ray tracing, such as realistic shadows and re­flec­tions.

Key Takeaways ⓘ

The key points at a glance:

• CUDA Cores are the versatile pro­cess­ing units of an NVIDIA GPU and handle tra­di­tion­al graphics work as well as general parallel cal­cu­la­tions.
• Tensor Cores ac­cel­er­ate matrix and tensor op­er­a­tions, which makes them the basis for training and running inference on AI models.
• RT Cores calculate ray tracing in hardware, above all BVH traversal and ray–triangle in­ter­sec­tion tests.
• Other man­u­fac­tur­ers follow the same principle under different names, such as Compute Units and Ray Ac­cel­er­a­tors at AMD or Xe-Cores and XMX Engines at Intel.
• Working together, the three core types divide the labor: the CUDA Cores render the image, the RT Cores supply the ray in­ter­sec­tions and the Tensor Cores refine the result with DLSS.

CUDA, Tensor and RT Cores compared

Many modern NVIDIA GPUs no longer rely on just one type of pro­cess­ing unit. Instead, they combine several spe­cial­ized hardware blocks on a single chip, with each one optimized for specific tasks.

This spe­cial­ized design allows the same GPU to handle a wide range of workloads: rendering games, pro­cess­ing AI models, sup­port­ing video editing and creating more realistic lighting and shadows through ray tracing. Each pro­cess­ing unit has its own role rather than handling every task equally.

The key tech­nolo­gies are CUDA Cores, Tensor Cores and RT Cores:

• CUDA Cores are versatile pro­cess­ing units designed for a wide range of tra­di­tion­al graphics and computing tasks.
• Tensor Cores spe­cial­ize in matrix cal­cu­la­tions, which are essential for AI and machine learning. They provide the spe­cial­ized pro­cess­ing power required for modern AI training and inference workloads.
• RT Cores ac­cel­er­ate dedicated ray tracing cal­cu­la­tions, such as de­ter­min­ing how light rays interact with objects in a 3D scene.

GPU tech­nolo­gies from different man­u­fac­tur­ers

CUDA Cores, Tensor Cores and RT Cores are terms specific to NVIDIA GPUs. Other GPU man­u­fac­tur­ers, such as AMD and Intel, use similar concepts but apply different names and technical im­ple­men­ta­tions. AMD, for example, uses terms such as Compute Units, AI Ac­cel­er­a­tors and Ray Ac­cel­er­a­tors (also known as Ray­trac­ing Ac­cel­er­a­tors). Intel refers to tech­nolo­gies including Xe-Cores, XMX Engines and Ray Tracing Units. The exact ar­chi­tec­ture varies between man­u­fac­tur­ers, but the general principle of dividing GPU workloads between spe­cial­ized pro­cess­ing units is similar.

What are CUDA Cores and how do they work?

CUDA Cores are the tra­di­tion­al pro­cess­ing units found in NVIDIA GPUs. They are grouped together in larger struc­tures known as streaming mul­ti­proces­sors and perform many of the GPU’s general-purpose cal­cu­la­tions.

They can be seen as the all-rounders of a GPU: flexible pro­cess­ing units capable of handling a wide range of standard math­e­mat­i­cal op­er­a­tions. They calculate shaders in games, process image data and run physics sim­u­la­tions, among other tasks. Their key strength is the parallel pro­cess­ing of floating-point numbers, par­tic­u­lar­ly in single-precision format (FP32). For this reason, CUDA Cores have tra­di­tion­al­ly been as­so­ci­at­ed with FP32 pro­cess­ing. However, since the RTX Blackwell gen­er­a­tion, this de­scrip­tion is less precise, as shader cores support both FP32 and INT32 op­er­a­tions. They can execute either floating-point or integer cal­cu­la­tions in each clock cycle.

The basis of CUDA’s ar­chi­tec­ture is SIMT (Single In­struc­tion, Multiple Threads). This execution model is similar to SIMD (Single In­struc­tion, Multiple Data), but it is designed for large numbers of threads running si­mul­ta­ne­ous­ly. In this approach, 32 threads are grouped together into a unit called a warp.

Threads within a warp typically execute the same in­struc­tion while working with their own in­de­pen­dent data and registers. For example, when a GPU needs to color a large number of pixels, it does not process the in­struc­tion “multiply the color value by 0.5” one pixel at a time. Instead, it dis­trib­utes the workload across many parallel threads and warps, allowing similar op­er­a­tions to be carried out si­mul­ta­ne­ous­ly. This makes GPUs par­tic­u­lar­ly efficient when pro­cess­ing large, con­sis­tent data sets.

Typical tasks for CUDA Cores

CUDA Cores are par­tic­u­lar­ly effective for workloads where many similar cal­cu­la­tions can be performed si­mul­ta­ne­ous­ly. Typical examples include:

• classic rendering and shading in games
• vertex, pixel and compute shader op­er­a­tions
• image pro­cess­ing, filters and post-pro­cess­ing effects
• physics sim­u­la­tions, such as particle, fluid and collision cal­cu­la­tions
• sci­en­tif­ic cal­cu­la­tions and numerical sim­u­la­tions
• general GPGPU tasks, where GPUs are used for computing beyond tra­di­tion­al graphics workloads
• data pre-pro­cess­ing for AI workflows
• parts of media pro­cess­ing pipelines, such as image and video pre-pro­cess­ing

IONOS CLOUD AI Model Hub

Your gateway to a secure mul­ti­modal AI platform

• One platform for the most powerful AI models
• Fair and trans­par­ent token-based pricing
• No vendor lock-in with open source

Try now

What are Tensor Cores and how do they ac­cel­er­ate AI?

With the rise of deep learning, GPU re­quire­ments also increased. The reason is that neural networks rely heavily on large numbers of matrix cal­cu­la­tions. When training and running inference on large AI models, enormous sets of numbers must be mul­ti­plied and added together re­peat­ed­ly.

CUDA Cores can perform these cal­cu­la­tions in principle. However, while they are highly flexible, they are not specif­i­cal­ly optimized for pro­cess­ing large matrix op­er­a­tions at maximum speed. This is where Tensor Cores provide an advantage. These spe­cial­ized pro­cess­ing units in modern NVIDIA GPUs are designed to handle matrix and tensor op­er­a­tions much more ef­fi­cient­ly than general-purpose compute units.

Tensor Cores ac­cel­er­ate a process known as GEMM (General Matrix Multiply), or general matrix–matrix mul­ti­pli­ca­tion. Instead of pro­cess­ing in­di­vid­ual cal­cu­la­tions sep­a­rate­ly, they can handle small matrix blocks at once by combining multiple mul­ti­pli­ca­tion and addition op­er­a­tions into dedicated hardware in­struc­tions.

A key factor behind the per­for­mance of Tensor Cores is mixed precision, which involves using different numerical formats for different parts of a cal­cu­la­tion. Depending on the GPU gen­er­a­tion and workload, Tensor Cores support formats such as TF32, BF16, FP16, FP8, FP6, FP4 and INT8. Using lower-precision formats can sig­nif­i­cant­ly increase through­put and reduce memory re­quire­ments.

The benefit is that smaller number formats require less memory, can be trans­ferred more ef­fi­cient­ly and allow higher pro­cess­ing through­put. At the same time, important in­ter­me­di­ate cal­cu­la­tions can be ac­cu­mu­lat­ed at higher precision to reduce the impact of rounding errors. For many AI workloads, this provides an effective balance: cal­cu­la­tions run sig­nif­i­cant­ly faster and more ef­fi­cient­ly while main­tain­ing largely com­pa­ra­ble results.

Typical tasks for Tensor Cores

Tensor Cores are primarily used for workloads that involve large numbers of matrix and tensor cal­cu­la­tions. Typical ap­pli­ca­tions include:

• training deep neural networks
• inference for large language models and gen­er­a­tive AI systems
• trans­former models, diffusion models and rec­om­men­da­tion systems
• AI-powered image upscaling and frame gen­er­a­tion, such as DLSS
• speech, image and object recog­ni­tion
• low-precision or quantized inference using formats such as INT8, FP8 or FP4
• sci­en­tif­ic ap­pli­ca­tions involving extensive linear algebra cal­cu­la­tions

What are RT Cores and how do they ac­cel­er­ate ray tracing?

The third spe­cial­ized unit, RT Cores, tackles one of the most com­pu­ta­tion­al­ly demanding graphics tasks: hardware-ac­cel­er­at­ed ray tracing. With ray tracing, light rays are traced through a scene to simulate shadows, re­flec­tions, re­frac­tions and global il­lu­mi­na­tion in a more phys­i­cal­ly accurate way. The com­pu­ta­tion­al workload is enormous because, for every in­di­vid­ual ray, the system must determine which geometric object it en­coun­ters first.

To make this process more efficient, ray tracing systems use so-called Bounding Volume Hi­er­ar­chies (BVH). This is a search structure that organizes a 3D scene using nested bounding boxes. Large sections of the scene are enclosed in broad boxes first. Only when a ray in­ter­sects one of these boxes does the system examine the smaller areas or in­di­vid­ual triangles inside. This means the GPU does not need to test every triangle sep­a­rate­ly. Instead, it can quickly eliminate ir­rel­e­vant parts of the scene and pro­gres­sive­ly narrow the search down to the objects a ray may actually hit.

This is where RT Cores come into play. They primarily ac­cel­er­ate two key tasks:

• BVH traversal: nav­i­gat­ing through the bounding box structure
• ray–triangle in­ter­sec­tion tests: de­ter­min­ing whether and where a ray in­ter­sects a specific polygon

The main advantage is that these cal­cu­la­tions are handled directly by dedicated RT Core hardware rather than using CUDA Cores. This reduces the workload on the general-purpose shader units, allowing them to focus on tasks such as shading and material cal­cu­la­tions at the same time. Without this spe­cial­ized pro­cess­ing, real-time ray tracing at current per­for­mance levels would not be possible.

Typical tasks for RT Cores

RT Cores are used whenever large numbers of rays need to be traced through a 3D scene and tested against its geometry. Typical ap­pli­ca­tions include:

• real-time ray tracing in games, such as more realistic re­flec­tions, shadows and ambient occlusion
• path tracing and global il­lu­mi­na­tion for highly realistic 3D en­vi­ron­ments
• rendering and vi­su­al­iza­tion in ar­chi­tec­ture, product design, film and animation
• phys­i­cal­ly based rendering, where light, materials and surfaces are simulated as re­al­is­ti­cal­ly as possible
• fast searches through 3D scenes using struc­tures such as BVHs
• sci­en­tif­ic vi­su­al­iza­tion involving large datasets rendered with ray tracing

How NVIDIA’s CUDA Cores, Tensor Cores and RT Cores work together

The strength of modern NVIDIA GPUs does not come from in­di­vid­ual pro­cess­ing units alone, but from their division of labor. CUDA Cores, Tensor Cores and RT Cores perform different tasks and work together to improve per­for­mance. This in­ter­ac­tion can be il­lus­trat­ed using a modern game scene with ray tracing and AI upscaling:

1. First, the CUDA Cores, together with other spe­cial­ized units, handle many tra­di­tion­al graphics tasks. These include geometry pro­cess­ing, ras­ter­i­za­tion, shaders, textures and material effects. They perform much of the core image pro­cess­ing required to create the final scene.
2. When ray tracing is enabled, the RT Cores take over specific cal­cu­la­tions. They ac­cel­er­ate the process of finding in­ter­sec­tions between light rays and objects in the scene by tra­vers­ing BVH struc­tures and de­ter­min­ing which triangles a ray hits. This in­for­ma­tion is then used for effects such as realistic shadows, re­flec­tions and lighting.
3. Finally, the Tensor Cores use AI-based methods to enhance the rendered image. With DLSS (Deep Learning Super Sampling), for example, a lower-res­o­lu­tion image is upscaled and improved using a neural network, producing a higher-quality result without requiring every pixel to be cal­cu­lat­ed tra­di­tion­al­ly. On GeForce RTX 50 Series GPUs, DLSS 4.5 adds Dynamic Multi Frame Gen­er­a­tion and a 6X mode that can generate up to five ad­di­tion­al frames for every rendered frame. DLSS 5 goes further with 3D-Guided Neural Rendering, which uses the Tensor Cores to improve lighting and material detail in real time.

By dividing tasks in this way, each pro­cess­ing unit handles the workloads it is best suited for, reducing com­pu­ta­tion time and improving overall ef­fi­cien­cy.

CUDA, Tensor and RT Cores: direct com­par­i­son

The three core types in modern NVIDIA GPUs mainly differ in their areas of re­spon­si­bil­i­ty, the math­e­mat­i­cal op­er­a­tions they are designed to ac­cel­er­ate and the workloads they are optimized for: