Every chapter of this book so far has treated address translation as something a single kind of agent performs. A CPU core walks its page tables, caches the result in a TLB, and relies on the operating system to tell it when a cached translation has become stale. Chapter 5 extended the picture to I/O devices behind an IOMMU, Chapter 10 added PCIe Address Translation Services and the Page Request Interface, and Chapter 11 examined how GPU drivers present a unified virtual address space to applications. Each of those chapters described a translator. None of them described the software that keeps several translators agreeing with one another while the process they serve keeps changing its own memory map.
That missing layer is the subject of this chapter. On a modern heterogeneous system a single process may have its virtual addresses translated by the CPU MMU, by an IOMMU acting on behalf of a PCIe accelerator, by a translation cache inside the accelerator itself, and by a GPU's private page tables that the GPU driver builds and maintains. When the process calls munmap(), when the kernel migrates a page to another NUMA node, when copy-on-write breaks a shared page after fork(), or when the reclaim path decides a page should be swapped out, every one of those translators must stop using the old translation before the old physical page can be reused. If any one of them keeps a stale entry, a device can read another process's data or silently corrupt freed memory. The hardware mechanisms that make this possible already exist and were described in earlier chapters. What coordinates them is the Linux kernel.
Linux provides two distinct frameworks for this coordination, and the central argument of this chapter is that they solve the problem in structurally different ways. Heterogeneous Memory Management (HMM) lets a device driver mirror a process's CPU page tables into a separate device page table, keeps that mirror synchronized through MMU notifiers, and adds a class of device-private memory to which anonymous pages can be migrated. Shared Virtual Addressing (SVA) instead lets the IOMMU walk the CPU's own page tables directly, tagging device traffic with a Process Address Space ID (PASID) so that the device sees exactly the address space the CPU sees. HMM gives the driver ownership of a second set of translations. SVA gives the IOMMU a reference to the first set. Every difference in fault handling, invalidation ordering, and failure behaviour discussed in the remainder of the chapter follows from that one distinction.
All kernel interfaces cited in this chapter were checked against the Linux v7.2.9 stable tree (released 3 October 2026), the latest stable release when the chapter was written. Where the v7.3 development series, at release candidate 6 at the time of writing, changes an interface, the text says so explicitly. Hardware mechanisms are cited against the Intel VT-d specification (revision 5.2), the AMD I/O Virtualization Technology specification (publication 48882), the Arm SMMUv3 architecture specification (IHI 0070), and the ratified RISC-V IOMMU specification (version 1.0). Where Linux does not yet implement a mechanism that a specification defines, the chapter states that gap rather than describing behaviour that no released kernel exhibits.
Readers who arrive here directly should know which earlier material the chapter assumes. PASID formats and the Intel scalable-mode PASID table are covered in Chapter 5 (Section 5.3.4), as are AMD guest page tables (Section 5.4.6), the Arm SMMU Page Request Interface (Section 5.5.6), and the trade-off between shared and separate IOMMU page tables (Section 5.6.4). The PCIe ATS translation request and completion protocol, together with PRI page-request messages, is covered in Chapter 10 (Section 10.3). CPU TLB shootdown, which every invalidation path in this chapter begins with or depends on, is covered in Chapter 4 (Section 4.7). NVIDIA's Unified Virtual Memory driver, the best-known managed-memory implementation, is covered in Chapter 11 (Section 11.3). CXL as a memory fabric is covered in Chapter 19. This chapter refers to those sections rather than repeating them.
Several adjacent topics are deliberately excluded. The chapter does not re-teach PASID table formats, ATS packet layouts, or the PRI state machine, all of which earlier chapters describe in detail. It does not survey Apple's unified memory architecture, which Chapter 11 covers and which does not use either Linux framework. It does not provide a general overview of CXL, and it limits its discussion of coherent fabrics to the specific question of what cache-coherent device memory changes for HMM and SVA. It is not a tutorial for the CUDA or HIP unified memory APIs; those APIs appear only where their behaviour depends on the kernel mechanisms described here. Finally, it makes no claims about the internal page-table organization of proprietary accelerators. Where a vendor has not published how its hardware behaves, the chapter says so and confines itself to what the in-tree Linux driver code, a published specification, or a peer-reviewed measurement establishes.
Before examining the kernel machinery it is worth being precise about the three programming models that sit on top of it, because each model assigns ownership of translations differently and therefore fails in a different way. The models are not vendor-specific. NVIDIA's CUDA Programming Guide (version 13.4.2, Section 4.1) organizes its own description of unified memory around the same distinction, and AMD's ROCm and Intel's oneAPI stacks expose equivalent choices.
In the copy model the device and the CPU have separate address spaces. The application allocates a buffer in device memory, allocates a second buffer in host memory, and moves data between them with explicit copy operations. When the device performs direct memory access to host memory, the driver maps the host buffer through the kernel DMA API, which installs entries in the IOMMU page table for that device and returns an I/O virtual address (IOVA) that has no relationship to the process's virtual address for the same bytes. Because the IOMMU mapping is built from physical pages that were resolved at mapping time, the driver must pin those pages so that the kernel cannot migrate, swap, or reclaim them while the mapping exists. Translation ownership is simple: the CPU MMU owns the process view, the IOMMU owns the device's IOVA view, and the two are reconciled only when the driver creates or destroys a mapping. The model fails when data structures contain pointers. A linked list or a tree built in host memory cannot be traversed by the device, because every pointer in it is a CPU virtual address that means nothing in the IOVA space. The application must either serialize the structure or rebuild it with device-side offsets.
In the managed model the application allocates memory through a vendor API such as cudaMallocManaged() or hipMallocManaged(). The returned pointer is valid on both the CPU and the device, and the driver moves pages between host and device memory on demand. Chapter 11 (Section 11.3) describes NVIDIA's implementation, in which a GPU access to a non-resident managed page raises a replayable fault that the driver services by migrating the page and updating the GPU page table. Translation ownership in this model belongs to the driver. The driver allocates the virtual range, decides where each page lives, and maintains both the CPU mapping and the device mapping for it. The model fails at its boundary: memory obtained from malloc(), from the stack, from a file mapping, or from a third-party library is not managed memory, and passing such a pointer to the device either faults or requires the copy model. Allen and Ge's analysis of NVIDIA's UVM driver (SC21) showed that the host-side driver and operating-system components of fault servicing are a significant share of the cost of this model, which matters for the comparison in Section 22.11.
In the system-allocated model any valid pointer in the process can be dereferenced by the device, including memory from malloc(), mmap(), and the stack. This is the model that HMM and SVA exist to support, and it is the only one of the three in which the CPU page table is the authoritative description of the device's address space. The CUDA guide distinguishes two implementations of it. On systems with software coherence the device driver uses Linux HMM, which the guide describes as a software-based memory management system providing the same programming model as hardware-coherent systems; NVIDIA documents this path as requiring Linux 6.1.24 or later (or 6.2.11, or 6.3 and later), a GPU of compute capability 7.5 or higher, and a driver of version 535 or later built with the open kernel modules. On systems with hardware coherence, of which NVIDIA's Grace Hopper is the documented example, the GPU uses Address Translation Services to obtain translations from what the guide calls a logically combined page table for the CPU and GPU. The first implementation is an HMM design; the second is an SVA design in the sense used by this chapter, because the device consumes translations derived from the CPU's own page tables rather than from a driver-built copy.
The three models also differ in what a single application-level operation costs. In the copy model an munmap() of host memory has no device-side consequence unless the region is currently DMA-mapped, in which case the pinning prevents the pages from being freed until the driver unmaps them. In the managed model the driver controls the virtual range and can refuse or defer operations on it. In the system-allocated model an munmap(), a madvise(MADV_DONTNEED), a NUMA balancing migration, or a copy-on-write break can occur at any time, on any page the device might be using, initiated by code that knows nothing about the device. The kernel must therefore notify every translator that holds a derived copy of the affected translations, wait until each has stopped using it, and only then allow the physical page to be reused. That obligation is the subject of Sections 22.3 through 22.7.
| Property | Copy | Managed | System-allocated |
|---|---|---|---|
| Device-usable pointers | Device allocations and DMA-mapped IOVAs only | Managed allocations only | Any valid process pointer |
| Owner of device translations | Driver, via DMA API mappings | Vendor driver | Kernel (HMM mirror via driver, or IOMMU via SVA) |
| Host pages pinned while device-mapped | Yes | Driver-dependent | No |
| Kernel framework | DMA API | Vendor driver (may use HMM internally) | HMM or SVA |
| Characteristic failure | Pointer-rich data structures | Memory not allocated through the API | Fault and invalidation cost under churn |
Figure 22.1 summarizes the three models by asking, for each, where the device-visible translation comes from and who is responsible for keeping it current.
HMM's first function, in the words of the kernel documentation (Documentation/mm/hmm.rst), is to share the address space by duplicating the CPU page table in the device page table, so that the same address points to the same physical memory. The documentation explains why this cannot be done entirely in common code: updating a device page table usually means writing device-specific unmap and cache-flush commands into a buffer and submitting it to the device. HMM therefore provides helpers for the common parts, tracking CPU page-table changes and resolving virtual ranges into page frames, and leaves the device-specific update to the driver.
A driver that wants to mirror part of a process's address space begins by registering an interval notifier with mmu_interval_notifier_insert(), passing the mm_struct, a start address, a length, and a table of operations. From that point on, whenever the core memory-management code is about to change page-table entries that overlap the registered interval, it calls the notifier's invalidate callback with a struct mmu_notifier_range describing the affected range and the reason for the change. The reason is carried in the range's event field, whose values in v7.2.9 are MMU_NOTIFY_UNMAP, MMU_NOTIFY_CLEAR, MMU_NOTIFY_PROTECTION_VMA, MMU_NOTIFY_PROTECTION_PAGE, MMU_NOTIFY_SOFT_DIRTY, MMU_NOTIFY_RELEASE, MMU_NOTIFY_MIGRATE, and MMU_NOTIFY_EXCLUSIVE.
The contract on that callback is strict. The mmu_notifier.h header states that upon return the caller must stop using any secondary page-table entries in the range, and the HMM documentation adds that the device must complete the update before the callback returns. The callback may sleep, but if the core code has marked the range non-blockable, as on some reclaim and out-of-memory paths, it must return false rather than sleep.
The notifier alone does not close the race between populating the mirror and invalidating it: a driver that looks up a CPU translation and then writes it to the device page table can be overtaken by an invalidation between the two steps. HMM closes this window with a sequence count per interval notifier. The driver samples it with mmu_interval_read_begin() before the lookup; the invalidate callback advances it with mmu_interval_set_seq(). After the lookup, the driver takes the same lock its callback takes and calls mmu_interval_read_retry() with the sampled value. If the sequence has moved, an invalidation ran concurrently and the driver starts again; if not, it writes the device page table while holding the lock, so any later invalidation waits for the write and then removes it. The documentation is explicit that the retry check must be made under the same lock the driver takes inside invalidate().
Figure 22.2 shows the complete populate-and-invalidate protocol as a sequence across the driver, the HMM core, the core memory-management code, and the interval notifier.
hmm_range_fault(), and writes its device page table only after mmu_interval_read_retry() succeeds under its update lock. A concurrent invalidation takes the same lock, advances the sequence, and removes device entries before the CPU mapping changes, so a stale translation is either retried or removed, never kept.The v7.2.9 header also defines an optional two-pass form, invalidate_start and invalidate_finish, in which the first pass may return a descriptor that the core code later passes to the second. If no descriptor is returned, invalidate_start must behave exactly like invalidate.
To populate the mirror the driver fills in a struct hmm_range: the interval notifier, the sampled notifier_seq, start and end addresses, an output array hmm_pfns, and two policy fields, default_flags and pfn_flags_mask, which request the same behaviour for every page, for example that each must be present and writable. Missing pages are faulted in through the generic handle_mm_fault() path exactly as if a CPU thread had touched them, so the result respects the process's VMA permissions, memory policy, and cgroup limits.
In the v7.2.9 stable tree the interface is hmm_range_fault(), and the caller must hold the mmap read lock across the call. The documented driver loop takes the lock, calls hmm_range_fault(), releases the lock, retries on -EBUSY (the range was invalidated during the walk), and then performs the mmu_interval_read_retry() check under its update lock. The header defines HMM_RANGE_DEFAULT_TIMEOUT as 1,000 milliseconds for bounding such waits.
The interface is changing. The v7.3 development series adds hmm_range_fault_unlocked_timeout(range, timeout), which its documentation calls the normal interface. The caller must not hold the mmap lock; the function takes it internally and lets handle_mm_fault() drop it, as fault handlers for userfaultfd regions require. When the lock is dropped or the range invalidated, the function refreshes range->notifier_seq and restarts itself. The timeout is in jiffies, zero meaning retry indefinitely; the function returns -EBUSY when the retry budget is exhausted and -EINTR if lock acquisition is interrupted or a fatal signal is pending. hmm_range_fault() remains for drivers that must hold the mmap lock across other work, with the documented limitation that it cannot support VMAs whose fault handlers drop the lock. Code for v7.2.9 must keep the caller-locked pattern; code for v7.3 and later should prefer the unlocked form.
/* Linux v7.2.9 pattern (Documentation/mm/hmm.rst), abbreviated */
range.notifier = &interval_sub;
range.start = start; range.end = end;
range.hmm_pfns = pfns;
range.default_flags = HMM_PFN_REQ_FAULT; /* every page must be present */
range.pfn_flags_mask = 0;
again:
range.notifier_seq = mmu_interval_read_begin(&interval_sub);
mmap_read_lock(mm);
ret = hmm_range_fault(&range);
mmap_read_unlock(mm);
if (ret == -EBUSY)
goto again;
if (ret)
return ret;
take_lock(driver->update); /* same lock as ->invalidate() */
if (mmu_interval_read_retry(&interval_sub, range.notifier_seq)) {
release_lock(driver->update);
goto again;
}
/* program device page table from pfns[] */
release_lock(driver->update);Four in-tree drivers in v7.2.9 use this path: nouveau (nouveau_svm.c), amdgpu (amdgpu_hmm.c) together with the AMD compute driver's interval-notifier ranges in kfd_svm.c, Intel Xe through the shared DRM helper drm_gpusvm.c, and the AMD XDNA NPU driver discussed in Section 22.9.
Mirroring has a price. The device page table is a second copy of state whose master lives in the CPU page table, and propagation is synchronous: the operation that triggered an invalidation cannot complete until every mirror has acknowledged it, so a device that needs milliseconds to drain in-flight work adds milliseconds to every munmap() touching its interval. In exchange, the device's page-table format, page sizes, and caching policy are entirely the driver's choice.
Mirroring lets a device follow pointers into host memory, but a discrete accelerator reading host memory across PCIe sees a fraction of the bandwidth of its local memory. HMM's second function lets the kernel migrate a process's anonymous pages into device memory while the process's virtual addresses stay unchanged. The design choice that makes this possible is unusual: device memory receives ordinary struct page descriptors, so the existing migration machinery can move pages into and out of it, even though the CPU cannot access that memory at all.
The kernel represents device memory through ZONE_DEVICE, with a struct dev_pagemap per region whose type field selects its semantics. The v7.2.9 header include/linux/memremap.h describes MEMORY_DEVICE_PRIVATE as device memory that the CPU can neither read nor write, backed nonetheless by struct page descriptors that must at certain points be treated as opaque. It describes MEMORY_DEVICE_COHERENT as device memory that is cache-coherent from both device and CPU, which Section 22.9 returns to. The other three types serve persistent memory, generic device-DAX, and peer-to-peer DMA and are outside the scope of this chapter.
A driver creates device-private pages by reserving a physical address range that does not collide with real memory, describing it in a dev_pagemap with type = MEMORY_DEVICE_PRIVATE and a struct dev_pagemap_ops, and calling memremap_pages() or its device-managed variant. The two operations that matter most are folio_free, called once a device folio's reference count reaches zero, and migrate_to_ram, used for device-private memory to migrate a faulting page back to CPU-accessible memory. The v7.2.9 table also includes memory_failure and folio_split.
From the CPU's side, the HMM documentation explains, a migrated page looks as though it had been swapped out. The CPU page-table entry for its virtual address A is replaced with a special swap-format entry that refers to the device page. Any CPU access to A faults, the fault handler recognizes the device-private entry, and it calls the owning driver's migrate_to_ram callback to bring the data back before the access is retried. Because the CPU never maps these pages, the documentation states that device memory can never be pinned by a device driver nor through GUP, and so it is always freed on process exit.
Migration in either direction uses three core calls with the driver's copy step in between. The driver first takes the mmap lock, because migrate_vma_setup() needs a struct vm_area_struct that stays valid throughout. It fills in a struct migrate_vma with the VMA, the range, source and destination PFN arrays, a flags field, and a pgmap_owner. The flags select eligible source pages: MIGRATE_VMA_SELECT_SYSTEM for system memory, MIGRATE_VMA_SELECT_DEVICE_PRIVATE and MIGRATE_VMA_SELECT_DEVICE_COHERENT for device memory, and, in v7.2.9, MIGRATE_VMA_SELECT_COMPOUND for compound pages. For device-private sources, pgmap_owner limits selection to the driver's own pages. The documentation notes that only anonymous private VMA ranges can currently be migrated to or from device-private memory.
Inside migrate_vma_setup() the core code brackets its page-table walk with mmu_notifier_invalidate_range_start() and _end(), notifying every mirror of the range with event = MMU_NOTIFY_MIGRATE and the range's owner set to the supplied pgmap_owner. A driver can compare that owner with its own and skip invalidating mappings it is about to update itself. Empty and zero-page entries yield a zero PFN in the source array, so the driver can clear rather than copy. Present pages are locked, isolated from the LRU if in system memory, unmapped, and replaced with a special migration entry.
The driver then skips any source entry whose MIGRATE_PFN_MIGRATE bit is clear, allocates and locks destination pages, records them with migrate_pfn(), invalidates its own device mappings, and copies the data. The core code has already handled the CPU side. migrate_vma_pages() commits the migration; for an empty or zero-page source this is when the new page is inserted into the CPU page table, and the documentation states that if a CPU thread faults on the same address only one of the new pages will be inserted, with the losing driver seeing MIGRATE_PFN_MIGRATE cleared. The driver updates its device page table for entries still migrating, granting write access where MIGRATE_PFN_WRITE is set, and rolls back the rest. Finally migrate_vma_finalize() replaces each migration entry with the new page-table entry and drops the page references, and the driver releases the mmap lock.
Migration is therefore best-effort per page. A request for a two-megabyte range may move most of its pages and leave some behind because they could not be isolated or because a CPU fault won the race, and the driver must tolerate a partially migrated range. In the tree, amdkfd migrates system pages to device memory in kfd_migrate.c with MIGRATE_VMA_SELECT_SYSTEM, and when migrating back selects MIGRATE_VMA_SELECT_DEVICE_COHERENT if the GPU is connected to the CPU over a coherent XGMI link and MIGRATE_VMA_SELECT_DEVICE_PRIVATE otherwise; drm_pagemap.c provides a shared implementation for DRM drivers. Figure 22.3 traces one page through the protocol, showing the CPU and device page-table state after each call.
migrate_to_ram() callback. Migration is best-effort per page, and a driver that loses a race with a CPU fault sees MIGRATE_PFN_MIGRATE cleared.Some devices need atomic access to a system-memory page that the CPU must not touch concurrently. HMM provides make_device_exclusive(), which, per the documentation, replaces the CPU mappings of the page with special swap entries. A CPU access to such an entry faults and is resolved by restoring the original mapping; the driver learns of this through an MMU notifier with event = MMU_NOTIFY_EXCLUSIVE and then no longer has exclusive access, which otherwise lasts until it drops the page lock and reference. The nouveau driver uses this mechanism to service GPU atomics to system memory.
The HMM documentation records a deliberate accounting choice: device memory is counted in a process's RSS and charged to its memory cgroup exactly as a regular page would be. The stated reasons are to avoid surprising applications that use device memory without knowing it and to guarantee that migration back to system memory cannot fail on a cgroup limit. The documented drawback is that the out-of-memory killer may kill a process that holds much device memory but little system memory, freeing little system memory. The documentation describes the choice as open to revision once there is more experience.
Shared Virtual Addressing takes the opposite approach to HMM. Rather than building a device-side copy of the CPU page tables, it lets the IOMMU walk the CPU page tables themselves. Every DMA request the device issues for the process carries a PASID, the 20-bit identifier defined by the PCIe specification (Chapter 5, Section 5.3.4), and the IOMMU uses that PASID to select a first-stage translation whose root is the same page-table root the CPU loads when it runs the process. The device therefore sees exactly the virtual address space the CPU sees, with the same permissions and the same page sizes, and no driver code ever writes a device page-table entry for a process address.
A driver establishes this sharing with iommu_sva_bind_device(dev, mm), which returns a struct iommu_sva handle, and obtains the PASID to program into the device with iommu_sva_get_pasid(). In v7.2.9 the bind operation, implemented in drivers/iommu/iommu-sva.c, does three things. It first checks that the process's page tables are compatible with DMA through arch_pgtable_dma_compat(), failing with -EBUSY otherwise. It then allocates a PASID for the mm if the process does not already have one, using iommu_alloc_global_pasid(); if the process already has a PASID that exceeds the device's PASID width, the bind fails with -EOVERFLOW, and if no PASID is available it fails with -ENOSPC. Finally it attaches an SVA-type IOMMU domain for that mm to the device at that PASID with iommu_attach_device_pasid(), reusing an existing domain if the same mm is already bound to another device behind a compatible IOMMU.
The PASID is therefore a property of the process, not of the device. Every device bound to the same process uses the same PASID, which is what allows a user-space work descriptor to carry a PASID that every device understands. The PASID also outlives individual bindings. iommu_sva_unbind_device() detaches the domain from the device and frees the domain when its last user is gone, but the PASID itself is released only in mm_pasid_drop(), which runs when the mm is destroyed. The unbind function's comment places two obligations on the driver: the device should not be issuing any more transactions for this PASID, and all outstanding page requests for the PASID must already have been flushed to the IOMMU. Section 22.10 returns to what happens when a driver fails to meet them.
Binding also installs the fault handler. The SVA domain's iopf_handler is set to iommu_sva_iopf_handler(), so that when the device reports a translation fault for the PASID, the I/O page-fault framework in drivers/iommu/io-pgfault.c routes it to a work item that resolves the fault against the bound mm. The fault path is described in Section 22.6.2.
Because the IOMMU walks the CPU's page tables, an SVA binding never needs to be told when an entry is added; the next walk will find it. It does need to be told when an entry is removed or downgraded, because the IOMMU's translation caches and the device's address translation cache may still hold the old translation. SVA does not use the interval-notifier mechanism that HMM uses for this. Instead each vendor IOMMU driver registers an MMU notifier that implements the arch_invalidate_secondary_tlbs operation. The header comment in mmu_notifier.h describes this operation as the way to manage a non-CPU TLB that shares page tables with the CPU, and says that a notifier implementing it should not also implement invalidate_range_start and invalidate_range_end, because the hook already catches every point at which an external TLB needs to be flushed. The architecture's TLB-flush code calls the hook whenever it invalidates CPU TLB entries for the mm; on x86, for example, flush_tlb_mm_range() calls it after the CPU flush has been issued. The hook is invoked while the page-table spinlock is held and must not sleep.
In v7.2.9 three vendor drivers implement the hook: the Intel driver in drivers/iommu/intel/svm.c, the AMD driver in drivers/iommu/amd/pasid.c, and the Arm SMMUv3 driver in drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c. Each translates the address range into the invalidation commands of its own hardware, as Section 22.7 details. The three drivers differ, however, in how much of the fault path they implement. The Intel and AMD drivers enable PCIe PRI on capable devices and service page requests from the Intel page request queue and the AMD PPR log respectively. The v7.2.9 SMMUv3 driver allocates an I/O page-fault queue only when the SMMU supports the stall model, and its PRI-queue handler logs any page request it receives as unexpected and answers it with a deny response. Recoverable SVA faults on Arm are therefore available in v7.2.9 to stall-capable platform devices, not to PCIe endpoints that rely on PRI, and a page request from a PCIe device on an Arm system running v7.2.9 is denied rather than resolved. The RISC-V IOMMU driver does not appear in that list. Its source in both v7.2.9 and v7.3-rc6 carries the comment that the SVA and PASID features have not yet been merged, and it issues no ATS commands. The RISC-V IOMMU specification defines the hardware needed for SVA, including a process-directory table indexed by a process_id and the commands discussed below, but at the time of writing no released Linux kernel uses them.
Intel platforms add a user-space work-submission path that depends on the process-wide PASID. As the kernel's x86 SVA documentation (Documentation/arch/x86/sva.rst) describes, the ENQCMD instruction atomically submits a work descriptor to a device, typically to a shared work queue, and the descriptor includes the PASID of the current process; the instruction returns a status indicating whether the hardware accepted the command. The instruction obtains that PASID from a thread-scoped model-specific register, IA32_PASID. Linux initializes the register lazily: a thread's first ENQCMD raises a general-protection fault because the register has not been set, the kernel's #GP handler notices that the process has a PASID, writes it into IA32_PASID, and resumes the thread, which retries the instruction successfully. Threads that never submit work never take the fault and keep a cleared register, which the documentation notes reduces context-switch overhead. Because clearing IA32_PASID in every thread at unbind time would be complex, the PASID is freed lazily, only on mm exit. Intel's Data Streaming Accelerator driver (drivers/dma/idxd/) is an in-tree user of this path; it binds the opening process with iommu_sva_bind_device() and exposes shared work queues to user space. The kernel documents this lazy-PASID scheme for x86 only; submission paths on the other architectures are outside the scope of this chapter.
Table 22.1 summarizes how each architecture provides the pieces SVA needs. The first three rows describe hardware defined by each specification; the last two rows describe Linux v7.2.9.
| Aspect | Intel VT-d | AMD IOMMU | Arm SMMUv3 | RISC-V IOMMU |
|---|---|---|---|---|
| Address-space tag | PASID, scalable-mode PASID table | PASID, GCR3 (guest CR3) table | SubstreamID, context-descriptor table | process_id, process-directory table |
| Fault request path | PCIe PRI into page request queue | PCIe PRI into PPR log | PCIe PRI (PCIe endpoints) or stall model (platform devices) | PCIe PRI into page-request queue |
| Fault response | Page group response descriptor | COMPLETE_PPR_REQUEST command | CMD_PRI_RESP or CMD_RESUME | ATS.PRGR command |
| Linux SVA driver (v7.2.9) | intel/svm.c | amd/pasid.c | arm-smmu-v3-sva.c | Not merged |
| Linux recoverable faults (v7.2.9) | PRI via page request queue (prq.c) | PRI via PPR log (ppr.c) | Stall model only; PRI requests answered with deny | None |
Figure 22.4 extends Table 22.1 with the invalidation commands examined in Section 22.7, so that the full set of per-vendor mechanisms can be read in one place.
The difference between mirroring and sharing is clearest when a device touches an address it has no translation for. This section follows that event through both frameworks and then compares who owns each piece of state. Both traces end in handle_mm_fault(), which is why the two frameworks can coexist in one kernel, but they reach it by different routes and leave different state behind.
A compute kernel on a GPU loads from virtual address A. The GPU's MMU misses in its TLB, walks the device page table the driver built, and finds no valid entry. How the device reports the miss is device-specific and defined by no Linux interface; nouveau, for example, services replayable faults that the GPU records in a fault buffer. In every case the fault reaches the device driver, not the IOMMU or the CPU's page-fault handler.
The driver identifies the faulting process and the interval notifier covering A, builds a struct hmm_range, and sets default_flags to require present pages, adding write permission for a store. It samples the notifier sequence and, on v7.2.9, calls hmm_range_fault() under the mmap read lock; on v7.3 it may call hmm_range_fault_unlocked_timeout() without the lock. HMM walks the CPU page table, reporting pages that are already present with sufficient permission and faulting the rest in through handle_mm_fault(), the function a CPU page fault would call. That function may allocate, read from a file, break copy-on-write, or swap in, and may itself trigger invalidations on the range; the walk then returns -EBUSY (v7.2.9) or restarts internally (v7.3).
With the hmm_pfns array filled, the driver takes its update lock and calls mmu_interval_read_retry(). If an invalidation intervened, it repeats the fault; otherwise it writes device page-table entries and releases the lock. If its policy places hot data in device memory, it may now or later migrate the pages with the protocol of Section 22.4, after which the device page table points at device memory and the CPU page table holds device-private entries. Finally the driver tells the device to replay the access. The device never consulted the CPU's page-table format, and the driver now holds a second copy of translation state that the interval notifier must keep consistent from this point on.
A PCIe accelerator bound with SVA reads virtual address A, tagged with the process's PASID. Holding no translation for A in its address translation cache (ATC), it sends an ATS translation request (Chapter 10, Section 10.3). The IOMMU looks up the PASID in the device's PASID-indexed structure, which is the scalable-mode PASID table on Intel, the GCR3 table on AMD, the context-descriptor table on Arm, or the process-directory table on RISC-V, and walks the CPU page table it names. Had A been mapped, the device would cache the completion and proceed with no software involvement, the common case and SVA's main advantage. Here A is unmapped, so the completion grants no access and the device, if it supports PRI, sends a page request naming the PASID, address, access type, and page request group.
The IOMMU writes the request to a memory-resident queue, the page request queue on Intel or the PPR log on AMD, and interrupts the CPU. The vendor driver converts it to a struct iopf_fault and calls iommu_report_device_fault(), which finds the domain attached at the PASID. Requests not marked last in their group are held as partial faults; when the last arrives, the group is handed to the SVA domain's handler, which queues it to a workqueue. The work item, iommu_sva_handle_iopf(), takes a reference on the mm and its mmap read lock for each fault, checks the requested access against the VMA's flags, and calls handle_mm_fault() with FAULT_FLAG_REMOTE plus write, instruction, or user flags as required. In v7.2.9, errors are sticky within a group.
iopf_group_response() reports the outcome, which the vendor driver issues as a page group response descriptor on Intel or a COMPLETE_PPR_REQUEST command on AMD. The device retries its translation request, the IOMMU walk finds the new CPU page-table entry, and the read completes. For a stall-capable platform device on Arm, the same framework is entered from the SMMU event queue and the response is a CMD_RESUME. No device driver code ran after binding, and the only new state is one CPU page-table entry plus its cached copies in the IOMMU and the ATC.
Figure 22.5 places the two traces in parallel lanes, coloured by which component performs each step.
hmm_range_fault(), and writes a second copy of the translation into its device page table. On the SVA path (right) the IOMMU walks the CPU page table, a PRI page request is resolved by the generic I/O page-fault framework, and the only new state is a CPU page-table entry that the IOMMU and the device ATC may then cache.Table 22.2 sets the two traces side by side. The rows that differ are the ones that determine how invalidation must work, which is the subject of Section 22.7.
| Question | HMM mirror | SVA shared tables |
|---|---|---|
| Who walks on a device TLB miss? | Device MMU walks the driver-built device page table | IOMMU walks the CPU page table selected by PASID |
| Who receives the fault? | Device driver, by a device-specific mechanism | IOMMU driver, via PRI queue or stall event, then io-pgfault |
| Who resolves the fault? | Driver calls hmm_range_fault(), which calls handle_mm_fault() | iommu_sva_handle_iopf() calls handle_mm_fault() |
| Who writes the device-visible translation? | Driver, into its device page table | Nobody; the IOMMU reads the CPU page table |
| Copies of translation state | CPU page table, CPU TLBs, device page table, device TLBs | CPU page table, CPU TLBs, IOMMU translation cache, device ATC |
| Who must invalidate device state? | Driver, in its interval-notifier callback | IOMMU driver, in arch_invalidate_secondary_tlbs |
| Can pages move into device memory? | Yes, device-private migration | No; device accesses system memory through the CPU's tables |
Adding a translation can be lazy, because a missing translation causes a fault that software resolves. Removing one cannot, because a stale translation causes no fault at all: the device reads or writes whatever physical page the stale entry names, even after the kernel has given that page to someone else. Invalidation, not fault handling, is therefore the correctness-critical half of both frameworks. This section describes what must happen before a page a device might be using can be reused, separating what Linux establishes from what each specification requires.
A translation used by a device can be held in up to five places, each invalidated by its own mechanism (Table 22.3). Read the table row by row: no mechanism in one row invalidates another row unless a specification says so. CPU TLB instructions such as INVLPG or SFENCE.VMA affect only CPU TLBs, and IOMMU invalidation commands do not reach a device's ATC unless the driver issues a separate ATS invalidation.
| Domain | Intel | AMD | Arm | RISC-V |
|---|---|---|---|---|
| CPU page table | PTE update under the page-table lock (architecture-independent core code) | |||
| CPU TLB | INVLPG/INVPCID locally; IPI shootdown to other CPUs | INVLPG locally; INVLPGB + TLBSYNC broadcast for processes with a global ASID, else IPI | TLBI (Inner Shareable) + DSB ISH | SFENCE.VMA locally; SBI remote fence to other harts; Svinval used when present |
| IOMMU translation cache | PASID-based IOTLB invalidation descriptor | INVALIDATE_IOMMU_PAGES (guest/PASID form) | CMD_TLBI_NH_VA by ASID | IOTINVAL.VMA |
| Device ATC | PASID-based device-TLB invalidation descriptor | INVALIDATE_IOTLB_PAGES | CMD_ATC_INV | ATS.INVAL |
| Completion barrier | Invalidation wait descriptor | COMPLETION_WAIT | CMD_SYNC | IOFENCE.C |
| Device-local TLB (non-ATS device MMU) | Driver-specific, performed inside the HMM interval-notifier callback and completed before it returns | |||
Two entries need comment. Linux v7.2.9 uses AMD's INVLPGB and TLBSYNC to broadcast CPU TLB invalidations for processes with a global ASID; a comment in arch/x86/mm/tlb.c names Intel's Remote Action Request as comparable, but v7.2.9 defines no feature flag for it, so Intel CPUs still use inter-processor interrupts. On Arm, an SMMU with Broadcast TLB Maintenance (BTM) can observe the CPU's broadcast TLBI for a shared ASID. The v7.2.9 SMMUv3 driver uses BTM only to choose a stream-table configuration, still issues explicit CMD_TLBI_NH_VA for SVA domains, and disables BTM on MMU-700, MMU-L1, and MMU-S3 because of errata. BTM never covers the ATC.
A safe invalidation passes through seven phases: begin invalidation, quiesce device access, modify the CPU mapping, invalidate each domain that may hold the old entry, await completions, end invalidation, and only then permit page reuse. The paths do not execute them in this textual order. HMM and SVA differ precisely in where the device-side phases fall relative to the CPU-side ones, as Figure 22.6 shows.
On the HMM mirror path the device phases come first. The core code calls mmu_notifier_invalidate_range_start() while, per the mmu_notifier.h contract, every page in the range is still mapped and referenced. Each overlapping interval notifier's invalidate callback advances its sequence, stops the device using the range, removes device page-table entries, flushes device TLBs, and waits for the flush before returning. Only then does the core code change the CPU entries and flush CPU TLBs, followed by mmu_notifier_invalidate_range_end(); the header notes that by then the pages may already have been freed, which is safe because every mirror stopped using them before its start callback returned. The header also forbids establishing new secondary entries inside the start-to-end window, which the sequence retry of Section 22.3 enforces.
On the SVA shared-table path the CPU phases come first. The core code changes the CPU entries, and the architecture's TLB-flush code, on x86 at the end of flush_tlb_mm_range(), calls mmu_notifier_arch_invalidate_secondary_tlbs(). Each bound IOMMU driver invalidates its translation cache for the PASID and then the ATC of each ATS-enabled device, waiting for completion, without sleeping. Because the hook runs on the TLB-flush path that precedes freeing, device-side invalidation completes before any page is reused. No quiesce phase is needed: the device held no private copy, and its next access walks the updated CPU page table.
CMD_SYNC between the two). On both paths every device-side invalidation completes before the physical page can be reused.The order between the IOMMU cache and the ATC is not arbitrary. If the ATC were invalidated first, the device could immediately request a new translation, the IOMMU could answer from a not-yet-invalidated cache entry, and the device would cache the stale translation again. The v7.2.9 SMMUv3 driver makes the rule explicit: it ends a command batch whenever an ATS command follows a non-ATS command, commenting that ATS must be after a sync of the S1/S2 invalidations. On Arm the sequence is therefore CMD_TLBI_NH_VA, then CMD_SYNC, then CMD_ATC_INV, then CMD_SYNC.
For each device bound to the mm, the AMD driver issues INVALIDATE_IOMMU_PAGES for the PASID, follows it with INVALIDATE_IOTLB_PAGES if the device has ATS enabled, and then issues COMPLETION_WAIT. The Intel driver walks the domain's cache tags, which it keeps grouped by IOMMU unit, accumulates the PASID-based IOTLB and device-TLB descriptors for the range into one batch per unit, and submits each batch with qi_submit_sync(), which appends an invalidation wait descriptor and waits for it to complete. On RISC-V the specification provides the building blocks: IOTINVAL.VMA for the IOMMU's cache, ATS.INVAL to send a PCIe invalidation request to the device, and IOFENCE.C, which the specification guarantees will not complete until all previously fetched commands have executed and completed, including outstanding ATS invalidations. Since no released Linux kernel implements SVA on the RISC-V IOMMU, the sequence a Linux driver will use there cannot yet be cited.
On the HMM path, a race between a fault and an invalidation is resolved by the sequence count. If the invalidation precedes the driver's mmu_interval_read_retry() check, the sequence has advanced and the driver faults again; if it arrives after the device write, it waits for the update lock and removes the entry just written. Either way the device never keeps a translation the CPU discarded. The cost is repeated retries under continuous invalidation, which is why callers bound the loop and the v7.3 interface takes a timeout.
On the SVA path there is no software copy to race with. A concurrent page request is resolved by handle_mm_fault() against whatever the CPU page table holds when the work item takes the mmap lock: an unmapped region yields an invalid response, and a migration or copy-on-write break yields the new mapping. The only remaining hazard, a stale ATC entry, is what the ordered IOMMU-then-ATC invalidation rules out.
Two shipping products bring CPU and GPU memory closer together than a PCIe-attached accelerator can, and both are frequently described as having "unified memory". They achieve it differently, and the differences map directly onto the HMM-versus-SVA distinction of this chapter. This section confines itself to what each vendor has documented and what peer-reviewed measurement has established.
The Grace Hopper superchip pairs an Arm-based Grace CPU with a Hopper GPU over NVLink-C2C, a cache-coherent chip-to-chip interconnect. Schieffer, Wahlgren, Ren, Faj, and Peng characterized a system with 480 GB of LPDDR5X attached to the CPU and 96 GB of HBM3 attached to the GPU (ICPP 2024), measuring about 3.4 TB/s of GPU memory bandwidth and 375 GB/s for host-to-device transfers. NVIDIA's CUDA Programming Guide classifies Grace Hopper as a system with full unified-memory support and hardware coherence, in which the GPU uses ATS to translate addresses through what the guide calls a logically combined page table for both CPUs and GPUs. In the terms of this chapter that is an SVA-style design: the GPU consumes translations derived from the CPU's page tables rather than from a driver-maintained mirror, and memory allocated with plain malloc() is directly accessible from GPU kernels.
The ICPP 2024 study measured the consequences. For most of the six HPC applications it evaluated, system-allocated memory outperformed CUDA managed memory, which the authors attribute to the GPU accessing CPU memory directly at cache-line granularity without invoking the page-fault mechanism. The exception was workloads that initialize their data on the GPU, for which managed memory performed better because of the cost of setting up page-table entries for system memory on first touch. Page size mattered: 64 KB system pages gave up to a fourfold speedup for GPU-side initialization compared with 4 KB pages, while sometimes reducing performance in the computation phase. Under memory oversubscription, system-allocated memory retained its relative advantage. The CUDA guide also documents that hardware-coherent systems can migrate pages based on access counters that track how often a GPU accesses memory located on another processor, and that file-backed memory is not migrated on that basis.
Grace Hopper therefore illustrates both of this chapter's themes at once. Translation is shared, so the per-fault software path that dominates HMM-style systems is avoided for memory the GPU merely reads or writes. Placement is still a policy decision, made by first touch and by access-counter-driven migration, and the measured performance depends heavily on where the data was first initialized.
The MI300A takes integration a step further by putting CPU and GPU in the same package with one physical memory. Smith et al. describe it in their ISCA 2024 industry paper as combining three CPU chiplets with a total of 24 Zen 4 cores and six accelerator complex dies with 228 compute units, sharing a unified HBM-based memory system of eight HBM stacks with 128 GB of capacity, 128 memory channels, and a peak theoretical bandwidth of about 5.3 TB/s. Each memory channel is paired with a 2 MB slice of Infinity Cache, 256 MB in total. The paper states that both kinds of processor can load and store directly to the HBM without copying data, that the CPUs are hardware-coherent using a probe-filter-based protocol, and that the GPUs are directory-based hardware-coherent within a socket and software-coherent with GPUs in other sockets.
The ISCA paper does not describe the MI300A's virtual-memory organization, GPU page-table format, or address-translation path, and this chapter does not infer them. What can be said from the Linux side is limited to the public driver: AMD's compute driver implements its shared-virtual-memory support with HMM interval notifiers and hmm_range_fault(), as Section 22.3 described, and its migration code selects device-coherent rather than device-private pages when the GPU is attached to the CPU over a coherent link. A preprint by Wahlgren and colleagues (arXiv:2508.12743, not yet peer-reviewed at the time of writing) reports a characterization of unified physical memory on MI300A, including GPU page-fault throughput and the effect of allocator choice on GPU TLB behaviour; readers should treat its figures as provisional until a reviewed version appears.
The two products bracket the design space. Grace Hopper keeps two physical memories and shares translation across a coherent link. MI300A has one physical memory, so the question of where a page lives largely disappears, and what remains is the question this chapter has been asking throughout: which translator holds which copy of the mapping, and who tells it when the mapping changes.
This section answers two narrow questions. First, what does a cache-coherent link between device and host change for HMM and SVA? Second, what evidence exists that neural processing units use these frameworks at all? Chapter 19 covers CXL as a memory fabric in general, and that material is not repeated here.
The kernel's answer to the first question is encoded in the distinction between two ZONE_DEVICE types. Device-private memory exists because the CPU cannot access it: every CPU touch must fault and migrate the page back to system memory. Device-coherent memory, which memremap.h describes as cache-coherent from both the device and CPU point of view and intended for platforms with an advanced system bus such as CXL, removes that restriction. A page that has migrated to device-coherent memory can remain mapped in the CPU page table, so the CPU can read and write it in place, and the migrate_to_ram fault path is no longer needed for correctness. The same header imposes the counterpart obligation: device-coherent memory must not be pinned, so that it can always be evicted. The migration helpers support it through MIGRATE_VMA_SELECT_DEVICE_COHERENT, and the amdkfd driver selects that flag when its GPU is attached to the CPU over a coherent link.
Coherence does not, however, remove any of the invalidation obligations of Section 22.7. Cache coherence keeps data consistent: a CPU store to a line is observed by a device that has cached the same line. It says nothing about translations. A device that has cached the translation from virtual address A to physical page P will keep using it after the kernel remaps A to another page, and coherence will dutifully keep its view of the old page P consistent while the device writes data the process no longer owns. Every translation a coherent device holds, whether in a driver-built mirror or in an ATS cache, must still be invalidated through the mechanisms of Table 22.3 before the old page is reused. Chapter 19 (Section 19.7) describes CXL 3.0 Back-Invalidation, with which a device that owns memory can invalidate copies of that memory's lines held in host caches; that mechanism operates on cached data, and translation invalidation remains a separate obligation.
Claims about neural processing units are easy to make and hard to verify, because most NPU memory systems are undocumented. This chapter therefore restricts itself to devices whose use of HMM or SVA is visible in the Linux v7.2.9 source.
The clearest example is the AMD XDNA driver in drivers/accel/amdxdna/, which supports the NPU integrated into AMD client processors such as the Ryzen AI 300 series. It uses both frameworks. When a process opens the device and the driver is not operating in its IOVA mode, it binds the process with iommu_sva_bind_device() and retrieves the process PASID with iommu_sva_get_pasid(), so that the NPU's DMA is translated through the process's own page tables. Separately, for buffer objects that wrap user-space memory, it registers an interval notifier with mmu_interval_notifier_insert_locked(), resolves the pages with hmm_range_fault() under the mmap read lock, and retries with its own timeout when the call is interrupted by invalidations. Its Kconfig entry selects HMM_MIRROR and depends on AMD_IOMMU. A single NPU driver thus exercises both the shared-table and the mirrored model described in this chapter, for different kinds of memory.
Two other in-tree accelerator frameworks rely on SVA without HMM. The Intel Data Streaming Accelerator driver binds the opening process to the device with iommu_sva_bind_device() and lets user space submit descriptors carrying the process PASID through ENQCMD, as described in Section 22.5.3. The UACCE framework in drivers/misc/uacce/, described in its Kconfig entry as an accelerator framework for user land, is used by HiSilicon's compression and cryptographic engine drivers. For devices registered with SVA support, it binds the opening process to each queue with iommu_sva_bind_device(), so that user space can access the hardware without kernel involvement in the data path and pass it process virtual addresses. These are not neural processing units, but they show that SVA is used in production drivers beyond GPUs.
For NPUs that do not appear in the tree, or whose drivers use neither framework, this chapter makes no claims. A proprietary NPU driver may implement its own mirroring, rely on pinned copy-model buffers, or use an approach not described in any public source; without a published architecture manual, an in-tree driver, or a peer-reviewed measurement, there is no basis on which to describe it.
The mechanisms in the preceding sections are correct when every participant behaves as specified. This section examines what happens when one does not: a device that misuses translations, a PASID released too early, an invalidation that never completes, a fault that cannot be resolved, or exhaustion of the resources the frameworks consume.
ATS lets a device issue translated requests, which carry a physical address obtained earlier from the IOMMU and which the IOMMU does not translate again. That is the source of ATS's performance benefit and its central weakness: a malicious or compromised device can issue a translated request for any physical address. Markettos et al. demonstrated with the Thunderclap FPGA platform (NDSS 2019) that a malicious peripheral interacting with an operating system's drivers could defeat IOMMU-based DMA protection on macOS, FreeBSD, and Linux, and Markuze, Morrison, and Tsafrir (ASPLOS 2016) showed that the conventional map-and-unmap use of the IOMMU is not truly secure, because it protects only at page granularity and because deferred unmaps leave a window in which devices can access in-use memory. Linux's response for ATS is blunt. In v7.2.9, pci_ats_supported() returns false for any device marked untrusted, a flag the PCI core propagates from an untrusted upstream bridge such as an external-facing port. Such devices never receive ATS, and every request they make is translated by the IOMMU.
HMM's trust model differs. The device page table is written by a kernel driver from page-frame numbers that HMM returned, so a correctly functioning device reaches only pages the driver mapped; where the device sits behind an IOMMU, the driver must also map those pages for DMA, for which the v7.2.9 HMM header provides helpers such as hmm_dma_map_alloc() and hmm_dma_map_pfn(). Isolation rests on the driver and the device MMU, with the IOMMU as a second line of defence.
Reusing a PASID while something still refers to its old meaning would deliver one process's DMA into another's address space. Linux guards against this in three ways. A process's PASID is released only by mm_pasid_drop() when the mm is destroyed. The documented contract of iommu_sva_unbind_device() requires that the device has stopped issuing transactions for the PASID and that all outstanding page requests for it have been flushed to the IOMMU. And the IOMMU driver calls iopf_queue_flush_dev() before releasing a PASID so that, in the words of its comment, all pending faults have been handled and will not hit the address space of the next process that uses it.
Process exit while a device is still bound is handled by the MMU notifier's release operation, which runs before the page tables are torn down. The Intel driver's intel_mm_release() clears the PASID-table entries and flushes the IOTLB and PASID caches, with a comment acknowledging that this might upset hardware still issuing requests. The SMMUv3 driver's arm_smmu_mm_release(), noting that DMA may still be running, keeps the context descriptor valid to avoid bad-descriptor events but disables translation. Either way, a device that keeps issuing requests after its process has exited receives faults, not access to freed memory.
An ATS invalidation depends on the device to answer, and each IOMMU architecture surfaces a missing answer differently. On Intel, an invalidation time-out error aborts all pending wait descriptors; the v7.2.9 driver clears the error, identifies the source device when the hardware reports it, returns -ETIMEDOUT if that device is no longer present, and otherwise resubmits. On Arm, an ATC-invalidation timeout is reported as a command-queue error while the consumer pointer rests on the CMD_SYNC, and the driver repeats the CMD_SYNC. The RISC-V specification reports an ATS.INVAL timeout when a subsequent IOFENCE.C executes and suggests resubmitting the invalidations, if necessary as individual ATS.INVAL and IOFENCE.C pairs to identify the device that timed out.
A hung device can therefore stall the memory-management operation waiting for it. On the SVA path the wait happens inside the secondary-TLB hook, which returns no value and so cannot report failure. On the HMM path it happens inside the interval-notifier callback, where returning before the device flush completes would let the page be freed while the device still maps it. Neither framework offers a safe way to abandon an invalidation; the safe outcomes are completion or removal of the device.
On the SVA path, iommu_sva_handle_mm() answers with IOMMU_PAGE_RESP_INVALID if the request carries no valid PASID, if the mm is already exiting, if no VMA covers the address, if the VMA forbids the access, or if handle_mm_fault() fails; because errors are sticky within a page request group, the rest of the group is then answered without being attempted. What the device does with an invalid response is device-specific. On the HMM path the error is returned by hmm_range_fault() and the driver decides how to report it. The -EBUSY return deserves care: in v7.2.9 it means the range was invalidated during the walk and the driver should retry, while in v7.3's unlocked interface it means the caller's retry budget is exhausted. A driver moving from one interface to the other must change how it interprets the same error code.
PASIDs come from a global space, and a bind fails with -ENOSPC when none are free or with -EOVERFLOW if the process's existing PASID exceeds what a newly bound device supports. Page request queues are fixed-size rings; the AMD driver, for example, restarts PPR logging after the IOMMU reports a log overflow. Every SVA fault is serviced by handle_mm_fault() under the process's mmap read lock, so a device that faults on many pages adds that work to the lock that the process's own CPU faults use. On the HMM side, device memory is charged to the process's memory cgroup (Section 22.4.4), and pages that are repeatedly migrated to the device and faulted back by the CPU pay for two migrations per round trip.
One constraint is specific to x86. Before allocating a PASID, the bind path calls arch_pgtable_dma_compat(), which on x86 in v7.2.9 returns false, failing the bind with -EBUSY, if the process has enabled Linear Address Masking unless it has set the flag that forces tagged SVA. A process that wants both LAM and SVA must opt in explicitly before binding.
Quantitative evidence about these frameworks is thinner than the attention they receive would suggest, and much of what circulates is vendor marketing or unreviewed benchmark posts. This section reports only figures from peer-reviewed measurement, clearly labels the one preprint it cites, and identifies the questions for which no published measurement exists.
The architectural case for sharing translation between CPU and GPU predates both Linux frameworks. Power, Hill, and Wood (HPCA 2014) showed that a GPU can support full x86-64 address translation, walking the same page-table format as the CPU, with a translation hierarchy designed for hundreds of GPU lanes, and Pichai, Hsu, and Bhattacharjee (ASPLOS 2014) examined the design of GPU MMUs for CPU/GPU unified address spaces. Vesely, Basu, Oskin, Loh, and Bhattacharjee (ISPASS 2016) reported observations on shared virtual memory as implemented on commercial heterogeneous hardware, together with opportunities for improving its architecture. These studies established that sharing a page-table format is feasible in hardware; the frameworks in this chapter are the operating-system half of the same idea.
For the managed model, Allen and Ge's in-depth analysis of NVIDIA's UVM driver (SC21) found that the cost of host operating-system components in fault servicing is significant and present across implementations. That observation applies directly to HMM-based designs, in which every device fault that cannot be satisfied from the existing mirror runs hmm_range_fault(), and therefore handle_mm_fault(), on the host, under the mmap lock.
For the system-allocated model on hardware-coherent systems, the ICPP 2024 Grace Hopper study summarized in Section 22.8 is the most direct evidence available. Its central finding, that system-allocated memory outperformed managed memory for most of the evaluated applications because the GPU could access CPU memory at cache-line granularity without invoking the page-fault mechanism, is a measured confirmation of the SVA-style advantage described in Section 22.6: when a translation can be obtained from the CPU's page table without a fault, no fault-handling software runs. Its counter-finding is equally important: workloads that first touch their data on the GPU paid for page-table setup and favoured managed memory, and the system page size changed GPU-side initialization time by up to a factor of four.
For AMD's MI300A, the preprint by Wahlgren and colleagues (arXiv:2508.12743) reports, among other results, a GPU page-fault throughput of about 1.1 million pages per second, a 2.2-fold improvement from prefaulting pages on the CPU, and up to 44% lower memory consumption when applications are ported to unified memory. These figures are cited here as provisional because the paper had not completed peer review at the time of writing.
For three questions central to this chapter, the research for this chapter found no published peer-reviewed measurement. The first is the cost of an munmap() or migration that must invalidate an HMM mirror, as a function of the device's drain latency. The second is the cost of the SVA secondary-TLB hook on each architecture, including the IOMMU-then-ATC ordering of Section 22.7.3, under realistic invalidation rates. The third is the cost of migration thrashing when the CPU and a device alternately touch the same device-private pages. Readers who need these numbers will have to measure them on their own platforms. The Intel IOMMU driver in v7.2.9 provides tracepoints for its cache-tag invalidations, including one emitted for every range flush, which make the second measurement straightforward on that platform; for the others, function tracing of the vendor driver's secondary-TLB hook and of the driver's interval-notifier callback is the most direct method available.
This chapter examined the kernel layer that keeps several address translators consistent when one process's address space is used by the CPU and by devices. Chapters 5 and 10 established the hardware; this chapter added the software contract that decides who holds each copy of a translation, who resolves a missing one, and who must remove a stale one before a page is reused.
Linux answers those questions with two frameworks that differ in one fundamental respect. HMM mirrors the CPU page table into a driver-owned device page table. The driver registers an interval notifier, resolves ranges with hmm_range_fault(), guards against concurrent invalidation with a per-notifier sequence count checked under its own update lock, and may migrate anonymous pages into device-private memory with the migrate_vma_setup(), migrate_vma_pages(), and migrate_vma_finalize() protocol. SVA instead binds a process to a device under a process-wide PASID, lets the IOMMU walk the CPU page table directly, resolves device faults through the generic I/O page-fault framework and handle_mm_fault(), and keeps IOMMU and ATC caches current through the arch_invalidate_secondary_tlbs hook. In HMM the device-side invalidation happens first, inside the notifier callback, before the CPU mapping changes; in SVA it happens after the CPU TLB flush, inside the hook, before pages are freed. In both, the device must stop using a translation before the page behind it can be reused, and neither framework offers a safe way to abandon an invalidation that a device never acknowledges.
The four IOMMU architectures provide the same building blocks under different names. Intel VT-d uses a scalable-mode PASID table, PASID-based IOTLB and device-TLB invalidation descriptors, and an invalidation wait descriptor, and adds ENQCMD with a lazily initialized IA32_PASID register for user-space submission. AMD uses a GCR3 table, INVALIDATE_IOMMU_PAGES and INVALIDATE_IOTLB_PAGES followed by COMPLETION_WAIT, and a PPR log for page requests. Arm SMMUv3 uses context descriptors, CMD_TLBI_NH_VA and CMD_ATC_INV separated by CMD_SYNC because ATC invalidation must follow a completed TLB invalidation, and may implement Broadcast TLB Maintenance, which Linux v7.2.9 does not rely on; in that release recoverable SVA faults on Arm are serviced only for stall-capable devices. RISC-V defines a process-directory table, IOTINVAL.VMA, ATS.INVAL, and IOFENCE.C in its ratified IOMMU specification, but no released Linux kernel yet implements SVA on it. On every architecture, CPU TLB maintenance reaches only CPU TLBs.
For systems programmers, the practical lessons are concrete. A driver that mirrors with HMM must make its invalidation callback complete the device flush before returning and must write device page-table entries only under the lock that callback takes. Code written for v7.2.9 must hold the mmap lock around hmm_range_fault(), while v7.3 introduces an unlocked, timeout-bounded alternative whose -EBUSY return has a different meaning. A driver using SVA must flush outstanding page requests before unbinding, and should expect ATS to be refused for untrusted devices. For AI infrastructure engineers, the Grace Hopper measurements show that shared translation removes fault-handling software from the common path while first-touch placement and page size still dominate performance, and published measurements of invalidation and migration-thrash costs do not yet exist.
Across this book, address translation has grown from a single structure in a single core into a system-wide agreement among many translators. The page table remains the one authoritative description of an address space; every TLB, IOMMU cache, device ATC, and device page table is a cache of it, and the correctness of the whole system rests on the discipline with which each of those caches is told, and confirms, that what it holds is no longer true.
T. Allen and R. Ge, “In-depth analyses of unified virtual memory system for GPU accelerated computing,” in Proc. Int. Conf. High Performance Computing, Networking, Storage and Analysis (SC21), 2021, pp. 1–15. doi:10.1145/3458817.3480855
Advanced Micro Devices, AMD I/O Virtualization Technology (IOMMU) Specification, Pub. 48882. [Online]. Available: https://www.amd.com/content/dam/amd/en/documents/processor-tech-docs/specifications/48882_IOMMU.pdf
Arm Limited, Arm System Memory Management Unit Architecture Specification, SMMU architecture version 3, Document IHI 0070. [Online]. Available: https://developer.arm.com/documentation/ihi0070/latest
Intel Corporation, Intel Virtualization Technology for Directed I/O Architecture Specification, Rev. 5.2, Doc. 831418, Aug. 2024. [Online]. Available: https://www.intel.com/content/www/us/en/content-details/831418/intel-virtualization-technology-for-directed-i-o-architecture-specification.html
Linux kernel developers, “Heterogeneous Memory Management (HMM),” Documentation/mm/hmm.rst, Linux v7.2.9 (stable), Oct. 2026; and v7.3-rc6 for hmm_range_fault_unlocked_timeout(). [Online]. Available: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/Documentation/mm/hmm.rst?h=v7.2.9
Linux kernel developers, “Shared Virtual Addressing (SVA) with ENQCMD,” Documentation/arch/x86/sva.rst, Linux v7.2.9. [Online]. Available: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/Documentation/arch/x86/sva.rst?h=v7.2.9
Linux kernel developers, Linux v7.2.9 source tree: include/linux/mmu_notifier.h, include/linux/hmm.h, include/linux/memremap.h, include/linux/migrate.h, mm/hmm.c, mm/migrate_device.c, drivers/iommu/iommu-sva.c, drivers/iommu/io-pgfault.c, drivers/iommu/intel/, drivers/iommu/amd/, drivers/iommu/arm/arm-smmu-v3/, drivers/iommu/riscv/, arch/x86/mm/tlb.c. [Online]. Available: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/?h=v7.2.9
A. T. Markettos, C. Rothwell, B. F. Gutstein, A. Pearce, P. G. Neumann, S. W. Moore, and R. N. M. Watson, “Thunderclap: Exploring vulnerabilities in operating system IOMMU protection via DMA from untrustworthy peripherals,” in Proc. Network and Distributed System Security Symp. (NDSS), 2019. [Online]. Available: https://www.ndss-symposium.org/ndss-paper/thunderclap-exploring-vulnerabilities-in-operating-system-iommu-protection-via-dma-from-untrustworthy-peripherals/
A. Markuze, A. Morrison, and D. Tsafrir, “True IOMMU protection from DMA attacks: When copy is faster than zero copy,” in Proc. 21st Int. Conf. Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2016. doi:10.1145/2872362.2872379
NVIDIA Corporation, “Unified Memory,” CUDA Programming Guide, v13.4.2, Sec. 4.1. [Online]. Available: https://docs.nvidia.com/cuda/cuda-programming-guide/04-special-topics/unified-memory.html
PCI-SIG, PCI Express Base Specification, Rev. 6.0, 2022 (Address Translation Services and Page Request Interface). [search]
B. Pichai, L. Hsu, and A. Bhattacharjee, “Architectural support for address translation on GPUs: Designing memory management units for CPU/GPUs with unified address spaces,” in Proc. 19th Int. Conf. Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2014, pp. 743–758. doi:10.1145/2541940.2541942
J. Power, M. D. Hill, and D. A. Wood, “Supporting x86-64 address translation for 100s of GPU lanes,” in Proc. IEEE 20th Int. Symp. High Performance Computer Architecture (HPCA), 2014, pp. 568–578. doi:10.1109/HPCA.2014.6835965
RISC-V International, RISC-V IOMMU Architecture Specification, Version 1.0 (ratified), Jun. 2023. [Online]. Available: https://github.com/riscv-non-isa/riscv-iommu
G. Schieffer, J. Wahlgren, J. Ren, J. Faj, and I. Peng, “Harnessing integrated CPU-GPU system memory for HPC: A first look into Grace Hopper,” in Proc. 53rd Int. Conf. Parallel Processing (ICPP), 2024, pp. 199–209. Extended version: arXiv:2407.07850. doi:10.1145/3673038.3673110
A. Smith, G. H. Loh, M. J. Schulte, M. Ignatowski, S. Naffziger, M. Mantor, M. Fowler, N. Kalyanasundharam, V. Alla, N. Malaya, J. L. Greathouse, E. Chapman, and R. Swaminathan, “Realizing the AMD exascale heterogeneous processor vision: Industry product,” in Proc. ACM/IEEE 51st Annu. Int. Symp. Computer Architecture (ISCA), 2024, pp. 876–889. doi:10.1109/ISCA59077.2024.00068
J. Vesely, A. Basu, M. Oskin, G. H. Loh, and A. Bhattacharjee, “Observations and opportunities in architecting shared virtual memory for heterogeneous systems,” in Proc. IEEE Int. Symp. Performance Analysis of Systems and Software (ISPASS), 2016, pp. 161–171. doi:10.1109/ISPASS.2016.7482091
J. Wahlgren, G. Schieffer, R. Shi, E. A. León, R. Pearce, M. Gokhale, and I. Peng, “Dissecting CPU-GPU unified physical memory on AMD MI300A APUs,” preprint, arXiv:2508.12743, 2025. doi:10.1109/IISWC66894.2025.00038