One of the biggest changes I've noticed over the past decade isn't a new processor or a new cache hierarchy. It's the contract between hardware and software. Modern hardware documentation increasingly tells you how to program a feature rather than how that feature actually works internally. That distinction rarely matters for application developers. It matters enormously if you're trying to build predictable systems.
I didn't appreciate this difference until I moved from embedded systems research to Intel server processors. Figuring out exactly where that contract had changed eventually became an entire research paper.
When the hardware contract was explicit
For most of my early research, hardware behaved the way the documentation suggested it would. I was working on embedded platforms, ARM-based SoCs, dedicated QoS hardware, memory controllers built for one job—where predictability itself was part of the product. The manuals described not only how to configure the hardware but enough of its implementation that I could reason about performance before writing a benchmark.
If I wanted to saturate a memory controller, evaluate a QoS mechanism, or isolate a shared resource, I already knew how requests flowed through the system. My synthetic benchmarks weren't discovering the hardware—they were validating my understanding of it.
Then I met Intel RDT
Intel's Resource Director Technology (RDT) looked equally straightforward. It provides Cache Allocation Technology (CAT) for partitioning the shared cache, Memory Bandwidth Allocation (MBA) for regulating DRAM bandwidth, and monitoring features including Cache Monitoring Technology (CMT) and Memory Bandwidth Monitoring (MBM).
The programming interface is exceptionally well documented. The manuals explain which model-specific registers to program, how Classes of Service are assigned, and how monitoring counters are read.
What they don't fully describe is the implementation behavior that matters when you're trying to build predictable systems. How effective are these mechanisms under real contention? Do they behave consistently across processor generations? Can the monitoring counters actually be trusted? Those questions motivated our RTNS paper because, as far as we could determine, nobody had systematically characterized how RDT behaved across processor generations.
The same interface, different hardware
We evaluated the same RDT mechanisms on two Xeon generations: Cascade Lake and the newer Ice Lake. CAT alone demonstrated why experimentation was necessary.
On Cascade Lake, assigning an application a private cache partition did not simply eliminate cache interference. Even with relatively few interfering cores, some workloads experienced a higher LLC miss rate than when CAT was disabled. The behavior was bounded and repeatable, but it wasn't what someone would naturally expect from reading only the programming interface.
Ice Lake behaved differently. Initially CAT produced stable cache-miss rates as contention increased. Beyond a particular number of interfering cores, however, cache misses rose sharply, approaching the same degradation observed without cache partitioning. The mechanism still worked, but its effectiveness depended on contention in ways that weren't obvious from the interface itself.
We hypothesized that reduced cache associativity and undocumented address hashing contributed to the observed behavior and tested that hypothesis using huge pages. Huge pages dramatically improved the Ice Lake results. On Cascade Lake they helped only partially, and only until contention reached a certain point. Even the mitigation depended on the processor generation.
Numbers without units
Memory Bandwidth Allocation exposed the same pattern. MBA accepts throttling levels—10%, 20%, and so on—but those percentages are not documented as corresponding to any concrete bandwidth value. To understand what each setting actually meant, we wrote a bandwidth-saturating benchmark and measured the bandwidth delivered at every available throttling level.
The results differed substantially between generations. On Cascade Lake, settings above roughly 40% delivered nearly identical bandwidth, leaving much of the available control range effectively indistinguishable. On Ice Lake, read-intensive workloads scaled much more linearly, while write-intensive workloads flattened early, resembling Cascade Lake. Reads and writes also could not be controlled independently despite having very different effects on contention.
The only reliable way to interpret an MBA setting was to measure it experimentally on the platform itself.
Monitoring that raised more questions
The monitoring mechanisms presented another challenge. We evaluated CMT using workloads with precisely controlled working-set sizes. Once huge pages were enabled, the reported cache occupancy became difficult to interpret, sometimes exceeding the application's actual working set. MBM also showed noticeable discrepancies compared to the theoretical amount of memory traffic generated by our synthetic benchmarks. The counters were useful, but they weren't accurate enough to simply accept at face value.
Why this isn't really Intel's fault
Initially I found this frustrating. Eventually I realized Intel wasn't solving the same problem I was.
Embedded platforms are frequently deployed into systems where predictability is itself a feature. Automotive controllers, industrial systems, and avionics platforms remain in service for years, sometimes decades. Customers need to understand timing behavior, arbitration policies, and QoS mechanisms because certification depends on them. Documenting implementation details makes the hardware more valuable.
Server processors live under completely different constraints. Intel wants software written today to continue running on the next generation even if the cache hierarchy, bandwidth controller, replacement policy, or memory scheduler are redesigned internally. The stable contract is the programming interface—not necessarily the implementation.
Once I started looking for it, I realized this wasn't unique to Intel. Modern GPUs expose excellent programming models like CUDA, yet researchers still publish papers reverse engineering cache behavior, warp scheduling, address mapping, and memory systems because those implementation details are intentionally abstracted away. The same trend appears across modern high-performance hardware: vendors increasingly document interfaces rather than implementations.
That abstraction is exactly what most software developers want. It allows hardware vendors to innovate underneath a stable API. But for systems researchers interested in predictability, it means understanding the hardware increasingly requires experimentation.
What changed
The main conclusion of our paper was practical: treat RDT as an interface that must be characterized on each target platform. The differences between Cascade Lake and Ice Lake alone were enough to invalidate assumptions that appeared reasonable after studying only one processor generation.
More broadly, it changed how I think about hardware documentation. I no longer assume the manual explains how the hardware works. I assume it explains the contract the hardware promises to preserve.
In embedded systems, I wrote benchmarks to validate my understanding of the hardware. On modern processors, I often write benchmarks to discover what the hardware actually does.