Most cloud resource managers do the same thing every single time an application runs: they watch it, wait for something to change, and react. Then the application exits, and everything they learned disappears. The next execution starts from zero, as if the system had never seen this workload before.
I spent years building systems that do the opposite, and it started with an uncomfortable realization: if an application is important enough to monitor forever, it's important enough to actually understand — once.
Performance and cost are the same problem
Cloud providers care about two things: how fast applications run, and how much it costs to run them. These aren't separate concerns. They're the same one, viewed from different angles.
When shared resources are managed poorly, applications interfere with each other under contention, and interference shows up as slowdown. Slower applications need more machines to hit the same service-level target. More machines mean more CPUs powered on, more DRAM refreshing, more racks, more cooling. Poor resource management doesn't just cost latency, it wastes hardware a company already paid for.
Last-level cache, memory bandwidth, and compute are some of the most contended, most expensive resources inside a modern server. Managing them well is one of the highest-leverage ways to raise utilization, which means fewer machines doing the same work, lower power draw, and a smaller footprint for the same amount of computing. That's not a sustainability slogan. It's an operations fact: idle or misallocated hardware burns energy whether or not anyone's watching the electricity bill.
What the field does today
Broadly, there are two philosophies in production use, and both fall short in their own way.
Static allocation
The simplest approach is to reserve resources ahead of time and leave them alone: partition the last-level cache with Intel CAT, size a fixed bandwidth quota, tune the split offline with Bayesian optimization, hand each workload a slice with a system like KPart.
This is predictable, and predictability counts for a lot in production. But applications don't behave the same way for their entire lifetime. A service might be cache-hungry in one phase and bandwidth-hungry in the next. A reservation sized for the worst moment sits mostly idle everywhere else. Static allocation buys predictability by giving up efficiency.
Dynamic allocation
The dominant answer to that problem has been to make the resource manager reactive. This is what most modern systems — Heracles, Caladan, CPA, EMBA, Balancer, LFoC, and plenty of others — actually do underneath, even though their mechanisms differ. They watch hardware counters: IPC, misses per kilo-instruction, bandwidth consumed, cache occupancy. When those numbers cross a threshold, the system reacts and reallocates.
Observe. Detect. React. Then observe again.
It's a real improvement over static reservations, and it's the mainstream approach for a reason. But look closely at that loop, and there's a structural problem hiding in it.
Why this is the wrong abstraction
The usual complaint about reactive systems is overhead, sampling counters costs cycles, and reacting takes time the application has already spent suffering under contention. That's true, but it's not the interesting problem.
The interesting problem is that these systems solve the same problem over and over, from scratch, every single time.
They watch an application, react to what it's doing, and then intentionally forget everything the moment it exits. The next execution starts from zero. For a workload that runs once, that's the only option available. For a database, a search backend, an inference pipeline, or a web service that executes the same code paths thousands or millions of times, it's a strange thing to insist on.
Monitoring at runtime is genuinely useful. But it should be there to validate a prediction, not to substitute for one made in advance.
What profile-driven management does instead
The alternative starts from a different assumption: recurring applications aren't random, and their behavior is worth learning once instead of rediscovering constantly.
Every execution becomes a chance to add to what the system already knows, instead of a blank slate. Profiles get richer over repeated runs. Eventually the system can see, before it happens, where memory demand is about to climb, where cache pressure builds, where a function is compute-bound versus memory-bound, when a bandwidth spike is coming.
Once that map exists, the runtime doesn't need to guess anymore, and it doesn't need to watch as closely either, because it already knows roughly what's coming next. The expensive work — the learning — happened offline, ahead of time, off the critical path. What's left at runtime is lightweight: follow the map, adjust when a function boundary is crossed, and fall back to closer monitoring only when something doesn't match what was predicted.
Two systems built on this idea
ProPer
ProPer profiles applications offline at function boundaries, using hardware performance counters gathered across many runs. It learns which functions are compute-, cache-, or memory-bound, then uses that to proactively repartition cache and memory bandwidth the moment execution crosses into a new function, before contention has a chance to hurt anything downstream.
E-WarP
E-WarP applies the same idea to a different resource. It builds a cumulative memory-demand profile offline, predicts how execution time will change under a candidate bandwidth budget, and configures bandwidth enforcement across CPUs and an accelerator together, before the workload ever competes for DRAM on real hardware.
Neither system is magic, and neither throws monitoring away entirely. They still watch the running application at runtime — but only to confirm a prediction, not to replace one.
The cloud already knows
None of this is really about my two systems. The broader claim is: recurring software isn't a mystery, and treating it like one on every single execution is a choice, not a requirement.
Cloud infrastructure already has enormous amounts of information about the applications running on it, how they behave, how they change, what they need and when. Almost all of that information gets thrown away the moment a process exits, because the prevailing philosophy is to rediscover everything at runtime instead of remembering it.
I think the future of resource management looks less like an endless observe-detect-react loop, and more like systems that already understand the applications they run. Today’s cloud infrastructure is increasingly dominated by recurring AI inference pipelines, databases, search engines, and other long-lived services whose execution patterns are far from random. These workloads execute the same code paths millions of times, exposing stable phases of computation, cache usage, and memory demand that can be learned over time. Rather than rebuilding that understanding on every execution through continuous monitoring, future resource managers should accumulate knowledge across runs, use it to anticipate resource demand, and rely on lightweight runtime validation to detect when reality diverges from the expected profile.
Why are we relearning the same application on every single execution? For the applications that matter most, at some point, we should just stop.