Hyper-V Virtual CPUs Explained

This is the most important note. Assigning 2 vCPUs to a system does not mean that Hyper-V plucks two cores out of the physical pool and permanently marries them to your virtual machine. I’ve seen IBM systems that do something like this, but I don’t believe that any other hypervisor does. Hyper-V certainly doesn’t. You can’t actually assign a physical core to a VM at all. So, does this mean that vendor request to dedicate a core just can’t be met? Well, not exactly. More on that toward the end.

Now, back when computers never, or almost never, came in multi-CPU multi-core boxes, we all knew that computers couldn’t really multitask. They had one CPU and one core, so there was only one possible thread of execution. But aside from the fancy graphical updates, Task Manager then looked pretty much like Task Manager now. You had a long list of running processes, all of them with a metric indicating what percentage of the CPUs time they were using.

Then, as in now, each line item you see is a process (or, new in the recent Task Manager versions, a process group). A process might consist of one or many threads. A thread is nothing more than a sequence of CPU instructions (key word: sequence).

What happens is that (in Windows, this started in 95 and NT) the operating system would stop a running thread, preserve its state, and then start another thread. After a bit of time, it would repeat those operations for the next thread. Remember that this is pre-emptive, meaning that it is the operating system that decides when a new thread will run.

The thread can beg for more, and you can set priorities that affect where a process goes in line, but the OS is in charge of thread scheduling.

The only difference today is that you have multiple cores and/or multiple CPUs in practically every system (as well as hyper-threading in Intel processors), so Windows can actually multi-task now.

Because of its role as a thread manager, Windows can be called a “supervisor” (very old terminology that you really never see anymore): a system that manages processes that are made up of threads.  Hyper-V is a hypervisor: a system that manages supervisors that manage processes that are made up of threads. Pretty easy to understand, right?

Task Manager doesn’t work the same way for Hyper-V, but the same thing is going on. There is a list of partitions, and inside those partitions are processes and threads. The thread scheduler works pretty much the same way. What follows is a rethought version of the original image that was submitted for the book, changed to avoid plagiarism:

 Of course, there are always going to be a lot more than just nine threads going at any given time. They’ll be queued up in the thread scheduler.

You probably know that you can affinitize threads in Windows so that they always run on a particular core or set of cores. As far as I know there’s no way to do that in Hyper-V with vCPUs. Doing so would be of questionable value anyway; dedicating a thread to a core is not the same thing as dedicating a core to a thread, which is what many people really want to try to do. You can’t prevent a core from running other threads in the Windows world.

The simplest answer is that Hyper-V makes the decision at the hypervisor level, but it doesn’t really let the guests have any input. Guest operating systems decide which of their threads they wish to operate. The image I presented is necessarily an oversimplification, as it’s not simple first-in-first-out. NUMA plays a role, for instance. Really understanding this topic requires a fairly deep dive into some complex ideas, and that level of depth is not really necessary for most administrators.

The first thing that matters is that (affinity aside) you never know where any given thread is going to actually execute. A thread that was paused to yield CPU time to another thread may very well be assigned to another core when it is resumed.

Did you ever wonder why an application consumes right at 50% of a dual core system and each core looks like it’s running at 50% usage? That behavior indicates a single-threaded application. Each time it is scheduled, it consumes 100% of the core that it’s on. The next time it’s scheduled, it goes to the other core and consumes 100% there.

When the performance is aggregated for Task Manager, that’s an even 50% utilization for the app. Since the cores are handing the thread off at each scheduling event and are mostly idle while the other core is running that app, they amount to 50% utilization for the measured time period. If you could reduce the period of measurement to capture individual time slices, you’d actually see the cores spiking to 100% and dropping to 0% (or whatever the other threads are using) in an alternating pattern.

What we’re really concerned with is the number of vCPUs assigned to a system and priority.

You should first notice that you can’t assign more vCPUs to a virtual machine than you have physical cores in your host.

So,a virtual machine’s CPU count means the maximum number of threads that it is allowed to operate on physical cores at any given time. I can’t set that virtual machine to have more than two vCPUs because the host only has two CPUs. Therefore, there is nowhere for a third thread to be scheduled.

But, if I had a 24-core system and left this VM at 2 vCPUs, then it would only ever send a maximum of two threads up to the hypervisor for scheduling. Other threads would be kept in the guest’s thread scheduler (the supervisor), waiting their turn.

Absolutely. Not only can you, but you’re almost definitely going to. It’s no different than the fact that I’ve got 40+ processes “running” on my dual core laptop right now. I can’t actually run more than two threads at a time, but I’m always going to have far more than two threads scheduled.

Windows has been doing this for a very long time now, and Windows is so good at it (usually) that most people don’t even pause to consider just what’s going on. Your VMs (supervisors) will bubble up threads to run and Hyper-V (hypervisor) will schedule them the way (mostly) that Windows has been scheduling them ever since it outgrew cooperative scheduling in Windows 3.x.

This is the question that’s on everyone’s mind. I’ll tell you straight: in the generic sense, this question has no answer.

Sure, way back when, people said 1:1. Some people still say that today. And you know, you can do it. It’s wasteful, but you can do it. I could run my current desktop configuration on a quad 16 core server and I’d never have any contention. But, I probably wouldn’t see much performance difference. Why? Because almost all my threads sit idle almost all the time. If something needs 0% CPU time, what does giving it its own core do? Nothing, that’s what.

Later, the answer was upgraded to 8 vCPUs per 1 physical core. OK, sure, good.

They went away because they were dumb. I mean, it was probably a good rule of thumb that was built out of aggregated observations and testing, but really, think about it. You know that mostly, operating threads will be evenly distributed across whatever hardware is available.

So then, the amount of physical CPUs needed doesn’t depend on how many virtual CPUs there are. It’s entirely dependent on what the operating threads need. And, even if you’ve got a bunch of heavy threads going, that doesn’t mean their systems will die as they get pre-empted by other heavy threads. It really is going to depend on how many other heavy threads they wait for.

I’m going to let you in on a dirty little secret about CPUs: Every single time a thread runs, no matter what it is, it drives the CPU at 100% (power-throttling changes the clock speed, not workload saturation). The CPU is a binary device; it’s either processing or it isn’t.

The 100% or 20% or 50% or whatever number you see is completely dependent on a time measurement. If you see it at 100%, it means that the CPU was completely active across the measured span of time. 20% means it was running a process 1/5th of the time and 4/5th of the time it was idle.

What this means is that a single thread can’t actually consume 100% of the CPU the way people think it can, because Windows/Hyper-V will pre-empt it when it’s another thread’s turn. You can actually have multiple “100%” CPU threads running on the same system.

The problem is that a normally responsive system expects some idle time, meaning that some threads will simply let their time slice go by, freeing it up so other threads get CPU access more quickly. When you have multiple threads always queuing for active CPU time, the overall system becomes less responsive because the other threads have to wait longer for their turns. Using additional cores will address this concern as it spreads the workload out.

What this means is, if you really want to know how many physical cores you need, then you need to know what your actual workload is going to be. If you don’t know, then go with the 8:1 or 12:1, because you’ll probably fine.

I don’t recommend you tinker with CPU settings unless you really understand what’s going on. Let the thread scheduler do its job. Just like setting CPU priorities on threads in Windows can get initiates into trouble in a hurry, fiddling with hypervisor vCPU settings can throw a wrench into the operations. In fact, I’ll confess that I haven’t spent a great deal of time testing it because I trust the hypervisor enough.

The first group of boxes is the reserve. The first box represents the percentage that I want to set, and its actual meaning depends on how many vCPUs I’ve given the VM. In this case, I have a 2 vCPU system on a dual core host, so the two boxes will be the same. If I set 10 percent reserve, that’s 10 percent of the total physical resources. If I drop this down to 1 vCPU, then 10 percent reserve becomes 5 percent physical. The second box, which is grayed out, will be calculated for you as you adjust the first box.

The reserve is a hard minimum… sort of. If the total of all reserve settings of all virtual machines on a given host exceeds 100%, then at least one virtual machine isn’t going to start. But, if a VM’s reserve is 0%, then it doesn’t count toward the 100% at all (seems pretty obvious, but you never know). But, if a VM with a 20% reserve is sitting idle, then other processes are allowed to use up to 100% of the available processor power… until such time as the VM with the reserve starts up. Then, once the CPU capacity is available, the reserved VM will be able to dominate up to 20% of the total computing power. Because time slices are so short, it’s effectively like it always has 20% available, but it does have to wait like everyone else.

So, that vendor that wants a dedicated CPU? If you really want to honor their wishes, this is how you do it. You enter whatever number in the top box that makes the second box the equivalent processor power of however many pCPUs/cores they think they need. If they want one whole CPU and you have a quad core host, then make the second box show 25%. Do you really have to? Well, I don’t know. Their software probably doesn’t need that kind of power, but if they can kick you off support for not listening to them, well… don’t get me in the middle of that. The real reason virtualization densities never hit what the hypervisor manufacturers say they can do is because of software vendors’ arbitrary rules, but that’s a rant for another day.

The next two boxes are the limit. Now that you understand the reserve, you can understand the limit. It’s a resource cap. It keeps a greedy VM’s hands out of the cookie jar.

The final box is the weight. As indicated, this is relative. Every VM set to 100 (the default) has the same pull with the scheduler, but they’re all beneath all the VMs that have 200, so on and so forth. If you’re going to tinker, this is safer than fiddling with reserves because you can’t ever prevent a VM from starting by changing relative weights. What the weight means is that when a bunch of VMs present threads to the hypervisor thread scheduler at once, the higher weighted VMs go first. That’s it, that’s all.

Hyper-Threading is an Intel-specific technology that lets a single core process two separate instructions in parallel (called pipelines). Neat, right? One problem: the pipelines run in lockstep.

If the instruction in pipeline one finishes before the thread in pipeline two, pipeline one sits and does nothing. But, that second pipeline shows up as another core. So the question is, do you count it? As far as I know, the official response is: No, Hyper-Threading should not be counted toward physical cores when considering hypervisor processing capabilities. Me, I’m a little more lenient. It’s not quite as good as another actual core, but it’s not useless either. Your mileage may vary.