What exactly is the difference between Paravirtualization (PV) and Hardware Virtualization (HVM) with regard to Xen?

This question continues to come up again and again. Rather than answer it in a private email or rather useless IRC chat room, it seems best to summarize it in a blog post.

Paravirtualization means that guests "cooperate" with the virtualization they run under. This means that paravirtualized virtual machines are aware they are running in a virtual environment, and have special drivers or awareness of that environment as they run.

In the case of Xen, all guests run under the guidance of a tiny Xen "hypervisor".

Think of a hypervisor as a microkernel (remember when those were big?), that is responsible for allocating RAM, acting as an intermediary for IO, routing hardware interrupts, and scheduling a fair share of CPU time to each virtual machine.

By default in Xen, one virtual machine talks to the PCI hardware, doing all IO for the others. In Xen parlance, this is "domain 0" (or "dom0"), which is a master OS that talks to the hardware on the box and provides IO resources like networking and disk space to the other domains on the physical machine. There are Linux, OpenSolaris, and FreeBSD dom0s now, it isn't just linux.

With Paravirtualization, your guest kernels need to be compiled to be aware of the Xen hypervisor, with a special Xen patch set. These kernels cannot run without the Xen hypervisor. They require Xen to operate.

The "new" thing out there is something that Xen coins Hardware Virtualized Machines (HVM). Normally, x86 based Operating Systems run with a kernel running in "ring0". Historically, only one Operating System can run as ring0 on a x86 based PC. Now, both Intel/AMD have added special VT/SVM CPU extensions that allow a special "privileged mode" of operation where a hypervisor can run multiple Operating Systems in ring0 at the same time.

Historically, without these VT/SVM instructions, you have to scan every code page for illegal instructions and/or trap instructions to emulate dangerous ones. This is how VMWare initially worked (today VMWare is a software/hardware hybrid that is aware of VT/SVM instructions). This is how QEMU, Parallels, Microsoft Virtual Server, and other virtualized PC platforms initially functioned.

With Hardware virtualization, you install an Operating System from CDROM just as if it is a physical machine. There is a BIOS, there is a VGA display (VNC/SDL), there are emulated IDE and RTL8139 network chipsets. The hardware is actually borrowed from the QEMU project, but thanks to the VT/SVM instructions, there is no need to scan the code or trap illegal CPU instructions the same way as the previous generation of PC virtualization had to.

The mainframe has had Hardware Virtualization since at least the OS/360 days. This is only something new to the PC platform.

While this is a fun essay, and I'd love to go on at length, I think this answers the initial question adequately.

If anyone else has any questions, please feel free to join us on ##xen on freenode, or drop me an email. Please don't be suprised if I post the answers here.

Xen documentation is sorely lacking. Lets try and change that, shall we?

I just finished backporting Xen 3.0.4 and a slew of 2.6.16.33 kernels to our standard platform (building patched debian packages along the way).

The new bits are an API change, better support for SMP and ACPI, some bug fixes, and framebuffer consoles for PVs (borrowing from HVM, it appears).

Here is the announcement from the list:

Folks, 

We're pleased to announce the official release of xen 3.0.4!

This is largely an opportunistic stabilising release for HVM guests, due to
the large amount of work in that area of the code since 3.0.3. These
enhancements have in particular improved support for SMP and ACPI Linux and
Windows operating systems.

Other highlights of this release include:
  - support for kexec/kdump of Xen and domain 0;
  - graphical framebuffer support for paravirtualised guests;
  - preview support for the new XenAPI management interfaces;
  - enhanced support for IA64 (IPF) and Power systems.

Since 3.0.4 is an interim release, certain features such as HVM save/restore
will now be part of Xen 3.0.5 which we expect to release in early 2007.

You can get the source using mercurial from:
  http://xenbits.xensource.com/xen-3.0.4-testing.hg

Source and binary tarballs, and RPMs, will be made available from:
  http://www.xensource.com/downloads

Cheers,
Keir (on behalf of the whole Xen dev team)