Qubes OS R3 Alpha preview: Odyssey HAL in action!

In a previous post I have outlined a new direction we're aiming with the Qubes project, which is a departure from using a “hardcoded” hypervisor with Qubes (as well as “hardcoded” Linux as Dom0, GUI domain, etc).

Today I'm happy to announce that we've already completed initial porting of the current Qubes OS into this Hypervisor-Abstraction-Layer-based framework. The new version of Qubes, that we call “R3 Alpha” for now, builds fine, installs fine, and even (mostly) works(!), as can be admired on the screenshot below :) It still uses Xen, of course, but this time in a non-hardcoded way, which allows to replace it easily with another hypervisor, as I discuss below.

Our Qubes Odyssey backend needed to support a specific hypervisor comprises essentially three parts:

1. A libvirt driver to support a given VMM. In our case we got it (almost) for free, because Xen 4.2 is well supported by libvirt. I wrote “almost” for free, because some patches to libvirt were still needed, mostly to get rid of some unjustified simplifying assumptions, such as that all the backends are always in Dom0, which is not the case for Qubes OS, of course. Some of those patches were accepted into upstream libvirt, some (still) not, so we had to fork libvirt.
2. A VMM-specific implementation of our vchan – a simple, socket-like, VMM shared memory-based communication stack between the VMs. Again, in case of Xen 4.2 we got that (almost) for free, because Xen 4.2 has now included a libxenvchan component, which is modified (improved and cleaned up) version of our original vchan (written in early Qubes days for older Xen versions) contributed and maintained by Daniel De Graff from the NSA.
3. Some minor configuration files, e.g. to tell libvirt which hypervisor protocol to use (in our case: xen:///), and VM configuration template files.

Now, if one wanted to switch Xen for some other hypervisor, such as e.g. the KVM, we would need to write a KVM Odyssey backend in a form of providing the above mentioned three elements. Again, libvirt driver we would get for free, configuration files would be trivial to write, and the only task which would require some coding would be the vchan for KVM.

Ok, one thing that is left out (non-HAL'ified) for now, is the xc_map_foreign_pages() Xen-specific function call within our GUI daemon.

Ideally such call could also be handled by the libvirt API, however it's not clear to us whether true zero-copy page access is really supported (and intended). If it is not, we will try to contribute a patch to libvirt to add such functionality, as it is generally useful for many things that involve high-speed inter-VM communication, of which our GUI virtualization is just one example. So, at this moment, one would need to add an ugly #if (BACKEND_VMM == ...) to the code above and use another VMM's function(s), equivalent to the xc_map_foreign_pages() on Xen.

But besides the above, essentially everything else should Just Work (TM). And that's pretty amazing, I think :) While I personally can't immediately see any security benefit of switching from Xen to KVM, it might appeal to some people for other reasons (Performance? Better hardware support?). The point is: this should be now easy to do.

If one wanted to support some Windows-based hypervisor, on the other hand, such as MS Hyper-V, or Virtual Box on top of Windows, then two more things will need to be taken care of:

1. Our core management stack (the core-admin repository), the core RPC services (mostly the qrexec daemon, currently part of core-admin-linux repo), and the libvirt code (core-libvirt, a forked original libvirt with some custom patches I mentioned above), all would need to build and run fine on Windows. While this is not a big problem for core-admin (it's all python) and core-libvirt (it is supposed to build and run on Windows fine), the qrexec daemon would need to be rewritten with Windows OS in mind. We're currently working on this step, BTW.
2. The GUI daemon would also need to be ported to run on Windows, instead of on top of X Server. This is somehow orthogonal to the need to get rid of the hardcoded xc_map_foreign_pages() function as mentioned above. This step might be optional, however, if we wanted to use a Linux-based (and so Xorg-based GUI server) as a GUI domain.

Once the above two pieces are made Windows-ready (note how I wrote Windows-ready, and not specific-VMM-ready), we can then use any Windows-based hypervisor we want (i.e. for which we have libvirt driver, and can write vchan).

This is again pretty amazing, because it means we don't need N*M variations of each component (where N is the number of VMMs, and M the number of host/GUI OSes to support) – but only N+M! This is similar to how modern compilers are designed using a language-specific frontends (C, C++, Pascal, C#, etc), and architecture-specific backends (x86, x64, ARM, etc), and an Intermediate Language for internal “grinding”, again achieving this N+M number of needed variants instead of N*M, which otherwise would be just totally impractical.

One other detail I would like to point out, and which is also visible on the screenshot above, is that we also got rid of using the Xen-specific Xenstore infrastructure (a registry-like tree-based infrastructure for inter-VM configuration and status exchange), and we replaced it with our own, vchan-based Qubes DB (core-qubesdb).

One interesting thing about Qubes DB is that it get rids of the (overly complex and unnecessary) permission system that is used by xenstore, and instead uses the most simple approach: each VM has its separate Qubes DB daemon, and so a totally separate configuration/state namespace. This is inline with the rest of the Qubes philosophy, which basically says that: permissions is dead, long live separation!

So, in Qubes OS we just isolate everything by default, unless a user/configuration specifically allows an exception – e.g. no file copy operation between domains is possible, unless the user expresses an explicit consent for it.

Many old-school security people can't imagine a system without permissions, but if we think about it more, we might get to a conclusion that: 1) permissions are complex and so often difficult to understand and set correctly, 2) require often complex code to parse and make security decisions, and 3) often are absolutely unneeded.

As a practical example of how permissions schemes might sometime trick even (otherwise somehow smart) developers into making a mistake consider this bug in Qubes we made a long time ago when setting permissions on some xenstore key, which resulted in some information leak (not much of a security problem in general, but still). And just today, Xen.org has published this advisory, that sounds pretty serious, again caused by bad permissions on some xenstore keys. (Yes, we do have updated Xen packages to fix that, of course).

Back to Qubes R3 Alpha, the first successful Qubes based on Odyssey HAL framework. As previously mentioned, we plan to make most of the framework open sourced, specifically all the non-Windows code. However, we're not publishing this Odyssey/R3 code at this moment, mainly for two reasons: 1) we don't want people to immediately start building other backends, such as to support KVM, right at this stage, because we still might want/need to modify some interfaces slightly, e.g. for our vchan, and we don't want to tide our hands now, and 2) the other reason is that we're still in the middle of “Beta” releases for Qubes R2, and we want people to rather focus on testing that, rather stable release, than jumping onto Qubes R3 alpha.

In other news: everybody seems to be genuinely surprised that unencrypted information can be intercepted and analyzed without user consent... Can it be that people will "discover" cryptography now? How many of you use PGP everyday? And how long will it take then to understand that cryptography without secure client devices is useless?