Saturday, March 03, 2012

Windows support coming to Qubes!

Ok, let's start with a screenshot :)

While the “Qubes 1.0” branch is currently in the final development and testing, we have already started working on the Next Big Feature, which is a support for HVM domains (hardware, or VT-x virtualized domains). This allows to run e.g. Windows VMs under Qubes. You might be wondering what so special about this, if Xen has been supporting HVM domains, and specifically Windows VMs for a long time, and Qubes uses Xen hypervisor, so why haven't we had Windows support since day one?

The are a couple of things that we don't like about HVM support in Xen (and also in other VMMs), which include: the need to run device emulator (AKA qemu) in Dom0, the need to use crappy VNC, or a similar protocol to access the VM's framebuffer, or alternatively, the crazy idea (from security point of view, that is) of using a pass-through graphics for a VM, the lack of support for disaggregated architecture where backends, e.g. network backends, run in other domains than Dom0. In fact even the Xen “stubdomain” feature, introduced a few years ago, that was supposed to be a solution allowing to move the qemu out of Dom0, in practice turned out to be quite disappointing, as the qemu in the stub domain still requires an accompanying process of another qemu in Dom0, somehow negating all the security benefits this architecture is supposed to bring... And not to mention the omni present assumption that backends run always in Dom0, hardcoded in a few places in the stubdomain code.

So, we didn't like it and that's why Qubes had no Windows support for long time. But this has now changed, as we have just finished the 1st stage implementation of HVM support in Qubes, the way we like it, without any security compromises. In our implementation we've completely eliminated all the qemu remains from Dom0 (it's running in a micro stub domain), the graphics virtualization fully integrates with our very slim GUI daemon (we didn't have to modify our GUI daemon at all!), using our Xen-optimized, zero-copy, minimalist GUI protocol, and the networking is also fully integrated with the Qubes diaggregated networking architecture that uses isolated domains for all the networking stacks and drivers. Of course, there are still some rough edges, such as no clipboard support, and the virtualization is currently in a “per-desktop” mode, rather than in a “per-window” mode, which is used for PV domains. But, rest assured, we are working on those things right now...

This code is currently not public, and the plan is to release it only after Qubes 1.0 release, either as an upgrade, or as Qubes 2.0. All the dom0 code for HVM support will likely remain GPL, while any Windows-specific code (agent code) will likely be proprietary.


Anonymous said...

Wonderful developments going on here! Can't wait for the first release with windows support, as I still need windows for office work and some hardware devices that require windows.

Will this Windows thing also be available for regular user (i.e. non-business license)?

Sam said...

Fantastic! can't wait to try it

Bruce Downs said...

You're trying to cheat minesweeper?!? For shame.

Just to clarify, when you say the code is not public, do you mean a pre-built iso is not public? Could I clone a development repo and build my own iso to try it out?

Anonymous said...

I love you Joanna. This is some awesome stuff you're doing here.

I may have to upgrade my system memory just to better play with this.

The time is coming for qubes to be my main system... I can feel it.

Anonymous said...

is it possible to test it ?

any link?

Very nice work love it!

Anonymous said...

Just love your work!

Can't wait the final release with windows...

Thank you for this good project

Joanna Rutkowska said...

No, the HVM code is currently not available, as it's in heavy development right now.

Anonymous said...

Is it true that you will be moving away from using any linux distro in the future for dom0? In other word you will be trimming down linux as much as possible?

Also I think I read in the news group you will be moving away from kde and going to gnome?

Noah said...

Are you going to contribute the changes back to xen?

Joanna Rutkowska said...

@Anonymous: yes, and yes.

netwho said...

Very impressive projct I have to say!
I'm in the XenClient world which is pretty much HVM oriented... and using vt-d for GPU's and PV drivers for network and storage exact the other way around.

Can't wait to see the windows code too!


AstralStorm said...

Actually, using passthrough graphics (a separate one) is reasonably secure if IOMMU is present and it's not the main GPU.

There are scant few opportunities to snoop on the rest of VMs.

Joanna Rutkowska said...

@AstralStorm: Yes, bu provided yo uthen never share this 2nd GPU among more than one VM (still might be useful -- e.g. to hardwire it to my "personal" VM). Would probably require some OEM magic to do the screen attach/detach between the integrated and discreet GPUs -- anybody has any idea how to do that?

anon said...

Hey Joanna,
your opinions on this "new" bootkit technique?

Researchers are:

Joanna Rutkowska said...

@anon: this is not the best place to ask random security-related questions. A better place would be qubes-devel...

Anyway, regarding this MBR rootkis -- looks rather pointless to me, especially that one can do something like this:

Eli said...

Have you and the qubes development team thought about introducing Mac/BSD virtualization support in the future?

Anonymous said...

Hi, Joanna.

Could you share as more as you can related to you blog portals/sites/forums/irc channel, it would be great to see such post, or see the links in the related div of front page of your blog.
I mean portals about hardware, security, intel and so on.


Anonymous said...

Have you considered using zram as standard and (optionally) not have a swap file on disk?


1. Running VMs will of course make the system more memory hungry, zram will effectively increase the available memory by 50% (?) (at the cost of increased processor usage)

2. On disk swap file is slow, by storing data in ram instead zram will be orders of magnitude faster.

3. On disk swap file is a security problem which requires encryption, whereas the processor cycles required for encryption can be more usefully used for compressing and storing data in RAM.

Joanna Rutkowska said...

But zram, as well as Xen's tmem (which allows for memory pages deduplication) is a Linux-only solution, so couldn't be easily adopted for Windows VMs, right?

Anonymous said...

If the memory compression takes place at host level, outside of the guest VMs, then won't it compress any & all guest memory usage?

tmem may well be better since zram had various features removed in order to be accepted in to the kernel (AFAIK).

Apologies for not talking about your topic of Windows guests. :-)

Joanna Rutkowska said...

But then you would need to run zram at the Xen-level, not at Dom0 level, I think...

Link said...

You need to stop google :).

Anonymous said...

Since one of my older laptops doesn't support VT-x/d, I've been looking for alternatives to XenClient and NxTop that will install and run in a fully PV mode. So when I learned that VT-x/d isn't an absolute requirement for Qubes, I gave it a try right away.

First, let me say that I'm impressed with how smoothly Qubes installed on my older laptop - no problems at all! Also, please accept my compliment for taking such an innovative approach to virtualization. Your virtual apps approach is very promising.

The above positive comments in mind, I must confess that, at least as far as I can tell, despite all its promise, Qubes doesn't seem to offer what I currently need. In particular, Qubes doesn't seem to offer any way (or at least any easy way) to run multiple virtual full DESKTOPS (not just apps) at the same time. Yes, I know, XenClient and NxTop already do that. But as I said, I'm looking for an alternative that doesn't require VT-x/d.

Am I missing something? Does Qubes offer an easy way to run multiple virtual desktops? If not, I hope you'll consider offering such a feature in future versions.


Joanna Rutkowska said...


As shown on the screenshots above, the full desktop support will be supported by the next version of Qubes (> 1.0), as part of supporting Windows VMs. Actually the plan is to have two options for running HVM VMs (so fully virtualized):

1) In full desktop mode -- this is what is shown on the screenshots and what is already running on my laptop,

2) In per-app mode, just like it is currently supported for Linux AppVMs.

The support for running HVM VMs, so Windows, will requires, however, VT-x support (and ideally also VT-d). This is only not needed for Linux AppVMs, but is required for Windows VMs.

Anonymous said...


Thanks so much for the reply and the explanation. I look forward to trying the next version when it comes out!

Best regards,

bradbury9 said...

Watching to next release milestone I see that it has many ineresting features:

- sandboxing audio card. It is mic support, right?
openGL for AppVM. Thats a difficult thing but a huge feature. Video editing, gaming, HD video playback with OpenGL...

I wonder about the release date. Could you gives us a hint?

Al said...

Any comments on Bromium vs Qubes?

Anonymous said...

Any comments on Bromium vs Qubes? Do you see their approach as a viable means to achieve security?

bradbury9 said...


Im in no way endorsed with ITL or cubeos, but I can tell that checking bromium whitepaper I find no info about how they isolate resources and what privileges have different resources (apart from 'least privileges'), so with that lack of info I guess it will be hard to answer. There they dont give much info on how their microVM works, how does it interface with resources.

Anonymous said...


Looks like the release date for "Milestone: Release 1.0" is just a few days away.

Will that be a public release? Hope so, because I'm really looking forward to giving it a try!

About the future of Qubes: What Linux distributions will be supported as guests? That is, will the choice be restricted to certain pre-built guests or will we be able to install whatever distributions we like for use as a guests?


Joanna Rutkowska said...

We will soon release "1.0 Release Candidate 1" -- likely not within the coming days, but within the coming weeks.

Qubes 1.0 will come with a default template based on Fedora 17. It is possible to build your own template, but that's not trivial currently. Qubes 2.0 will allow to install fully virtualized OSes, such as Windows, in a matter similar to VMWare Workstation. It will also have special support for Windows VMs, similar to how we currently handle Linux VMs (seamless GUI integration, template sharing, etc).

BaGgAcFrEaK said...


as there is no info on how to install on usb hdds and how this effects already installed oses on internal hdds and no contact form is on the org www site. I just came across this blog. Would be nice to get some info on this questions and by the way is there a forum for this os somewhere?

ThXs! Great work so far!

David Sundstrom said...


Will your support for Windows include pushing individual application windows to your display VM as you've done for X, or will you always push a complete desktop as a single X window?

I am working on a Xen display VM implementation that is trying to solve a similar problem of providing a unified desktop. We've considered the same approach you've taken for X, but a big requirement we have is that the unified desktop "look and feel" like the native guest desktop. This is further complicated by the fact we allow different guest OS s to be run. At minimum, if we relax that last requirement, we'd still be forced to run Win 7 as the display VM OS, since that is the predominant guest OS people will run and they will want the look and feel of a Win 7 shell.

Our current thinking is that we'd send over the entire guest desktop as a framebuffer and then "clip out" application windows. Similar to RemoteApp (RAIL) in RDP, but without implementing any local window management. All keyboard/mouse would be passed to the correct VM depending on which window has focus, and we'd allow the guest VM to manage its own windows.

Wondering what Qubes will do for WIndows 7. There seem to be two different approaches - either mirror the guest desktop and cut out the app windows or send over the app window buffers and do the desktop management in the display VM. The former allows us to preserve the look and feel of the guest desktop window manager, but the latter allows us to do more fancy desktop composing and unified window management.

Joanna Rutkowska said...

@David: In Qubes 2 Beta 1 (that is to be released soon) we will do only full desktop virtualization. But in the next releases we will be aiming towards the per-app GUI virtualization, done by extracting the composition buffer in the VM.