Tuesday, June 01, 2010

Disposable VMs

While we're still busy with some last few tickets left for Qubes Alpha 2 milestone, Rafal has already started working on a new feature for Qubes Beta 1: on Disposable VMs. I think this is really gonna be a killer feature, and I wanted to say a few words about it.

Disposable VMs will be very lightweight VMs that can be created and booted in a very short time, say < 1s, with a sole purpose of hosting only one application, e.g. a PDF viewer, or a Media Player.

To understand why Disposable VMs are important, imagine the following situation -- you receive an email from a customer that contains a PDF attachment, say an invoice or a contract. Obviously you're opening and reading the message in an email client running in your "work" AppVM (or "work-email" AppVM, if you're paranoid), just because it is a work-related correspondence, arriving at your professional email address (for many reasons it is good to use different email addresses for job-related activities and for personal life).

However, chances of somebody compromising your email client by just sending you a maliciously crafted message that would exploit your body or subject parsers are very small, if you have disabled full HTML parser for message bodies (which I think most security-concious people do anyway). Perhaps a more effective attack vector would be for somebody to 0wn your email server first, and then try to exploit IMAP/POP/SMTP protocol parser in your email client. But hey, in that case, they already would get access to all your emails on the corporate server, without exploiting your email client (well, they could however gain access to your PGP keys this way -- if this bothers you, you might want to use smartcards for PGP keys). There is also a possibility to do a Man-In-The-Middle attack and try to exploit SSL protocol early parsers, but this could be prevented using a separate VPN AppVM in Qubes.

But now you would like to open this PDF that a customer just sent you. It's quite reasonable to be afraid that the PDF might be malicious and might try to exploit your PDF viewer, and then try to steal your emails or other things you keep in the "work" AppVM (or "work-email" AppVM). It doesn't matter if you trust the sender, as the sender's OS might very well be compromised by some malware and might be infecting all outgoing PDFs without the user consent.

You could try opening the PDF in one of your non-sensitive VMs, e.g. the "random" VM that you use for causal Web browsing, to make sure that even if the PDF is malicious, that it won't get access to any sensitive data. But what if the PDF is not malicious, and what if it contains some confidential data? In that case you might throw the baby out with the bath water (your "random" VM might have been already compromised and now it would be able to steal the secrets from your PDF file).

A disposable VM is an ideal solution here. You create a clean, disposable VM, just for the purpose of viewing the PDF. Then, once you're done, you just throw it away. If the PDF was malicious it could done harm only to its own disposable VM, that doesn't contain anything except... this very PDF. At the same time, the disposable VM is always started in a clean state, so there is no way somebody could steal the document. Only the document can steal itself :)

That all sounds easy, but to make it practical we need a very efficient implementation of disposable VMs, and a good system integration, so the experience was seamless to the user. E.g. the user should only be required to right-click on a file and choose "Open in a Disposable VM", and Qubes should take care about everything else: creating the VM, starting it, copying the file to the VM, and starting a MIME-associated application for this type of file (e.g. PDF) in the VM. And this all in time below 1s!

Basic support for Disposable VMs is planned for Beta 1, which is scheduled sometime at the end of the summer holidays. But I can tell that's just the beginning. The ultimate goal, from the user's point of view, would be to make Qubes OS to look and behave just like a regular mainstream OS like Linux, or Windows, or even Mac, but still with all the strong security that Qubes architecture provides, deployed behind the scene. Seamless support for Disposable VM is one of the first steps to achieve this goal.

Special credits go to Matt Piotrowski, who just left Berkeley University, and whose recently published thesis was a direct inspiration to implement disposable VMs in Qubes. While we did mention "one-time" VMs in our architecture document back in January (see chapter 4.6), it really was Matt's paper that convinced me we should really have them in Qubes. Virtics, a proof-of-concept implementation written by Matt, shares lots of similarities with Qubes, like e.g. architecture and implementation of the GUI virtualiztion. There are also differences though, and I refer readers to the Matt's paper for more details.


Anonymous said...

Don't regular VMs already offer that feature?
You can revert to a clean baseline (with snapshots).
The only downside I see is the length of time to open up the VM (usually a minute).


Joanna Rutkowska said...


1 minute startup time is not the "only" downside -- it is a *big* downside.

Image you have 10 PDF files in your folder called "contracts" -- all the files were sent to you by your (various) customers. Now, you want to browser through all of those PDF files. Sure, you could create a regular VM (in Qubes it indeed would take about 1 minute using the qvm-create command) and then copy all those PDF files into this newly created VM. But if one of those PDFs were malicious, it could now compromise this VM you just created, and then steal all the other PDF files.

Anonymous said...

I suppose this is an obvious idea, and may be mentioned in the paper (I have not read it yet), but could you start up a pool of disposable VMs and use them for quickly opening PDFs or whatever?

Anonymous said...

Doesn't Linux (2.6.32+) allow precisely that with cgroups+lxc? Zero "startup" time.

Bruce Downs said...

Will the disposable VMs be like a xen stubdom? A Mini-OS?

Can you share how you plan to manage a < 1s start up time?

Craig said...

I don't see why you would use this over a capability-based approach. Creating an entire VM instance when a more secure API is all that is needed seems like an awful waste of resources.

Joanna Rutkowska said...

Certainly you can do that, and we have even been considering this, but you need to sacrifice your precious RAM for this. So, it's an important tradeoff. Assuming the user would like to view 10 PDF files in a row, if you don't have a really fast VM creation mechanism, you would not catch up with filling the pool.

What do you mean? Linux knows nothing about Qubes/Xen VMs...?

No, a disposable VM would be like any other Qubes AppVM, i.e. having full read-only access to the template VM's root filesystem. The main difference: non-persistent private storage, super-fast-creation time.

We already discuss it in our architecture document, but in short:
1) any monolithic kernel-imposed security mechanism (such as native Linux ones) are by-passable by any kernel/driver exploit.
2) Linux native security mechanisms (even including SELinux) don't provide GUI-level isolation, i.e. if two apps are allowed to talk to the same X server, each should be considered to be able to control the other one (or steal the other one's secretes).

manu ^^ said...

@anonymous that mentioned lxc;

No, there's a big difference between a virtual machine and an isolated container.

Richard said...

Correct if I am wrong, but isn't this what VMWare is offering with their ThinApp solution? Or Citrix XenApp, Microsoft App-V and Novell ZENWorks? A little virtual bubble that the program can run in, isolated from the host OS.

Joanna Rutkowska said...

@Richard: No, wrong guess, try again :)

engur said...

Do you have any plan about different OS based Disposable VMs? e.g. I want to open my cad file in Windows based DVM, open pdf in ubuntu-Linux based DVM and open my media file in a specialized mediacenter linux distro based DVM?

Anonymous said...

I like this concept.....now if there was a way to work on improving the start up time in the VM.

What would be the challenges of this working in a Debian/Ubuntu based distribution? Its not that I have anything against Fedora, I am just more used to using these types of distros.

Will said...

"any monolithic kernel-imposed security mechanism (such as native Linux ones) are by-passable by any kernel/driver exploit."

How is a VM fundamentally different in this respect? 'any VM security mechanism is by-passably by any VM exploit'?

And if that's not true, then why aren't the monolithic kernel-imposed security mechanisms implemented in terms of VMs?

Joanna Rutkowska said...


The primary difference is that a typical monolithic kernel consists of some few-tens-of-million lines of C code, while a bare-metal hypervisor like Xen of only 1-2 hundred thousand lines of code. That's a few orders of magnitude difference. To make it worse: out of those tens of million lines of code in a monolithic kernel, large percent is comprised by *3rd party* drivers, which have much lower code quality than "core" kernel code.

A bare metal hypervisor can be so thin, because it doesn't provide most of the interfaces that a normal kernel provides, e.g. hypervisor knows nothing about devices such as disks or network cards, so don't provide APIs for file and networking, etc.

How many exploitable bugs in bare metal hypervisors have you heard about in the last 5 years? I have heard about just one, and even that one was in some optional code for Xen.

agux said...

I haven't read about this. So, please, make me know if this is a shilly question:

Isn't this like a sandbox? If I have understood, this VM opens the file isolated. Doesn't it?

Joanna Rutkowska said...


A VM is a type of sandbox, because sandbox is a generic term that also covers all various VMs.

But the VMs used in Qubes offer orders of magnitude stronger isolation than other sandboxes used by other OSes. Why is this so? Well, you would need to read our architecture spec paper to learn why.

LGB said...

A VM can be a bit too much for this purpose, I think. Isn't it possible to monitor all the syscalls, etc, so an application can't do anything? I feel that this method tries to workaround the "openness" of a regular OS, that an application can do just too much. It would be much nicer to define some kind of "profiles" what an application can do. From some view point, apparmor does this for example, surely maybe it's not the full solution right now, though. But having dozens of VMs when runnign dozens of applications: hmmm ... Maybe it's time to try to find a better solution at OS level, that there can't be a "full access" for an application even not as non-root user, 99% of apps should get something really restricted set of software/hardware resources from the OS ...

Joanna Rutkowska said...


You should not think about VMs in Qubes like about other VMs you might know from other systems (e.g. VMware, etc). They are much more lightweight, they use smart root filesystem sharing, they use PV-virtualization, which means no need for I/O emulation (no qemu!), etc.

Of course, one can try to do syscall filtering, and X protocol filtering (don't forget about the GUI!) in a normal OS kernel, but than you only add complexity on top of an already complex (and buggy!) kernel. How can you know there would be no bug in the sycall/X filtering code (let me remind again our Xen heap overflow exploit against NSA-written flask _security_ code for Xen!)?

In Qubes, instead of bothering how to correctly implement syscall or X filtering, we use the Xen hypervisor to do isolation for us. The VM<->Xen interface is very thin -- Xen doesn't provide all the usual services that typical kernel must provide (filesystem, networking, etc). And we also have a smart architecture to make other components of the system (networking, storage) non-security-sensitive. This means a compromise of networking or storage subsystem in Qubes, doesn't buy much to the attacker (expect for obvious DoS).

Ian said...

A few years ago we were trying to use VMware to build a sandbox for all the downloaded files, quite similar to the disposable VM you described here. Although the tool itself worked, needless to say speed was its major drawback. I strongly agree that a disposable VM would be a killer app for Qubes. That said, I would envision a wider adoption of this technique: anti-virus companies would kill for something like this in their product!?