Qubes R3/Odyssey initial source code release

Back in 2013 we've started the work on generalizing Qubes architecture, which we code-named “Odyssey”, to allow for use of multiple hypervisors instead of just Xen via Hypervisor Abstraction Layer (“HAL” -> “Space Odyssey”, get it? ;). The concept has been described in this post, which I recommend to re-read if you're more interested in understanding our goals.

We have been wandering here and there since that time. Lots of work has been invested in the light-weight Qubes edition for Windows, which, sadly, turned out to be a failure.

We have also done a lot of work in the meantime to polish Qubes R2 and bring it to the state of the final release, which happened earlier this fall.

We have also been heavily researching possibilities of other cool projects based on this flexible new architecture. Some of which you might hear about in the coming months, others turned out to be dead ends.

Today we're finally releasing the Qubes R3 source code to the public. The code builds fine (see here for building instruction), produces install-able ISO, and, if that was not enough, even seems to be working, mostly fine, when installed :)

However, we don't recommend users to switch to it, and we intend this release for developers only, specifically those who would like to start working towards porting of other hypervisors, or other containerization technologies, like LXC, to Qubes R3. I highly recommend these devlopers to discuss what they try to achieve on the qubes-devel mailing list, before they start the actual coding.

Currently the only implemented and supported backend is Xen, of course, specifically the Xen 4.4, currently the latest version. It should be now trivial to switch to future versions as they become available, although, a decision to rush with that might not be such a no-brainer from the security point of view. We should remember that the hypervisor, unlike Linux kernel, is not someting you would like to change every month or so. Ideally we should aim for having a stable version of Xen for desktops that would work for years without needing any updates.

But use of other hypervisors might open up lots of interesting possibilities: imagine e.g. Qubes Live USB edition that has backends for 1) Xen, 2) KVM, and 3) LXC, and choose automatically the most secure one which is still supported on the given laptop.

Major features of the current release, compared to Qubes R2:

• Hypervisor Abstraction Layer for all the core management stack (but still missing for the GUI daemon, see below)

• New implementation of vchan and qrexec. As you might know our original vchan has been rewritten and improved (better performance and flexibility) and included in the upstream Xen starting from v4.2. Now we're switching to this upstream libvchan. Also, qrexec has been slightly rewritten to utilize some new features of this libvchan, which results in much better performance for inter-VM traffic (like a few orders of magnitude better!) Especially important for things such as USB virtualization that we're testing right now (not to be confused with USB controller pass-though).

There is still some work going on which we would like to complete before we officially decide to release Qubes OS 3.0-rc1 ISO, and this includes:

• Rewrite of some internal code for the core management stack, which includes internal API of the python classes. This should mostly be of no interest to users, and even most developers working on Qubes.

• Initial code for Qubes Admin API and port of Qubes Manager to use it.

Further down the road (Qubes OS 3.1) we plan to work on some really exciting things:

• More flexibility to qrexec policy (more on that in a separate post)
• More flexibility to Qubes Admin API (expose it to slelect other VMs)
• Split of Dom0 into (semi-depriviliged) GUI domain and minimal Admin domain. This would be great opportunity to also add the missing HAL support for the GUI daemon.

One of the immediate application of these features above would be to introduce support for remote management of Qubes installations, an absolutely necessary feature for corporate adoption of Qubes.

Also note how all these tasks are independent of the actual hypervisor support, meaning it's perfectly possible for other developers to work on porting other hypervisors to Qubes in the meantime.

The possibilities seems to be endless now. Join us and help us with The Revolution! :)

Announcing Qubes OS Release 2!

Today we're releasing Qubes OS R2! I'm not gonna write about all the cool features in this release because you can find all this in our wiki and previous announcements (R2-beta1, R2-beta2, R2-beta3, R2-rc1, and R2-rc2). Suffice to say that we've come a long way over those 4+ years from a primitive proof of concept to a powerful desktop OS which, I believe, it is today.

One of the biggest difficulties we have been facing with Qubes since the very beginning, has been the amount of this extra, not-so-exciting, not directly security-related work, but so much needed to ensure things actually work. Yet, the line between what is, and what is not-security related, is sometimes very thin and one can easily cross it if not being careful.

It's great that we're receiving more and more community contributions. This includes not only bug fixes, but also invaluable efforts related to documentation, HCL maintenance, as well as some really non-trivial new features (advanced backups support, Debian and Arch templates, TorVM, Whonix port, etc). Thanks!

I'm also happy to announce that Caspar Bowden, a well known privacy advocate, expert on EU data protection law, member of the board of Tor, former Microsoft Chief Privacy Adviser, etc, will be taking a role as Qubes Policy Adviser, helping us to make Qubes OS more suitable for a wider audience of people interested in privacy, and be liaising  with other projects that would like to build privacy services with Qubes as a base.

And there is still a lot in front of us. Using the obligatory car analogy, I would say Qubes OS is currently like a racing car that just went into production as a road vehicle: one hell of an engine under-the-hood, and powerful new technologies until now unavailable even for professional use, yet lacking leather interior with 12-speaker audio system, and still with a manual transmission... This is just the beginning for making security by isolation on the desktop as "driveable" as a [insert your fav make of German fine cars] :)

Exciting stuff is coming next: the Release 3 (“Odyssey”) and more, stay tuned!

Thanks to everyone who has made Qubes OS possible, as well as all the upstream projects without which we would probably never even try this journey: Xen, Linux, Xorg, and many others!

Physical separation vs. Software compartmentalization

Many people believe the Holy Grail of secure isolation is to use two or more physically separate machines. This belief seems so natural, that we often don't give it much thought. After all, what better isolation could we possible get than physical "airgap"?

I argue with this point of view in this new paper.

I think a good place for in-depth technical discussions around the topics discussed in the paper would be our qubes-devel mailing list.

Qubes OS R2 rc2, Debian template, SSLed Wiki, BadUSB, and more...

Today we're release the second release candidate (rc2) for Qubes OS R2. There are currently no more open tickets for the final R2 release, and we hope that what we release today is stable enough and so will be identical, or nearly identical, to the final R2 ISO, which we plan to release after the summer holidays. Download and installation instructions are here.

After Qubes rc1 release a few months ago we have been hit by a number of problems related to unreliable VM start-ups. The most prevalent problem has been traced down to an upstream bug in systemd, which just happened to be manifesting on Qubes OS due to specific conditions imposed by our startup scripts.

Actually, it has not been the first time when some things related to VM bootup or initialization didn't work quite well on Qubes, a side effect of heavy optimizations and stripping down we do in order to make the VMs as light weight as possible. E.g. we don't start most of the Desktop Environment which otherwise is assumed to be running by various desktop-related applications and services. In most cases these are really NOTOURBUG kind of problems, yet we just happen to be unlucky they manifest on Qubes. We do need more help from the community with testing, debugging and patching such NOTOURBUG problems in the upstream. The more people use Qubes OS, the higher the chances such problems will be addressed much quicker. Ideally, in the future, we could partner with a Linux distro that would include Qubes AppVM as one of the test cases.

Speaking of different Linux distros --  we have also recently built and released an experimental (“beta”) Debian template for Qubes AppVMs, a popular request expressed by our users for quite some time. It can be readily installed with just one command, as described in the wiki. It is supposed to behave as a first class Qubes AppVM with all the Qubes signature VM integration features, such as seamless GUI virtualization, secure clipboard, secure file copy, and other integration, all working out of the box. Special thanks to our community contributors for providing most of the patches required for porting of our agents and other scripts to Debian. This template is currently provided via our templates-community repo, but it nevertheless has been built and signed by ITL, and is also configured to fetch updates (for Qubes tools) from our server, but we look forward for somebody from the community to take over from us the maintenance (building, testing) of the updates for this template.

Also in our "Templates Appstore" you can find now an experimental “minimal” fedora-based template, which might be used by more advanced users to build customized special-purpose VMs and templates.

We have also moved our Wiki server to a bigger EC2 instance so it could better handle the increased traffic and also added a real CA-signed SSL certificate! But I encourage people to read why this is mostly irrelevant from the security standpoint and why they should still be checking signatures on the ISOs.

We also got a new logo (actually we never really had our own logo before). This also means Qubes now got its own distinct set of themes for installer, plymouth and, of course, a bunch of cool wallpapers with Qubes logo nicely engraved on them. However, it turned out that convincing KDE to set our wallpaper as a default one exceeds the collective mental abilities of ITL, and so one needs to right-click on the desktop and choose one of the Qubes-branded wallpapers manually after install or upgrade.

Every once in a while people (re-)discover that monolithic kernel-based desktop operating systems are not the best solution whenever the user even remotely cares about security...

Yes, USB inherent insecurity, as well as widespread GUI insecurity, or networking stack insecurity, trivial physical insecurities, or sick permissions model as used in most desktop systems, have all been known facts for years. The recognition of these problems has been the primary motivator for us to start the work on Qubes OS back in 2009/2010.

And yes, Qubes running on an appropriate hardware (specifically with Intel VT-d) can solve most of these problems. Correction: Qubes OS can allow the user or administrator to solve these problems, as unfortunately this still requires some configuration decisions made by the human operator. So today Qubes R2 is like a sports manual transmission, which requires a bit of skill to get most out of it. In the near future I see no reason why we should not be offering the "automatic 8-speed transmission" edition of Qubes OS. We just need more time to get there. The R3 release (Odyssey-based), whose early code is planned to be released just after the "final" R2, so sometime in September, is all about bringing us closer to that "automatic transmission" version.

With my 10+ years of experience as a system-level security researcher, I believe there is no other way to go. Don't get deluded that safe languages or formally verified microkernels could solve these problems. Security by Isolation, done sensibly, is the only way to go (of course it doesn't preclude making use of some formally verified components, like e.g. microkernel in place of Xen, at least in some editions of Qubes).

Finally one more announcement for today: after writing this blog for 8 years, I've suddenly felt like I might need to try also some new form of expression... And so, for a few days, I now have a twitter account (@rootkovska), which I hope to use for updates on Qubes, as well as more general commentary on various things happening in IT security.

Qubes OS R2 rc1 has been released!

Today we're releasing Qubes OS R2 rc1 (release candidate), which is expected to be the last milestone before the final Qubes OS R2 release. As mentioned previously today's release is bringing mainly UI improvements and polishing and lots of bugfixes, as well as some last new features: 

• Both Dom0 and VMs have been upgraded to Fedora 20.

• Support for full templates download via two new repo definitions: templates-itl and templates-community. With a bit of imagination we could call it Qubes “AppStore” for VMs :) Currently we have only published one template there – the new default fc20-based template, but we plan to upload more templates in the coming weeks (such as the community-produced Arch Linux and Debian templates). Even though we have a separate repo for community contributed templates, we still plan on building those templates ourselves, from (contributed) sources.

• Support for running Windows AppVMs in “full desktop” mode with support for arbitrary window resizing (which automatically adjusts the resolution in the VMs).

• Support for on-the-fly switching between the “full desktop” and “seamless” modes for Windows AppVMs.

The last two features require, of course, our proprietary Qubes Windows Tools to be installed in the Windows AppVMs to work, which new version we have also published to the new repositories for R2rc1.

We support smooth upgrading for current Qubes R2 Beta 3 users – the procedure is very simple, yet it will take some hours because of the Dom0 distro upgrading.

As can be seen in our ticketing system, there really are only few minor cosmetic tasks left before the final Qubes R2 release. It is expected that upgrade from today's release to the final R2 will be very simple and quick – just standard updates installation.

As usual, the detailed installation and upgrade instructions, as well as the HCL, can be found here. Note however, that the HCL for the today's release will take some days/weeks to compile, as we need to wait for reports from the community, and so for this time the HCL for the previous release (R2 Beta 3) should be used instead. It is reasonable to expect that the new HCL will be a subset of the previous one.

Also, as usual, please keep in mind that we don't control the servers from which the ISO is being served and so please always make sure to verify the digital signature on the downloaded ISO before installing it.

Please direct all the technical questions or comments regarding Qubes OS to our mailing lists.

Enjoy!

Shattering the myths of Windows security

When I originally described the flexible Qubes Odyssey framework several months ago, I mentioned that we would even consider to use “Windows Native Isolation” mechanisms as a primitive type of isolation provider (“hypervisor”) for some basic edition of Qubes for Windows. The idea has been very attractive indeed, because with minimal effort we could allow people to install and run such Qubes WNI on their normal, consumer Windows laptops.

Sure, the inter-process isolation provided by a monolithic kernel such as Windows or Linux could never be compared to the inter-VM isolation offered even by the most lousy hypervisors. This is simply because the sizes of the interfaces exposed to untrusted entities (processes in case of a monolithic kernel; VMs in case of a hypervisor) are just incomparable. Just think about all those Windows system calls and GDI calls which any process can call and which contains probably thousands of bugs still waiting to be discovered by some kid with IDA. And think about those tens of thousands of drivers, which also expose (often unsecured) IOCTLs, as well as parsing the incoming packets, USB devices infos, filesystem metadata, etc. And then think about various additional services exposed by system processes, which are not part of the kernel, but which are still trusted and privileged. And now think about the typical interface that needs to be exposed to a typical VM: it's “just” the virtualized CPU, some emulated devices (some old-fashined Pentium-era chipset, SVGA graphics adapter, etc) and virtualized memory.

Anyway, knowing all this, I still believed that Qubes WNI would make a whole lot of sense. This is because Qubes WNI would still offer a significant boost over the “Just Windows” default security, which is (still) essentially equivalent to the MS-DOS security model.  And this is a real pity, because Windows OS has long implemented very sophisticated security mechanisms, such as complex ACLs applicable to nearly any object, as well as recent mechanisms such as UIPI/UAC, etc. So, why not use all those sophisticated security to bring some real-world security to Windows desktops!

And, best of all, once people start using Qubes WNI, and they liked it, they could then pretty seamlessly upgrade to Xen-based Qubes OS, or perhaps Hyper-V-based Qubes OS (when we implement it) and their system would look and behave very similarly. Albeit with orders of magnitude stronger security. Finally, if we could get our Odyssey Framework to be flexible enough to support both Qubes WNI, as well as Xen-based Qubes OS, we should then be able to support any hypervisor or other isolation mechanism in the future.

And so we decided to build the Qubes WNI. Lots of work we invested in building Qubes WNI was actually WNI-independent, because it e.g. covered adjusting the core Odyssey framework to be more flexible (after all “WNI” is quite a non-standard hypervisor) as well as some components that were Windows-specific, but not WNI-specific (e.g. could very well be used on Hyper-V based Qubes OS in the future). But we also invested lots of time into evaluating all those Windows security mechanisms in order to achieve our specific goals (e.g. proper GUI isolation, networking isolation, kernel object spaces isolation, etc)...

Sadly this all has turned out to be a story without a happy end, as we have finally came to the conclusion that consumer Windows OS, with all those one-would-think sophisticated security mechanisms, is just not usable for any real-world domain isolation.

And today we publish a technical paper about our findings on Windows security model and mechanisms and why we concluded they are inadequate in practice. The paper has been written by Rafał Wojdyła who joined ITL a few months ago with the main task of implementing Qubes WNI. I think most people will be able to learn a thing or two about Windows security model by reading this paper.

Also, we still do have this little hope that somebody will read the paper and then write to us: “Oh, you're guys so dumb, you could just use this and that mechanism, to solve all your problems with WNI!” :)

The paper can be downloaded from here.