The Invisible Things Lab's blog

Thoughts on DeepSafe

2012-01-21T18:01:00.002+01:00

Several people asked me recently what Ithough about DeepSafe.So, below I present my opinion...

First, for any AV system (or Host IPS,or Personal Firewall, etc) to work effectively, there are three problems that must be addressed:

How to protect the AV agent (code and data) from tampering (from the rest of the OS)?
How can the AV agent get reliable access to (sensitive pieces of) the system memory and registers, and/or provide reliable memory protection for the (sensitive pieces of) the OS.
What are those "sensitive pieces of” memory that should be monitored or protected?

From reading various PR materials, itseems like the #1 above is the primary differentiation factor forDeepSafe (DS). So, let's consider this problem in the context of e.g.a Windows OS. In order to protect its code and data, DS uses, as itis heavily advertised, Intel VT-x virtualization technology. Now,that sounds really secure -- after all what can be more secure than ahardware virtualization, right? ;)

But VT-x (including EPT) is only aboutCPU virtualization, which in our case translates to protecting the DSmemory and registers from CPU-originating accesses. But, as everyregular to this blog knows, there is also another method of accessingmemory on any PC system, and this is through DMA transactions fromdevices. The OS (so also the kernel malware) is free to program oneof the many devices in the system to issue DMA reads or writes to anyphysical memory itwants...

Now, in order to protect some portionof the system memory (DRAM, cache) against DMA accesses, we have theIntel VT-d technology... So, one would think that DS must be alsousing VT-d in order to protect itself.

Very well, let's assume then that theDeepSafe is not a total ripoff,and that it implements also VT-d protection for its agent, although Ihaven't found this mentioned in any of the public papers or pressmaterials I found on the web...

This, however, would be a bit complexto do correctly, because the OS (so, also the kernel malware) stillhas a full control over the chipset (MCH), which is the entity...that controls the VT-d.

Now, in order toprevent the OS (or the kernel malware) from playing with the chipsetfor fun and profit, and e.g. disabling VT-d protection, DS would haveto virtualize the chipset.

If you look at some consumer VMMs, suchas VMware or Xen/Qemu, you would see that they all virtualize thechipset for their guests (of course), but that the chipset theyprovide this way is some kind of an ancient Pentium MCH. I don'tthink any of the consumers would be especially happy if they foundout that after installing DS on their brand new 2012 laptop, Windowssuddenly see a Pentium-era chipset... And this is not without areason – chipsets, specifically MCHs, are one of the most complexdevices, perhaps only beaten by GPUs in this category. There arevirtually hundreds of configuration registers exposed by the chipset,some of them control the VT-d, some other control system memory mapsand permissions, PCIe configuration, and many other things that Ieven have no idea about, and this all makes virtualizing the chipseta very challenging task.

So, it's either that McAfee and Intelfound some interesting way of how to securely virtualize the chipsetwhile preserving all of its (very rich) functionality, or thatthey... don't bother with VT-d protection and chipset virtualizationat all, assuming that even without VT-d, DeepSafe is good enough and“rises the bar” anyway (sarcasm intended).

(Can somebody from McAfee or Intelconfirm in the comments below what does DP really do?)

Anyway, let's assume they dohave VT-d protection and they do virtualize the chipsetsomehow...

Now, we're moving on to the #2 pointfrom the list of tasks above -- about the reliable

memory access or reliable protection.

So, let say that the DS agent decidedthat some part of the system memory, e.g. the IDT table, is sensitiveand should be monitored/protected. So it sets up EPT traps to triggeran VT-x/EPT intercept on any access to that memory (or IDT baseregister), in order to find kernel malware that tried to modify IDT.That sounds really nice, but what if the malware uses DMA to modifyIDT? DS would not be able to catch such access! (So far we consideredthe, hypothetical, use of VT-d only to protect the DS agent code).

One might think that DS is programmingVT-d to sandbox each and every device in the system (so includingGPU, USB controllers, NICs, SATA, etc) so they never beallowed to touch any of those sensitive parts of the system, such asIDT. Let's assume they do it this way...

And here we've arrived to the lastpoint from the list at the beginning: which of the system memoryconstitutes those "sensitive pieces" that should beprotected/monitored? IDT? Sure. What about all the code sections ofthe all the kernel modules? Yes. Are we fine now? Well, no, becausethe malware can hook some pointers other than the well known IDT.Some public NDIS data structure? Ok, we can add those to theprotected areas. But, what about some undocumented NDIS structures?And this is just NDIS subsystem, one of the many subsystems in theWindows kernel... When we think about it, it should be intuitivelyobvious that in a general purpose Operating System like Windows, itis not possible (at least for 3^rd party) to make asatisfactory list of all the sensitive pieces of memory that shouldbe monitored/protected, in order to detect all the systemcompromises.

Greg Hoglund, Jamie Butler, AlexTereshkin, and myself, have been researching this area actively inthe early years of this millennium. In addition to the Alex's paperlinked above, you might also check out one of my last slides fromthisperiod.

I don't think anything has changedsince that time. It was also the reason why I gave up on writingWindows compromise detectors, or forensic tools, and moved on toresearching other ways to secure OSes, which finally gave birth toQubes OS, many years later.

So, back to DS -- in order to provide asomehow satisfactory protection level for your general purpose OS,such as Windows, it would need to:

Use VT-d to protect its own memory,

Virtualize the chipset, at least some (sensitive) parts of it,

Program VT-d permissions for each device to exclude all the sensitive areas in the system that should be protected, and also protect one device from DMAing into/from another device memory (e.g. NIC stealing GPU framebuffer, or inserting instructions to the GPU instruction buffer, or keystrokes to USB controller buffer). Ideally, this could be done by programming VT-d to grant each device only access to its own DMA buffer, but as far as I know, this would be very hard to implement, if not impossible for a 3rd party, on a Windows OS (in contrast to Linux, which mostly support this model). Please correct me, if the recent Windows version allows for such use of VT-d.

Finally, and the most hard thing to solve, how to define all the "sensitive pieces of memory" in the system that should be protected and/or monitored? Although this is a somehow more generic problem, not specific to DS, but applying to any A/V, HIPS, or forensic tool.

So, is DeepSafe another piece of crap not worth any special attention, or has McAfeeand Intel came up with some novel methods, e.g. for chipsetvirtualization, and other problems? Unless I see some technical info to backup the latter, I would have to assume,unfortunately, the former. But I would like to be mistaken – afterall DeepSafe seems to be just a new incarnation of my Bluepill ;)

Trusted Execution In Untrusted Cloud

2011-12-13T20:25:00.000+01:00

Wouldn't it be nice if we couldactually own our data and programs in the cloud? By “owning” hereI mean to have control over their confidentiality and integrity. Whenit comes to confidentiality and integrity for the data,it's not much of a rocket since, as the classic crypto (andsecure client systems) is all that we need. Ihave already wrote about it in an earlier post.

But itwould also be nice, if we could somehow get the same confidentialityand integrity assurance for our programsthat we upload for the execution in the cloud...

Forexample, a company might want take theirdatabase application, that deal with all sorts of corporate criticalsensitive data, and then upload and safely run this application on e.g.Amazon's EC2, or maybe even to some China-based EC2-clone. Currently there is really nothingthat could stop the provider, who has a full control over the kernelor the hypervisor under which our application (or our VM) executes, from reading the contents of our process' memory and stealingthe secrets from there. This is all easy stuff to do from the technical point ofview, and this is also not just my own paranoia...

Plus,there are the usual concerns, such as: is the infrastructure of thecloud provider really that safe and secure, as it is advertised? Howdo we know nobody found an exploitable bug in the hypervisor and wasnot able to compromise other customer's VMs from within theattacker-hired VM? Perhaps the same question applies if we didn't decided to outsource the apps to a 3^rdparty cloud, but in case of a 3^rdparty clouds we really don't know about what measures have beenapplied. E.g. does the physical server on which my VMs are hosted also used to host some foreign customers? From China maybe? Youget the point.

Sometimesall we really need is just integrity, e.g. if we wanted to host anopen source code revision system, e.g. a git repository or a fileserver. Remember the kernel.org incident?On a side note, I find the Jonathan Corbet's self-comforting remarkson how there was really nothing to worry about, to be strikinglynaive... I could easily think of a few examples of how theattacker(s) could have exploited this incident, so that Linus &co. would never (not soon) find out. But that's another story...

But, how can one protect a running process, or a VM, from apotentially compromised OS, or a hypervisor/VMM?

Tosome extent, at least theoretically, Intel Trusted ExecutionTechnology (TXT), could be used to implement such protection.Intel TXT can attest to a remote entity, in that case this would bethe cloud customer, about the hash of the hypervisor (or kernel) that has beenloaded on the platform. This means it should be possible for the userto know that the cloud provider uses the unmodified Xen 4.1.1 binaryas the hypervisor and not some modified version, with a built-in FBI backdoor formemory inspection. Ok, it's a poor example, because the Xenarchitecture (and any other commercially used VMM) allow the administrator who controls Dom0 (or equivalent) to essentiallyinspect and modify all the memory in the system, also that belongingto other VMs, and no special backdoors in the hypervisor are needed for this.

But let's assume hypothetically that Xen 5.0 would change thatarchitecture, and so theDom0 would not be able to access any other VM's memory anymore. Additionally,if we also assumed that the Xen hypervisor was secure, so that it wasnot possible to exploit any flaw in the hypervior, then we should be fine.Of course, assuming also there were also no flaws in the TXT implementation, and that the SMM was properly sandboxed, or that we trusted (some partsof) the BIOS (these are really complex problems to solve in practice, but I knowthere is some work going on in this area, so there is some hope).

Such aTXT-bases solution, although a step forward, still requires us to trust the cloud provider abit... First, TXT doesn't protect against bus-level physical attacks– think of an attacker who replaces the DRAM dies with some kind ofDRAM emulator – a device that looks like DRAM to the host, but onthe other end allows full inspection/modification of its contents(well, ok, this is still a bit tricky, because of the lack ofsynchronization, but doable).

Additionallyfor Remote Attestation to make any sense, we must somehow know thatwe “talk to” a real TPM, and not to some software-emulated TPM.The idea here is that only a “real” TPM would have access to aprivate key, called Endorsement Key, used for signing during RemoteAttestation procedure (or used during the generation of the AIK key,that can be used alternatively for Remote Attestation). But thenagain who generates (and so: owns) the private endorsement keys? Well,the TPM manufacturer, that can be... some Asian company that we notnecessarily want to trust that much...

Now we see it would really be advantageous for customers, if Intel decidedto return to the practice of implementing TPM internally inside the chipset, as they did in the past for their Series 4 chipsets (e.g.Q45). This would also protect against the LCP bus-level attacksagainst TPM (although somebody told me recently that TPM in currentsystems cannot be so easily attacked from LCP bus, because of someauthentication protocol being used there – I really don't know, asphysical attacks have not been the area we ever looked atextensively; any comments on that?).

Butthen again, the problem of DRAM content sniffing always remains,although I would consider this to be a complex and expensive attack.So, it seems to me that most governments would be able to bypass suchTXT-ensured guarantees in order to “tap” the user's programsexecuting in the cloud provides that operate within theirjurisdictions. But at least this could stop malicious companies fromstaring up fake cloud services with an intent to easily harvest somesensitive data from unsuspecting users.

It seems that the only way to solve the above problem of DRAM sniffing attacks is to add some protection at the processor level. We can imagine two solutions that processor vendors could implement:

First,they could opt for adding an in-processor hardware mechanism forencrypting all the data that leave the processor, to ensure thateverything the is kept in the DRAM is encrypted (and, of course, alsointegrity-protected), with some private key that never leave the processor. This could be seen as an extension to the Intel TXT.

This would mean, however, we still needed to relay on:1) the hypervisor to not contain bugs, 2) the whole VMM architectureto properly protect VM's memory, specifically against the Dom0, 3)Intel TXT to not be buggy either, 4) SMM being properly sandboxed, oralternatively to trust (some parts of) the BIOS and SMI handler, 5)TPM's EK key to be non-compromised and verifiable as genuine, and 6)TPM bus attacks made impossible (those two could be achieved bymoving the TPM back onto the chipset, as mentioned above), and finally, 7) on theencryption key used by the processor for data encryption to be safelykept in the processor.

That'sstill quite a lot of things to trust, and it requires quite a lot of work to make it practically really secure...

The other option is a bit more crazy, but also more powerful. Theidea is that the processor might allow to create untrustedsupervisors (or hypervisors).Bringing this down to x86 nomenclature, it would mean that kernelmode (or VT-x root) code cannot sniff or inject code into (crypto-protected) memory of the usermode processes (or VT-x guests).This idea is not as crazy as you might think, and there has even beensome academic work done in this area.Of course, there are many catches here, as this would requirespecifically written and designed applications. And if we everconsidered to use this technology also for client systems (how niceit would be if we could just get rid of some 200-300 kLOC of the Xenhypervisor from the TCB in Qubes OS!), the challenges are evenbigger, mostly relating to safe and secure trusted output (screen)and, especially, input (keyboard, mouse).

Ifthis worked out, then we would need to trust just one element: theprocessor. But we need to trust it anyway.Of course, we also need to trust some software stack, e.g. thecompilers we use at home to build our application, and the librariesit uses, but that's somehow an unrelated issue. What is important isthat we now would be able to choose that (important) software stackourselves, and don't care about all the other software used by thecloud provider.

As Iwrote above, the processor is this final element we always need torust. In practice this comes down to also trusting the US government :) But we might imagine users consciously choosing e.g.China-based, or Russia-based cloud providers and require(cryptographically) to run their hosted programs on US-madeprocessors. I guess this could provide reasonable politically-based safety. And there is also ARM, with its licensable processor cores, where, I canimagine, the licensee (e.g. an EU state) would be able to put theirown private key, not known to any other government (here I assume thelicensee also audits the processor RTL for any signs of backdoors). I'm not sure if it would bepossible to hide such a private key from a foundry in Hong Kong, orsomewhere, but luckily there are also some foundries within the EU.

In anycase, it seems like we could make our cloud computing orders ofmagnitude safer and more secure than what is now. Let's see whetherthe industry will follow this path...

Exploring new lands on Intel CPUs (SINIT code execution hijacking)

2011-12-06T10:48:00.001+01:00

Today we're releasing a new paper where we describe exploiting a bug in Intel SINIT authenticated code module that allows for arbitrary code execution in what we call an “SINIT mode”. So, to the already pretty-well explored “lands” on Intel processors, that include ring 3 (usermode), ring 0 (kernelmode), ring “-1” (VT-x root), and ring “-2” (SMM), we're now adding a new “island”, the SINIT mode, a previously unexplored territory inhabited so far only by the Intel-blessed opcodes.

What is really interesting about the attack are the consequences of SINIT mode hijacking, which include ability to bypass Intel TXT, LCP, and also compromise system SMRAM.

It's also interesting how difficult was this vulnerability for Intel to patch, as they had to release not only updated SINIT modules, but also updated microcode for all the affected processors, and also work with the BIOS vendors so they release updated BIOSes that would be unconditionally loading this updated microcode (plus provide anti-rollback mechanisms for both the BIOS and microcode). Quite an undertaking...

You can get the paper here.

Intel also published an advisory yesterday, which can be downloaded from their website here. The advisory is peculiar in a few ways, however...

First, the advisory (I'm referring to the revision 1.0) never explicitly mentions that the attack allows to bypass TXT launch itself, only that the attack “may compromise certain SINIT ACM functionality, including launch control policy and additionally lead to compromise of System Management Mode (SMM). Intel also recommend to disable TXT altogether in the BIOS, as a preventive measure, in case the user doesn't “actively running Intel® TXT”... This reminds me how various vendors started actively disabling Intel VT-x after certain virtualization rootkits have been demonstrated some 5 years ago, and how many laptops still ship with this technology disabled today (or VT-d at least) to the questionable delight of many users.

Second, the advisory assigns only an “Important” rating to this vulnerability, even though another Intel advisory, published some two years ago for a problem also reported by us, and which which was strictly a subset of the current vulnerability in terms of powers that it gave to the attacker (in other words the current vulnerability provides the attacker with everything that the previous one did, plus much more), was given a “Critical” rating... This is called evolution, I guess, and I wonder what would be considered critical by Intel these days?

UPDATE (Dec 7th, 2011): Intel has just released an updated advisory (release 1.1) that now explicitly states that the vulnerability also bypasses Intel TXT.

This is the last paper co-authored with Rafal Wojtczuk, who recently decided to try some new things and to leave ITL. Rafal has been the most talented exploit writer I have worked with, and I will surely miss his ingenious insights, such as e.g. how to practically win an absolutely hopeless race condition with ICMP-delivered MSI! But then again, how many times can one break Intel technologies, before getting bored? At the same time ITL is really transforming now into a development company, with all our efforts around Qubes and architecting, rather than on breaking. I wish Rafal all the best with his new endeavors, and thank him for all the excellent contributions he made while working for ITL over the past 3+ years.

Playing with Qubes Networking for Fun and Profit

2011-09-28T16:36:00.002+02:00

Today, I would like to showcase some of the cool things that one can do with the Qubes networking infrastructure, specifically with all the new features that have been brought by the just released Qubes Beta 2. This will cover the use of multiple Net VMs for creating isolated networks, the use of a Proxy VM for creating a transparent Tor Proxy VM, as well as demonstration of how to use a Standalone VM with manually assigned devices, to create a “WiFi pen-testing” VM, which surely represents the “for fun” aspect of this post.

Qubes Networking Intro

From the networking point of view there are three types of VMs in Qubes:

Net VMs, that have networking devices assigned to them, such as e.g. a WiFi or Ethernet card. Each Net VM contains a Xen network backend that is used to provide networking to all VMs that are connected to this Net VM.
Regular VMs (AppVMs) that use the networking provided by Net VMs (so they have Xen network frontends that provide virtual interfaces that are backed by the backend in the corresponding Net VM.
Proxy VMs that combine both of the above: to Net VMs they look like regular AppVMs, because they are consumers of the networking they provide, but to other AppVMs they act as if they were Net VMs themselves, allowing other VMs to connect to them. Of course the Proxy VMs do not have directly assigned networking devices – they use the networking provided by the Net VM that they connect to. One can chain many Proxy VMs, as we will see below.

The virtual interfaces in client VMs are called ethX, and are provided by the xen_netfront kernel module, and the corresponding interfaces in the Net/Proxy VM are called vifX.Y and are created by the xen_netback module.

Each Net and Proxy VM implements NAT, specifically masquerading, for all the connected VMs. Additionally to this SNAT, each Net or Proxy VM provides also DNAT redirection for DNS resolutions, so that each VM behind a Proxy or Net VM thinks that it uses a DNS in the Net/Proxy VM, but in fact all the DNS request are DNAT-ed by all the Proxy and Net VMs down the original DNS that is provided to the final Net VM. This smart trick allows us to avoid running a DNS caching server in Proxy/Net VMs.

Also, any VM-to-VM traffic, among the VMs connected to the same Net/Proxy VM is blocked by default.

Additionally, each Proxy VM enforces system-wide firewaling rules, specifically the rules for all the directly connected VMs. Those firewalling rules are centrally managed in Dom0 and exposed to each Proxy VM through Xen store. One useful application of this firewalling mechanism is to limit certain VMs to only specific type of white-listed traffic to minimize likelihood of user mistakes. A good example could be a work VM that might be limited to network connectivity only with the select corporate servers and denied all other traffic. This way, when the user receives an email message with an embedded http link (possibly leading to a malicious website) and accidentally clicks on it, nothing wrong happens.

The current infrastructure doesn't support IPv6 routing, but we will likely add this support in the upcoming Beta 3.

The default networking topology in Qubes OS

When you proceed with the default installation of Qubes Beta 2, then your initial networking topology looks like on the diagram below:

The default network configuration in Qubes.

So, by default there is one Net VM, called 'netvm', that is automatically assigned all the networking devices in the system. There is also one Proxy VM, called 'firewallvm' that is directly connected to the default Net VM, and which provides networking to all other VMs in the system. This Proxy VM is used for firewall rules enforcement. Each such service VM consumes 200MB of RAM by default.

Network-isolated VMs

For some VMs it might be desirable to completely disconnect them from any kind of networking access. This can be easy done using the following command (issued from Dom0's konsole):

[dom0]$ qvm-prefs -s netvm none

For example I have a 'vault' VM that I use for keeping my master PGP keys, and other secrets, and this machine is not connected to any network.

Using multiple Net VMs for physically isolated networks

In some scenarios the machine might be connected to two or more physically separate networks (e.g. safe corporate intranet, reachable via ethernet cable on the user's desk, and the unsafe and evil Internet, reachable via WiFi card).

It is easy to use more than one Net VMs in Qubes, and assign different networking devices to different Net VMs, and also decide which VMs are connected to which Net VMs. The diagram below presents an exemplary such setup:

A simple setup with two isolated networks, and one fully isolated domain ('vault').

We could created such a setup using the following commands (issued in Dom0):

[dom0]$ qvm-create netvm1 --net --label red

[dom0]$ qvm-create netvm2 --net --label yellow

Currently qvm-create when used with the --net option automatically assigns all networking devices to the just created VM, so in the example above you would want to remove extra devices from each Net VM using qvm-pci -d, leaving only those you really want, e.g.:

[dom0]$ qvm-pci -l netvm1 # to get a list of currently assigned devices

[dom0]$ qvm-pci -d netvm1 02:00.0

Now we should create the Firewall VMs:

[dom0]$ qvm-create firewallvm1 --proxy --label green

[dom0]$ qvm-create firewallvm2 --proxy --label green

... and connect them to proper Net VMs:

[dom0]$ qvm-prefs -s firewallvm1 netvm netvm1

[dom0]$ qvm-prefs -s firewallvm2 netvm netvm2

And now, for any other VM, just set the appropriate Net VM (either firewallvm1 or firewallvm2, or 'none), to get it assigned to either of the isolated networks, e.g.:

[dom0]$ qvm-prefs -s banking netvm firewallvm1

[dom0]$ qvm-prefs -s xfiles netvm firewallvm2

[dom0]$ qvm-prefs -s vault netvm none

...

This configuration provides very strong isolation between the VMs belonging to network #1, and the VMs belonging to network #2. Specifically, this becomes significant if we fear about potential remotely exploitable bugs in the client code of the core TCP/IP stack (in this case the Net VM could potentially compromise all the connected VMs -- but the same problem applies to even physically separated machines that use the same network).

Setting up Tor Proxy using a Proxy VM

Let's now play a bit with Proxy VMs and see how we can use it to create a simple Tor proxy VM. Such a VM would provide anonymized networking to all its clients, so would allow to easily create VMs for anonymous Internet access. The simple setup we would like to prepare is depicted on the figure below:

The 'torvm' Proxy VM provides anonymized networking to 'anon-web' and 'anon-bitcoin' VMs. All the traffic generated by the VMs behind 'torvm' is either fed into the Tor network, or discarded. Furthermore, any app running in those VMs is not able to read any global system identifiers, such as the external IP, external MAC address, etc.

Our Tor proxy would forward only the Tor traffic, so we don't have tofear about some Tor-not-aware applications, or even intentionallymalicious ones to compromise the privacy of our connection. This isbecause such applications have no way to generate traffic to theoutside world without going through our Tor proxy (unless they couldexploit a hypothetical vulnerability in the Tor process running inthe Tor VM). Also, the applications running in any VM behind the Torproxy are not able to determine any globally identifiable IDs, suchas the user's external IP address, the real MAC address used by realNICs, etc.

Interestingly just after writing the above paragraph, I discoveredthat one of our xenstore keys had wrong permissions and, as a result,any VM could read it and get to know the actual external IP (the keyis used by a Net VM to communicate the external IP configuration tothe connected Proxy VMs, so they could know when to update thefirewall configuration). The fix for this problem is here,and the update (qubes-core-dom0-1.6.32) is now available for Dom0(just do qvm-dom0-updateto get it installed).

So, this represents a rather strong setup for use with Tor. Let's nowhave a look at how to practically create such a configuration, stepby step.

First, let's create the VM that will become our Tor proxy:

[dom0]$qvm-create torvm --proxy --label green

This will create a Proxy VM named 'torvm', based on the defaulttemplate. We will need to now start the template VM and install theTor client there:

[dom0]$ qvm-run-a fedora-14-x64 gnome-terminal

Alternatively, if we didn't trust the Tor client rpm package to benon-malicious, specifically for its installation scripts to be nonmalicious, we could have based this on a different template, e.g. oneused for less trusted VMs, or we could installed the Tor client in/usr/local,that is backed by the VM's private storage, but this would requirecompiling Tor from sources.

Now, in the just started template VM,lets install the Tor client and (optionally) the Vidalia graphicalfrontend:

[fedora-14-x64]$sudo yum install tor vidalia

And then power offthe template VM. Now, every VM based on this template, started afterthe template shutdown, will also see the Tor binary in itsfilesystem.

Let's now configureour torvm to properly start Tor proxying at boot:

[dom0]$ qvm-run-a torvm gnome-terminal

Now, we will createthe following script for starting up the Tor transparent proxy andsetting up traffic redirection using iptables:

[torvm]$ vim/rw/config/start_tor_proxy.sh

and now paste thefollowing into this file:

#!/bin/sh
killall tor
QUBES_IP=$(xenstore-readqubes_ip)
TOR_TRANS_PORT=9040

if [ X$QUBES_IP== X ]; then
echo"Error getting QUBES IP!"
echo"Not starting Tor, but setting the traffic redirection anyway toprevent leaks."
QUBES_IP="127.0.0.1"
else
/usr/bin/tor\
--SocksPort0 \
--TransListenAddress$QUBES_IP --TransPort $TOR_TRANS_PORT \
--DNSListenAddress$QUBES_IP --DNSPort 53 \
--RunAsDaemon1 --ControlPort 9051 \
|| echo"Error starting Tor!"

fi

echo “0” >/proc/sys/net/ipv4/ip_forward
/sbin/iptables-t nat -F
/sbin/iptables-t nat -A PREROUTING -i vif+ -p udp --dport 53 -j DNAT--to-destination $QUBES_IP:53
/sbin/iptables-t nat -A PREROUTING -i vif+ -p tcp -j DNAT --to-destination$QUBES_IP:$TOR_TRANS_PORT
/sbin/iptables-I INPUT 1 -i vif+ -p udp --dport 53 -j ACCEPT
/sbin/iptables-I INPUT 2 -i vif+ -p tcp --dport 9040 -j ACCEPT
/sbin/iptables-F FORWARD

echo “1” >/proc/sys/net/ipv4/ip_forward

Except for the“QUBES_IP=$(xenstore-readqubes_ip)” line that reads the torvm'sIP address, there is nothing Qubes-specific in the above listing.It's just a standard way of setting up transparent Tor proxy.

It is importantthat this file be located in the /rwdirectory, as this directory is backed by the VM's private storageand will survive VM reboots. The VM's root file-system is read-onlyand all the changes to it are lost on VM shutdown (VM gets anillusion of the root fs being writeable thanks to Copy-On-Writemechanism, but the actual COW backing device is cleared upon each VMshutdown).

We should alsomodify the /rw/config/rc.localscript, to ensure that our Tor proxy is automatically started -- justpaste the following into this script:

#!/bin/sh

# Uncommentthis if you would like to use a custom torrc file:
#rm -f/rw/config/log
#ln -sf/rw/config/torrc /etc/tor/torrc

chkconfigqubes_netwatcher off
chkconfigqubes_firewall off
/rw/config/start_tor_proxy.sh

Finally we shouldalso provide a script that would restart our proxy in case the userdynamically switched the NetVM, which would result in the completelydifferent routing. This could be done by creating a script withpredefined name qubes_ip_change_hookwithin /rw/config/directory:

#!/bin/sh
/rw/config/start_tor_proxy.sh

Make sure that all the scripts are executable (chmod +x). And that's all.Now, shutdown the torvm:

[dom0]$ qvm-run--shutdown --wait torvm

From now on, everytime you start the torvm (or when Qubes starts it in response tostart of some other VM that uses torvm as its Net VM), the Tortransparent proxy should be automatically started.

Let's test this bycreating a VM that would be using the just created Tor proxy:

[dom0]$qvm-create anon-web --label black

[dom0]$qvm-prefs -s anon-web netvm torvm

Now, every time youstart the anon-web VM (e.g. by clicking on the Web browser icon inthe anon-web's start menu), Qubes will also ensure that torvm is upand running, and this in turn would configure all the Tor proxyingfor this VM.

Fo additionalcontrol one might want to use Vidalia, the graphical front end forTor (this should be installed within the template VM that has beenused for torvm). We could easily start Vidalia by just typing:

[dom0]$ qvm-run-a torvm vidalia

We should howevermake sure to disable "Start the Tor software when vidaliastarts" option in Settings/General in Vidalia. Otherwise,Vidalia might kill your original Tor (that has transparent proxyopen) and start own without transparent proxy enabled.

The web browser runs in the 'anon-web' VM that uses 'torvm' for networking access, and thus all the traffic generated by 'anon-web' is routed through the Tor network, or discarded if it's a different traffic than TCP or DNS.

Of course one case easily create more VMs that would be using torvmas their Net VM, as so would have anonymized network access. Thebeauty of this solution is that in case one of my anonymized VM getscompromised, others do not. Plus, the already mentioned benefit, thatno matter whether apps in those VMs are buggy, or even intentionallymalicious, they would not be able to leak out the user's external IPaddress.

Creating a WiFipen-testing VM

Finally let's have some fun and create a WiFi pen-testing VM. Thedesired config is depicted below:

Because we would like to use all sorts of ~~l33t h4x0r t00lz~~pen-testing security software in this VM, it would make sense tocreate it as a StandaloneVM, which means thatit would get its own copy of the whole file-system (as opposed tojust the home directory, /rwand /usr/local,as it is the case with regular Qubes VMs). This would ease theinstallation of all the extra software we would need there, and alsoensure that even if the install/build scripts were malicious, thedamages would be contained only to this very VM and nothing else.Also, for some reason the standard Linux WiFi stack and drivers stilldon't support injection on (all?) most of the WiFi cards out of thebox, so we would need to patch the actual kernel drivers -- yetanother reason to use a Standalone VM in this case.

So, let's create the VM first, and assign a WiFi card to it:

[dom0]$qvm-create wififun --standalone --label yellow

[dom0]$qvm-prefs -s wififun memory 800 #ensure at least this mem at startup

[dom0]$qvm-prefs -s wififun kernel none #use own copy of kernel and modules

[dom0]$ qvm-pci-a wififun

Youcan easily find the BDF address of any device using the lspcicommand in Dom0 -- this would be something like e.g. “02:00.0”.You should make sure that this WiFi card is not used by any other VM,specifically by your default Net VM (called 'netvm' in a standardQubes installation). Ideally you could just use a dedicated ExpressCard-based WiFi card, leaving the built in WiFi assigned to yourdefault Net VM.

Because it's aStandalone VM, Qubes will make a copy of the whole root filesystem,and thus it would eat about 5GB of your disk (normal VMs would takeonly as much space as their private fs takes up).

Let's now start the VM...

[dom0]$ qvm-run-a wififun gnome-terminal

... and then install the prerequisite software there, starting withdownloading the reasonably new compat-wireless sources, together withthe required injection patches, and then building and installing thenew kernel modules. All actions below are now executed within the VM.This stuff here is really nothing Qubes- or Xen-specific -- one woulddo more or less the same on any Linux in order to get injectionworking (so, treat this as a free bonus WiFi hacking tutorial onLinux).

[wififun]$ wgethttp://linuxwireless.org/download/compat-wireless-2.6/compat-wireless-2011-07-14.tar.bz2

[wififun]$ wgethttp://patches.aircrack-ng.org/channel-negative-one-maxim.patch

[wififun]$wgethttp://patches.aircrack-ng.org/mac80211-2.6.29-fix-tx-ctl-no-ack-retry-count.patch

[wififun]$wgethttp://patches.aircrack-ng.org/mac80211.compat08082009.wl_frag+ack_v1.patch

[wififun]$ sudoyum install kernel-devel patch gcc

[wififun]$ tarxjf compat-wireless-2011-07-14.tar.bz2

[wififun]$ cdcompat-wireless-2011-07-14

[wififun]$patch -p1 < ../channel-negative-one-maxim.patch

[wififun]$patch -p1 < ../mac80211-2.6.29-fix-tx-ctl-no-ack-retry-count.patch

[wififun]$patch -p1 < ../mac80211.compat08082009.wl_frag+ack_v1.patch

[wififun]$ make

[wififun]$ sudomake unload

[wififun]$ sudomake install

Now, lets reboot the VM to ensure thatall the patched drivers will get properly loaded on each VM boot:

[dom0]$ qvm-run--shutdown --wait wififun

[dom0]$ qvm-run-a wififun gnome-terminal

Let's first see if the WiFi driver got properly loaded and if theinterface has been created (look for wlanXinterface):

[wififun]$ifconfig -a

If yes, then proceed with the steps below (if not, then have a lookinto dmesg and see what was the problem):

[wififun]$ sudobash

[wififun]# yuminstall aircrack-ng dnsmasq

[wififun]#airmon-ng start wlan0

[wififun]#iptables -F INPUT

[wififun]#iptables -F FORWARD

[wififun]# echo“1” > /proc/sys/net/ipv4/ip_forward

Note that you don't need to add anyexplicit masquerading rules, as they are applied by default on QubesVMs (you can take a look at the nat table in the VM if youwant to see by yourself).

Edit the /etc/dnsmasq.conf,so that it contains at least the following:

interface=at0

dhcp-range=192.168.0.50,192.168.0.150,12h

and then start the dnsmasq daemon -- wewill use it for providing DHCP to our fake AP (the at0 interface willbe created by airbase-ng and emulates the “uplink” of atraditional AP):

[wififun]#/etc/init.d/dnsmasq start

And finally the fake AP:

[wififun]#airbase-ng -e free_wifi mon0

and on another console (before anyclient connects, but after airbase-nggot started), configure the at0interface (make sure it matches what you wrote into dnsmasq.conf):

[wififun]#ifconfig at0 192.168.0.1 up

(you can also add an udev rule to thatautomatically).

and just to verify it really isworking:

[wififun]#tcpdump -i at0

... and now, just wait for a client toconnect to your AP. What you do next is only limited by yourimagination... But hey, this article is about Qubes networking andnot about 0wning client systems ;)

Here's an innocent example usingMoxie's sslstrip (amazing this attack still works so well at the endof 2011...):

My 'wififun' VM in action using a simple sslstrip attack, that surprisingly still works pretty nice...

Please note that as your wififun VM is a regular Qubes VM, it isautomatically connected to the default Net VM, which in turn providesnetworking to it. That's why it is so easy to create a fullyfunctioning fake AP.

When using custom driver domains, there are currently some catchesyou should be aware:

Catch #1: Whenyou start a driver domain lateafter system boot, so after some days of uptime and extensive use ofVMs, Xen might not be able to allocate enough continues (in terms ofMFNs) memory for a driver domain. And PV driver domains, unlikenormal domains or HVM driver domains, do require MFN-continuous memory for their DMA buffers (HVM domains do not need that, becauseIOMMU can create an illusion of this; even though IOMMU is also usedfor PV driver domains, for protection, it doesn't actively translatebus addresses into GMFNs).

This is usually not a bigproblem in practice, because in most cases all the driver domains arestarted early at system boot, when there is still plenty ofnon-fragmented memory available. However it might become a problemwhen one wishes to start e.g. the WiFi pen-testing at some latertime. The work around is to close as many VMs as possible beforestarting such driver domain, and then also reducing, for a moment,the amount of memory assigned to Dom0:

[dom0]$xm mem-set 0 1600m

andthen starting the driver domain should be fine. Now we can start allother domains, and that should no longer be problematic for thealready running driver domain.

Catch #2: Somenetwork cards, notably Express Cards, might not work well with the3.0.4 pvops kernel that we use in all VMs by default. In that caseyou might want to try to use the 2.6.38.3 xenlinux kernel in yourWiFi fun VM -- to do that, follow these steps:

[dom0]$sudo qvm-dom0-update kernel-qubes-vm-2.6.38.3-10.xenlinux.qubes

[dom0]$cp /var/lib/qubes/vm-kernels/2.6.38.3/*/var/lib/qubes/appvms/wififun/kernels/

[dom0]$qvm-prefs wififun -s kernelopts "swiotlb=force"

And then, in the VM:

[wififun]$ sudoyum install kernel-devel-2.6.38.3-10.xenlinux.qubes

And rebuild the compat-wireless,unload, install modules, and then load drivers again.

Summary

As you can see, Qubes Beta 2 now offers a very advanced networkinginfrastructure that allows more advanced users to create verysophisticated configurations, allowing for pretty good isolationbetween various domains and networks. Qubes leaves it up to the user(or admin) to figure out what would be the best configuration -- mostusers would be happy with the default simple setup with just one NetVM and one Firewall VM, while others would go for much more advancedsetups.

A bit more advanced networking setup. The usbvm has a 3G modem assigned, and it is possible to dynamically switch between the Net VMs without restarting any other VMs.

Qubes Beta 2 Released!

2011-09-19T12:52:00.000+02:00

I'm proud to announce that we have just released Qubes Beta 2! You can view installation instructions and download the ISO here.

We faced quite a few serious problems with this release that were caused by an upgrade to Xen 4.1 (from Xen 3.4) that we used in Beta 1. But finally we managed to solve all those problems and all in all I'm very happy with this release. It includes many performance optimizations compared to Beta 1 (CPU- and memory-wise) and also many bugfixes.

We also introduced a couple of new features:

Generic mechanism for inter-domain services with a centralized policy enforcement (more)

Network-less update mechanism for Dom0 (more)

VM management improvements: easy device assignment for driver domains, dynamic netvm switching, flexible VM kernel configuration, etc (see the new qvm-prefs utility)

Easy management of appmenus (shortcuts in the Start Menu)

Update to Xen 4.1 that offers, among other things, better VT-d support and more lightweight management stack (we have ported Qubes to use the new xl now, instead of the slow and heavy xend), and also to 2.6.38-xenlinux kernel for Dom0, and to 3.0.4 pvops kernel for VMs (better hardware compatibility, better power management)

I will write some more posts shortly that would present in detail some of the new features and what cool things one could do with them.

We have also created a dedicated wiki page that enumerates all the security-critical code for Qubes OS. We hope this page would be useful for security researchers that might attempt to find weaknesses in Qubes OS either in our code or in the 3rd party code that we rely on (Xen hypervisor, select Xen backends). Whether your motives are noble (gaining immortal fame, helping create a secure client OS), or not (proving ITL wrong), we would appreciate your efforts! And you might even get a job at ITL.

Speaking of which, I'm happy to announce that Marek Marczykowski, who has effectively become the key Qubes developer over the past few months, has now officially joined ITL :)

Anti Evil Maid

2011-09-07T23:56:00.004+02:00

Anti Evil Maid is an implementation of a TPM-based static trusted boot with a primary goal to prevent Evil Maid attacks.

The adjective trusted, in trusted boot, means that the goal of the mechanism is to somehow attest to a user that only desired (trusted) components have been loaded and executed during the system boot. It's a common mistake to confuse it with what is sometimes called secure boot, whose purpure is to prevent any unauthorized component from executing. Secure boot is problematic to implement in practice, because there must be a way to tell which components are authorized for execution. This might be done using digital signatures and some kind of CA infrastructure, but this gets us into problems such as who should run the CA, what should be the policy for issuing certificates, etc.

The adjective static means that the whole chain of trust is anchored in a special code that executes before all other code on the platform, and which is kept in a non re-flashable memory, whose sole purpure is to make the initial measurement of the next component that is going to be executed, which is the BIOS code. This special code, also known as Core Root of Trust for Measurement (CRTM), might be part of the BIOS (but kept on a special read-only memory, or implemented by some other entity that executes before the BIOS reset vector, such as e.g. Intel ME or the processor microcode even. Once measured, the BIOS code is executed, and it is now its turn to measures the platform configuration, Option ROM code, and MBR. Then the loader (stored in the MBR), such as Trusted GRUB, takes over and measures its own next stages (other than the MBR sector), and the hypervisor, kernel, and initramfs images that are to be loaded, together with their configuration (e.g. kernel arguments).

As explained above, trusted boot can only retrospectively tell the user whether correct (trusted) software has booted or not, but cannot prevent any software from executing. But how can it communicate anything reliably to the user, if it might have just been compromised? This is possible thanks to the TPM unseal operation that releases secrets to software only if correct software has booted (as indicated by correct hashes in select PCR registers).

So the idea is that if a user can see correct secret message (or perhaps a photo) being displayed on the screen, then it means that correct software must have booted, or otherwise the TPM would not release (unseal) the secret. Of course we assume the adversary had no other way to sniff this secret and couldn't simply hardcode it into the Evil Maid – more on this later.

Another way to look at it is to realize that Anti Evil Maid is all about authenticating machine to the user, as opposed to the usual case of authenticating the user to the machine/OS (login and password, decryption key, token, etc). We proceed with booting the machine and entering sensitive information, only after we get confidence it is still our trusted machine and not some compromised one.

Installing Anti Evil Maid

Anti Evil Maid should work for any Linux system that uses dracut/initramfs, which includes Qubes, Fedora and probably many other distros. You can find the Anti Evil Maid source code in a git repository here. You can also download a tarball with sources and prebuilt rpm packages from here (they all should be signed with the Qubes signing key). Qubes Beta 2, that is coming soon, will have those RPMs already per-installed.

To install Anti Evil Maid, follow the instructions in the README file.

Some Practical considerations

If you decided to use no password for your TPM SRK key (so, you passed '-z' to tpm_takeownership, see the README), then you should definitely install Anti Evil Maid on a removable USB stick. Otherwise, if you installed it on your disk boot partition, the attacker would be able to just boot your computer and note down the secret passphrase that will be displayed on the screen. Then the attacker can compromise your BIOS/MBR/kernel images however she likes, and just hardcode the secret passphrase to make it look like if your system was fine.

If you decided to use custom TPM SRK password (so, you did not pass -z to tpm_takeownership), then you can install Anti Evil Maid onto your regular boot partition. The attacker would not be able to see your secret passphrase without knowing the SRK password. Now, the attacker can try another Evil Maid attack to steal this password, but this attack is easy to spot and prevent (see the discussion in the next section).

However, there is still a good argument to install Anti Evil Maid on a separate USB stick rather than on your built-in disk boot partition. This is because you can use Anti Evil Maid as a provider of a keyfile to your LUKS disk encryption (as an additional file unsealable by the TPM). This way you could also stop adversary that is able to sniff your keystrokes (e.g. using hidden camera, or electromagnetic leak), and capture your disk decryption passphrase (see the discussion in the next section).

In any case it probably would be a good idea to make a backup stick that you might want to use in case you lose or somehow damage your primary stick. In that case you should have a way to figure out if your system has been compromised in the meantime or not. Use another stick, with another passphrase, and keep it in a vault for this occasion.

Finally, be aware that, depending on which PCRs you decided to seal your secrets to, you might be unable to see the secret even after you changed some minor thing in your BIOS config, such as e.g. the order of boot devices. Every time you change something in your system that affects the boot process, you would need to reseal your secrets to new PCR values as described in the installation instructions.

Attacks prevented by Anti Evil Maid

The classic Evil Maid attack is fully prevented.

If the attacker is able to steal your Anti Evil Maid stick, and the attacker gets access to your computer, then the attacker would be able to learn your secret passphrase by just booting from the stolen stick. This is not fatal, because user should get alarmed seeing that the stick has been stolen, and use the backup stick to verify the system (with a different secret messages, of course), and later create a new stick for every day use with a new secret message.

A variation of the above attack is when the attacker silently copies the content of the stick, so that the user cannot realize that someone got access to the stick. Attacker then uses the copied stick to boot the user's computer and this way can learn the secret passphrase. Now, the attacker can infect the computer with Evil Maid, and can also bypass Anti Evil Maid verification by just hardcoding the secret message into Evil Maid. So, even though TPM would know that incorrect software has booted, and even though it would not unseal the secret, the user would have no way of knowing this (as the secret would still be displayed on screen).

In order to protect against this attack, one might want to use a non-default SRK password – see the installation instructions. Now an extra SRK password would be needed to unseal any secret from the TPM (in addition to PCRs being correct). So the attacker, who doesn't know the SRK password, is now not able to see the secret message and cannot prepare the Evil Maid Attack (doesn't know what secret passphrase to hardcode there).

The attacker might want to perform an additional Evil Maid attack targeted at capturing this SRK password, e.g. by infecting the user's stick. This, however, could be immediately detected by the user, because the user would see that after entering the correct SRK password, there was no correct secret passphrase displayed. The user should then assume the stick got compromised together with the SRK password, and should start the machine from the backup stick, verify that the backup secret is correct, and then create new AEM stick for daily usage.

If an attacker is able to capture the user's keystrokes (hidden camera, electromagnetic leaks), the attacker doesn't need Evil Maid attack anymore, and so doesn't need to bother with compromising the system boot anymore. This is because the attacker can just sniff the disk decryption password, and then steal the laptop and will get full access to all user data.

In order to prevent such a “keystroke sniffing” attack, one can use an additional sealed secret on the Anti Evil Maid stick that would be used as a keyfile for LUKS (in addition to passphrase). In this case the knowledge of the sniffed LUKS passphrase would not be enough for the attacker to decrypt the disk. This has not been implemented, although would be a simple modification to dracut-antievilmaid module. If you decided to use this approach, don't forget to also create a backup passphrase that doesn't need a keyfile, so that you don't lock yourself from access to your data in case you lose your stick, or upgrade your BIOS, or something! You have been warned, anyway.

Attacks that are still possible

An adversary that is able to both: sniff your keystrokes (hidden camera, electromagnetic leak) and is also able to copy/steal/seize your Anti Evil Maid stick, can not be stopped. If a non-democratic government is your adversary, perhaps because you're a freedom fighter in one of those dark countries, then you likely cannot ignore this type of attacks. The only thing you can do, I think, is to use some kind of easy-to-destroy USB stick for keeping Anti Evil Maid. A digestible USB stick, anyone?

Another type of attack that is not addressed by Anti Evil Maid is an attack that works by removing the “gears” from your laptop (the motherboard and disk at the very least), putting there a fake board with a transmitter that connects back to the attacker's system via some radio link and proxies all the keyboard/screen events and USB ports back to the original “gears” that execute now under supervision of the attacker. Another way of thinking about this attack is as if we took the motherboard and disk away, but kept all the cables connecting them with the laptop's keyboard, screen, and other ports, such as USB (yes, very long cables). The attacker then waits until the user boots the machine, passes the machine-to-user authentications (however sophisticated it was), and finally enters the disk decryption key. In practice I wouldn't worry that much about such an attack, but just mentioning it here for completeness.

Finally, if our adversary is able to extract secret keys from the TPM somehow, e.g. using electron microscope, or via some secret backdoor in the TPM, or alternatively is able to install some hardware device on the motherboard that would be performing TPM reset without resetting the platform, then such an attacker would be able to install Evil Maid program and avoid its detection by SRTM. Still, this doesn't automatically give access to the user data, as the attacker would need to obtain the decryption key first (e.g. using Evil Maid attack).

Implementation Specific Attacks

In the discussion above we assumed that the trusted boot has been correctly implemented. This might not be true, especially in case of the BIOS. In that case we would be talking about attacks against a particular implementation of your BIOS (or TrustedGRUB), and not against Anti Evil Maid approach.

One typical problem might be related to how CRTM is implemented – if it is kept in a regular BIOS reflashable memory, than the attacker who can find a way to reflash the BIOS (which might be trivial in case your BIOS doesn't check digital signatures on updates) would be able to install Evil Maid in the BIOS but pretend that all hashes are correct, because the attacker controls the root of trust.

Another possible implementation problem might be similar to the attack we used some years ago to reflash a secure Intel BIOS (that verified digital signatures on updates) by presenting a malformed input to the BIOS that caused a buffer overflow and allowed to execute arbitrary code within the BIOS. For such an attack to work, however, the BIOS should not measure the input that is used as an attack vector. I think this was the situation with the logo picture that was used in our attack. Otherwise, even if there was a buffer overflow, the chain of trust would be broken and thus the attack detected. In other words, the possibility of such an attack seems to be rather slim in practice.

What about Intel TXT?

Intel TXT takes an alternative approach to trusted boot. It relies on a Dynamic instead of Static Root of Trust for Measurement (DRTM vs. SRTM), which is implemented by the SENTER instruction and special dynamic PCR registers that can be set to zero only by SENTER. Intel TXT doesn't rely anymore on the BIOS or CRTM. This offers a huge advantage that one doesn't need to trust the BIOS, nor the boot loader, and yet can still perform a trusted boot. Amazing, huh?

Unfortunately, this amazing property doesn't hold in practice. As we have demonstrated almost 3 years ago (!), it is not really true that Intel TXT can remove the BIOS away from the chain of trust. This is because Intel TXT is prone to attacks through a compromised SMM, and anybody who managed to compromise the BIOS would be trivially able to also compromise the SMM (because it is the BIOS that is supposed to provide the SMI handler).

Thus, if one compares SRTM with Intel TXT, then the conclusion is that Intel TXT cannot be more secure than SRTM. This is because if an attacker can compromise the BIOS, then the attacker can also bypass Intel TXT (via a SMM attack). On the other hand, a BIOS compromise alone doesn't automatically allow to bypass SRTM, as it has been discussed in a paragraph above.

It really is a pity, because otherwise Intel TXT would be just a great technology. Shame on you Intel, really!

Alternative approaches to mitigate Evil Maid Attacks

Various people suggested other methods to prevent Evil Maid attacks, so lets quickly recap and discuss some of them...

The most straight forward approach suggested by most people, has been to disable booting from external devices in BIOS, together with locking the BIOS setup with an admin password.

There are two problems with such an approach. First, all the BIOSes have a long history of so called default passwords (AKA maintenance passwords). You don't want to rely on the lack of BIOS default passwords when protecting your sensitive data, do you?

Second, even if your BIOS doesn't have a backdoor (maintenance password), it is still possible to just take your disk away and connect to another laptop and infect its boot partition.

Another suggested approach has been to keep your boot partition on a separate USB stick. This solution obviously doesn't take into account the fact that the attacker might install Evil Maid into your BIOS. Many consumer laptop BIOSes do not require digital signatures on BIOS firmware updates (my Sony Vaio Z, a rather high-end machine, is among them), making it simple to install Evil Maid there (the most trivial attack is to make the BIOS always boot from the HDD instead of whatever other device the user wanted to boot from).

Finally, some people pointed out that many modern laptops comes with SATA disks that offer ability to “lock” the disk so that it could only be used with a specific SATA controller. Using this, combined with setting your BIOS to only boot from your internal disk, plus locking access to BIOS setup, should provide reasonable protection. This solution, of course, doesn't solve the problem of a potential maintenance password in your BIOS. Also being skeptical and paranoid as I am, I would not trust this mechanism to be really robust – I would expect it would be fairly simple to unlock the disk so that it could be paired with another, unauthorized controller, and that this probably is a matter of NOP-ing a few instructions in the controller firmware... In fact it seems like you can buy software to unlock this mechanism for some $50... And apparently (and not very surprisingly) some drives seems to continue on the 'default passwords' tradition.

FAQ

Q: Bitlocker implemented this already several years ago, right?
A: No.

Q: But, two-factor authentication can also be used to prevent Evil Maid, right?
A: No.

Q: Does it make any sense to use Anti Evil Maid without a full disk encryption?
A: No.

Q: Are you going to answer 'no' for each question I ask?
A: No.

Q: Why there are no negative indicators (e.g. a big scary warning) when the unseal process fails?
A: The lack of negative indicators is intentional. The user should keep in mind that if somebody compromised their computer, then the attacker would be able to display whatever she wants on the screen, and especially to skip displaying of any warning messages. The only thing the attacker would not be able to display would be the secret message. Thus, it would make no sense to use negative indicators, as they would likely not work in case of a real attack. One solution here would be to use the unsealed secret as a keyfile for disk encryption (as discussed above), which would make it impossible to decrypt the user disk (and so generally proceed with the boot) without successfully unsealing the secret from the TPM.

Interview about Qubes OS

2011-08-30T23:06:00.001+02:00

Here is a recent interview with me for Tom's Hardware, where I talk about Qubes, why virtualization alone does not automatically bring much security, and why we need it for secure systems anyway, and all that kind of stuff. Nothing really new, but still might be of interest to some readers.

As for Qubes Beta 2 release -- it really is coming, but we've faced recently some very nasty, race-condition-related problems with new Xen (we bravely switched to Xen 4.1 in Beta 2) that seem to occur on machines with very fast SSDs and we're currently trying to see if we can solve them, or should we instead revert back to Xen 3.4 that we used previously in Beta 1. Except for that, Beta 2 is mostly ready, so we should be releasing it within coming weeks.

My SSTIC 2011 slides

2011-06-10T15:02:00.002+02:00

A few days ago I had a privilege to give an opening keynote at the SSTIC conference in Rennes, France, which is believed by many to be the most important security conference in France. You can find my slides here.

SSTIC seems to be a very interesting conference indeed, with a strong emphasis on system-level security, which is quite unusual these days where most conferences focus on networking, apps, and web-apps. What a pity all those interestingly-looking talks have been encoded in an obscure language used only by some 3% of the population of the planet...

Anyway, it was a pleasure to talk to some ANSSI people I met before the conference (one of the organizers of the event) who really seemed to understand well the challenges we face with building secure operating systems, and generally seemed well versed in the topic. Perhaps some other nations should learn from France, instead of proposing ridiculous and superficial means that can't really solve any real problem.

From Slides to Silicon in 3 years!

2011-06-03T17:16:00.003+02:00

Remember our Xen 0wning Trilogy at Black Hat in summer 2008, specifically the presentation on Detecting & Preventing the Xen Hypervisor Subversions?

One of the things we were discussing there was a proposal to include an additional restriction to Intel processors that would disallow execution of usermode pages from within supervisor mode (ring0). Such a feature, we argued, apart from obviously making many ring3-to-ring0 exploits much harder, including the very Xen heap overflow exploit we presented in the slides, would also bring us closer to efficient runtime code integrity checkers for kernels and hypervisors, as discussed in the slides.

Slide #97, Detecting and Preventing Xen Hypervisor Subversions, Black Hat USA, July, 2008

Fast forward 3 years. On June 1^st, 2011, an Intel engineer is submitting a patch for Xen to support a mysterious new processor feature called SMEP (Supervisor Mode Execution Protection). He writes the feature is not yet documented in SDM, but soon will be. In fact, the May 2011 update of Intel SDM already contains the details:

Intel SDM, vol. 3a, May 2011, source: intel.com

Some other people spotted this feature earlier, because of another patch submitted by another Intel engineer to Linux kernel a few weeks ago. Here's a good write up by Dan Rosenberg discussing how this patch makes writing Linux kernel exploits harder, and how it's still possible to write them.

The SMEP feature still doesn't seem to be present in the processors available on the market, including the latest Sand Bridge processors, but there's no question it's coming, now that the feature made it into SDM.

It is quite rewarding to see your idea implemented in a processor... I guess this is how physicists feel when they introduce a new particle as part of a new quantum model, and later discover evidences to support the existence of this very particle in the wild...

USB Security Challenges

2011-06-01T01:25:00.001+02:00

When we think about “USB Security” there are lots of things that come to mind...

First there are all the physical attacks that could be conducted with the help of USB devices. These are generally not so interesting, because if one includes physical attacks in the threat model, then it really opens up lots of possibilities of various attacks, and generally a physical attacker always wins. Still, there are a few very cheap and easy physical attacks that one would like to avoid, or make harder, such as the Evil Maid Attacks or the Cold Boot Attacks. Strictly speaking these are not problems inherent to USB itself, but rather with lack of Trusted Boot, or OS not cleaning properly secrets from memory upon shutdown. They are just made simple thanks to bootable USB sticks.

Much more interesting USB-related physical attacks are those that take advantage of the specifics of the USB standard. One example here would be a malicious USB device that exposes intentionally malformed info about itself in order to exploit a potential flaw in a USB Host Controller driver that processes this info upon each new USB device connect. Or a malicious USB device that would trick the OS (Windows at least) into downloading a known buggy USB driver (or even an intentionally malicious driver, legally submitted to WHQL by the attacker) and then exploit the driver.

Another class of physical attacks made possible by the USB specification are malicious USB devices that pretend to be a keyboard or mouse. The input devices, such as keyboard, are actually the most security sensitive devices, because an attacker who controls the keyboard can do everything the user can do, which basically means: can do everything, at least with regards to the user's data.

Finally, the USB, as the names stands, is a bus interconnect, which means all the USB devices sharing the same USB controller are capable of sniffing and spoofing signals on the bus. This is one of the key differences between USB and PCI Express standards, where the latter uses a peer-to-peer interconnect architecture.

Ok, so these all above were physical attacks. Let's now look at, much more fatal, software attacks.

The infamous class of attacks exploiting various autorun or auto-preview behaviors is the most known example, but also the easiest, at least in theory, to protect against.

Much more interesting are software attacks that attempt to exploit potential flaws in the USB stacks – similarly like the physical attacks mentioned above, just that this time not requiring any hardware-level modifications to the USB device. Exposing a malformed partition table is a great example of such an attack. Even if we have all the autorun mechanisms disabled, still, when we're inserting a storage medium the OS always attempts to parse the partition table in order to e.g. create devices symbolizing each partition/volume (e.g. /dev/sdbX devices).

Now, this is really a problematic attack, because the malformed partition table can be written onto a fully legitimate USB stick by malware. Imagine e.g. you have two physically separated machines (air-gapped), belonging to two different security domains, and you want to transfer files from one to another. You insert the USB stick into the first machine, copy files, and then insert the stick to the second machine. If the first machine was compromised, it could have altered the partition table on the USB stick, and now when this stick is inserted into the other machine its malformed partition table might exploit a buffer overflow in the code used by the second OS to parse the stick's partition information. Air-gapped systems, huh? We avoid this attack vector in Qubes by using a special inter-domain file copy mechanism that doesn't require any metadata parsing.

A variation of the above attack would be to expose a malicious file system metadata, but this time the target OS would have to actually mount the partition for the attack to work (and, of course, there would have to be bugs in the OS file system parsing code, although these seem to be quite common on most OSes).

Having quickly summarized the USB security-related threats, let's now think about how we could design an OS to mitigate most of those attacks, and at the very least the software-based attacks. This is, in fact, precisely the challenge we've been facing in Qubes, so the divagations below necessarily focus mostly on the Qubes architecture.

First we should realize that USB devices, unlike PCI Express devices, cannot be independently delegated to different domains (VMs). This is because IOMMU technologies, such as Intel VT-d, operate only on PCIe device granularity. This means we can only delegate a whole USB controller to a domain, including all of the USB devices connected to this controller/hub.

Imagine now two internal devices, both connected via internal USB bus: a keyboard, and a 3G wireless modem. Chances are high that you will have those two devices connected to the same USB controller – usually one controller is used for all the internal devices, like those I just mentioned, plus camera, fingerprint reader, etc, and the other controller is used for all the externally visible USB connectors (at least this is true for modern systems: Intel Series 5 chipsets and newer).

We would like to be able to delegate the 3G modem to the NetVM (an untrusted domain on Qubes where all the networking drivers and stacks are kept; it's considered untrusted because its compromise is equivalent to a compromise of a WiFi network or home router, or some other router, and any reasonable person always assumes that the network is compromised, and deals with that using crypto, such as SSL or SSH). But assigning the USB controller, to which the 3G modem is connected to, to the NetVM, would also assign the USB keyboard to the NetVM! And this is precisely what we don't want to do, because control over the keyboard is equivalent to the control over the whole system!

Currently, in Qubes Beta 1, we keep all the USB controllers assigned to Dom0. This, however, causes two annoyances:

First, the user cannot use any of the USB-connected networking devices, such as 3G modems (because there is no networking in Dom0).

Second, if somebody connects a USB disk and later delegates it to some domain (this could easily be done via block-attach mechanism, supported by the same backend that handles storage virtualization for domains), and this domain turns out to be compromised, it might alter e.g. the stick's partition table and later attack Dom0 as explained above.

We can eliminate the second problem by modifying the Dom0's kernel to not parse the partition table of any removable devices automatically, and instead expect some kind of explicit consent from user to actually do that (we still must allow to mount USB disks in Dom0 to allow easy backups of all domains at once).

To allow the use of USB-connected networking devices in NetVM, we could use a PVUSB backend that can virtualize single USB devices without moving the whole USB controller to the domain. But that would require introducing a whole lot of new code to Dom0 – code that would be directly reachable from VMs (in other words that would be processing lots of untrusted input coming from untrusted domains).

So another option is to delegate all the non-security-critical USB controllers, i.e. those controllers that don't have any security-sensitive USB devices connected, such as keyboard, to a dedicated “USB” domain, and later share the USB devices via PVUSB backend from this USB domain. This time, the extra PVUSB backend runs in the USB domain, not in Dom0, so we don't worry that much about potential bugs in this backend. Of course, this way you cannot delegate the USB controller to which the keyboard, and potentially also other security-critical devices, such as camera, are connected to, which in practice rules out the integrated3G modem. Fortunately many modern laptops do not use USB-connected keyboard and touchpad (they use PS/2-connected keyboards instead), and the face camera can be easily disabled with a piece of sticker (although that sucks, because it means we cannot really use the camera).

With this approach (a dedicated USB domain) you can now delegate your 3G modem to the NetVM, and other USB devices, such as removable disks to other domains, e.g. for file exchange. This seems the most reasonable setup, although it requires that either 1) your laptop doesn't have USB-connected keyboard, or 2) you don't use internal USB devices connected to the same controller that your USB keyboard/touchpad from other domains than Dom0 (in practice: no 3G modem in NetVM).

As we can see proper handling of USB devices is quite a challenge for OS architects. It might have been much less of a challenge if the engineers designing the USB, chipsets, and motherboards were a bit more security-conscious. Even such simple practice as never mixing security critical devices (keyboard, touchpad, camera, fingerprint reader), with non-security ones (3G modem), onto the same USB controller, would help tremendously. Or ability to somehow dynamically configure their connectivity, e.g. in BIOS?

(Un)Trusting the Cloud

2011-05-28T14:56:00.003+02:00

Everybody loves The Cloud these days, and it is not hard to understand why. When every person owns computers (devices), the cloud is really hard to beat when it comes to syncing all your digital life back and forth between all those devices, and also sharing with your family members, friends, and colleagues at work. From task lists, through calendars, through health & fitness data, to work-related documents. And I'm not even mentioning all the unencrypted email that is out there.

One doesn't need to be especially smart or security conscious to realize how much this might be a threat to security and privacy. How much easier would it be to attack somebody's laptop if I knew precisely in which hotel and when he or she is planning to stay? How much more expensive would my health and life insurance be, if they could get a look at my health and fitness progress? Etc.

But we're willing to sacrifice our privacy and security in exchange for easy of syncing and sharing of our data. We decide to trust The Cloud. What specifically does that mean?

First, it means we trust the particular cloud-based service vendor, such as the provides of our training monitoring app and service. We trust that this vendor is: 1) non-malicious and ethical, and so is not going to sell our private data to some other entity, e.g. insurance company, and 2) that the software written by this vendor is somehow secure, so it would not be easy for an attacker to break into their cloud service and download all the user's data (and then sell to health insurance companies).

Next, we trust the cloud infrastructure provider, such as Amazon EC2. We trust that the cloud provider is 1) non-malicious and ethical, and that they won't really read the memory of the virtual machine on which the previously mentioned cloud-service is running (and won't make it available to a local government officials, e.g. in China), and 2) that they secured their infrastructure properly (e.g. it wouldn't be easy for one customer to “escape” from a VM and read all the memory of the VMs belonging to other customers).

Finally we trust all the infrastructure that is in the middle between us and the service provider, such as e.g. the networking protocols, are safe to use (e.g. we trust all the engineers working in any of the ISP we use won't sniff/spoof our communication, e.g. by using some fake or quasi-fake SSL certs).

So, that's a hell of a lot of trusting! And the stake is high. Do we really need to make such a sacrifice? Do we really need to hand in all our private data to all those organizations? Of course we don't!

First, notice that in majority of cases, the cloud is only used basically as a on-line storage. No processing, just dump storage. Indeed, what kind of server-side processing does your task list or calender require? Or your freestyle swimming results? Or your conference slides? None.

And we know for very long how to safely keep secrets on untrusted storage, don't we? This is achieved via encryption (and digital signatures for integrity/authenticity). So, the idea is very simple: let's encrypt all the data before we send them to the cloud. The point here is, the encryption must be done by the app that is running on our client device. Not in the cloud, of course.

Ok, so let's say I have my calendar records encrypted in the cloud, how do I share it with my other devices and other people, such as my partner and colleagues at work? Very simple – you encrypt each record with a random symmetric key and then, for every other device or person who you want to grant access to your calendar you make the symmetric key available to this person, by encrypting it with their public key (if you're paranoid, you can even verify fingerprints using some out-band communication channel, such as phone, to ensure the cloud/service provider didn't do MITM attack on you). What if you want to share only some events (or some details) with some group of people (e.g. only your availability info)? Very simple – just encrypt those records you want to share in non-full access with some other symmetric key and publish only this key to those people/devices you want to grant such non-full access.

Implementing the above would require writing new end-user apps, or plugins for existing apps (such as Outlook), so that they do encryption/decryption/signing/verification before sending the data out to the cloud. But what stops the malicious vendor from offering apps that would be leaking out our secrets, e.g. the keys? Well, nothing actually. But this time, the vendor would need to explicitly build in some kind of backdoor into the app. The same could be done with any other vendor, and any other, non-cloud-based app. After all, how do we know that MS Word, which is not cloud-based yet, is not sending out fragments of our texts to Agent Smith? Note how different this is from a situation when the vendor already owns all our data, unencrypted, brought legitimately to their servers, and all they need to do is to read them from their own disks. No need to plant and distribute any backdoors!

In practice few vendors would be risking their reputation and would be willing to build in a backdoor into an app that is then made available to customers. Because every backdoor in such client-exposed code will sooner or later be found (You would really not believe what great lengths all those young people aimed with disassembler and debugger would go to, to win an economy class ticket to the middle of desert in the hottest summer season, just to be able to deliver a presentation on how evil/stupid a company X is ;).

One problem is, however, with accessing our encrypted cloud over a Web Browser. In contrast to apps, the web browser content is much less identifiable. An app can have a digital signature – everybody know its an App v 1.1, published by X. As explained above it would be rather stupid for X to plant a backdoor into such an app. But a Web-delivered Javascript is much more tentative, and it's very possible for X to e.g. deliver various versions of scripts to different customers. Digital signature on client-side scripts, paired with ability to whitelist allowed client-side-scripts, would likely solve this problem.

So, why we still haven't got client-side-encrypted cloud-services? The question is rhetorical, of course. Most vendors actually loves the idea of having unlimited access to their customers data. Do you think Google would be happy to give up an opportunity to data mine all your data? This might affect their ad business, health research, or just Secret Plan To 0wn The World. After our dead body, I can almost hear them yelling! After all they have just came up with Chrome OS to bring even more data into their data mining machine...

To sum it up, there is no technical reason we must entrust all those people with our most private data. Sooner or later somebody will start selling client-side-encrypted cloud services, and I would be the first person to sign up for it. Hopefully it will happen sooner than later (to late?).

This post also hopefully shows, again, one more aspect – that we can, relatively easy, move most of the IT infrastructure out of the “TCB” (Trusted Computing Base, used as metaphor here). In other words, we can design our systems and services so that we don't need to trust a whole lot of things, including servers and the networking infrastructure (except for its reliability, but not for its security). But, there always remains one element that we must trust – these are our client devices. If they are compromised, the attacker can steal everything.

Strangely most people still don't get it, or get it backwards. Just the fact that “information is not stored on the iPad but kept safe on the corporate network”, doesn't change anything! Really. If the attacker owns your iPad, then she also can do anything that the legitimate user could do from this iPad. So if you could get to the company's secret trade data from your iPad's Receiver, so would be able to do the malware/attacker.

The App-oriented UI Model and its Security Implications

2011-05-21T20:17:00.001+02:00

Most of the desktop OSes today, such as Windows or Mac, expose and encourage a File-oriented UI model. You pick a file in the file manager, click it, and then the file manager automagically determines the best app to handle the file, starts the app, and passes the file to it.

Back in the MS-DOS days we used a different model: an app-oriented model – you started an app first, e.g. Word Perfect, or Lotus 1-2-3, and then you opened a file from within the app (Norton Commander and similar programs somehow changed that later).

Interestingly this very same app-oriented model is now becoming popular again thanks to systems such as iOS and Android. There is no such thing as a global File Explorer or Finder on an iPad. Only the apps. One must first pick an app, and then it's the application's responsibility to expose an option for opening one of your “files”, if the app supports it (e.g. the calendar or task list apps would always open your default calendar or task list without asking for anything).

I actually like this app-oriented model a lot! It's much less confusing to the user. Just think about all those attacks in the past where an attacker could prepare a file with some innocently-looking extension but which in fact was an MZ executable. Or how many times people are not even aware which app they use! One might argue that user should not be distracted by such “unimportant” things as what app he or she uses for her work, but I disagree. Apparently Apple, and millions of iPhone and iPad users, disagree too.

But the main reason why I like this app-oriented model is because it just fits greatly into the Security by Isolation philosophy.

Just think about it: if it's possible to get users to consciously select an app, and we now know it is possible thanks to the millions of app-oriented devices sold, then it should be not much more difficult to get them to also consciously select the domain or area, such as “work”, or “personal”, which they wish to use. Just imagine that instead of one “Mail” app, you would have two apps (and two icons): “Mail Work”, and “Mail Personal”.

There are some technicalities here – such as e.g. how to isolate apps between each other? Do we need to build another layer of isolation in a form of VMs to isolate “Mail Work” from “Mail Personal”, or should the (new) OSes and the (new) APIs be designed in such a way, that they were thin and secure, and allow for very good isolation between processes without using virtualization?

In Qubes we must use this additional layer of abstraction (virtualization), because we want to use Linux apps (and in the future also Windows apps), and they require huge POSIX/X API (and Win32 API) to work correctly. And those APIs are not easily isolate-able. So we use VMs as “API providers”. Same with isolating networking drivers and stacks – we need Linux kernel API to get those drivers and stacks running, so that's why we use a Linux-based “NetVM” for isolating networking. For this reason we expect users to explicitly define domains, such as “work”, “personal”, etc. This is because we cannot afford to run every single app in a separate AppVM (more precisely we cannot afford to create a working copy of this huge POSIX/X API for each app).

But we could very well imagine a well constructed API for apps that would just be easily isolate-able (I'm not saying iOS or Android has such an API), and so there would be no need to define domains explicitly. Still, we would need a possibility to define more than one instance of each app – such as the previously mentioned “Mail Work” and “Mail Personal”.

The app-oriented model seems to be the future. And so seems the Security by Isolation philosophy!

Following the White Rabbit: Software Attacks Against Intel VT-d

2011-05-13T19:04:00.001+02:00

Today we publish a new paper which is a result of our several month long in-depth evaluation of Intel VT-d technology. To quote the abstract:

We discuss three software attacks that might allow for escaping from a VT-d-protected driver domain in a virtualization system. We then focus on one of those attacks, and demonstrate practical and reliable code execution exploit against a Xen system. Finally, we discuss how new hardware from Intel offers a potential for protection against our attacks in the form of Interrupt Remapping (for client systems available only on the very latest Sandy Bridge processors). But we also discuss how this protection could be circumvented on a Xen system under certain circumstances...

I think the attack is likely the most complex and surprising out of all the things we have presented so far. Parts of it are even funny (if you share our weird sense of humor), such as the use of ICMP ping to generate MSIs. The paper also covers the vendors' response. You can download the paper here.

The Linux Security Circus: On GUI isolation

2011-04-23T16:52:00.008+02:00

There certainly is one thing that most Linux users don't realize about their Linux systems... this is the lack of GUI-level isolation, and how it essentially nullifies all the desktop security. I wrote about it a few times, I spoke about it a few times, yet I still come across people who don't realize it all the time.

So, let me stress this one more time: if you have two GUI applications, e.g. an OpenOffice Word Processor, and a stupid Tetris game, both of which granted access to your screen (your X server), then there is no isolation between those two apps. Even if they run as different user accounts! Even if they are somehow sandboxed by SELinux or whatever! None, zero, null, nil!

The X server architecture, designed long time ago by some happy hippies who just thought all the ~~people~~ apps are good and non-malicious, simply allows any GUI application to control any other one. No bugs, no exploits, no tricks, are required. This is all by design. One application can sniff or inject keystrokes to another one, can take snapshots of the screen occupied by windows belonging to another one, etc.

If you don't believe me, I suggest you do a simple experiment. Open a terminal window, as normal user, and run xinput list, which is a standard diagnostic program for Xorg (on Fedora you will likely need to install it first: yum install xorg-x11-apps):

$ xinput list

It will show you all the pointer and keyboard devices that your Xorg knows about. Note the ID of the device listed as “AT keyboard” and then run (as normal user!):

$ xinput test id

It should now start displaying the scancodes for all the keys you press on the keyboard. If it doesn't, it means you used a wrong device ID.

Now, for the best, start another terminal window, and switch to root (e.g. using su, or sudo). Notice how the xinput running as user is able to sniff all your keystrokes, including root password (for su), and then all the keystrokes you enter in your root session. Start some GUI app as root, or as different user, again notice how your xinput can sniff all the keystrokes you enter to this other app!

Yes, I can understand what is happening in your mind and heart right now... Don't worry, others have also passed through it. Feel free to hate me, throw out insults at me, etc. I don't mind, really (I just won't moderate them). When you calm down, continue reading.

In Qubes the above problem doesn't exist, because each domain (each AppVM) has it own local, isolated, dummy X server. The main X server, that runs in Dom0 and that handles the real display is never exposed to any of the AppVMs directly (AppVMs cannot connect to it via the X protocol). For details see this technical overview.

You can repeat the same experiment in Qubes. You just need to use the ID of the “qubesdev” device, as shown by xinput list (should be 7). Run the xinput in one of your domains, e.g. in the “red” one. Because we actually use the same device for both mouse and keystrokes, you should now see both the key scancodes, as well as all the mouse events. Notice how your xinput is able to sniff all the events that are destined for other apps belonging to the same domain where you run xinput, and how it is unable to sniff anything targeted to other domains, or Dom0.

BTW, Windows is the only one mainstream OS I'm aware of, that actually attempts to implement some form of GUI-level isolation, starting from Windows Vista. See e.g. this ancient article I wrote in the days when I used Vista at my primary laptop. Of course, it's still easy to bypass this isolation, because of the huge interface that is exposed to each GUI client (that also includes GPU API). Nevertheless, they at least attempt to prevent this at the architecture level.

Why the US "password revolution" won't work

2011-04-16T14:29:00.001+02:00

So, I've been reading this article this morning on how the US "private and public" institutions are going to revolutionize the way we authenticate on the web. The "ground breaking" idea, also illustrated on this NIST animation, is to use 3rd party authorities that would first verify your identity somehow ("Can we see your id?", "What is your Mam's maiden name?", etc), and then would issue you some kind of a token that you would later use for authentication on the web. A token would be e.g. a smart card, or a USB stick (probably they just mean a smart card with USB connector, whatever), or even a "phone application".

The idea is that the user will not have to "remember" all those passwords for all the various websites, which apparently is a problem in practice, because most users never heard about password manager apps, and so they actually try to remember all those passwords, or even try to use the same one all over the place. Using one password for more than one website is obviously wrong and people should be told not to do that. But an easy way to solve this is to just get people to use password managers.

But the key problem that they try to solve, which is identity theft, is just not gonna be solved by this "password revolution". This is because if somebody has compromised my laptop, then it really doesn't matter if I use passwords, or smart cards, or whatever other multi-factor authentication mechanism -- none of them will help if the attacker controls my operating system.

Most people cannot just get it -- this is because they lack understanding of how computers and operating systems work. They don't understand that the operating system can impersonate the user at will! This is because the operating system fully controls the keyboard, the mouse, and the screen.

So, imagine you use your super-secure smart card token for authentication to your bank. So, before you log into your bank account, and perhaps before you make any transaction on the banking website, you must insert your smart card somewhere (e.g. into smart card reader, or into USB port, etc). Before you insert your token, no one can impersonate you on the bank website. So far, so good! But then, once you inserted your token, it's all lost! The compromised OS could have saved your PIN to this card when you used it previously (even if you configured it not to do so!) and now, immediately, it could use the inserted card to authenticate as you to the bank and start issuing transactions on your behalf. And you won't even notice this all, because in the meantime it will show you a faked screen of your banking account. After all, it fully controls the screen.

The bottom line is that we cannot secure our digital lives, if our client operating systems could not be secured first. And today, the operating systems we use on our laptops, such as Windows, or Mac, or Ubuntu, are just trivial to be compromised by the attackers. After all, if that wasn't true we wouldn't have all those problems with identity theft. But introduction of tokens won't make our operating systems any more secure!

What we need instead are technologies that allow to build next-generation trusted operating systems. Technologies such as Intel TXT or VT-d. And we need OS vendors to actually start using them.

You can say I'm biased, because of our work on Qubes OS. But then, consider this -- perhaps we would never invest so much money and resources into this project, if we believed there are other ways to bring security to our digital life.

Qubes Beta 1 has been released!

2011-04-12T08:49:00.001+02:00

I'm very proud to announce that we have just released Qubes Beta 1! Some new features that have come into this release include:

Installer (finally!),

Improved template sharing mechanism: service VMs can now be based on a common template, and you can now easily create many net- and proxy- VMs; template upgrades now don't require shutting down all the VMs;

Standalone VMs, convenient for development, as well as for installing the least trusted software,

Built in, easy to use firewall VM(s),

Seamless integration of virtualized tray icons (check the screen shots!)

Redesigned file-copy between domains (easier, more secure),

Default template based on Fedora 14 (x64)

Reasonably complete User Guide.

... and many other improvements and bug fixes!

To download the installation ISO go to this page.

You can also install Qubes on an external USB disk - this might be a convenient option if you want to just try it out, without the need to "sacrifice" your laptop.

This release is very stable, but we feel that it still requires some more polish, mostly with regards to user interface. We're planning to release at least one more beta, in about 2 months, where we will focus mostly on UI improvements, and also on upgrading Xen and kernel in Dom0 to allow for better hardware support.

The final Qubes 1.0 is planned after the summer holidays. Once we reach this milestone, further work will likely fork into two branches:

The "commercial branch" which will focus on adding various extensions on top of Qubes 1. One specific commercial extension that we think would be a killer is support for Windows-based domains (AppVMs),

The "open source branch" that will continue on implementing even more revolutionary architecture and features, such as untrusted storage domains, safe GPU multiplexing, trusted boot, etc. In the end this should lead to Qubes 2.0 sometime in 2012 or 2013.

Cross your fingers!

Partitioning my digital life into security domains

2011-03-13T12:11:00.007+01:00

The diagram below illustrates how I have decomposed my digital life into security domains. This is a quite sophisticated scheme and most users would probably want something simpler, but I think it's still interesting to discuss it. The domains are implemented as lightweight AppVMs on Qubes OS. The diagram also shows what type of networking connectivity each domain is allowed.

Let's discuss this diagram bit by bit. The three basic domains are work (the green label), personal (the yellow label), and red (for doing all the untrusted, insensitive things) – these are marked on the diagram with bold frames.

A quick digression on domain labels (colors) – in Qubes OS each domain, apart form having a distinct name, is also assigned a label, which basically is one of the several per-defined colors. These colors, which are used for drawing window decorations by the trusted Window Manager (color frames), are supposed to be user friendly, easy noticeable, indicators of how trusted a given window is. It's totally up to the user how he or she interprets these colors. For me, it has been somehow obvious to associate the red color with something that is untrusted and dangerous (the “red light” -- stop! danger!), green with something that is safe and trusted, while yellow and orange with something in the middle. I have also extended this scheme, to also include blue, and black, which I interpret as indicating progressively more trusted domains than green, with black being something ultimately trusted.

Back to my domains: the work domain is where I have access to my work email, where I keep my work PGP keys, where I prepare reports, slides, papers, etc. I also keep various contracts and NDAs here (yes, these are PDFs, but received from trusted parties via encrypted and signed email – otherwise I open them in Disposable VMs). The work domain has only network access to my work email server (SMTP/IMAP4 over SSL), and nothing more.

For other work-related tasks that require some Web access, such as editing Qubes Wiki, or accepting LinkedIn invites, or downloading cool pictures from fotolia.com for my presentations, or specs and manuals from intel.com, for all this I use work-pub domain, which I have assigned the yellow label, meaning I consider it only somehow trusted, and I would certainly never put my PGP keys there, or any work-related confidential information.

The personal domain is where all my non-work related stuff, such as personal email and calendar, holiday photos, videos, etc, are held. It doesn't really have access to the Web, but if I was into social networking I would then probably allow HTTPS to something like Facebook.

Being somehow on the paranoid side, I also have a special very-personal domain, which I use for the communication with my partner when I'm away from home. We use PGP, of course, and I have a separate PGP keys for this purpose. While we don't discuss any secret and sensitive stuff there, we still prefer to keep our intimate conversations private.

I use shopping for accessing all the internet e-commerce sites. Basically what defines this domain is access to my credit card numbers and my personal address (for shipping). Because I don't really have a dedicated “corporate” credit card, I do all the shopping in this domain, from groceries, through sports equipment, on hotel/plane reservations ending. If I had separate business credit cards, then I would probably split my shopping domain into personal-shopping and work-shopping. I also have banking domain, which I use only for managing my bank account (which again combines both my personal and company accounts).

I also have a few specialized work-related domains, that I rarely use. The work-admin domain is used to manage almost all of the ITL servers, such as our webserver, Qubes repo & wiki servers, email server and DNS management, etc. This domain is allowed only SSH traffic to those server, and HTTPS to a few Web-based management servers. The work-blog is used to manage this very blog you're reading now. The reason why it is separate from work-admin or work, is because I'm over paranoid, and because I fear that if somebody compromises the blogger service, and subsequently exploits a bug in my browser that I use for editing my blog, than I don't want this person to be able to also get admin access to all the ITL servers. Similarly, if somebody somehow compromised e.g. the Amazon Web Management Console, and then exploited my browser in work-admin, then I would like at least to retain access to my blog. If I used twitter, I would probably also manage it from this work-blog domain, unless it was a personal twitter account, in which case I would run it from personal.

The qubes-dev domain is used for all the Qubes development, merging other developers' branches (after I verify signatures on their tags, of course!), building RPMs/ISOs (yes, Qubes Beta 1 will ship as a DVD ISO!), and finally signing them. Because the signing keys are there, this domain is very sensitive. This domain is allowed only SSH network access to our Qubes git server. Again, even if somebody compromised this Git server, it still would not be a big problem for us, because we sign and verify all the tags in each others repos (unless somebody could also modify the SSH/Git daemons running there so that they subsequently exploit a hypothetical bug in my git client software when I connect to the server).

I also decided to keep all the accounting-related stuff in a separate domain – whenever I get an invoice I copy it to the accounting domain. The rationale for this is that I trust those PDFs much less than I trust the PDFs I keep in my work domain.

Once a year I move the old stuff from my work domain, such as old email spools, old contracts and NDAs, to the work-archives domain. This is to minimize the potential impact of the potential attack on my work domain (my work domain could be attacked e.g. by exploiting a hypothetical bug in Thunderbird's protocol handshake using a MITM attack, or a hypothetical bug in GPG).

The vault domain is an ultimately trusted one where I generate and keep all my passwords (using keepass) and master GPG keys. Of course, this vault domain has no networking access. Most of those passwords, such as the email server access password is also kept in the specific domains which uses them, such as the work domain, and more specifically in the Thunderbird client (there is absolutely no point in not allowing e.g. Thunderbird to remember the password – if it got compromised it would just steal it the next time I manually enter it)

And finally, there is the previously mentioned red domain (I have tried to call it junk or random in the past, but I think red is still a better name after all). The red domain is totally untrusted – if it gets compromised, I don't care – I would just recreate it within seconds. I don't even back it up! Basically I do there everything that doesn't fit into other domains, and which doesn't require me to provide any sensitive information. I don't differentiate between work-related and personal-related surfing even – I don't care about anonymity for all those tasks I do there. If I was concerned about anonymity, I would create a separate anonymous domain, and proxy all the traffic through a tor proxy from there.

Now, this all looks nice and easy ;) but there is one thing that complicates the above picture...

Data flows between the domains

The diagram below shows the same domains, but additionally with arrows symbolizing typical data flows between them.

You can see that most of the usual data flows are from more trusted domains to less trusted domains – e.g. copy and pasting a URL that I receive via email in my work domain, so that I could open it in my untrusted browser in red, or moving invoices from my work domain (where I receive them via email) to the accounting domain.

But there are, unfortunately, also some transfers from less trusted domains to more trusted ones. One example is copy and pasting an interesting URL that I just stumbled upon when surfing in the red domain, and that I would like to share with a college at work, or a friend, and so I need to copy and paste it to my email client in either work (colleague) or personal (friend) domain.

Now, copying data from less trusted domains to more trusted ones presents a significant problem. While one could think that pasting an URL into Thunderbird email editor is a pretty harmless operation, it's still is an untrusted input – and we don't really know what the red domain really pasted into its local clipboard, and so what we will paste into the work domain's Thunderbird email editor (perhaps 64kB of some junk that will overflow some undo buffer in the editor?). And even more scary is the example with copying the cool-looking graphics file from the Web into my work domain so that I could use it in my presentation slides (e.g. an xkcd . Attacks originating through malicious JPEGs or other graphics format, and exploiting bugs in rendering code have been known for more than a decade.

But this problem – how to handle data flows from less trusted systems to more trusted ones – is not easily solvable in practice, unfortunately...

Some people who design and build high-security systems for use by military and government takes a somehow opposite approach – they say they are not concerned about less-trusted-to-more-trusted data transfers as long as they could assure there is no way to perform a transfer in the opposite direction. So, if we could build a system that guarantees that a more trusted domain can never transfer data to a less trusted domain (even if both of those domains are compromised!), then they are happy to allow one-way “up transfers”. In practice this means we need to eliminate all the covert channels between two cooperating domains. The word cooperating is a key word here, and which makes this whole idea not practical at all, IMHO.

Elimination of the covert channels between cooperating domains is indeed required in this scheme, because the assumption is that the data transfer from the less trusted domain could have indeed compromised the more trusted domain. But this, at least, should not result in any data leak back to the originating domain, and later to the less-classified network, which this less-trusted domain is presumably connected to. One of the assumptions here is that the user of such a system is connected to more than one, isolated networks. Even in that case, elimination of all the covert channels between domains (or at least minimizing their bandwith to something unusable – what is unusable, really?) is a big challenge, and can probably only could be done when we're ready to significantly sacrifice the system's performance (smart scheduling tricks are needed to minimize temporal covert channels).

I would like to make it clear that we are not interested in eliminating cooperative covert channels between domains in Qubes any time in the near future, and perhaps in the long term as well. I just don't believe into such approach, and I also don't like that this approach does nothing to preserve the integrity of the more-trusted domain – it only focuses on the isolation aspect. So, perhaps the attacker might not be able to leak secrets back to the less trusted domain, but he or she can do everything else in this more trusted domain. What good is isolation, if we don't maintain integrity?

An alternative solution to handling the less-trusted-to-more-trusted data transfers, is to have trusted “converters” or “verifiers” that could handle specific file types, such as JPEGs, and ensure we get a non-malicious file in the destination domain. While this might remind the bad-old A/V technology, it is something different. Here, the trusted converters would likely be some programs written in a safe language, running in another trusted domain, rather than a big ugly A/V with a huge database of signatures of “bad” patterns of what might appear in a JPEG file. The obvious problem with such an approach is that somebody must write those converters, and write them for all file types that we wish to allow to be transferred to more trusted domains. Perhaps doable in the longer-term, and perhaps we will do it in some future version of Qubes...

Right now we are ignoring this problem, and we say that all less-trusted-to-more-trusted transfers are to be done on the user's own risk :) You're welcome to submit trusted converters for your favorite file type(s) in the meantime!

Copying files between domains

Speaking of copying files between domains, there is another security catch here. If we imagined two physically separated machines that share no common network resources, the only way to move files between those two air-gaped machines would be via something like a USB stick or a CDROM or DVD disc. But inserting a USB drive or CDROM into a machine triggers a whole lot of actions: from parsing device-provided information, loading required drivers (for USB), parsing the driver's partition table, mounting and finally parsing the filesystem. Each of this stage requires the machine's OS to perform a lot of untrusted input processing, and the potential attack space here is quite large. So, even if we could limit ourselves to copy only harmless files between machines/domains (perhaps they were somehow verified by a trusted party in-between, as discussed above), still there is a huge opportunity that the originating domain could compromise the target domain.

In Qubes Alpha we have been using a similar file copy mechanism, using a virtual stick for file copy between domains. In Qubes Beta 1 we will provide a new scheme based on same shared memory channel that we use for GUI virtualization – the technical details of this solution will be available soon in our wiki. The most sensitive element in this new scheme is the un-cpio-like utility that runs in the target domain and unpacks the incoming blob into the pre-defined directory tree (e.g. /home/user/incoming/from-{domainname}/). We believe we can write pretty safe un-cpio-like utility, in contrast to secure all the previously mentioned elements (USB device parsing, partition parsing, fs parsing). The Qubes Beta 1 is planned to be released at the end of March, BTW.

Partitioning enforcement and easy of use

For any security partitioning scheme to make sense in real life, it is necessary to have some enforcement mechanism that would ensure that the user doesn't mistakenly bypass it. Specifically for this purpose we have come up with special, previously-mentioned firewalling support in Qubes Beta 1, that I will cover in a separate article soon.

Anther thing is to make the partitioning easy to use. For instance, I would like to be able to setup a hint in the policy, that when I click on an URL in an email I received in my work domain that it should be automatically opened in the red domain's default Web browser. Currently we don't do that in Qubes, but we're thinking about doing it in the near future.

Summary

Partitioning one's digital life into security domains is certainly not an easy process and requires some thinking. This process is also very user-specific. The partitioning scheme that I've come up for myself is quite sophisticated, and most people would probably want something much simpler. In case of corporate deployments, the scheme would be designed by CIO or IT admins, and enforced on users automatically. Much bigger problem are home and small business users, who would need to come up with the partitioning themselves. Perhaps in future versions of Qubes we will provide some ready to use templates for select "typical" groups of users.

My documents got lost/stolen [offtopic]

2011-03-08T14:40:00.007+01:00

I just realized yesterday that my wallet has disappeared, with all my credit cards, Polish ID card, and driver's license inside. Most likely somebody stole it. No strange transactions have been observed on my credit card accounts yet, but these are generally not much of a concern, thanks to credit card insurance. What is more troubling, is that perhaps some other woman is currently using my stolen ID and the driver's license doing nasty things on my account.

Apparently there is little one can do in Poland (EU?) in order to invalidate a stolen ID card. While there is an inter-bank Polish-wide database of stolen ID cards, it is being used only by banks, so it can only prevent other people from applying for small loans (for bigger loans, one would need more documents). But there are so many other things one could do, such as renting a car (and then committing a crime with it), signing up a deal with a mobile carrier (and then committing a cyber crime using this phone, or just making a really huge bill), or perhaps buying an SSL cert...

With apparently no better option left, I decided to write this blog post -- hopefully somebody will find it, e.g. before issuing a Class 2 SSL cert to the fake Joanna Rutkowska.

Here are the numbers of my lost/stolen documents:

AFS739530
**********5058

Luckily I have had my ID details written down somewhere, and the driver's license number I extracted from my Hertz profile.

A scene at a police department in Warsaw:

Hi, I would like to report my wallet being lost or stolen...
Madam, was your wallet stolen, or have you lost it?
Officer, how could I possibly know this...? If I lost it, do you really think I would remember the very moment of losing it?
Madam, you must be sure whether it was a crime or not!
...

A scene on the hotline, calling my mobile provider (note that I decided to use the word stolen this time):

Hi, my documents have been stolen -- I would like that you indicate my ID card as invalid in your system (that you hopefully share with other telcom operators)..
You should report such an incident to the police, Madam...
Right, but I guess that neither you, nor any other mobile provides in Poland will consult a Police database before signing up a contract with a strange person who might be using my stolen documents, correct?
Oh, but we will not sign a contract with a strange person who uses your documents! Only with you!
And how would you know it was not me, if that person was similarly aged and looking, and was using my stolen ID and driver's license?
...

Update on Qubes

2010-12-06T18:23:00.004+01:00

It's been a bit quiet on the Qubes development front for the last 2 months. The reason for this was that Rafal and myself got fully engaged in a new commercial research project. After all, we do need to make money somehow, so that we could later spend them on funding Qubes development :)

But this new engagement is actually closely related to what we do with Qubes (i.e. how new hardware technologies allow to build more secure OSes), so it's not like we're abandoning Qubes, as the experience we get with this research project will surely be useful for us when designing and implementing the Qubes 2.0 architecture.

In order to continue with Qubes, we've decided to hire some Linux programmers, while Rafal and I will continue with our research project over the coming months. We've decided to start a cooperation with another Polish computer outfit, TLS Technologies, who specializes in advanced systems design and implementation.

There are a couple of people people from TLS engaged in Qubes, and you will soon "meet" them on qubes-devel, in our wiki, and of course, you will see their contributions in our git repos.

The plan is to have Beta 1 released sometime in January ~~2010~~2011. The two important features that will be implemented first, and that will make it into Beta 1 (apart for the long-awaited installer) are: Firewall VMs, and support for templates for service VMs. Stay tuned for more details soon!

If everything goes smoothly, then we should expect Qubes 1.0 sometime at the end of Q1 2011...

Qubes Alpha 3!

2010-10-06T16:00:00.003+02:00

We have just uploaded the new packages for the Qubes Alpha 3 milestone. A lot of under the hood work went into this release, including:

Support for fast-booting Disposable VM (see also implementation description here)

Dynamic memory balancing between AppVMs

Redesigned networking and NetVM support (for VT-d system)

Reasonably stable S3 sleep support (suspend-to-RAM), that works even with a NetVM!

Improved GUI virtualization (all known bugs fixed finally!)

Disposable VMs are really a killer feature IMO. The screenshot below shows the user's experience:

The user righ-clicks on a PDF file, chooses "Open in Disposable VM", and then waits 1... 2... 3... 4 seconds (assuming a reasonably modern laptop) and the document automagically opens in a fresh new Disposable VM. If you make some changes to the document (e.g. if it was a PDF form, and you edited it), those changes will propagate back to the original file in the original AppVM.

So, within 4-5 seconds, Qubes creates a new VM, boots it up (actually refreshes from a savefile), copies the file in question to the VM, and finally opens the application that is a registered MIME handler for this type of documents, e.g. a PDF viewer. We're pretty confident this time could be further decreased down to some 2 seconds, or maybe even less. This is planned for some later Beta release.

Dynamic memory balancing allows to better utilize system physical memory by moving it between running AppVMs in realtime, according to the VM's real needs. This allows to run more VMs, compared to a scheme with static memory allocation, and also dramatically eliminates system hiccups, that otherwise occur often in a static scheme when one of the VMs is short of memory and initiates swapping.

The screenshot above shows the memory usage on my 6GB laptop when writing this blog post. As you can see I can easily run a dozen of AppVMs (most users will not need that many, but I'm a bit more paranoid I guess ;) and could probably even start a few more if there was such a need (e.g. open some Disposable VMs). Of course, this all depends on the actual type of workload the user runs in each VM - most of my AppVMs run just one or two applications, usually a Web browser (Firefox), but some, e.g. the work, and personal AppVMs run much more memory-hungry applications such as Open Office, or Picasa Photo Browser. I very rarely see more than 1 GB of memory allocated to a single VM, though. Generally speaking, the new memory management in Qubes works pretty nice.

Currently, the biggest slow-down factor for Qubes is somehow poor disk performance, most likely caused by the joint impact of the Xen backend, Linux dm, and kcryptd (we use the simplest possible Xen block backend for security reasons, will move to more sophisticated backends when we introduce untrusted storage domain in Qubes 2.0).

Now, most of the under-the-hood work for Qubes 1.0 seems to be complete, and now it time for all the polishing of the user experience, which will be the main focus of the upcoming Beta development. Just reminding that we're currently looking to hire developers for this effort.

The Installation instructions can be found here. Enjoy!

ITL is hiring!

2010-09-28T13:55:00.007+02:00

We're looking to hire one or two full time developers, who will be working on the open source version of Qubes OS, with the primary task of advancing it from Alpha to Beta stage, and then finally to a production quality version.

We're looking to hire developers, not necessarily security researchers! Specifically we expect the following from candidates:

Many years of experience with Linux/GNU development, including system-level and kernel-level Linux development, documented by the actual projects,
Familiarity with virtualization technologies, and specifically with Xen hypervisor,
Basic understanding of the Qubes architecture and excitement about the project :)
Product-oriented approach (polishing, testing, packaging, understanding of user needs),
Good communication skills in written English

In return we offer the following benefits:

Decent, full-time salary,
Opportunity to be part of a renown security team,
Opportunity to work on an exciting product,
Work on a GPLed project with all the benefits it gives to the developer (visibility, rights to the code)

If you're interested in joining our team, please send a message to joanna at invisiblethingslab.com.

Please do not send typical resumes: don't write about schools you finished, certificates you obtained, driving license, scuba trainings, etc. We are only interested in a short bio (keep it below 100 words please), and links to your past or current projects. Include your geographic location.

While it would be great if you were based in Warsaw (or somewhere in Poland), as it would allow for regular face-to-face meetings, this is not a critical factor. ITL doesn't have a physical office, and everybody work from their apartments, so there is no need to relocate to Warsaw, in case you happened to be based somewhere else.

On Thin Clients Security

2010-09-13T16:35:00.006+02:00

I'm constantly being asked about it, and so I thought I would write a handy blog post, so I could just referrer to it in the future, when yet anther person asks me if I think the use of Thin Clients is a game-changer to desktop security...

It is not! Thin Clients do not improve your desktop security in any way, and that's because:

You still run a regular full-blown OS, such as Widows and all the regular applications, such as those buggy PDF readers, Web browsers, etc - it's just you run them all on some corporate server, rather on your laptop. The fact that you run the OS on the corporate server, doesn't make it any less prone to compromises, compared to if you run it locally on your laptop.

A compromise of your laptop, even if it's just a dump terminal, is still fatal! This is because if your laptop's kernel (or MBR, or BIOS, or some PCI device's firmware, or GPU) is compromised, the attacker can intercept/steal/spoof all the data that you work on remotely, because it is still your laptop that processes the input (keystrokes, mouse events) and output (pixels). So, an Evil Maid attack on your laptop when you use it as a Thin Client, would be just as devastating, as it is otherwise (and don't fool yourselves that crypto tokens can help)

We really need secure end-user systems, even if we just want to use them as dump terminals only! There is really no way we could skip this step (and e.g. focus only on infrastructure, or services security).

(Un)Trusting your GUI Subsystem

2010-09-09T18:21:00.002+02:00

Why do we need secure desktop systems? Why support from hardware is necessary to build secure desktop OSes? Does virtualization make things more, or less complex? Why Dynamic RTM (Intel TXT) is better than Static RTM? Can we have untrusted GUI domain/subsystem?

I tried to cover those questions in my recent keynote at ETISS, and you can grab the slides here.

Particularly, the slide #18 presents the idealistic view of an OS that could be achieved through the use of hardware virtualization and trusted boot technologies. It might look very similar to many other pictures of virtualized systems one can see these days, but what makes it special is that all the dark gray boxes represent untrusted domains (so, their compromise is not security-critical, except for the potential of a denial-of-service).

No OS currently implements this architecture, even Qubes. We still have Storage and GUI subsystem in Dom0 (so they are both trusted), although we already know (we think) how to implement the untrusted storage domain (this is described in detail in the arch spec), and the main reason we don't have it now is that TXT market adoption is so poor, that very few people could make use of it.

The GUI subsystem is, however, a much bigger challenge. When we think about, it should really feel impossible to have an untrusted GUI subsystem, because the GUI subsystem really "sees" all the pixmaps that are to be displayed to the user, so also all the confidential emails, documents, etc. The GUI is different in nature than the networking subsystem, where we can use encrypted protocols to prevent the netvm from sniffing or meaningfully intercepting the application-generated traffic, or the storage subsystem, where we can use fs-encryption and trusted boot technologies to keep the storage domain off from reading or modifying the files used by apps in a meaningful ways. We cannot really encrypt the pixmaps (in the apps, or AppVMs), because for this to work we would need to have graphics cards that would be able to do the decryption and key exchange (note how this is different from the case of an untrusted storage domain, where there is no need for internal hardware encryption!), and the idea of putting, essentially an HTTPS webserver on your GPU is doubtful at best, because it would essentially move the target from the GUI domain to the GPU, and there is really no reason why lots-of-code in the GPU were any harder to attack than lots-of-code in the GUI domain...

So we came out recently with an idea of a Split I/O model that is also presented in my slides, where we separate the user input (keyboard, mouse), and keep it still in dom0 (trusted domain), from the output (GUI, audio), which is moved into an untrusted GUI domain. We obviously need to make sure that the GUI domain cannot "talk" to other domains, to make sure it cannot "leak out" the secrets that it "sees" while processing the various pixmaps. For this we need to have the hypervisor ensure that all the inter-domain shared pages mapped into the GUI domain are read-only for the GUI domain, and this would imply that we need the GUI protocol, exposed by the GUI domain to other AppVMs, to be unidirectional.

There are more challenges though, e.g. how to keep the bandwith of timing covert channels, such as those through the CPU caches, between the GUI domain and other AppVMs on a reasonably low level (please note the distinction between a covert channel, which require cooperation of two domains, and a side-channel, which requires just one domain to be malicious - the latter are much more of a theoretical problem, and are of a concern only in some very high security military systems, while the former are easy to implement in practice usually, and present a practical problem in this very scenario).

Another problem, that was immediately pointed out by the ETISS audience, is that an attacker, who compromised the GUI domain, can manipulate the pixmaps that are being processed in the GUI subsystem to present false picture to the user (remember, the attacker should have no way to send them out anywhere). This includes attacks such as button relabeling ("OK" becomes "Cancel" and the other way around), content manipulation ("$1,000,000" instead of "$100", and vice-versa), security labels spoofing ("red"-labeled windows becoming "green"-labeled), and so on. It's an open question how practical these attacks are, at least when we consider automated attacks, as they require ability to extract some semantics from the pixmaps (where is the button, where is the decoration), as well as understanding the user's actions, intentions, and behavior (just automatically relabeling my Friefox label to "green" would be a poor attack, as I would immediately realize something is going wrong). Nevertheless this is a problem, and I'm not sure how this could be solved with the current hardware architecture.

But do we really need untrusted GUI domain? That depends. Currently in Qubes the GUI subsystem is located in dom0, and thus it is fully trusted, and this also means that a potential compromise of the GUI subsystem is considered fatal. We try to make an attack on GUI as hard as possible, and this is the reason we have designed and implemented special, very simple GUI protocol that is exposed to other AppVMs (instead of e.g. using the X protocol or VNC). But if we wanted to add some more "features", such as 3D hardware acceleration for the apps (3D acceleration is already available to the Window Manager in Qubes, but not for the apps), then we would not be able to keep the GUI protocol so simple anymore, and this might result in introducing exploitable fatal bugs. So, in that case it would be great to have untrusted GUI domain, because we would be able to provide feature-rich GUI protocols, with all the OpenGL-ish like things, without worrying that somebody might exploit the GUI backend. We would also not need to worry about putting all the various 3rd party software in the GUI domain, such as KDE, Xorg, and various 3rd party GPU drivers, like e.g. NVIDIA's closed source ones, and that some of it might be malicious.

So, generally, yes, we would like to have untrusted GUI domain - we can live without it, but then we will not have all the fancy 3D acceleration for games, and also need to carefully choose and verify the GUI-related software (which is lots of software).

But perhaps in the next 5 years everybody will have a computer with a few dozens of cores, and also the CPU-to-DRAM bandwidth will be orders of magnitude faster than today, and so there will be no longer a need to offload graphic intensive work to a specialized GPU, because one of our 64 cores will happily do the work? Wouldn't that be a nicer architecture, also for many other reasons (e.g. better utilization of power/circuit real estate)? In that case nobody will need OpenGL, and so there will be no need for a richer GUI protocol than what is already implemented in Qubes...

It's quite exciting to see what will happen (and what we will come up for Qubes) :)

BTW, some people might confuse X server de-privileging efforts, i.e. making the X server run without root privileges, which is being done in some Linux distros and BSDs, with what had been described in this article, namely making the GUI subsystem untrusted. Please note that a de-priviliged X server doesn't really solve any major security problems related to GUI subsystem, as whoever controls ("0wns") the X server (depriviliged or not) can steal or manipulate all the data that this X server is processing/displaying. Apparently there are some reasons why people want to run Xorg as non-root, but in case of typical desktop OSes this provides little security benefit (unless you want to run a few X servers with different user accounts, and on different vt's, which most people would never do anyway).

Qubes, Qubes Pro, and the Future...

2010-09-02T10:38:00.005+02:00

The work on Qubes OS has been extremely exciting and also very challenging for us. While most of the work we have been doing so far relates to solving various technical, under-the-hood challenges, the more important goals in the long-term are related more to mitigating the so called "human factor", i.e. making the system not only easy to use, but tolerant to user absentmindedness. This includes e.g. ensuring the user uses a correct AppVM (e.g. do the banking in the "banking" AppVM, and not in the "random web browsing" AppVM, and also not the other way around: don't do random surfing in the "banking" AppVM), and generally making the whole isolation between AppVMs as seamless as possible, but without sacrificing the security at the same time.

This is becoming very important, as the technical level of security in Qubes is already very high, and so the "human factor" might easily become a low hanging fruit for the attacker. (In contrast to other OSes)

But for Qubes to become something more than just an interesting OS for Linux geeks and security enthusiasts, it is also critical to have better application support. Right now Qubes lets users run Linux apps, because each AppVM is Linux-based. But, and let's not be afraid to admit this: Linux sucks when it comes to application support! (Take Open Office as an example - it not only looks like MS Office 97, but is also terribly user-unfriendly, especially their presentation program, the Impress. Why is it so difficult to make it look and behave more like Apple Keynote?)

There is only one way to provide better application support to Qubes: make it support Windows-based, or Mac-based, AppVMs. Just imagine that: being able to run most of your Windows (or Mac) applications, but at the same time benefit from the Qubes strong isolation and seamless integration on one common desktop...

In order to implement support for Windows-based AppVMs (or alternatively Mac-based AppVM) we would need to engage significant resources (5+ very skilled developers, working full time for 1+ year), and so we're currently looking for an investor that would be able to provide funding for such an endeavor. The idea is to create a dedicated spin-off company that would focus entirely on Qubes and Qubes Pro, and in the future will make a profit from selling Qubes Pro licenses. Qubes Pro will become a commercial product, still based on the open source Qubes, but adding support for Windows-based or Mac-based AppVMs. I would be happy to discuss the details and business plan via email with interested potential investors.

Speaking about the future of Qubes: next week I will speak at the European Trusted Infrastructure Summer School, where I will talk about some general stuff like why we need secure desktop systems and why trusted computing might be a way to go, but will also dive a little bit into some new things we plan for Qubes 2.0, such as storage domain and split I/O graphics model. The conference features some very reputable speakers in system-level security field, such as David Grawrock (the father of Intel TXT and TPM), and Loic Duflot (our venerable competitor in the filed of offensive system-level research), so I consider a honour to deliver an opening keynote there (Check the agenda here).

I will have my Qubes laptop with me, of course, so if anybody is interested to see Qubes OS live (including Disposable VMs!), I would be happy to do a quick demo on the spot.

The MS-DOS Security Model

2010-08-19T21:55:00.004+02:00

Back in the '80s, there was an operating system called MS-DOS. This ancient OS, some readers might not even remember it today, had a very simple security model: every application had access to all the user files and other applications.

Today, over two decades later, overwhelming majority of people still use the very same security model... Why? Because on any modern, mainstream OS, be that Linux, Mac, or Windows, all the user applications still have full access to all the user's files, and can manipulate all the other user's applications.

Does it mean we haven't progressed anywhere from the MS-DOS age? Not quite. Modern OSes do have various anti-exploitation mechanisms, such as ASLR, NX, guard pages (well, Linux has it since last week at least), and even some more.

But in my opinion there has been too much focus on anti-exploitation, and on bug finding, (and on patching, of course), while almost nothing has been done on the OS architecture level.

Does anybody know why Linux Desktops offer ability to create different user accounts? What a stupid question, I hear you saying - different accounts allow to run some applications isolated from user's other applications! Really? No! The X server, by design, allows any GUI application to mess with all the other GUI applications being displayed by the same X server (on the same desktop). So, what good it is to have a "random_web_browsing" user, if the Firefox run under this user account would still be able to sniff or inject keystrokes to all my other GUI applications, take screenshots of them, etc...?

[Yes, I know, the user accounts allows also to theoretically share a single desktop computer among more than one physical users (also known as: people), but, come on, these days it's that a single person has many computers, and not the other way around.]

One might argue that the progress in the anti-exploitation, and also safe languages, would make it nearly impossible to e.g. exploit a Web browser in the next few years, so there would be no need to have a "random_web_browsing" user in the first place. But, we need isolation not only to protect ourselves when somebody exploits one of our application (e.g. a Web Browser, or a PDF viewer), but also, and perhaps most importantly, to protect from maliciously written applications.

Take summer holiday example: imagine you're a scuba diver - now, being also a decently geeky person, no doubt you will want to have some dive log manager application to store the history of your dives on a computer. There are a dozen of such applications on the web, so all you need to do is to pick one (you know, the one with the nicest screenshots), and... well you need to install it on your laptop now. But, hey, why this little, made by nobody-knows-who, dive application should be given unlimited access to all your personal files, work email, bank account, and god-know-what-else-you-keep-on-your-laptop? Anti-exploitation technology would do exactly nothing to prevent your files in this case.

Aha, it would be so nice if we could just create a user "diving", and run the app under this account. In the future, you could throw in some advanced deco planning application into the same account, still separated from all the other applications.

But, sorry, that would not work, because the X server doesn't provide isolation on the GUI-level. So, again, why should anybody bother creating any additional user accounts on a Linux Desktop?

Windows Vista made a little step forward in this area by introducing integrity levels, that, at least theoretically, were supposed to prevent GUI applications from messing with each other. But they didn't scale well (IIRC there were just 3 or 4 integrity levels available), and it still isn't really clear if Microsoft treats them seriously.

So, why do we have user accounts on Linux Desktops and Macs is beyond me (I guess Mac's X server doesn't implement any GUI-level isolation either - if I'm wrong, please point me out to the appropriate reference)?

And we haven't even touched the problems that might arise from the attacker exploiting a bug in the (over-complex) GUI server/API, or in the (big fat) kernel (with hundreds of drivers). In order for those attacks to become really interesting (like the Rafal's attack we presented yesterday), the user would have to already be using e.g. different X servers (and switch between them using Ctrl-Shift-Fn), or some sandboxing mechanisms, such as SELinux sandbox, or, in case of Vista, a scheme similar to this one.

Skeletons Hidden in the Linux Closet: r00ting your Linux Desktop for Fun and Profit

2010-08-17T17:18:00.005+02:00

A couple of months ago, while working on Qubes GUI virtualization, Rafal has come up with an interesting privilege escalation attack on Linux (a user-to-root escalation), that exploits a bug in... well, actually it doesn't exploit any concrete bug, which makes it so much more interesting.

The attack allows a (unpriviliged) user process that has access to the X server (so, any GUI application) to unconditionally escalate to root (but again, it doesn't take advantage of any bug in the X server!). In other words: any GUI application (think e.g. sandboxed PDF viewer), if compromised (e.g. via malicious PDF document) can bypass all the Linux fancy security mechanisms, and escalate to root, and compromise the whole system. The attack allows even to escape from the SELinux's "sandbox -X" jail. To make it worse, the attack has been possible for at least several years, most likely since the introduction of kernel 2.6.

You can find the details of the attack, as well as the discussion of possible solutions, including the one that has eventually been implemented, in the Rafal's paper.

One important aspect the attack demonstrates, is how difficult it is to bring security to a desktop platform, where one of the biggest challenges is to let applications talk to the GUI layer (e.g. X server in case of Linux), which usually involves a very fat GUI protocol (think X protocol, or Win32 GUI API) and a very complex GUI server, but at the same time keep things secure. This was one of the key priories for us when designing Qubes OS architecture. (So, we believe Qubes is much more secure than other sandboxing mechanisms, such as BSD jails, or SELinux-based sandboxes, because it not only eliminates kernel-level exploits, but also dramatically slims down GUI-level attacks).

The kernel-level "patch" has been implemented last week by Linus Torvalds, and pushed upstream into recent stable kernels. RedHat has also released an advisory for this attack, where they rated its severity as "high".

ps. Congrats to Brad Spengler for some good guessing :)

Qubes Alpha 2 released!

2010-07-01T16:16:00.004+02:00

The Alpha 2 is out!
New screenshots are here :)

Disposable VMs

2010-06-01T23:41:00.004+02:00

While we're still busy with some last few tickets left for Qubes Alpha 2 milestone, Rafal has already started working on a new feature for Qubes Beta 1: on Disposable VMs. I think this is really gonna be a killer feature, and I wanted to say a few words about it.

Disposable VMs will be very lightweight VMs that can be created and booted in a very short time, say < 1s, with a sole purpose of hosting only one application, e.g. a PDF viewer, or a Media Player.

To understand why Disposable VMs are important, imagine the following situation -- you receive an email from a customer that contains a PDF attachment, say an invoice or a contract. Obviously you're opening and reading the message in an email client running in your "work" AppVM (or "work-email" AppVM, if you're paranoid), just because it is a work-related correspondence, arriving at your professional email address (for many reasons it is good to use different email addresses for job-related activities and for personal life).

However, chances of somebody compromising your email client by just sending you a maliciously crafted message that would exploit your body or subject parsers are very small, if you have disabled full HTML parser for message bodies (which I think most security-concious people do anyway). Perhaps a more effective attack vector would be for somebody to 0wn your email server first, and then try to exploit IMAP/POP/SMTP protocol parser in your email client. But hey, in that case, they already would get access to all your emails on the corporate server, without exploiting your email client (well, they could however gain access to your PGP keys this way -- if this bothers you, you might want to use smartcards for PGP keys). There is also a possibility to do a Man-In-The-Middle attack and try to exploit SSL protocol early parsers, but this could be prevented using a separate VPN AppVM in Qubes.

But now you would like to open this PDF that a customer just sent you. It's quite reasonable to be afraid that the PDF might be malicious and might try to exploit your PDF viewer, and then try to steal your emails or other things you keep in the "work" AppVM (or "work-email" AppVM). It doesn't matter if you trust the sender, as the sender's OS might very well be compromised by some malware and might be infecting all outgoing PDFs without the user consent.

You could try opening the PDF in one of your non-sensitive VMs, e.g. the "random" VM that you use for causal Web browsing, to make sure that even if the PDF is malicious, that it won't get access to any sensitive data. But what if the PDF is not malicious, and what if it contains some confidential data? In that case you might throw the baby out with the bath water (your "random" VM might have been already compromised and now it would be able to steal the secrets from your PDF file).

A disposable VM is an ideal solution here. You create a clean, disposable VM, just for the purpose of viewing the PDF. Then, once you're done, you just throw it away. If the PDF was malicious it could done harm only to its own disposable VM, that doesn't contain anything except... this very PDF. At the same time, the disposable VM is always started in a clean state, so there is no way somebody could steal the document. Only the document can steal itself :)

That all sounds easy, but to make it practical we need a very efficient implementation of disposable VMs, and a good system integration, so the experience was seamless to the user. E.g. the user should only be required to right-click on a file and choose "Open in a Disposable VM", and Qubes should take care about everything else: creating the VM, starting it, copying the file to the VM, and starting a MIME-associated application for this type of file (e.g. PDF) in the VM. And this all in time below 1s!

Basic support for Disposable VMs is planned for Beta 1, which is scheduled sometime at the end of the summer holidays. But I can tell that's just the beginning. The ultimate goal, from the user's point of view, would be to make Qubes OS to look and behave just like a regular mainstream OS like Linux, or Windows, or even Mac, but still with all the strong security that Qubes architecture provides, deployed behind the scene. Seamless support for Disposable VM is one of the first steps to achieve this goal.

Special credits go to Matt Piotrowski, who just left Berkeley University, and whose recently published thesis was a direct inspiration to implement disposable VMs in Qubes. While we did mention "one-time" VMs in our architecture document back in January (see chapter 4.6), it really was Matt's paper that convinced me we should really have them in Qubes. Virtics, a proof-of-concept implementation written by Matt, shares lots of similarities with Qubes, like e.g. architecture and implementation of the GUI virtualiztion. There are also differences though, and I refer readers to the Matt's paper for more details.

On Formally Verified Microkernels (and on attacking them)

2010-05-03T16:11:00.005+02:00

Update May 14th, 2010: Gerwin Klein, a project lead for L4.verified, has posted some insightful comments. Also it's worth reading their website here that clearly explains what assumptions they make, and what they really prove, and what they don't.

You must have heard about it before: formally verified microkernels that offer 100% security... Why don't we use such a microkernel in Qubes then? (The difference between a micro-kernel and a type I hypervisor is blurry. Especially in case of a type I hypervisor used for running para-virtualized VMs, such as Xen used in Qubes. So I would call Xen a micro-kernel in this case, although it can also run fully-virtualized VMs, in which case it should be called a hypervisor I think.)

In order to formally prove some property of any piece of code, you need to first assume certain things. One such thing is the correctness of a compiler, so that you can be sure that all the properties you proved for the source code, still hold true for the binary generated from this source code. But let's say it's a feasible assumption -- we do have mature compilers indeed.

Another important assumption you need, and this is especially important in proving kernels/microkernels/hypervisors, is the model of the hardware your kernel interacts with. Not necessarily all the hardware, but at least the CPU (e.g. MMU, mode transitions, etc) and the Chipset.

While the CPUs are rather well understood today, and their architecture (we're talking IA32 here) doesn't change so dramatically from season to season. The chipsets, however, are a whole different story. If you take a spec for any modern chipset, let's say only the MCH part, the one closer to the processor (on Core i5/i7 even integrated on the same die), there are virtually hundreds of configuration registers there. Those registers are used for all sorts of different purposes -- they configure DRAM parameters, PCIe bridges, various system memory map characteristics (e.g. the memory reclaiming feature), access to the infamous SMM memory, and finally VT-d and TXT configuration.

So, how are all those details modeled in microkernels formal verification process? Well, as far as I'm aware, they are not! They are simply ignored. The nice way of saying this in academic papers is to say that "we trust the hardware". This, however, might be incorrectly understood by readers to mean "we don't consider physical attacks". But this is not equal! And I will give a practical example in a moment.

I can bet that even the chipset manufactures (think e.g. Intel) do not have formal models for their chipsets (again, I will give a good example to support this thesis below).

But why are the chipsets so important? Perhaps they are configured "safe by default" on power on, so even if we don't model all the configuration registers, and their effects on the system, and if we won't be playing with them, maybe it's safe to assume all will be fine then?

Well, it might be that way, if we could have secure microkernels without IOMMU/VT-d and without some trusted boot mechanism.

But we need IOMMU. Without IOMMU there is no security benefit of having a microkernel vs. having a good-old monolithic kernel. Let me repeat this statement again: there is no point in building a microkernel-based system, if we don't correctly use IOMMU to sandbox all the drivers.

Now, setting up IOMMU/VT-d permissions require programming the chipset's registers, and is by no means a trivial task (see the the Intel VT-d spec to get an impression, if you don't believe me). Correctly setting up IOMMU is one of the most security-critical tasks to be done by a hypervisor/microkernel, and so it would be logical to expect that they also formally prove that this part is done flawlessly...

The next thing is the trusted boot. I will argue that without proper trusted boot implementation, the system cannot be made secure. And I'm not talking about physical attacks, like Evil Maid. I'm talking about true, remote, software attacks. If you haven't read it already, please go back and read my very recent post on "Remotely Attacking Network Cards". Building on Loic's and Yves-Alexis' recent research, I describe there a scenario how we could take their attack further to compromise even such a securely designed system as Qubes. And this could be possible, because of a flaw in TXT implementation. And, indeed, we demonstrated an attack on Intel Trusted Execution Technology that exploits one such flaw before.

Let's quickly sketch the whole attack in points:

The attacker attacks a flaw in the network card processing code (Loic and Yves-Alexis)

The attacker replaces the NIC's firmware in EEPROM to survive the reboot (Loic and Yves-Alexis)

The new firmware attacks the system trusted boot via a flaw in Intel TXT (ITL)

If the system uses SRTM instead, it's even easier -- see the previous post (ITL)

If you have new SINIT module that patched our attack, there is still an avenue to attack TXT via SMM (ITL)

The microkernel/hypervisor gets compromised with a rootkit and the attacker gets full control over the system:o

And this is the practical example I mentioned above. I'm sure readers understand that this is just one example, of what could go wrong on the hardware level (and be reachable to a software-only attacker). Don't ignore hardware security! Even for software attacks!

A good question to ask is: would a system with a formally verified microkernel also be vulnerable to such an attack? And the answer is yes! Yes, unless we could model and prove correctness of the whole chipset and the CPU. But nobody can do that today, because it is impossible to build such a model. If it was, I'm pretty sure Intel would already have such a model and they would not release an SINIT module with this stupid implementation bug we found and exploited in our attack.

So, we see an example of a practical attack that could be used to fully compromise a well designed system, even if it had a formally verified microkernel/hypervisor. Compromise it remotely, over the network!

So, are all those whole microkernel/hypervisor formal verification attempts just a waste of time? Are they only good for academics so that they could write more papers for conferences? Or for some companies to use them in marketing?

Perhaps the formal verification of system software will never be able to catch up with the pace of hardware development... By the time people will learn how to build models (and how to solve them) for hardware used today, the hardware manufactures, in the meantime, will present a few new generations of the hardware. For which the academics will need another 5 years to catch up, and so on.

Perhaps the industry will take a different approach. Perhaps in the coming years we will get hardware that would allow us to create untrusted hypervisors/kernels that would not be able to read/write usermode pages (Hey Howard;)? This is currently not possible with the hardware we have, but, hey, why would a hypervisor need access to the Firefox pages?

And how this all will affect Qubes? Well, the Qubes project is not about building a hypervisor or a microkernel. Qubes is about how to take a secure hypervisor/microkernel, and how to build the rest of the system in a secure, and easy to use, way, using the isolation properties that this hypervisor/microkernel is expected to provide. So, whatever kernels we will have in the future (better formally verified, e.g. including the hardware in the model), or based on some exciting new hardware features, still Qubes architecture would make perfect sense, I think.

Evolution

2010-05-01T15:45:00.005+02:00

If you have been following my research over the last several years (even in the days before ITL), you will undoubtedly notice how much I have changed the profile over that time...

Several years ago, myself and Alex Tereshkin (who later became ITL employee #1), were known mostly as rootkit researchers. It was back in the days when the word "rootkit" was not as much well known as it is today (It became well known sometime in the late 2005, and I remember when I was applying for a US Visa that year, the immigration officer in the Warsaw embassy asked me what I did professionally and when I replied that I was a security researcher specializing in rootkits, he was very happy to tell me that he just read about those "rootkits" somewhere, although he was not very much worried about them, because he was a Mac user...)

But then, in the coming years, we decided to explore other areas, like virtualization, trusted computing, chipset security, and even touched on the CPU security briefly. Many valuable contributions in those areas have come from Rafal Wojtczuk, who joined our team some two years ago.

And then, finally, we became ready to actually build something meaningful. Not just yet another nonsense trivial-to-break "security product", but something that have had a potential to really improve user's security. And so, the Qubes project idea has been born, and soon it became ITL's highest priority project.

So, these days we don't do any reverse engineering or malware analysis any more. We'd rather design systems so they be immune to rootkits by design (e.g. by significant TCB reduction), rather then analyze each and every new rootkit sample caught in the wild and try to come up with a detector for it.

Of course, this all doesn't mean we're giving up on our offensive research. There is still a chance you will hear about some new attacks from us. But this would surely be limited only to the attacks that we consider relevant in an environment that is already designed with security in mind, like Qubes :) So, e.g. an attack against VT-d, or some CPU exploit, or a Xen exploit, might be extremely interesting. But don't expect to see any research on how to e.g. compromise Windows 7 or Mac kernel or break out of their primitive sandboxes -- these systems are so badly designed from a security standpoint, that coming up with a yet-another attack against them makes little sense from a scientific point of view.

Naturally, I'm all excited about this all: that I've been exploring new areas, and that my work has eventually started becoming meaningful. But that is, of course, only mine subjective opinion. Specifically, this turned out not be the case for Alex, who simply enjoys reverse engineering and compiler hacking just for the sake of doing it (Alex did some excellent job on metamorphic code generators, that are years ahead of what you can read at public conferences). Unfortunately, with the current new course we took at ITL, Alex started getting less and less chances to apply his skills, and faced a decision whether to stay at ITL and do other things, i.e. other than reversing or compiler hacking, or to quit and continue doing what he has always liked to do.

The reader has probably figured out by now that Alex decided to quit ITL. I fully understand his decision and wish him all the best in his new adventures!

You should still be able to reach Alex using his old ITL's email address (alex@), or directly via his new email: alex.tereshkin at gmail.com.

Remotely Attacking Network Cards (or why we do need VT-d and TXT)

2010-04-30T14:03:00.005+02:00

I've finally found some time to study Loic Duflot's and Yves-Alexis Perez's recent presentation from the last month on remotely attacking network cards. You can get the slides here.

In short, they're exploiting a buffer overflow in the network card's firmware by sending malicious packets to the card, and then they gain full control over the card's firmware, so they can e.g. issue DMA to/from the host memory, effectively fully controlling the host (that's another example of "Ring -3 rootkit" I would say). The buffer overflow is in some exotic management protocol (that I think is disabled by default, but that's irrelevant) implemented by the NIC's firmware (the NIC has its own RISC processor, and memory, and stack, which they overflow, etc.).

I like this research very much, because it demonstrates several important things:

First, it shows that it is definitely a good idea to isolate/sandbox all the OS networking code using IOMMU/VT-d. And this is exactly what we do in Qubes.

Second, the attack provides a real-world example of why Static Root for Trust Measurement (SRTM) is inferior to Dynamic RTM (DRTM), e.g. Intel TXT. To understand why, let's make the following assumptions:
1) The OS/VMM properly uses IOMMU to isolate the network card(s), just like e.g. Qubes does.
2) Once the attacker got control over the NIC firmware, the attacker can also modify the persistent storage (EEPROM) where this firmware is kept. This has been confirmed by Loic in a private email exchange.
3) The system implements trusted boot via SRTM, i.e. using just BIOS and TPM, without Intel TXT.

Now, the attacker can modify the firmware in the EEPROM and this will allow the attacker to survive the platform reboot. The card's firmware will start executing early in the boot process, definitely before the OS/VMM gets loaded. Now, the compromised NIC, because it is capable of doing DMA to the host memory, can compromise the image of the VMM in a short time window between the time it got measured and loaded by the (trusted) OS loader, e.g. Trusted GRUB, but still before the time VMM had a chance to setup proper IOMMU/VT-d protections for itself.

Of course, in practice, it might be tricky for the compromised NIC firmware to precisely know this time window when it should send a compromising DMA write request. If the DMA was issued too early, then the trusted OS loader would calculate a wrong hash and put a wrong value into a PCR register, which would later prevent the system from completing the boot, and prevent the attack. If the DMA was issued too late, the IOMMU/VT-d protections would already be in-place, and the attack would again be unsuccessful. But, hey, much harder obstacles have been worked around by smart exploit writes in the past, so don't comfort yourself that the attack is hard. If it's possible, it means this technology is flawed, period.

And this is where DRTM, AKA Intel TXT, shows its advantage over simple SRTM. When you load a hypervisor using TXT, the SENTER instruction would first apply the VT-d protections around the hypervsior image, then do the measurements, and only then load it, with VT-d protections still in-place.

The above is the theory. A few months ago we demonstrated an attack against this scheme, but the attack was exploiting a flaw in the TXT implementation, not in its design, so it didn't render TXT useless as a technology.

A much bigger problem with Intel TXT is, that Intel still has done nothing to prevent SMM-based attacks against TXT. This is what we demonstrated about 1.5 years(!) ago. Our research stressed that TXT without protection from SMM is essentially useless. Intel then promised to come up with a spec on how to write an STM, and how TXT should work with STM (when to measure/load it, etc), but nothing has been released by Intel for all this time AFAIK...

Now, without STM (which is supposed to provide protection from potentially compromised SMM), the TXT cannot really prevent Loic and friends from owning the system, even if it uses such a securely designed OS as Qubes. This is because Loic would be able to modify e.g. the MBR while the system boots (thanks to DMA ability of the infected NIC firmware), and then attack an SMM from this MBR (I can bet lots of money Loic & co. would easily find a few other SMM exploits in any recent BIOS if they only wanted to), and then having infected the SMM, they will be able to compromise TXT-loaded hypervisor, and finally compromise the whole system.

I know there are some people from various governments reading this blog. If you really want to have secure systems, consider pushing on Intel to finally do something about the SMM-based attacks against TXT. Beware, Intel will try to tell you that, using TXT LCP you can seal your secrets to only "trusted" SMM images and would try to convince you it's a way to prevent SMM attacks on TXT. It is not. Only true SMM sandboxing is a proper way to address this problem.

Anyway, congrats to Loic and colleagues for yet another very interesting and meaningful system-level research!

Introducing Qubes OS

2010-04-07T12:58:00.008+02:00

For the last 6 months we have been busy with a new project: Qubes. Qubes is an open source OS based on Xen, X, and Linux, designed to provide strong isolation for desktop computing. The link to the project website is at the end of the post.

The system is currently in the alpha stage, but if you're determined it's actually usable. For example I have switched to Qubes around a month ago, and two weeks ago I even decided to wipe and reinstall my Mac Book, which used to be my primary laptop previously. Now I use my old Mac Book only for making the slides (Apple Keynote really has no competition) and Web page for Qubes :) And I use Qubes for pretty much all the other daily tasks, from work, shopping, banking, random browsing, to Qubes development itself (it takes part in the "qubes" AppVM).

Just remember to make backups regularly if you decided to use Qubes for anything else than testing and development.

So, enough of introduction, you will find lots of details (including a 40-page PDF describing the system architecture) at the Qubes project website. Enjoy!

Update 7-Apr-2010 15:56 CEST: The server seems to be overloaded a bit by the traffic... If you are planning to install the OS, I guess it would be wise to postpone downloading the installation packages until later this week, when the first wave of visitors goes away.

Update 7-Apr-2010 16:31 CEST: The Wiki doesn't work due to lack of free memory... Talking to my provider about buying some more RAM. Sorry for the inconvenience.

Update 7-Apr-2010 18:28 CEST: The server has been brought offline for RAM upgrade. Should be back online in some 15 minutes...

http://qubes-os.org

Priorities

2010-01-16T12:25:00.004+01:00

It’s interesting how many people don’t realize what are the priorities in computer security... There are many fields to secure: server security, web applications security, network security, and finally desktop security. Over the last years I met SO many people that always expressed surprise why I would like to focus on desktop systems security? They usually argue that today, as everybody knows, it is the Network that is what computing is all about and that we should focus on securing infrastructure, and forget about the desktops, which are always to be insecure. The network is the computer, as somebody said.

What those people forget about, is that it is always the desktop that ultimately gets access to all the user’s secretes -- all the passwords, all the keys, all the corporate documents, all the nude holiday pictures, all the secret love letters, all the credit card numbers, and many more.

However secure were all the services (remote servers and network protocols) that we use, if our desktop gets compromised it’s all lost. The recent incident with Google is just yet another example of that. Our desktop systems are the most crucial piece of the whole puzzle.

It’s funny how many people think that by using some thin client solution on their desktops they can solve the problem. Of course they cannot! Just the fact that your OS executes on a server, rather then on your hardware, doesn’t make it any less prone to all the attacks that were otherwise possible when the software executed on your system.

The attempts to secure desktops have been failing for so many years. While recently there is some attempt to minimize likelihood of remote attacks via Web browsers (or generally to focus on application security), this is still just the tip of the iceberg -- there are so many other attack avenue that none of the popular OSes even tries to address, that I consider myself a brave person (not to say stupid) that I actually use my laptop everyday and keep some sensitive information on it ;)

Ok, so that’s a nice piece of complaining you say, but what are we, at ITL, gonna do about it? Well, we just gonna sit and patiently wait for better OSes to appear some day... Oh, hell, we won’t!

Happy New Year :)


<please ignore>
9933 F096 8820 0E23 1AF4  078D 8BDB D97D BDEA 9E9D
</>

Another TXT Attack

2009-12-21T19:11:00.005+01:00

Earlier this year our team has presented an attack against Intel TXT that exploited a design problem with SMM mode being over privileged on PC platforms and able to interfere with the SENTER instruction. The Intel response was two-fold: to patch the SMM implementation bugs we used for the attack (this patch was for both the NVACPI SMM attacks, as well as for the SMM caching attack), and also to start (intensify?) working on STM specification, that is, we heard, planned to be published sometime in the near future. STM is a thin hypervisor concept that is supposed to provide protection against (potentially) malicious SMMs.

Today we present a totally different attack that allows an attacker to trick the SENTER instruction into misconfiguring the VT-d engine, so that it doesn’t protect the newly loaded hypervisor or kernel. This attack exploits an implementation flaw in a SINIT AC module. This new attack also allows for full TXT circumvention, using a software-only attack. This attack doesn't require any SMM bugs to succeed and is totally independent from the previous one.

The press release is here.

The full paper is here.

The advisory published by Intel today can be found here.

Enjoy.

Evil Maid goes after TrueCrypt!

2009-10-16T00:30:00.005+02:00

From time to time it’s good to take a break from all the ultra-low-level stuff, like e.g. chipset or TXT hacking, and do something simple, yet still important. Recently Alex Tereshkin and I got some spare time and we implemented the Evil Maid Attack against TrueCrypt system disk encryption in a form of a small bootable USB stick image that allows to perform the attack in an easy “plug-and-play” way. The whole infection process takes about 1 minute, and it’s well suited to be used by hotel maids.

The Attack
Let’s quickly recap the Evil Maid Attack. The scenario we consider is when somebody left an encrypted laptop e.g. in a hotel room. Let’s assume the laptop uses full disk encryption like e.g. this provided by TrueCrypt or PGP Whole Disk Encryption.

Many people believe, including some well known security experts, that it is advisable to fully power down your laptop when you use full disk encryption in order to prevent attacks via FireWire/PCMCIA or ”Coldboot” attacks.

So, let’s assume we have a reasonably paranoid user, that uses full disk encryption on his or her laptop, and also powers it down every time they leave it alone in a hotel room, or somewhere else.

Now, this is where our Evil Maid stick comes into play. All the attacker needs to do is to sneak into the user’s hotel room and boot the laptop from the Evil Maid USB Stick. After some 1-2 minutes, the target laptop’s gets infected with Evil Maid Sniffer that will record the disk encryption passphrase when the user enters it next time. As any smart user might have guessed already, this part is ideally suited to be performed by hotel maids, or people pretending to be them.

So, after our victim gets back to the hotel room and powers up his or her laptop, the passphrase will be recorded and e.g. stored somewhere on the disk, or maybe transmitted over the network (not implemented in current version).

Now we can safely steal/confiscate the user’s laptop, as we know how to decrypt it. End of story.

Quick Start
Download the USB image here. In order to “burn” the Evil Maid use the following commands on Linux (you need to be root to do dd):


dd if=evilmaidusb.img of=/dev/sdX

Where /dev/sdX should be replaced with the device representing your USB stick, e.g. /dev/sdb. Please be careful, as choosing a wrong device might result in damaging your hard disk or other media! Also, make sure to use the device representing the whole disk (e.g. /dev/sdb), rather than a disk partition (e.g. /dev/sdb1).

On Windows you would need to get a dd-like program, e.g. this one, and the command would look more or less like this one (depending on the actual dd implementation you use):


dd if=evilmaidusb.img of=\\?\Device\HarddiskX\Partition0 bs=1M

where HarddiskX should be replaced with the actual device the represents your stick.

After preparing the Evil Maid USB stick, you’re ready to test it against some TrueCrypt-encrypted laptop (more technically: a laptop that uses TrueCrypt system disk encryption). Just boot the laptop from the stick, confirm you want to run the tool (press ‘E’) and the TrueCrypt loader on your laptop should be infected.

Now, Evil Maid will be logging the passphrases provided during the boot time. To retrieve the recorded passphrase just boot again from the Evil Maid USB -- it should detect that the target is already infected and display the sniffed password.

The current implementation of Evil Maid always stores the last passphrase entered, assuming this is the correct one, in case the user entered the passphrase incorrectly at earlier attempts.

NOTE: It’s probably illegal to use Evil Maid to obtain password from other people without their consent. You should always obtain permission from other people before testing Evil Maid against their laptops!

CAUTION: The provided USB image and source code should be considered proof-of-concept only. Use this code at your own risk, and never run it against a production system. Invisible Things Lab cannot be held responsible for any potential damages this code or its derivates might cause.

How the Evil Maid USB works
The provided implementation is extremely simple. It first reads the first 63 sectors of the primary disk (/dev/sda) and checks (looking at the first sector) if the code there looks like a valid TrueCrypt loader. If it does, the rest of the code is unpacked (using gzip) and hooked. Evil Maid hooks the TC’s function that asks user for the passphrase, so that the hook records whatever passphrase is provided to this function. We also take care about adjusting some fields in the MBR, like the boot loader size and its checksum. After the hooking is done, the loader is packed again and written back to the disk.

You can get the source code for the Evil Maid infector here.

Possible Workarounds
So, how should we protect against such Evil Maid attacks? There are a few approaches...

1. Protect your laptop when you leave it alone
Several months ago I had a discussion with one of the TrueCrypt developers about possible means of preventing the Evil Maid Attack, perhaps using TPM (see below). Our dialog went like this (reproduced here with permission from the TrueCrypt developer):

TrueCrypt Developer: We generally disregard "janitor" attacks since they inherently make the machine untrusted. We never consider the feasibility of hardware attacks; we simply have to assume the worst. After an attacker has "worked" with your hardware, you have to stop using it for sensitive data. It is impossible for TPM to prevent hardware attacks (for example, using hardware key loggers, which are readily available to average Joe users in computer shops, etc.)

Joanna Rutkowska: And how can you determine that the attacker have or have not "worked" with your hardware? Do you carry your laptop with you all the time?

TrueCrypt Developer: Given the scope of our product, how the user ensures physical security is not our problem. Anyway, to answer your question (as a side note), you could use e.g. a proper safety case with a proper lock (or, when you cannot have it with you, store it in a good strongbox).

Joanna Rutkowska: If I could arrange for a proper lock or an impenetrable strongbox, then why in the world should I need encryption?

TrueCrypt Developer: Your question was: "And how can you determine that the attacker has or has not worked with your hardware?" My answer was a good safety case or strongbox with a good lock. If you use it, then you will notice that the attacker has accessed your notebook inside (as the case or strongbox will be damaged and it cannot be replaced because you had the correct key with you). If the safety case or strongbox can be opened without getting damaged & unusable, then it's not a good safety case or strongbox. ;-)

That's a fair point, but this means that for the security of our data we must relay on the infeasibility to open our strongbox lock in a "clean" way, i.e. without visually damaging it. Plus it means we need to carry a good strongbox with us to any travel we go. I think we need a better solution...

Note that TrueCrypt authors do mention the possibility of physical attacks in the documentation:

If an attacker can physically access the computer hardware and you use it after the attacker has physically accessed it, then TrueCrypt may become unable to secure data on the computer. This is because the attacker may modify the hardware or attach a malicious hardware component to it (such as a hardware keystroke logger) that will capture the password or encryption key (e.g. when you mount a TrueCrypt volume) or otherwise compromise the security of the computer.

However, they do not explicitly warn users of a possibility of something as simple and cheap as the Evil Maid Attack. Sure, they write "or otherwise compromise the security of the computer", which does indeed cover e.g. the Evil Maid Attack, but my bet is that very few users would realize what it really means. The examples of physical attacks given in the documentation, e.g. modifying the hardware or attaching a malicious hardware, is something that most users would disregard as too expensive an attack to be afraid of. But note that our Evil Maid attack is an example of a “physical” attack, that doesn’t require any hardware modification and is extremely cheap.

Of course it is a valid point, that if we allow a possibility of a physical attack, then the attacker can e.g. install a hardware keylogger. But doing that is really not so easy as we discuss in the next paragraph. On the other hand, spending two minutes to boot the machine from an Evil Maid USB stick is just trivial and is very cheap (the price of the USB stick, plus the tip for the maid).

2. The Trusted Computing Approach
As explained a few months ago on this blog, a reasonably good solution against Evil Maid attack seems to be to take advantage of either static or dynamic root of trust offered by TPM. The first approach (SRTM) is what has been implemented in Vista Bitlocker. However Bitlocker doesn’t try to authenticate to the user (e.g. via displaying a custom picture shot by the user, with the picture decrypted using a key unsealed from a TPM), so it’s still possible to create a similar attack against Bitlocker, but with a bit different user experience. Namely the Evil Maid for Bitlocker would have to display a fake Bitlocker prompt (that could be identical to the real Bitlocker prompt), but after obtaining a correct password from the user Evil Maid would not be able to pass the execution to the real Bitlocker code, as the SRTM chain will be broken. Instead, Evil Maid would have to pretend that the password was wrong, uninstall itself, and then reboot the platform. Thus, a Bitlocker user that is confident that he or she entered the correct password, but the OS didn’t boot correctly, should destroy the laptop.

The dynamic root of trust approach (DRTM) is possible thanks to Intel TXT technology, but currently there is no full disk encryption software that would make use of it. One can try to implement it using Intel’s tboot and some Linux disk encryption, e.g. LUKS.

Please also note that even if we assume somebody “cracked” the TPM chip (e.g. using an electron microscope, or NSA backdoor), that doesn’t mean this person can automatically get access to the encrypted disk contents. This is not the case, as the TPM is used only for ensuring trusted boot. After cracking the TPM, the attacker would still have to mount an Evil Maid attack in order to obtain the passphrase or key. Without TPM this attack is always possible.

Are those trusted computing-based approaches 100% foolproof? Of course not. As signalized in the previous paragraph, if an attacker was able to mount a hardware-based keylogger into your laptop (which is non-trivial, but possible), then the attacker would be able to capture your passphrase regardless of the trusted boot. A user can prevent such an attack by using two-factor authentication (RSA challenge-response implemented in a USB token) or e.g. one-time passwords, so that there is no benefit for the attacker to capture the keystrokes. But the attacker might go to the extreme and e.g. replace the DRAM, or even the CPU with malicious DRAM or CPU that would sniff and store the decryption key for later access. We’re talking here about attack that very few entities can probably afford (think NSA), but nevertheless they are theoretically possible. (Note that an attack with inserting a malicious PCI device that would try to sniff the key using DMA can be prevented using TXT+VT-d technology).

However, just because the NSA can theoretically replace your CPU with a malicious one, doesn’t mean TPM-based solutions are useless. As for the great majority of other people that do not happen to be on the Terrorist Top 10, these represent a reasonable solution that could prevent Evil Maid attacks, and, when combined with a proper two-factor authentication, also simple hardware based attacks, e.g. keylogger, cameras, remote keystroke sniffing using laser, etc. I really cannot think of a more reasonable solution here.

3. The Poor Man’s Solution
Personally I would love to see TrueCrypt implementing TPM-based trusted boot for its loader, but, well, what can I do? Keep bothering TrueCrypt developers with Evil Maid attacks and hope they will eventually consider implementing TPM support...

So, in the meantime we have come up with a temporary poor man’s solution that we use at our lab. We call it Disk Hasher. It’s a bootable Linux-based USB stick that can be configured in quite a flexible way to calculate hashes of selected disk sectors and partitions. The correct hashes are stored also on the stick (of course everything is encrypted with a custom laptop-specific passphrase). We use this stick to verify the unencrypted portions of our laptops (typically the first 63 sectors of sda, and also the whole /boot partition in case of Linux-based laptops where we use LUKS/dm-crypt).

Of course there are many problems with such a solution. E.g. somebody who can get access to my Disk Hasher USB (e.g. when I’m in a swimming pool), can infect it in such a way that it would report correct hashes, even though the disk of my laptop would be “evilmaided”...

Another problem with Disk Hasher solution is that it only looks at the disk, but cannot validate e.g. the BIOS. So if the attacker found a way to bypass the BIOS reflashing protection on my laptop, then he or she can install a rootkit there that would sniff my passphrase or the decryption key (in case I used one time passwords).

Nevertheless, our Disk Hasher stick seems like a reasonable solution and we use it often internally at ITL to validate our laptops. In fact this is the most we can do, if we want to use TrueCrypt, PGP WDE, or LUKS/dm-crypt.

FAQ

Q: Is this Evil Maid Attack some l33t new h4ck?
Nope, the concept behind the Evil Maid Attack is neither new, nor l33t in any way.

Q: So, why did you write it?
Because we believe it demonstrates an important problem, and we would like more attention to be paid in the industry to solving it.

Q: I’m using two-factor authentication, am I protected against EM?
While a two-factor authentication or one time passwords are generally a good idea (e.g. they can prevent various keylogger attacks), they alone do not provide protection from Evil Maid-like attacks, because the attacker might modify his or her sniffer to look for the final decryption key (that would be calculated after the 2-factor authentication completes).

Q: How is Evil Maid different from Stoned-Bootkit?
The Stoned Bootkit, released a few months ago by an individual describing himself as “Software Dev. Guru in Vienna”, is also claimed to be capable of "bypassing TrueCrypt", which we take to mean a capability to sniff TC's passphrases or keys. Still, the biggest difference between Stoned Bootkit and Evil Maid USB is that in case of our attack you don’t need to start the victim's OS in order to install Evil Maid, all you need to do is to boot from a USB stick, wait about 1 minute for the minimal Linux to start, and then press ‘E’, wait some 2 more seconds, and you’re done. With the Stoned Bootkit, according to the author’s description, you need to get admin access to the target OS in order to install it, so you either need to know the Windows admin password first, or use some exploit to get the installer executing on the target OS. Alternatively, you can install it from a bootable Windows CD, but this, according to the author, works only against unencrypted volumes, so no use in case of TrueCrypt compromise.

Q: I've disabled boot from USB in BIOS and my BIOS is password protected, am I protected against EM?
No. Taking out your HDD, hooking it up to a USB enclosure case and later installing it back to your laptop increases the attack time by some 5-15 minutes at most. A maid has to carry her own laptop to do this though.

Q: What about using a HDD with built-in hardware-based encryption?
We haven’t tested such encryption systems, so we don’t know. There are many open questions here: how is the passphrase obtained from the user? Using software stored on the disk or in the BIOS? If on the disk, is this portion of disk made read-only? If so, does it mean it is non-updatable? Even if it is truly read-only, if the attacker can reflash the BIOS, then he or she can install a passphrase sniffer there in the BIOS. Of course that would make the attack non-trivial and much more expensive than the original Evil Maid USB we presented here.

Q: Which TrueCrypt versions are supported by the current Evil Maid USB?
We have tested our Evil Maid USB against TrueCrypt versions 6.0a - 6.2a (the latest version currently available). Of course, if the “shape” of the TrueCrypt loader changed dramatically in the future, then Evil Maid USB would require updating.

Q: Why did you choose TrueCrypt and not some other product?
Because we believe TrueCrypt is a great product, we use it often in our lab, and we would love to see it getting some better protection against such attacks.

Q: Why there is no TPM support in TrueCrypt?
The TrueCrypt Foundation published official generalized response to TPM-related feature requests here.

Acknowledgments
Thanks to the ennead@truecrypt.org for all the polemics we had which allowed me to better gather my thoughts on the topic. The same thanks to Alex and Rafal, for all the polemics I have had with them (it's customary for ITL to spend a lot of time finding bugs in each other's reasoning).

Intel Security Summit: the slides

2009-09-22T16:55:00.005+02:00

Last week I was invited to Hillsboro to speak at the Intel's internal conference on security. My presentation title was "A Quest To The Core: Thoughts on present and future attacks on system core technologies", and my goal was to somehow make a quick summary of the recent research our team has done over the last 12 months or so, and explain why we're so keen on hacking the low-level system components, while all the rest of the world is excited about browser and flash player bugs.

The slides (converted to PDF) can be found here. As you will see, I decided to remove most of the slides from the "Future" chapter. One reason for that was that we didn't want to hint ~~Loic~~ our competition as to some of our new toys we're working on;) The other reason was that, I think, the value of presenting only thoughts about attacks, i.e. unproven thoughts, or, should I even say, feelings about future attacks, has little research value, and while I can understand such information being important to Intel, I don't see how others could benefit from them.

I must say it was nice and interesting to meet in person with various Intel architects, i.e. the people that actually design and create our basic "universe" we all operate in. You can always change the OS (or even write your own!), but still you must stick to the rules, or "laws", of the platform (unless you can break them ;)

About Apple’s Security Foundations, Or Lack Of Thereof...

2009-09-02T16:19:00.004+02:00

Every once in a while it’s healthy to reinstall your system... I know, I know, it’s almost a heresy to say that, but that’s reality in the world where our systems are totally unverifiable. In fact I don’t even attempt to verify if my Mac laptop has been compromised in any way (most system files are not signed anyway). But sometimes, you got this feeling that something might be wrong and you decide to reinstall to start your (digital) life all over again :)

So, every time I (re)install a Mac-based system, I end up cursing horribly at Apple’s architects. Why? Because in the Apple World they seem to totally ignore the concept of files integrity, to such extent that it’s virtually impossible to get any assurance that the programs I install are in any way authentic (i.e. not tampered by some 3rd party, e.g. by somebody controlling my Internet connection).

Take any Apple installer package, e.g. Thunderbird. In most cases an installer package on Mac is a .dmg file, that represents an installation disk image. Now, when you open such a file under Mac, the OS will never display any information about if this file is somehow signed (e.g. by who) or not. In fact, I’m pretty sure it’s never signed. What you end up with, is a .dmg file that you just downloaded over plaintext HTTP and you have absolutely no way of verifying if it is the original file the vendor really published. And you’re just about to grant admin privileges to the installer program that is inside this file -- after all it’s an installer, so must got root privileges, right (well, not quite maybe)? Beautiful...

Interestingly, this very same Thunderbird installer, but for Windows, is correctly signed, and Windows, correctly, displays that information (together with the ability to examine the certificate) and allows the user to make a choice of whether to allow it to run or not.

Sure, the certificate doesn’t guarantee that Mozilla didn’t put a nasty backdoor in there, nor that the file was not compromised due to Mozilla’s internal server compromise. Or that the certificate (the private key) wasn’t somehow stolen from Mozilla, or that the issuing authority didn’t make a mistake and maybe issued this certificate to some random guy, who just happened to be named Mozilla.

But the certificate provides liability. If it indeed turns out that this very Thunderbird installer was somehow malicious, I could take this signed file to the court and sue either Mozilla, or the certification authority for all the damages it might have done to me. Without the certificate I cannot do that, because I (and nobody) cannot know if the file was tampered while being downloaded (e.g. malicious ISP) or maybe because my system was already compromised.

But in case of Apple, we have no such choice -- we need to take the risk every time we download a program from the Internet. We must bet the security of our whole system, that at this very moment nobody is tampering with out (unsecured) HTTP connection, and also that nobody compromised the vendor’s Web Server, and, of course, we hope that the vendor didn’t put any malicious code into its product (as we could not sue them for it).

So that sucks. That sucks terribly! Without ability to check the integrity of programs we want to install, we cannot build any solid foundations. It’s funny how people divagate whether Apple implemented ASLR correctly in Snow Leopard, or not? Or whether NX is bypassable. It’s meaningless to dive into such advanced topics, if we cannot even assure that at the day 0 our system is clean. We need to start building our systems from the ground up, and not starting from the roof! Ability to assure the software we install is not tampered seems like a reasonable very first step. (Sure it could be compromised 5 minutes later, and to protect against this we should have other mechanisms, like e.g. mentioned above ASLR and NX).

And Apple should not blame the vendors for such a situation (“Vendors would never pay $300 for a certificate”, blah, blah), as it is just enough to have a look at the Windows versions of the same products, and that most of them do have signed installers (gee, even open-source TrueCrypt, has a signed installer for Windows!).

One should say that a few vendors, seeing this problem on Mac, do publish PGP signatures for their installation files. This includes e.g. PGP Desktop for Mac, KeePassX, TrueCrypt for Mac, and a few others. But these are just exceptions and I wonder how many users will be disciplined (and savvy) enough to correctly verify those PGP signatures (in general it requires you to download the vendor keys many months before, keep it in your ring, to minimize possibility that somebody alters both the installer files and the keys you download). Some other vendors offer pseudo-integrity by displaying MD5/SHA1 sums on their websites. That would make some sense only if the website on which the hashes are displayed was itself SSL-protected (still the file signature is a better option), as otherwise we can be sure that the attacker that is tampering with the installer file, will also take care about adjusting the hash on the website... But of course this never is the case -- have a look e.g. at the VMWare download page for the Mac Fusion (one need to register first). Very smart, VMWare! (Needles to say, the VMWare Workstation installer for Windows is properly signed).

BTW, anybody checked if the Apple updates are digitally signed somehow?

All I wrote here in this post is just trivial. It should be just obvious for every decently educated software engineer. Believe me it’s really is much more fun for me to write about things like new attacks on chipsets or virtualization. But I have this little hope that maybe somebody at Apple will read this little post and fix their OS. Because I really like Apple products for their aesthetics...

PDF signing and beyond

2009-08-26T21:48:00.004+02:00

Today I got an advertising email from GlobalSign (where I previously bought a code signing certificate for Vista kernel drivers some years ago) highlighting their new (?) type of certificates for signing of Adobe PDF files. It made me curious, because, frankly, I've been recently more and more missing this feature. After a quick online research it turned out that this whole Adobe Certified Documents Services (CDS) seem to be nothing new, as apparently even Adobe Reader 6.0 had support for verifying those CDS certificates. The certificates are also available from other popular certification authorities like e.g. Entrust and Verisign, and a couple of others.

So, I immediately felt stupid that I haven't been aware of such a great feature, which apparently is out there for a few years now. Why I thought it was so great a feature? Consider the following scenario…

At our Invisible Things Lab resources page we offer a handful of files to download — slides and some proof of concept code. The website is served over a plaintext HTTP. This means that if you're downloading anything over a public WiFi (hotel, airport lounge, etc) you never know if the PDF you actually get has not been infected somewhere in the middle, e.g. by a guy in the lobby that is messing with the hotel WiFi.

So, one might argue that I should have paid a few hundred bucks and get an SSL certificate for my website and start serving it over HTTPS. But here's the problem — I, as zillions of other small businesses and individuals, host my website on some 5-dollar-a-month one-of-the-thousands hosting provider. I have zero knowledge about what people work there and if they can be trusted, and I also know nothing (and have zero impact) on how secure (or not, for that matter) the server is. (Same applies to my cell phone carrier, ISP, etc, BTW).

Now, the SSL certificate for the website "knows" nothing about how the files on my website should look like, in particular if they are compromised or not. All the SSL certificate does is to give assurance to the remote client that he or she downloaded the actual files that were on the server in the moment of downloading — whether they were the original ones authored by me, or perhaps maliciously modified by somebody who got access to the server.

So, the solution with an SSL certificate would work only if I trusted my web server, which could be assumed only if I run my own dedicated server. That, however, would be an overkill for a small company like ITL, especially that our business is not based on our web presence — in fact the website is maintained mainly for other researchers and students, who can easily download our papers and code from there, and also for the reporters so they can e.g. download a press release from there.

Surprisingly, the website has never been compromised, probably because it doesn't present an interesting target for any skilled person (or maybe exceptionally skilled people work at the hosting provider?). But I cannot know for sure, as I don't constantly monitor all the hashes of all the files, as this would require… well a dedicated server that would be running an SHA1 calculating script in a loop for 24/7 :)

Of course, zillions of other websites works this very same way and present the very same problems.

Now, ability to sign PDFs would be just a great solution here, because I could sign all those files with my certificate, and then all the people downloading stuff from ITL could know they are getting original PDFs that were created on one of the ITL members desktop computers, no matter how compromised the web server or the network connection is.

For the same reasons, I would welcome if others started doing the same, as currently I simply must assume every PDF I download from the net (and PDFs account for the majority of file downloads I do) to be potentially malicious. So, I always open them in my Red or Yellow VM (depending on the source of the download), and only if it "looks good" (very fuzzy term, I know), I might decide to move it to my host desktop (it's easier to work with PDFs on your host, and actually you should use your host desktop for something).

(Yes, I know, Kostya Kortchinsky, or Rafal, can sometimes escape from VMWare, but still I believe that today the best isolation I can get on a desktop, without sacrificing much convince, is via a type II hypervisor. It's horribly inelegant, but well, that's life).

So, I read some more about this Adobe CDS, being all excited about it, and ready to spend a few hundred euros on a certificate, only to realize that it doesn't look as good as I thought.

First disappointment comes from the fact that you must create a PDF using Adobe Acrobat software (not the Reader, but the commercial one). I've created all my PDFs using either Office (in the past) or iWork (today), and none of them seem to offer a way to digitally sign the PDF. I would like to get a simple tool, say pdfsign.exe, that I could use to sign any PDF I have, no matter how I generated it. Also, not surprisingly, the Mac native PDF viewer (Preview) doesn't seem to recognize the digital signature, and I bet some Linux PDF viewers do not as well.

Worst of all, even the Acrobat Reader 9, that I tested under Windows, and that correctly displayed all the CDS information, does one unbelievably stupid thing — it parses and renders the whole PDF before displaying the signature info. So, if you downloaded a malicious PDF, Acrobat Reader will happily open it and parse, without asking you a question of whether you would like to open it (as it is perhaps unsigned). At least I was unable to find an option that would force it to do that. So, if this PDF contained an exploit for the reader, it surely would get executed. Compare this with the (correct) behavior of Vista UAC where it presents the executable signature details before executing it.

You can see how your software works with Adobe PDF signatures, e.g. by looking at this exemplary file signed by GlobalSign.

So, Adobe CDS, in the form they are today, seem to be pretty useless, as far as protection from potentially malicious PDFs is considered (they surely have other positive applications, e.g. to certify about authenticity of e.g. a diploma).

But wouldn't it be great to have such a file signing mechanism globally adopted and not only for PDFs, but for any sort of files, including ZIPs, tgz's, heck, even plain text files? And have our main OSes generically recognize those signatures and display unified prompts of whether we want to allow an application to to open the file or not? Perhaps, in some situations, we could even define policies for specific applications. This seems easy to do from the technical point of view — we just need to "hook" (oh, God, did I say "hook"?) high-level OS API's like e.g. open() or CreateFile().

What about PGP and possibility of using this for signing any sort of files? Well, we use PGP a lot at ITL, but mainly for securing peer-to-peer communication (e.g. between us and our clients). There really is no good way to publish one's PGP key — the concept of Web of Trust might be good for some closed groups of people, but not for publishing files "to the world". And, of course, the first thing that an attacker who subverted PDFs on our website will do is to also subvert the PGP key displayed on the website. I also tried once to publish a PGP key to a key server, but got discouraged immediately after I noticed it didn't use SSL for submission. BTW, anybody knows if the key servers today use SSL? If not, how the trust is established? Maybe email clients, e.g. Thunderbird, come with built in PGP keys for select key servers?

So, I guess that was the main point of writing this post — to express how madly I would welcome a generic, OS-based, non-obligatory, signature verification for files, based on PKI :)

Ah, before a dozen of people jumps to the comment box to tell me that digital signatures do not assure non-maliciousness of anything — please don't do that, because I actually know that. In fact, it is not possible to assure non-maliciousness of pretty much anything, especially without strictly defining an ethical system we would like to use first. What the signatures provide is the liability, so that I know who to sue, in case my naked holiday pictures got leaked to the public because of some malicious PDF exploiting my system. In that case I can sue either the actual person who signed the PDF (if this person is identifiable) or the certification authority who issued the certificate to a wrong (unidentifiable) person.

Vegas Toys (Part I): The Ring -3 Tools

2009-08-25T11:59:00.007+02:00

We've just published the proof of concept code for the Alex's and Rafal's "Ring -3 Rootkits" talk, presented last month at the Black Hat conference in Vegas. You can download the code from our website here. It's highly recommended that one (re)reads the slides before playing with the code.

In short, the code demonstrates injection of an arbitrary ARC4 code into a vPro-compatible chipset AMT/ME memory using the chipset memory reclaiming attack. Check the README and the slides for more information.

The actual ARC4 code we distribute here is very simple: it sets a DMA write transaction to the host memory every ca. 15 seconds in order to write the "ITL" string at the predefined physical addresses (increased by 4 with every iteration). Of course one can do DMA read as well.

The ability to do DMA from the ARC4 code to/from the host memory is, in fact, all that is necessary to write a sophisticated rootkit or any sort of malware, from funny jokers to sophisticated secret sniffers. Your imagination (and good pattern searching) is the only limit here.

The OS, nor any software running on the host OS, cannot access our rootkit code, unless, of course, it used the same remapping attack we used to insert our code there :) But the rootkit might even cut off this way by locking down the remapping registers, so fixing the vulnerability on the fly, after exploiting it (of course it would be insane for any AV to use our remapping attack in order to scan ME space, but just for completeness;)

An OS might attempt to protect itself from DMA accesses from the rootkit in the chipset by carefully setting VT-d protections. Xen 3.3/3.4, for example, sets VT-d protections in such a way that our rootkit cannot access the Xen hypervisor memory. We can, however, access all the other parts of the system which includes all the domains memory (i.e. where all the interesting data are located). Still, it should be possible to modify Xen so that it set VT-d mappings in such a strict way, that the AMT code (and the AMT rootkit) could not access any useful information in any of the domains. This, in fact, would be a good idea anyway, as it would also prevent any sort of hardware-based backdoors (except for the backdoors in the CPU).

An AMT rootkit can, however, get around such a savvy OS because it can modify the OS's VT-d initialization code before it sets the VT-d protections. Alternatively, if the protections are set before the rootkit was activated, the rootkit can force the system to reboot and boot it from the AMT Virtual CDROM (In fact AMT has been designed to be able to do exactly that), which would contain rootkit agent code that would modify the OS/VMM to-be-loaded image, so that it doesn't setup VT-d properly.

Of course, the proper solution against such an attack would be to use e.g. Intel TXT to assure trusted boot of the system. In theory this should work. In practice, as you might recall, we have already shown how to bypass Intel TXT. This TXT bypass attack still works on most (all?) hardware, as there is still no STM available in the wild (all that is needed for the attack is to have a working SMM attack, and last month we showed 2 such attacks — see the slides for the BIOS talk).

Intel has released a patch a day before our presentation at Black Hat. This is a cumulative patch that is also targeting a few other, unrelated, problems, like e.g. the SMM caching attack (also reported by Loic), the SMM nvacpi attack, and the Q45 BIOS reflashing attack (for which the code will be also published shortly).

Some of you might remember that Intel has patched this very remapping bug last year, after our Xen 0wning Trilogy presentations, where we used the very same bug to get around Xen hypervisor protections. However, Intel forgot about one small detail — namely it was perfectly possible for malware to downgrade BIOS to the previous, pre-Black-Hat-2008 version, without any user consent (after all this old BIO file was also digitally signed by Intel). So, with just one additional reboot (but without a user intervention needed) malware could still use the old remapping bug, this time to get access to the AMT memory. The recent patch mentioned above solves this problem by displaying a prompt during reflash boot, if reflashing to an older version of BIOS. So now it requires user intervention (a physical presence). This "downgrade protection" works, however, only if we have administrator password enabled in BIOS.

We could get into the AMT memory on Q35, however, even if the downgrade attack was not possible. In that case we could use our BIOS reflashing exploit (the other Black Hat presentation).

However, this situation looks differently on Intel latest Q45 chipsets (that also have AMT). As explained in the presentation, we were unable to get access to the AMT memory on those chipsets, even though we can reflash the BIOS there, and consequently, even though we can get rid of all the chipset locks (e.g. the remapping locks). Still, the remapping doesn't seem to work for this one memory range, where the AMT code resides.

This suggest Intel added some additional hardware to the Q45 chipset (and other Series 4 chipsets) to prevent this very type of attacks. But we're not giving up on Q45 yet, and we will be trying other attacks, as soon as we recover from the holiday laziness ;)

Finally, the nice picture of the Q35 chipset (MCH), where our rootkit lives :) The ARC4 processor is somewhere inside...

Black Hat 2009 Slides

2009-07-30T23:18:00.006+02:00

The wait is over. The slides are here. The press release is here. Unless you're a chipset/BIOS engineer kind of person, I strongly recommend reading the press release first, before opening the slides.

So, the "Ring -3 Rootkit" presentation is about vPro/AMT chipset compromises. The "Attacking Intel BIOS" presentation is about exploiting a heap overflow in BIOS environment in order to bypass reflashing protection, that otherwise allows only Intel-signed updates to be flashed.

We will publish the code some time after get back from Vegas.

Enjoy.

ps. Let me remind my dear readers that all the files hosted on the ITL website are not digitally signed and are served over a plaintext connection (HTTP). In addition, the ITL's website is hosted on a 3rd party provider's server, on which we have totally no control (which is the reason why we don't buy an SSL certificate for the website). Never trust unsigned files that you download from the Internet. ITL cannot be liable for any damages caused by the files downloaded from our website, unless they are digitally signed.

Interview

2009-07-17T00:12:00.004+02:00

Alan Dang from Tom's Hardware did an interview with me. I talk there about quite a lot of things, many of which I would probably write about on this blog sooner or later (or already had), so I thought it might be of interest to the readers of this blog.

Virtualization (In)Security Training in Vegas

2009-06-12T14:08:00.003+02:00

VM escapes, hypervisor compromises (via "classic" rootkits, as well as Bluepill-like rootkits), hypervisor protection strategies, SMM attacks, TXT bypassing, and more — these are some of the topics that will be covered by our brand new training on Virtualization (In)Security at the upcoming Black Hat USA.

The training offers quite a unique chance, I think, to absorb the results of 1+ year of the research done by our team within... just 2 days. This will be provided via detailed lectures and unique hands-on exercises.

Unlike our previous training on stealth malware (that will also be offered this year, BTW), this time we will offer attendees a bit of hope :) We will be stressing that some of the new hardware technologies (Intel TXT, VT, TPM), if used properly, have potential to dramatically increase security of our computer systems. Sure, we will be showing attacks against those technologies (e.g. TXT), but nevertheless we will be stressing that this is the proper way to go in the long run.

Interestingly, I'm not aware of any similar training of this kind, that would be covering the security issues related to virtualization systems and bare metal hypervisors. Hope we will not get into troubles with the Antitrust Commission for monopolizing this field ;)

The training brochure (something for your boss) is here.

The detailed agenda spanning 2 full days can be downloaded here.

The Black Hat signup page is here.

Quest to The Core

2009-06-09T15:10:00.004+02:00

If you think SMM rootkits or PCI backdoors is low-level, then you should certainly see our talks in Vegas — ITL is going to define what does the "low-level" adjective really mean at the end of the decade ;)

In case you haven't noticed it at the Black Hat website yet — Alex and Rafal will be giving two presentations in Vegas:

1) Introducing Ring -3 Rootkits (description)

2) Attacking Intel® BIOS (description)

Let me stress that we have been in touch with Intel for quite some time about the above attacks, and that Intel is planning to release appropriate fixes a few weeks before our presentations at Black Hat.

There is more than just this coming at this year's Black Hat — most notably we will also be debuting with our Virtualization (In)Security Training. I will write a separate post about this training (containing a detailed agenda) in the coming days, so stay tuned.

Quite exciting.

More Thoughts on CPU backdoors

2009-06-02T00:16:00.003+02:00

I've recently exchanged a few emails with Loic Duflot about CPU-based backdoors. It turned out that he recently wrote a paper about hypothetical CPU-backdoors and also implemented some proof-of-concept ones using QEMU (for he doesn't happen to own a private CPU production line). The paper can be bought here. (Loic is an academic, and so he must follow some of the strange customs in the academic world, one of them being that papers are not freely published, but rather being sold on a publisher website… Heck, even we, the ultimately commercialized researchers, still publish our papers and code for free).

Let me stress that what Loic writes about in the paper are only hypothetical backdoors, i.e. no actual backdoors have been found on any real CPU (ever, AFAIK!). What he does is he considers how Intel or AMD could implement a backdoor, and then he simulate this process by using QEMU and implementing those backdoors inside QEMU.

Loic also focuses on local privilege escalation backdoors only. You should however not underestimate a good local privilege escalation — such things could be used to break out of any virtual machine, like VMWare, or potentially even out of a software VMs like e.g. Java VM.

The backdoors Loic considers are somewhat similar in principle to the simple pseudo-code one-liner backdoor I used in my previous post about hardware backdoors, only more complicated in the actual implementation, as he took care about a few important details, that I naturally didn't concern. (BTW, the main message of my previous post about was how cool technology this VT-d is, being able to prevent PCI-based backdoors, and not about how doomed we are because of Intel- or AMD-induced potential backdoors).

Some people believe that processor backdoors do not exist in reality, because if they did, the competing CPU makers would be able to find them in each others' products, and later would likely cause a "leak" to the public about such backdoors (think: Black PR). Here people make an assumption that AMD or Intel is technically capable of reversing each others processors, which seems to be a natural consequence of them being able to produce them.

I don't think I fully agree with such an assumption though. Just the fact that you are capable of designing and producing a CPU, doesn't mean you can also reverse engineer it. Just the fact that Adobe can write a few hundred megabyte application, doesn't mean they are automatically capable of also reverse engineering similar applications of that size. Even if we assumed that it is technically feasible to use some electron microscope to scan and map all the electronic elements from the processor, there is still a problem of interpreting of how all those hundreds of millions of transistors actually work.

Anyway, a few more thoughts about properties of a hypothetical backdoors that Intel or AMD might use (be using).

First, I think that in such a backdoor scenario everything besides the "trigger" would be encrypted. The trigger is something that you must execute first, in order to activate the backdoor (e.g. the CMP instruction with particular, i.e. magic, values of some registers, say EAX, EBX, ECX, EDX). Only then the backdoor gets activated and e.g. the processor auto-magically escalates into Ring 0. Loic considers this in more detail in his paper. So, my point is that all the attacker's code that executes afterwards, think of it as of a shellcode for the backdoor, that is specific for the OS, is fetched by the processor in an encrypted form and decrypted only internally inside the CPU. That should be trivial to implement, while at the same time should complicate any potential forensic analysis afterwards — it would be highly non-trivial to understand what the backdoor actually have done.

Another crucial thing for a processor backdoor, I think, should be some sort of an anti-reply attack protection. Normally, if a smart admin had been recording all the network traffic, and also all the executables that ever got executed on the host, chances are that he or she would catch the triggering code and the shellcode (which might be encrypted, but still). So, no matter how subtle the trigger is, it is still quite possible that a curious admin will eventually find out that some tetris.exe somehow managed to breakout of a hardware VM and did something strange, e.g. installed a rootkit in a hypervisor (or some Java code somehow was able to send over all our DOCX files from our home directory).

Eventually the curious admin will find out that strange CPU instruction (the trigger) after which all the strange things had happened. Now, if the admin was able to take this code and replicate it, post it to Daily Dave, then, assuming his message would pass through the Moderator (Hi Dave), he would effectively compromise the processor vendor's reputation.

An anti-replay mechanism could ideally be some sort of a challenge-response protocol used in a trigger. So, instead having you always to put 0xdeadbeaf, 0xbabecafe, and 0x41414141 into EAX, EBX and EDX and execute some magic instruction (say CMP), you would have to put a magic that is a result of some crypto operation, taking current date and magic key as input:

Magic = MAGIC (Date, IntelSecretKey).

The obvious problem is how the processor can obtain current date? It would have to talk to the south-bridge at best, which is 1) nontrivial, and 2) observable on a bus, and 3) spoof'able.

A much better idea would be to equip a processor with some sort of an eeprom memory, say big enough to hold one 64-bit or maybe 128-bit value. Each processor would get a different value flashed there when leaving the factory. Now, in order to trigger the backdoor, the processor vendor (or backdoor operator, think: NSA) would have to do the following:

1) First execute some code that would read this unique value stored in eeprom for the particular target processor, and send this back to them,

2) Now, they could generate the actual magic for the trigger:

Magic = MAGIC (UniqeValueInEeprom, IntelSecretKey)

3) ...and send the actual code to execute the backdoor and shellcode, with the correct trigger embedded, based on the magic value.

Now, the point is that the processor will automatically increment the unique number stored in the eeprom, so the same backdoor-exploiting code would not work twice for the same processor (while at the same time it would be easy for NSA to send another exploit, as they know what the next value in the eeprom should be). Also, such a customized exploit would not work on any other CPU, as the assumption was that each CPU gets a different value at the factory, so again it would not be possible to replicate the attack and proved that the particular code has ever done something wrong.

So, the moment I learn that processors have built-in eeprom memory, I will start thinking seriously there are backdoors out there :)

One thing that bothers me with all those divagations about hypothetical backdoors in processors is that I find them pretty useless in at the end of the day. After all, by talking about those backdoors, and how they might be created, we do not make it any easier to protect against them, as there simply is no possible defense here. Also this doesn't make it any easier for us to build such backdoors (if we wanted to become the bad guys for a change). It might only be of an interest to Intel or AMD, or whatever else processor maker, but I somewhat feel they have already spent much more time thinking about it, and chances are they probably can only laugh at what we are saying here, seeing how unsophisticated our proposed backdoors are. So, my Dear Reader, I think you've been just wasting time reading this post ;) Sorry for tricking you into this and I hope to write something more practical next time :)

Thoughts About Trusted Computing

2009-05-28T00:58:00.004+02:00

Here are the slides about Trusted Computing I used for my presentations at the EuSecWest today, and at the Confidence conference last week.

As this was supposed to be a keynote, the slides are much less technical then our other slides, and also there are no new attacks presented there. Still, I hope they might be useful as some sort of an "alternative" introduction to Trusted Computing :)

A cool presentation I saw today was about PCI-based backdoors by Christophe Devine and Guillaume Vissian. They basically took a general-purpose FPGA programmable PC-card (AKA PCMCIA), flashed it with an FPGA "program" that implemented a simple state machine. The purpose of the state machine was to wait until its DMA engine gets initialized and then to modify certain bytes in the host memory, that happened to be part of the winlogon.exe process (IIRC they changed XOR AL, AL into MOV AL, 1, or something like that, at the end of some password verification function inside the winlogon.exe process). The slides should be available soon on the conference website. I also hope they will publish all the source code needed to flash your own personal "winlogon unlocker".

The live demo was really impressive — they showed a winlogon screen, tried to login a few times with wrong passwords, of course all the attempts failed, then they inserted their magic, $300 worth, PC-card, and… 2 seconds later they could log in using any password they wanted.

While not necessary being a breakthrough, as everybody has known such things could be done for years, I think it is still important that somebody eventually implemented this, discussed the technical details (FPGA-related), and also showed how to implement it with a cheap generic "reflashable" hardware without using a soldering iron.

Of course I have also discussed in my presentation how to prevent PCI-based backdoors (like the one discussed here) using VT-d, but this defense is currently only available if you use Xen 3.3 or later, and also requires that you manually create driver domain partitions and come up with a reasonable scheme for assigning devices to driver domains. All in all 99.9% of users are not (and will not be anytime soon) protected against such attacks. Oh, wait, there is actually a relatively simple software-based workaround (besides putting a glue into your PC-card slot, which is not a very subtle one)… I wonder who else will find out :)

Trusting Hardware

2009-03-25T14:37:00.008+01:00

So, you're a decent paranoid person, running only open source software on your box: Linux, GNU, etc. You have the feeling you could, if you only wanted to, review every single line of code (of course you will probably never do this, but anyway). You might be even more paranoid and also try running an open source BIOS. You feel satisfied and cannot understand all those stupid people running closed source systems like e.g. Windows. Right?

But here's where you are stuck — you still must trust your hardware. Trust that your hardware vendor has not e.g. built in a backdoor into your network card micro-controller…

So, if we buy a laptop from vendor X, that might be based in some not-fully-democratic country, how do we know they didn't put backdoors there? And not only to spy on Americans, also to spy on their own citizens? When was the last time you reverse-engineered all the PCI devices on your motherboard?

Scared? Good!

Enters the game-changer: IOMMU (known as VT-d on Intel). With proper OS/VMM design, this technology can address the very problem of most of the hardware backdoors. A good example of a practical system that allows for that is Xen 3.3, which supports VT-d and allows you to move drivers into a separate, unprivileged driver domain(s). This way each PCI device can be limited to DMA only to the memory region occupied by its own driver.

The network card's microcontroller can still compromise the network card driver, but nothing else. Assuming we are using only encrypted communication, there is not much an attacker can gain by compromising this network card driver, besides doing a DoS. Similarly for the disk driver — if we use full disk encryption (which is a good idea anyway), there is not much an attacker can gain from compromising the low-level disk driver.

Obviously the design of such a system (especially used for desktop computing) is not trivial ans needs to be thoroughly thought out. But it is possible today(!), thanks to those new virtualization technologies.

It seems than, that we could protect ourselves against potentially malicious hardware. With one exception however… we still need to trust the CPU and also the memory controller (AKA northbridge AKA chipset), that implements that IOMMU.

On AMD systems, the memory controller has long been integrated into the processor. Also Intel's recent Nehalem processors integrate the memory controller on the same die.

This all means we need to trust only one vendor (Intel or AMD) and only one component, i.e. The Processor. But should we blindly trust them? After all it would be trivial for Intel or AMD to build in a backdoor into their processor. Even something as simple as:

if (rax == MAGIC_1 && rcx == MAGIC_2) jmp [rbx]

Just a few more gates in the CPU I guess (there are apparently already about 780 million gates on Core i7, so a few more should not make much difference), and no performance penalty. Exploitable remotely on most systems and any more complex program I guess. Yet, totally undetectable for anybody without an electron microscope (and tons of skills and knowledge).

And this is just the simplest example that comes to mind within just a few minutes. I'm sure one could come up with something even more universal and reliable. The fact is — if you are the CPU vendor, it is trivial for you to build in an effective backdoor.

It's funny how various people, e.g. European government institutions, are afraid of using closed source software, e.g. Windows, because they are afraid of Microsoft putting backdoors there. Yet, they are not concerned about using processors made by some other US companies. It is significantly more risky for Microsoft to put a backdoor into its software, where even a skilled teenager equipped with IDA Pro can find it, than it is for Intel or AMD, where effectively nobody can find it.

So, I wonder whether various government and large corporate customers from outside the US will start asking Intel and AMD to provide them with the exact blueprints of their processors. After all they already require Microsoft to provide them with the source code under an NDA, right? So, why not the "source code" for the processor?

Unfortunately there is nothing that could stop a processor vendor to provide its customers with a different blueprints than those that are used to actually "burn" the processors. So, the additional requirement would be needed that they also allow to audit their manufacturing process. Another solution would be to hire some group of independent researchers, equip them with an electron microscope and let them reverse engineer some randomly chosen processors… Hmmm, I even know a team that would love to do that ;)

A quick summary in case you get lost already:

On most systems we are not protected against hardware backdoors, e.g. in the network card controller.
New technologies, e.g. Intel VT-d, can allow to protect against potentially malicious hardware (requires specially designed OS, e.g. specially configured Xen)…
… except for the potential backdoors in the processor.
If we don't trust Microsoft, why should we trust Intel or AMD?

BTW, in May I will be speaking at the Confidence conference in Krakow, Poland. This is gonna be a keynote, so don't expect new attacks to be revealed, but rather some more philosophical stuff about trusted computing (why it is not evil) and problems like the one discussed today. See you there!

The Sky Is Falling?

2009-03-20T17:20:00.003+01:00

A few reporters asked me if our recent paper on SMM attacking via CPU cache poisoning means the sky is really falling now?

Interestingly, not many people seem to have noticed that this is the 3rd attack against SMM our team has found in the last 10 months. OMG :o

But anyway, does the fact we can easily compromise the SMM today, and write SMM-based malware, does that mean the sky is falling for the average computer user?

No! The sky has actually fallen many years ago… Default users with admin privileges, monolithic kernels everywhere, most software unsigned and downloadable over plaintext HTTP — these are the main reasons we cannot trust our systems today. And those pathetic attempts to fix it, e.g. via restricting admin users on Vista, but still requiring full admin rights to install any piece of stupid software. Or selling people illusion of security via A/V programs, that cannot even protect themselves properly…

It's also funny how so many people focus on solving the security problems by "Security by Correctness" or "Security by Obscurity" approaches — patches, patches, NX and ASLR — all good, but it is not gonna work as an ultimate protection (if it could, it would worked out already).

On the other hand, there are some emerging technologies out there that could allow us to implement effective "Security by Isolation" approach. Such technologies as VT-x/AMD-V, VT-d/IOMMU or Intel TXT and TPM.

So we, at ITL, focus on analyzing those new technologies, even though almost nobody uses them today. Because those technologies could actually make the difference. Unlike A/V programs or Patch Tuesdays, those technologies can change the level of sophistication required for the attacker dramatically.

The attacks we focus on are important for those new technologies — e.g. today Intel TXT is pretty much useless without protection from SMM attacks. And currently there is no such protection, which sucks. SMM rootkits sound sexy, but, frankly, the bad guys are doing just fine using traditional kernel mode malware (due to the fact that A/V is not effective). Of course, SMM rootkits are just yet another annoyance for the traditional A/V programs, which is good, but they might not be the most important consequence of SMM attacks.

So, should the average Joe Dow care about our SMM attacks? Absolutely not!

Attacking SMM Memory via Intel® CPU Cache Poisoning

2009-03-19T17:02:00.001+01:00

As promised, the paper and the proof of concept code has just been posted on the ITL website here.

A quote from the paper:

In this paper we have described practical exploitation of the CPU cache poisoning in order to read or write into (otherwise protected) SMRAM memory. We have implemented two working exploits: one for dumping the content of SMRAM and the other one for arbitrary code execution in SMRAM. This is the third attack on SMM memory our team has found within the last 10 months, affecting Intel-based systems. It seems that current state of firmware security, even in case of such reputable vendors as Intel, is quite unsatisfying.

The potential consequence of attacks on SMM might include SMM rootkits [9], hypervisor compromises [8], or OS kernel protection bypassing [2].

Don't worry, the shellcode we use in the exploit is totally harmless (have really no idea how some people concluded we were going to release an SMM rootkit today?) — it only increases an internal counter on every SMI and jumps back to the original handler. If you want something more fancy, AKA SMM rootkits, you might want to re-read Sherri's and Shawn's last year's Black Hat paper and try writing something they describe there.

The attack presented in the paper has been fixed on some systems according to Intel. We have however found out that even the relatively new boards, like e.g. Intel DQ35 are still vulnerable (the very recent Intel DQ45 doesn't seem to be vulnerable though). The exploit attached is for DQ35 board — the offsets would have to be changed to work on other boards (please do not ask how to do this).

Keep in mind this is a different SMM attack than the one we mentioned during our last month's Black Hat presentation on TXT bypassing (the VU#127284). We are planning to present that other attack at the upcoming Black Hat Vegas. Hopefully this will not be the only one thing that ITL will entertain you with in Vegas — Alex and Rafal are already working now on something even cooler (and even lower level) for the show, so cross your fingers!

And good luck to Loic with his presentation that is about to start just now...

Independent Attack Discoveries

2009-03-13T13:22:00.004+01:00

Next week's Thursday, March 19th, 1600 UTC, we will publish a paper (+ exploits) on exploiting Intel® CPU cache mechanisms.

The attack allows for privilege escalation from Ring 0 to the SMM on many recent motherboards with Intel CPUs. Interestingly, the very same attack will be presented by another researcher, Loic Duflot, at the CanSecWest conference in Vancouver, Canada, on... Thursday 19th, 1600 UTC. BTW, this is a different SMM-targeting attack than the one we mentioned during our recent TXT talk and that is scheduled to be presented later this year.

Here's the full story (there is also a moral at the end) …

Just after our presentation at the Black Hat last month, we (i.e. Rafal and I) have been independently approached by some person (or two different persons — we haven't figured that out actually — there were some ca. 30 people willing to ask us questions after the talk, so it's hard to remember all the faces), who was very curious about our SMM attacks (whose details we haven't discussed, of course, because Intel is still working on a fix). This person(s) started asking various questions about the attacks and one of the questions, that was asked to both me and Rafal, was if the attack used caching. Later that day, during a private ITL dinner, one of us brought this issue, and we started thinking if it was indeed possible to perform an SMM attack via CPU caching. By the end of the dinner we have sketched out the attack, and later when we got back to Poland, Rafal implemented a working exploit with code execution in SMM in a matter of just a few hours. (I think I used way too many parenthesis in this paragraph).

So, being the good and responsible guys that we are, we immediately reported the new bug to Intel (actually talking to Intel's PSIRT is getting more and more routined for us in the recent months ;). And this is how we learnt that Loic came up with the same attack (back then there was no talk description at the conference website) — apparently he approached Intel about this back in October 2008, so 3-4 months before us — and also that he's planning to present it at the CanSecWest conference in March. So, we contacted Loic and agreed to do coordinated disclosure next Thursday.

Interestingly, however, none of us was even close to being the first discoverer of the underlying problem that our attacks exploit. In fact, the first mention of the possible attack using caching for compromising SMM has been discussed in certain documents authored as early as the end of 2005 (!) by nobody else than... Intel's own employees. Stay tuned for the details in our upcoming paper.

Conclusion

If there is a bug somewhere and if it stays unpatched for enough time, it is almost guaranteed that various people will (re)discover and exploit it, sooner or later. So, don't blame researchers that they find and publish information about bugs — they actually do a favor to our society. Remember the guy who asked us if our attack used caching? I bet he (or his associates) also have had exploits for this caching bug, but apparently didn't notify the vendor. Hmm, what they might have been doing with the exploit? When was the last time you scanned your system for SMM rootkits? ;)

Anyways, congrats to Loic for being the first one who wrote exploits for this bug. Also congrats to Intel employees who originally noticed the problem back in 2005.

Attacking Intel TXT: paper and slides

2009-02-19T22:23:00.005+01:00

The new press release covering the basic details about our TXT attack is here.

The paper is here.

The slides converted to a PDF format are here. There is also an original version of slides in the Keynote format here for the Mac people. And for all the other people who don't use Mac, but still value the aesthetics (?!), I have also generated a QuickTime clickable movie out from the Keynote slides -- it can be found here, but it weighs 80MB.

Enjoy.

Nesting VMMs, Reloaded.

2009-02-10T18:30:00.007+01:00

Besides breaking the Intel's stuff, we have also been doing other things over at our lab. Thought I would share this cool screenshot of a Virtual PC running inside Xen. More precisely what you see on the pic is: Windows XP running inside Virtual PC, that runs inside Vista, which itself runs inside a Xen's HVM VM. Both the Virtual PC and Xen are using the Intel's hardware virtualization (VT-x is always used for HVM guests on Xen).

Our Nested Xen patch is part of a work done for a customer, so it is not going to be published. Besides it is currently a bit unstable ;) It's just a prototype that shows such a thing could be done.

Closed Source Conspiracy

2009-01-26T17:56:00.004+01:00

Many people in the industry have an innate fear of closed source (AKA proprietary software), which especially applies to everything crypto-related.

The usual arguments go this way: this (proprietary) crypto software is bad, because the vendor might have put some backdoors in there. And: only the open source crypto software, which can be reviewed by anyone, can be trusted! So, after my recent post, quite a few people wrote to me and asked how I could defend such an evil thing as BitLocker, which is proprietary, and, even worse, comes from Microsoft?

I personally think this way of reasoning sucks. In majority of cases, the fact something is distributed without the accompanying source code does not prevent others from analyzing the code. We do have advanced disassemblers and debuggers, and it is really not that difficult to make use of them as many people think.

Of course, some heavily obfuscated programs can be extremely difficult to analyze. Also, analyzing a chipset's firmware, when you do not even know the underlying CPU architecture and the I/O map might be hard. But these are special cases and do not apply to majority of software, that usually is not obfuscated at all.

It seems like the argument of Backdoored Proprietary Software usually comes from the open-source people, who are used to unlimited accesses to the source code, and consequently do not usually have much experience with advanced reverse engineer techniques, simply because they do not need them in their happy "Open Source Life". It's all Darwinism, after all ;)

On the other hand, some things are hard to analyze, regardless of whether the source code is available or not, think: crypto. Also, how many of you who actively use open source crypto software, e.g. TrueCrypt or GnuPG, have actually reviewed the source code? Anyone?

You might be thinking — maybe I haven't looked at the source code myself, but because it is open source, zillions of other users already have reviewed it. And if there was some backdoor in there, they would undoubtedly have found it already! Well, for all those open source fetishists, who blindly negate the value of anything that is not open source, I have only one word to say: Debian.

Keep in mind: I do not say closed source is more secure than open source — I only resist the open-source fundamentalism, that defines every proprietary software as inherently insecure, and everything open source as ultimately secure.

So, how should one (e.g. a government institution) verify security-level of a given crypto software, e.g. to ensure there are no built-in backdoors in there? I personally doubt it could be performed by one team, as it just usually happens that the same people who might be exceptionally skilled in code review, system-level security, etc, at the same time are average cryptographers and vice-versa.

Imagine e.g. that you need to find out if there are any weaknesses in your system drive encryption software, something like BitLocker. Even if you get access to the source code, you still would have to analyze a lot of system-level details — how is the trusted boot implemented (SRTM? DRTM? TPM interaction?), which system software is trusted, how the implementation withstands various not-crypto-related attacks (e.g. some of the attacks I described in my previous post), etc…

But this all is just system-level evaluation. What should come later is to analyze the actual crypto algorithms and protocols. Those later tasks fall into cryptography field and not into system-level security discipline, and consequently should be performed by some other team, the crypto experts.

So, no doubt, it is not an easy task, and the fact if there is or there is not C/C++ source code available, is usually one of the minor headaches (a good example is our attack on TXT, where we were able to discover bugs in Intel's specific system software, which, of course, is not open source).

Why do I miss Microsoft BitLocker?

2009-01-21T18:52:00.009+01:00

In the previous post, I wrote the only one thing I really miss after I've switched from Vista to Mac is the BitLocker Driver Encryption. I thought it might be interesting for others to understand my position, so below I describe why I think BitLocker is so cool, and why I think other system disk encryption software sucks.

So, it's all about the Trusted Boot. BitLocker does make use of a trusted boot process, while all the other system encryption software I'm aware of, does not. But why the trusted boot feature is so useful? Let's start with a quick digression about what the trusted boot process is…

Trusted boot can be implemented using either a Static Root of Trust or a Dynamic Root of Trust.

The Static Root of Trust approach (also known as Static Root of Trust Measurement or SRTM) is pretty straightforward — the system starts booting from some immutable piece of firmware code that we assume is always trusted (hence the static root) and that initiates the measurement process, in which each component measures the next one in a chain. So, e.g. this immutable piece of firmware will first calculate the hash of the BIOS and extend a TPM's PCR register with the value of this hash. Then the BIOS does the same with the PCI EEPROMs and the MBR, before handling execution to them. Then the bootloader measures the OS loader before executing it. And so on.

An alternative method to implementing trusted boot is to use Dynamic Root of Trust (often called Dynamic Root of Trust Measurement or DRTM). Intel's TXT technology, formerly LaGrande, is an example of a DRTM (more precisely: TXT is more than just DRTM, but DRTM is the central concept on which TXT is built). We will be talking a lot about TXT next month at Black Hat in DC :) This will include discussion of why DRTM might sometimes be preferred over SRTM and, of course, what are the challenges with both.

Essentially, both SRTM and DRTM, in the context of a trusted boot, are supposed to provide the same: assurance the system that just booted is actually the system that we wanted to boot (i.e. the trusted one) and not some modified system (e.g. compromised by an MBR virus).

BitLocker uses the Static Root of Trust Measurement. SRTM can really make sense when we combine it with either TPM sealing or attestation feature. BitLocker uses the former to make sure that only the trusted system can get access to the disk decryption key. In other words: BitLocker relies on the TPM that it will unseal (release) the decryption key (needed to decrypt the system partition) when and only when the state of chosen PCR registers is the same is it was when the decryption key was sealed into the TPM.

Ok, why is this trusted boot process so important for the system disk encryption software? Because it protects against a simple two-stage attack:

You leave your laptop (can be even fully powered down) in a hotel room and go down for a breakfast… Meanwhile an Evil Maid enters your room. She holds an Evil USB stick in her hand and plugs it into your laptop and presses the power button. The system starts and boots from the USB. An Evil version of something similar to our BluePillBoot gets installed into the MBR (or to a PCI EEPROM). This Evil Program has only one task — to sniff out the encryption software's password/PIN and then report it back to the maid next time she comes...
So, you come back to your room to brush your teeth after the breakfast. Obviously you cannot refrain from not turning on your laptop for a while. You just need to enter your disk encryption passphrase/PIN/whatever. Your encryption software happily displays the prompt, like if nothing happened. After all how could it possibly know the Evil Program, like BluePillBoot, has just been loaded a moment ago from the MBR or a PCI EEPROM? It can not! So, you enter the valid password, your system gets the decryption key and you can get access to your encrypted system...
But then you have an appointment at the hotel SPA (at least this little fun you can have on a business trip, right?). Obviously you don't want to look so geeky and you won't take your laptop with you to the SPA, will you? The Evil Maid just waited for this occasion… She sneaks again into your room and powers up your laptop. She presses a magic key combo, which results in the Evil Program displaying the sniffed decryption password. Now, depending on their level of subtleness, she could either steal your whole laptop or only some more important data from the laptop. Your system disk encryption software is completely useless against her now.

(Yes, I know that's 3 bullets, but the Evil Maid had to sneak into your room only twice:)

So, why the BitLocker would not allow for this simple attack? Because the BitLocker software should actually be able to know that the system gets compromised (by the Evil Program) since the last boot. BitLocker should then refuse to display a password prompt. And even if it didn't and asked the user for the password, still it should not be able to get the actual decryption key out from the TPM, because the values in the certain PCR register(s) will be wrong (they will now account for the modified hashes of the MBR or PCI EEPROM or BIOS). The bottom line is: the maid is not getting the decryption key (just as the user now), which is what this is all about.

At least this is how the BitLocker should work. I shall add a disclaimer here, that neither myself, nor anybody from my team, have looked into the BitLocker implementation. We have not, because, as of yet, there have been no customers interested in this kind of BitLocker implementation evaluation. Also, I should add that Microsoft has not paid me to write this article. I simply hope this might positively stimulate other vendors, like e.g. TrueCrypt (Hi David!), or Apple, to add SRTM or, better yet, DRTM, to their system encryption products.

Of course, when we consider an idiot-attack, that involves simply garbbing somebody's laptop and running away with it (i.e. without any prior preparation like our Evil Maid did), then probably all system disk encryption software would be just good enough (assuming it doesn't have any bugs in the crypto code).

Some people might argue that using a BIOS password would be just as good as using trusted boot. After all, if we disable booting from alternate media in BIOS (e.g. from USB sticks) and lock down the BIOS using a password (i.e. using the Power-On password, not just the BIOS supervisor password), then the above two-stage attacks should not be feasible. Those people might argue, that even if the Evil Maid had cleared the CMOS memory (by removing the CMOS battery from the motherboard), still they would be able to notice that something is wrong — the BIOS would not longer be asking for the password, or the password would be different from what they used before.

That is a valid point, but relaying on the BIOS password to provide security for all your data might not be such a good idea. First problem is that all the BIOSes have had a long history of various default or "maintenance" passwords (I actually do not know how the situation looks today with those default passwords). Another problem is that the attacker might first clear the CMOS memory, and then modify her Evil MBR program to also display a fake BIOS password prompt, that would accept anything the user enters. This way the user will not be alerted that something is wrong and will be willing to provide the real password for drive decryption when prompted later by the actual drive encryption software.

One might ask why can't the attacker use the similar attack against BitLocker? Even if the real BitLocker uses trusted boot process, we can still infect the MBR, display the fake BitLocker PIN prompt and this way get into the possession of the user's PIN.

This attack, however, can be spotted by an inquisitive user — after all, if he or she knows they provided the correct PIN, then it would be suspicious not to see the system being booted (and it won't boot, because the fake BitLocker will not be able to retrieve the password from the TPM). If the fake BitLocker wanted to boot the OS (so that user didn't suspect anything), it would have to remove itself from the system and then reboot the system. Again this should alert the user that something wrong is going on.

There is also a much more elegant way of defending against the above attack (with fake BitLocker prompt) — but I'd rather let Microsoft to figure it out by themselves ;)

By the way, contrary to a popular belief the BitLocker doesn't protect your computer from boot-stage infections, e.g. MBR viruses or BIOS/PCI rootkits. As we have been pointing out since the first edition of our Understanding Stealth Malware training at Black Hat in August 2007, BitLocker should not be thought as of a system integrity protection. This is because it is trivial, for any malware that already got access to the running Vista, to re-seal the BitLocker key to arbitrary new system firmware/MBR configuration. Everybody can try it by going to Control Panel/BitLocker Driver Encryption, then clicking on the "Turn Off BitLocker" and choosing "Disable BitLocker Drive Encryption". This will simply save your disk decryption key in plaintext, allowing you to e.g. reflash your BIOS, boot Vista again and then to enable BitLocker again (this would generate a new key and seal it to the TPM with the new PCR values).

This functionality has been provided obviously to allow user to update his or her firmware. But what is important to keep in mind is that this process of disabling BitLocker doesn't involve entering any special password or PIN (e.g. the BitLocker's PIN). It just enough that you are the default user with admin rights or some malware running in this context. Pity they decided on the simplest solution here. But still BitLocker is simply the one coolest thing in Vista and something I really miss on all other OSes...

Attacking Intel® Trusted Execution Technology

2009-01-05T16:30:00.005+01:00

Press people: please read our press release first and also refer to the disclaimer at the end of this blog post. Thank you!

Update: 1/5/2009 19:21 CEST: minor typos/spelling corrections. Thanks to Jarred for point out some of the typos.

A word about Trusted Computing
The term Trusted Computing and related technologies, like Palladium, Trusted Platform Module, LaGrande, have always caused lots of controversy in the IT world. Most of the fear, however, has been a result of the lack of understanding of how a particular technology really works.

Nevertheless, Trusted Computing is becoming part of our lives, whether we want it or not. These days almost every new laptop comes with an on-board Trusted Platform Module (TPM). Microsoft's Palladium initiative have been renamed so many times in the recent years, that probably even people working at Microsoft are confused now. Nevertheless, some of the Palladium technologies made their way into Vista, and Microsoft BitLocker is, without doubt, the most successful, widely deployed product that is based on the idea of Trusted Computing. (In fact the Bitlocker is the only one thing that I really have been missing since I switched from Vista to Mac some time ago).

On the hardware side, besides the famed TPM, we also have had the LaGrande technology, that is often connected with things such as Remote Attestation, Protected Execution and other scary terms…

A word about Trusted Execution Technology
LaGrande, recently renamed Trusted Execution Technology (TXT), is Intel's response to the Trusted Computing trend. TXT is currently part of the vPro™ brand, and for about a year now users can buy a vPro/TXT compatible hardware in regular computer stores (the first one was the DQ35J desktop board with certain Core 2 Duo processors, which I was able to buy at the end of 2007 — remember that TXT requires support from both the CPU and the chipset).

TXT is not an alternative to TPM, in fact TXT heavily relies on the TPM to provide basic services like e.g. secure storage of measurements done by the TXT. Also, Palladium, or whatever it is called these days, is not a competition to TXT. Intel TXT can provide building blocks to e.g. Vista Bitlocker, arguably making it more secure then it is now (Current Bitlocker implementation, AFAIK, relies on a so called Static Root of Trust for Measurement, which requires TPM, but not TXT).

What kind of measurement would TXT like to store in our TPM? Well, the whole TXT is, in fact, all about making and storing software measurements, or, using a more familiar language, secure hashes of certain software components.

The sole purpose of Intel TXT technology is to provide a trusted way for loading and executing system software, e.g. Operating System kernel or Virtualization Machine Monitor. What is extraordinary here is that TXT doesn't make any assumptions about the state of the system before loading the software, thus making it possible for a user to ensure secure load of an OS or VMM, even in a potentially compromised machine.

In other words, our system can be all full of boot sector viruses and BIOS rootkits, and god-knows-what-else, and still TXT should allow to load a clean VMM (or OS kernel) in a secure way, immune to all those rootkits present in the system in a moment just before the load process. This TXT-supported load process is called Late Launch, and is implemented via a special new CPU instruction called SENTER.

It's a good place to mention that AMD has its own version of the late launch implemented via SKINIT instruction. We haven't looked at the AMD technology thoroughly yet, so I will refrain from commenting on this.

The late launch is a pretty amazing thing, when we think about. It promises to effectively provide all the benefits of a computer restart without actually restarting it.

It is hard to overemphasize the potential impact that a technology such as TXT could have on computer security. One can immediately see that it could eliminate all the system-level persistent malware — in other words we can easily build systems (VMMs or even standard OSes) that would be immune to attacks that try to compromise system binaries on disk, or attack the system right from the bootloader or BIOS. Combining this with VT-x and VT-d technologies, system developers (for the first time, at least as far as the "PC" platform is considered) have gotten extremely strong tools into their hands that should allow them to create really secure VMMs and OSes…

Hopefully by now, my Dear Reader, you should have the feeling what kind of an animal Intel TXT is and how desperately the world needs it...

And now, we are going to move on and show practical attacks on current TXT implementations... :)

Attacking Intel TXT!
Ok, not in this post today, but rather at the upcoming Black Hat conference in Washington, DC in February. Over the recent months, Rafal and I have been looking at the Intel TXT technology as part of a work done for a customer, to see if this could be used to improve security of a product, from a typical user's perspective. We figured out that it definitely could, but that there are also some issues…

And those "issues" gave us a starting point in developing a proof-of-concept (albeit very reliable) exploit that shows how we can bypass trusted boot process implemented by Intel's tboot.

Tboot, which is also part of (scroll down to the end of the page) the Xen hypervisor, can be though of as a reference implementation of TXT-based system loader, that could be used to securely load either the Xen hypervisor or the Linux kernel, when run on a vPro/TXT compatible hardware.

[copy-and-paste from the press release follows]

Our attack comprises two stages. The first stage requires an implementation flaw in a specific system software. The second stage of the attack is possible thanks to a certain design decision made in the current TXT release.

While evaluating the effectiveness of the Intel® TXT technology, as part of a work done for a customer, we have identified several implementation flaws in the Intel's system software, which allowed to conduct the above mentioned stage-one attack. We have provided Intel with extensive description of the flaws in December 2008, and Intel is currently working on fixing those vulnerabilities.

We have also been in touch with Intel about the possibility of conducting the second-stage attack since November 2008. In December, after providing Intel with the details about the first-stage attack, Intel promised to release, in the coming weeks, an updated TXT specification for developers that would explain how to design their TXT-based loaders in such a way that they are immune to our attack. Intel claims the current Intel® TXT release does contain the basic building blocks that could be used to prevent our second-stage attack and the release of the additional specification would make it feasible in practice.

More details in February in DC :)

TXT useless?
Some people are skeptical about the TXT technology, and not only because of the Irrational Fear of the Trusted Computing (IFTC), but rather because they point out to the complexity of the whole technology. The complexity is bad, because 1) it leaves more space for potential attacks, and 2) it discourages developers (ISVs) from using the technology in their products (e.g. neither Microsoft, nor VMWare make use of TXT in any of their bare-metal hypervisors, even though TXT is very well suited for this kind of software).

It is true that TXT is a very complex technology (the SENTER instruction is probably the masterpiece of the CISC architecture!), but I personally like it. In my opinion this is the first technology available for the PC platform that has the potential to really change something, much more then the NX-feature did a few years ago. Before people will run to the comment box — if you would like to argue about the usefulness/uselessness of Trusted Computing/TXT, please base your opinions on technical facts (read the spec!) and not on your feelings!

Disclaimer (for press)

Starting January 2009, we (at Invisible Things Lab), decided to issue press releases in addition to this blog. The general rule is: press releases are written for journalists, while the blog is mainly written for other researchers, security enthusiast, etc.

The wording of our press releases is carefully chosen to minimize the potential of a possible misinterpretation. The press releases carry less information, but, we think, are better suited for a more general public, that doesn't have background in computer science, programming and security.

The blog is written in a much more casual way, without thinking for half an hour on every sentence. The articles on this blog might present some facts as extremely exciting, because e.g. for me, a person deeply involved in a system-level security research, they indeed might be very exciting, which might not be the case for a general audience. I sometimes might also use shortcuts, metaphors, or irony, and other figures of speech, that might not necessarily be obvious for a more general public.

If you are a journalist and you think you just found something very sensational on my blog, I would suggest that you double-check with me, before writing about it.

Thank you for your cooperation.
Joanna Rutkowska,
Founder and CEO,
Invisible Things Lab.

Microsoft executive "rebuts" our research!

2008-09-07T09:54:00.004+02:00

Ah, there is no feeling like seeing your name in the news when drinking your morning coffee... In this piece some Steve Riley, a senior security strategist at Microsoft, decided to "rebute" our recent Black Hat presentations research results.

Mr. Riley had been quoted by ZDnet as saying:

"Her [Joanna Rutkowska] insistence is that you can replace the hypervisor without anybody knowing... Our assertion is that this is incorrect," Riley told the audience. "First of all, to do these attacks you need to become administrator at the root. So that's going to be, on an appropriately configured machine, an exceedingly difficult thing to happen."

Apparently, Mr. Riley has never seen our Black Hat presentations (or slides at least) that he is referring to (oh, wait, that is the typical case with all our "refuters", how come?)...

First, we never said anything about replacing the hypervisor. I really have no idea how this idea was born in Mr. Riley's head? Replacing the hypervisor - that would indeed be insane for us to do!

Second, it is not true that the attacker needs to become an administrator "at the root" (he mean the root partition or administrative domain here I assume). The attack we presented in our second speech, that exploited a heap overflow in the Xen hypervisor FLASK module, could have been conducted from the unprivileged domain, as we demonstrated during the presentation.

Mr. Riley continues with his vision:

"Because you [the attacker] didn't subject your own replacement hypervisor through the thorough design review that ours did, I'll bet your hypervisor is probably not going to implement 100 percent of the functionality as the original one," Riley said. "There will be a gap or two and we will be able to detect that."

Well, if he only took the effort of looking into our slides, he would realize that, in case of XenBluePill, we were slipping it beneath (not replacing!) the original hypervisor, and then run the original one as nested. So, all the functionality of the original hypervisor was preserved.

Mr. Riley also shares some other ground breaking thoughts in this article, but I think we can leave them uncommented ;)

This situation is pretty funny actually - we have here the words and feelings of some Microsoft executive vs. our three technical presentations, all the code that we released for those presentations, and also a few of our demos. Yet, it's apparently still worth getting into the news and reporting what the feeling of Mr. Riley are...

Let me, however, write one more time, that I'm (still) not a Microsoft hater. There are many people at Microsoft that I respect: Brandon Baker, Neil Clift, the LSD guys, Mark Russinovich, and probably a few more that I just haven't had occasion to meet in person or maybe forgot about at the moment. It's thus even more sad that people like Mr. Riley are also associated with Microsoft, even more they are the face of Microsoft for the majority of people. Throwing a party in Vegas and Amsterdam once a year certainly is not enough to change the Microsoft's image in this case...

Interestingly, if Mr. Riley only attended our Xen 0wning Trilogy at Black Hat, then he would notice that we were actually very positive about Hyper-V. Of course, I pointed out that Xen 3.3 certainly has a more secure architecture right now, but I also said that I knew (from talking to some MS engineers from the virtualization group) that Hyper-V is going to implement similar features in the next version(s) and that this is very good. I also prized the fact it has only about 100k LOC (vs. about 300k LOC in Xen 3.3).

So, Mr. Senior Security Strategist, I suggest you do your homework more carefully next time before throwing mud at others and trying to negate the value of their work (and all the efforts of Microsoft's PR people).

On a separate note, I found it quite unprofessional that ZDNet's Liam Tung and Tom Espiner, the authors of the news, didn't ask me for a commentary before publishing this. Not to mention that they also misspelled Rafal's name and forgot to mention about Alex, the third co-author of the presentations.

Xen 0wning Trilogy: code, demos and q35 attack details posted

2008-09-06T13:59:00.001+02:00

We have posted all the code that we used last month during our Black Hat presentations about Xen security, and you can get it here. This includes the full source code for:
1) The generic Xen Loadable Modules framework
2) Implementation of the two Xen Hypervisor Rootkits
3) The Q35 exploit
4) The FLASK heap overflow exploit
5) The BluePillBoot (with nested virtualization support on SVM)
6) The XenBluePill (with nested virtualization support on SVM)

Beware the code is by far not user-friendly, it requires advanced Linux/Xen, C and system-level programming skills in order to tweak some constants and run it successfully on your system. Do not send us questions how to compile/run it, as we don’t have time to answer such questions. Also do not send questions how the code works – if you can’t figure it out by reading our slides and the source code, then it means you should probably spend more time on this yourself. On the other hand, we would appreciate any constructive feedback.

The code is our gift to the research community. There is no warranty and Invisible Things Lab takes no responsibility for any potential damage that this code might cause (e.g. by rebooting your machine) or any potential malicious usage of this code, or any other code built on top of this code. We believe that by publishing this code we help to create more secure systems in the future.

Additionally, we also posted the full version of our second Black Hat talk, which now includes all the slides about the Q35 bug and how we exploited it. Those slides had to be previously removed during our Black Hat presentation, as the patch was still unavailable during that time.

The three approaches to computer security

2008-09-02T13:39:00.002+02:00

If we looked at the computer systems and how they try to provide security, I think we could categorize those attempts into three broad categories:

1) Security by Correctness
2) Security by Isolation
3) Security by Obscurity

Let's discuss those categories in more detail below.

Security by Correctness

The assumption here is obvious: if we can produce software that doesn't have bugs (nor any maliciously behaving code), then we don't have security problems at all. The only problem is that we don't have any tools to make sure that a given code is correct (in terms of implementation, design and ethical behavior). But if we look at various efforts in computer science, we will notice a lot of effort has been made to achieve Security by Correctness: "safe" languages, code verifiers (although not sound ones, just heuristic based), developer's education, manual code audit, etc. Microsoft's famed Secure Development Life-cycle is all about Security by Correctness. The only problem is: all those approaches sometimes work and sometimes do not, sometimes they miss some bug and also there are problems that I simple don't believe can be addresses by automatic code verifiers or even safe languages, like e.g. logic/design bugs or deciding on wheatear a given code behaves maliciously or not (after all this is an ethical problem in many cases, not a computer science problem).

To sum it: I think that in some more or less distant future (some people think abuout a timeframe of 50 years or so), we would get rid of all the implementation bugs, thanks to safe languages and/or sound code verifiers. But I don't believe we could assure correctness of software on any higher level of abstraction then implementation level.

Security by Isolation

Because of the problems with effectively implementing Security by Correctness approach, people, from the very beginning, has also taken another approach, which is based on isolation. The idea is to split a computer system into smaller pieces and make sure that each piece is separated from the other ones, so that if it gets compromised/malfunctions, then it cannot affect the other entities in the system. Early UNIX's user accounts and separate process address spaces, things that are now present in every modern OS, are examples of Security by Isolation.

Simple as it sound, in practice the isolation approach turned out to be very tricky to implement. One problem is how to partition the system into meaningful pieces and how to set permissions for each piece. The other problem is implementation - e.g. if we take a contemporary consumer OS, like Vista, Linux or Mac OSX, all of them have monolithic kernels, meaning that a simple bug in any of the kernel components (think: hundreds of 3rd party drivers running there), allows to bypass of the isolation mechanisms provided by the kernel to the rest of the system (process separation, ACLs, etc).

Obviously the problem is because the kernels are monolithic. Why not implement Security by Isolation on a kernel level then? Well, I would personally love that approach, but the industry simply took another course and decided that monolithic kernels are better then micro-kernels, because it's easier to write the code for them and (arguably) they offer better performance.

Many believe, including myself, that this landscape can be changed by the virtualization technology. Thin bare-metal hypervisor, like e.g. Xen, can act like a micro kernel and enforce isolation between other components in the system - e.g. we can move drivers into a separate domain and isolate them from the rest of the system. But again there are challenges here on both the design- as well as the implementation-level. For example, we should not put all the drivers into the same domain, as this would provide little improvement in security. Also, how to make sure that the hypervisor itself is not buggy?

Security by Obscurity (or Security by Randomization)

Finally we have the Security by Obscurity approach that is based on the assumption that we cannot get rid of all the bugs (like in Security by Isolation approach), but at least we can make exploitation of those bugs very hard. So, it's all about making our system unfriendly to the attacker.

Examples of this approach include Address Space Layout Randomization (ASLR, present in all newer OSes, like Linux, Vista, OSX), StackGuard-like protections (again used by most contemporary OSes), pointer encryption (Windows and Linux) and probably some other mechanisms that I can't remember at the moment. Probably the most extreme example of Security by Obscurity would be to use a compiler that generates heavily obfuscated binaries from the source code and creates a unique (on a binary level) instances of the same system. Alex did his PhD on this topic and his an expert on compilers and obfuscators.

The obvious disadvantage of this approach is that it doesn't prevent the bugs from being exploited - it only make the meaningful exploitation very hard or even impossible. But if one is concerned also about e.g. DoS attacks, then Security by Obscurity will not prevent them in most cases. The other problem with obfuscating the code is the performance (compiler cannot optimize the code for speed) and maintenance (if we got a crash dump on an "obfuscated" Windows box, we couldn't count on help from the technical support). Finally there is a problem of proving that the whole scheme is correct and that our obfuscator (or e.g. ASLR engine) doesn't introduce bugs to the generated code and that we will not get random crashes later (that we would be most likely unable to debug, as the code will be obfuscated).

I wonder if the above categorization is complete and if I haven't forgotten about something. If you know an example of a security approach that doesn't fit here (besides blacklisiting), please let me know!