Virtualization (In)Security Training in Vegas

VM escapes, hypervisor compromises (via "classic" rootkits, as well as Bluepill-like rootkits), hypervisor protection strategies, SMM attacks, TXT bypassing, and more — these are some of the topics that will be covered by our brand new training on Virtualization (In)Security at the upcoming Black Hat USA.

The training offers quite a unique chance, I think, to absorb the results of 1+ year of the research done by our team within... just 2 days. This will be provided via detailed lectures and unique hands-on exercises.

Unlike our previous training on stealth malware (that will also be offered this year, BTW), this time we will offer attendees a bit of hope :) We will be stressing that some of the new hardware technologies (Intel TXT, VT, TPM), if used properly, have potential to dramatically increase security of our computer systems. Sure, we will be showing attacks against those technologies (e.g. TXT), but nevertheless we will be stressing that this is the proper way to go in the long run.

Interestingly, I'm not aware of any similar training of this kind, that would be covering the security issues related to virtualization systems and bare metal hypervisors. Hope we will not get into troubles with the Antitrust Commission for monopolizing this field ;)

The training brochure (something for your boss) is here.

The detailed agenda spanning 2 full days can be downloaded here.

The Black Hat signup page is here.

Quest to The Core

If you think SMM rootkits or PCI backdoors is low-level, then you should certainly see our talks in Vegas — ITL is going to define what does the "low-level" adjective really mean at the end of the decade ;)

In case you haven't noticed it at the Black Hat website yet — Alex and Rafal will be giving two presentations in Vegas:

1) Introducing Ring -3 Rootkits (description)

2) Attacking Intel® BIOS (description)

Let me stress that we have been in touch with Intel for quite some time about the above attacks, and that Intel is planning to release appropriate fixes a few weeks before our presentations at Black Hat.

There is more than just this coming at this year's Black Hat — most notably we will also be debuting with our Virtualization (In)Security Training. I will write a separate post about this training (containing a detailed agenda) in the coming days, so stay tuned.

Quite exciting.

More Thoughts on CPU backdoors

I've recently exchanged a few emails with Loic Duflot about CPU-based backdoors. It turned out that he recently wrote a paper about hypothetical CPU-backdoors and also implemented some proof-of-concept ones using QEMU (for he doesn't happen to own a private CPU production line). The paper can be bought here. (Loic is an academic, and so he must follow some of the strange customs in the academic world, one of them being that papers are not freely published, but rather being sold on a publisher website… Heck, even we, the ultimately commercialized researchers, still publish our papers and code for free).

Let me stress that what Loic writes about in the paper are only hypothetical backdoors, i.e. no actual backdoors have been found on any real CPU (ever, AFAIK!). What he does is he considers how Intel or AMD could implement a backdoor, and then he simulate this process by using QEMU and implementing those backdoors inside QEMU.

Loic also focuses on local privilege escalation backdoors only. You should however not underestimate a good local privilege escalation — such things could be used to break out of any virtual machine, like VMWare, or potentially even out of a software VMs like e.g. Java VM.

The backdoors Loic considers are somewhat similar in principle to the simple pseudo-code one-liner backdoor I used in my previous post about hardware backdoors, only more complicated in the actual implementation, as he took care about a few important details, that I naturally didn't concern. (BTW, the main message of my previous post about was how cool technology this VT-d is, being able to prevent PCI-based backdoors, and not about how doomed we are because of Intel- or AMD-induced potential backdoors).

Some people believe that processor backdoors do not exist in reality, because if they did, the competing CPU makers would be able to find them in each others' products, and later would likely cause a "leak" to the public about such backdoors (think: Black PR). Here people make an assumption that AMD or Intel is technically capable of reversing each others processors, which seems to be a natural consequence of them being able to produce them.

I don't think I fully agree with such an assumption though. Just the fact that you are capable of designing and producing a CPU, doesn't mean you can also reverse engineer it. Just the fact that Adobe can write a few hundred megabyte application, doesn't mean they are automatically capable of also reverse engineering similar applications of that size. Even if we assumed that it is technically feasible to use some electron microscope to scan and map all the electronic elements from the processor, there is still a problem of interpreting of how all those hundreds of millions of transistors actually work.

Anyway, a few more thoughts about properties of a hypothetical backdoors that Intel or AMD might use (be using).

First, I think that in such a backdoor scenario everything besides the "trigger" would be encrypted. The trigger is something that you must execute first, in order to activate the backdoor (e.g. the CMP instruction with particular, i.e. magic, values of some registers, say EAX, EBX, ECX, EDX). Only then the backdoor gets activated and e.g. the processor auto-magically escalates into Ring 0. Loic considers this in more detail in his paper. So, my point is that all the attacker's code that executes afterwards, think of it as of a shellcode for the backdoor, that is specific for the OS, is fetched by the processor in an encrypted form and decrypted only internally inside the CPU. That should be trivial to implement, while at the same time should complicate any potential forensic analysis afterwards — it would be highly non-trivial to understand what the backdoor actually have done.

Another crucial thing for a processor backdoor, I think, should be some sort of an anti-reply attack protection. Normally, if a smart admin had been recording all the network traffic, and also all the executables that ever got executed on the host, chances are that he or she would catch the triggering code and the shellcode (which might be encrypted, but still). So, no matter how subtle the trigger is, it is still quite possible that a curious admin will eventually find out that some tetris.exe somehow managed to breakout of a hardware VM and did something strange, e.g. installed a rootkit in a hypervisor (or some Java code somehow was able to send over all our DOCX files from our home directory).

Eventually the curious admin will find out that strange CPU instruction (the trigger) after which all the strange things had happened. Now, if the admin was able to take this code and replicate it, post it to Daily Dave, then, assuming his message would pass through the Moderator (Hi Dave), he would effectively compromise the processor vendor's reputation.

An anti-replay mechanism could ideally be some sort of a challenge-response protocol used in a trigger. So, instead having you always to put 0xdeadbeaf, 0xbabecafe, and 0x41414141 into EAX, EBX and EDX and execute some magic instruction (say CMP), you would have to put a magic that is a result of some crypto operation, taking current date and magic key as input:

Magic = MAGIC (Date, IntelSecretKey).

The obvious problem is how the processor can obtain current date? It would have to talk to the south-bridge at best, which is 1) nontrivial, and 2) observable on a bus, and 3) spoof'able.

A much better idea would be to equip a processor with some sort of an eeprom memory, say big enough to hold one 64-bit or maybe 128-bit value. Each processor would get a different value flashed there when leaving the factory. Now, in order to trigger the backdoor, the processor vendor (or backdoor operator, think: NSA) would have to do the following:

1) First execute some code that would read this unique value stored in eeprom for the particular target processor, and send this back to them,

2) Now, they could generate the actual magic for the trigger:

Magic = MAGIC (UniqeValueInEeprom, IntelSecretKey)

3) ...and send the actual code to execute the backdoor and shellcode, with the correct trigger embedded, based on the magic value.

Now, the point is that the processor will automatically increment the unique number stored in the eeprom, so the same backdoor-exploiting code would not work twice for the same processor (while at the same time it would be easy for NSA to send another exploit, as they know what the next value in the eeprom should be). Also, such a customized exploit would not work on any other CPU, as the assumption was that each CPU gets a different value at the factory, so again it would not be possible to replicate the attack and proved that the particular code has ever done something wrong.

So, the moment I learn that processors have built-in eeprom memory, I will start thinking seriously there are backdoors out there :)

One thing that bothers me with all those divagations about hypothetical backdoors in processors is that I find them pretty useless in at the end of the day. After all, by talking about those backdoors, and how they might be created, we do not make it any easier to protect against them, as there simply is no possible defense here. Also this doesn't make it any easier for us to build such backdoors (if we wanted to become the bad guys for a change). It might only be of an interest to Intel or AMD, or whatever else processor maker, but I somewhat feel they have already spent much more time thinking about it, and chances are they probably can only laugh at what we are saying here, seeing how unsophisticated our proposed backdoors are. So, my Dear Reader, I think you've been just wasting time reading this post ;) Sorry for tricking you into this and I hope to write something more practical next time :)

Thoughts About Trusted Computing

Here are the slides about Trusted Computing I used for my presentations at the EuSecWest today, and at the Confidence conference last week.

As this was supposed to be a keynote, the slides are much less technical then our other slides, and also there are no new attacks presented there. Still, I hope they might be useful as some sort of an "alternative" introduction to Trusted Computing :)

A cool presentation I saw today was about PCI-based backdoors by Christophe Devine and Guillaume Vissian. They basically took a general-purpose FPGA programmable PC-card (AKA PCMCIA), flashed it with an FPGA "program" that implemented a simple state machine. The purpose of the state machine was to wait until its DMA engine gets initialized and then to modify certain bytes in the host memory, that happened to be part of the winlogon.exe process (IIRC they changed XOR AL, AL into MOV AL, 1, or something like that, at the end of some password verification function inside the winlogon.exe process). The slides should be available soon on the conference website. I also hope they will publish all the source code needed to flash your own personal "winlogon unlocker".

The live demo was really impressive — they showed a winlogon screen, tried to login a few times with wrong passwords, of course all the attempts failed, then they inserted their magic, $300 worth, PC-card, and… 2 seconds later they could log in using any password they wanted.

While not necessary being a breakthrough, as everybody has known such things could be done for years, I think it is still important that somebody eventually implemented this, discussed the technical details (FPGA-related), and also showed how to implement it with a cheap generic "reflashable" hardware without using a soldering iron.

Of course I have also discussed in my presentation how to prevent PCI-based backdoors (like the one discussed here) using VT-d, but this defense is currently only available if you use Xen 3.3 or later, and also requires that you manually create driver domain partitions and come up with a reasonable scheme for assigning devices to driver domains. All in all 99.9% of users are not (and will not be anytime soon) protected against such attacks. Oh, wait, there is actually a relatively simple software-based workaround (besides putting a glue into your PC-card slot, which is not a very subtle one)… I wonder who else will find out :)

Trusting Hardware

So, you're a decent paranoid person, running only open source software on your box: Linux, GNU, etc. You have the feeling you could, if you only wanted to, review every single line of code (of course you will probably never do this, but anyway). You might be even more paranoid and also try running an open source BIOS. You feel satisfied and cannot understand all those stupid people running closed source systems like e.g. Windows. Right?

But here's where you are stuck — you still must trust your hardware. Trust that your hardware vendor has not e.g. built in a backdoor into your network card micro-controller…

So, if we buy a laptop from vendor X, that might be based in some not-fully-democratic country, how do we know they didn't put backdoors there? And not only to spy on Americans, also to spy on their own citizens? When was the last time you reverse-engineered all the PCI devices on your motherboard?

Scared? Good!

Enters the game-changer: IOMMU (known as VT-d on Intel). With proper OS/VMM design, this technology can address the very problem of most of the hardware backdoors. A good example of a practical system that allows for that is Xen 3.3, which supports VT-d and allows you to move drivers into a separate, unprivileged driver domain(s). This way each PCI device can be limited to DMA only to the memory region occupied by its own driver.

The network card's microcontroller can still compromise the network card driver, but nothing else. Assuming we are using only encrypted communication, there is not much an attacker can gain by compromising this network card driver, besides doing a DoS. Similarly for the disk driver — if we use full disk encryption (which is a good idea anyway), there is not much an attacker can gain from compromising the low-level disk driver.

Obviously the design of such a system (especially used for desktop computing) is not trivial ans needs to be thoroughly thought out. But it is possible today(!), thanks to those new virtualization technologies.

It seems than, that we could protect ourselves against potentially malicious hardware. With one exception however… we still need to trust the CPU and also the memory controller (AKA northbridge AKA chipset), that implements that IOMMU.

On AMD systems, the memory controller has long been integrated into the processor. Also Intel's recent Nehalem processors integrate the memory controller on the same die.

This all means we need to trust only one vendor (Intel or AMD) and only one component, i.e. The Processor. But should we blindly trust them? After all it would be trivial for Intel or AMD to build in a backdoor into their processor. Even something as simple as:

if (rax == MAGIC_1 && rcx == MAGIC_2) jmp [rbx]

Just a few more gates in the CPU I guess (there are apparently already about 780 million gates on Core i7, so a few more should not make much difference), and no performance penalty. Exploitable remotely on most systems and any more complex program I guess. Yet, totally undetectable for anybody without an electron microscope (and tons of skills and knowledge).

And this is just the simplest example that comes to mind within just a few minutes. I'm sure one could come up with something even more universal and reliable. The fact is — if you are the CPU vendor, it is trivial for you to build in an effective backdoor.

It's funny how various people, e.g. European government institutions, are afraid of using closed source software, e.g. Windows, because they are afraid of Microsoft putting backdoors there. Yet, they are not concerned about using processors made by some other US companies. It is significantly more risky for Microsoft to put a backdoor into its software, where even a skilled teenager equipped with IDA Pro can find it, than it is for Intel or AMD, where effectively nobody can find it.

So, I wonder whether various government and large corporate customers from outside the US will start asking Intel and AMD to provide them with the exact blueprints of their processors. After all they already require Microsoft to provide them with the source code under an NDA, right? So, why not the "source code" for the processor?

Unfortunately there is nothing that could stop a processor vendor to provide its customers with a different blueprints than those that are used to actually "burn" the processors. So, the additional requirement would be needed that they also allow to audit their manufacturing process. Another solution would be to hire some group of independent researchers, equip them with an electron microscope and let them reverse engineer some randomly chosen processors… Hmmm, I even know a team that would love to do that ;)

A quick summary in case you get lost already:

1. On most systems we are not protected against hardware backdoors, e.g. in the network card controller.
2. New technologies, e.g. Intel VT-d, can allow to protect against potentially malicious hardware (requires specially designed OS, e.g. specially configured Xen)…
3. … except for the potential backdoors in the processor.
4. If we don't trust Microsoft, why should we trust Intel or AMD?

BTW, in May I will be speaking at the Confidence conference in Krakow, Poland. This is gonna be a keynote, so don't expect new attacks to be revealed, but rather some more philosophical stuff about trusted computing (why it is not evil) and problems like the one discussed today. See you there!

The Sky Is Falling?

A few reporters asked me if our recent paper on SMM attacking via CPU cache poisoning means the sky is really falling now?

Interestingly, not many people seem to have noticed that this is the 3rd attack against SMM our team has found in the last 10 months. OMG :o

But anyway, does the fact we can easily compromise the SMM today, and write SMM-based malware, does that mean the sky is falling for the average computer user?

No! The sky has actually fallen many years ago… Default users with admin privileges, monolithic kernels everywhere, most software unsigned and downloadable over plaintext HTTP — these are the main reasons we cannot trust our systems today. And those pathetic attempts to fix it, e.g. via restricting admin users on Vista, but still requiring full admin rights to install any piece of stupid software. Or selling people illusion of security via A/V programs, that cannot even protect themselves properly…

It's also funny how so many people focus on solving the security problems by "Security by Correctness" or "Security by Obscurity" approaches — patches, patches, NX and ASLR — all good, but it is not gonna work as an ultimate protection (if it could, it would worked out already).

On the other hand, there are some emerging technologies out there that could allow us to implement effective "Security by Isolation" approach. Such technologies as VT-x/AMD-V, VT-d/IOMMU or Intel TXT and TPM.

So we, at ITL, focus on analyzing those new technologies, even though almost nobody uses them today. Because those technologies could actually make the difference. Unlike A/V programs or Patch Tuesdays, those technologies can change the level of sophistication required for the attacker dramatically.

The attacks we focus on are important for those new technologies — e.g. today Intel TXT is pretty much useless without protection from SMM attacks. And currently there is no such protection, which sucks. SMM rootkits sound sexy, but, frankly, the bad guys are doing just fine using traditional kernel mode malware (due to the fact that A/V is not effective). Of course, SMM rootkits are just yet another annoyance for the traditional A/V programs, which is good, but they might not be the most important consequence of SMM attacks.

So, should the average Joe Dow care about our SMM attacks? Absolutely not!

Attacking SMM Memory via Intel® CPU Cache Poisoning

As promised, the paper and the proof of concept code has just been posted on the ITL website here.

A quote from the paper:

In this paper we have described practical exploitation of the CPU cache poisoning in order to read or write into (otherwise protected) SMRAM memory. We have implemented two working exploits: one for dumping the content of SMRAM and the other one for arbitrary code execution in SMRAM. This is the third attack on SMM memory our team has found within the last 10 months, affecting Intel-based systems. It seems that current state of firmware security, even in case of such reputable vendors as Intel, is quite unsatisfying.

The potential consequence of attacks on SMM might include SMM rootkits [9], hypervisor compromises [8], or OS kernel protection bypassing [2].

Don't worry, the shellcode we use in the exploit is totally harmless (have really no idea how some people concluded we were going to release an SMM rootkit today?) — it only increases an internal counter on every SMI and jumps back to the original handler. If you want something more fancy, AKA SMM rootkits, you might want to re-read Sherri's and Shawn's last year's Black Hat paper and try writing something they describe there.

The attack presented in the paper has been fixed on some systems according to Intel. We have however found out that even the relatively new boards, like e.g. Intel DQ35 are still vulnerable (the very recent Intel DQ45 doesn't seem to be vulnerable though). The exploit attached is for DQ35 board — the offsets would have to be changed to work on other boards (please do not ask how to do this).

Keep in mind this is a different SMM attack than the one we mentioned during our last month's Black Hat presentation on TXT bypassing (the VU#127284). We are planning to present that other attack at the upcoming Black Hat Vegas. Hopefully this will not be the only one thing that ITL will entertain you with in Vegas — Alex and Rafal are already working now on something even cooler (and even lower level) for the show, so cross your fingers!

And good luck to Loic with his presentation that is about to start just now...