Vegas Training 2008

Last year we debuted with our Understanding Stealth Malware training at the Black Hat Vegas. We had about 70 participants and I think it was a reasonable success, especially that the training was announced very late. Since then we have done a couple of on-site classes and also have been continually updating the training.

During our 2nd public edition, at Black Hat Europe 2008 in March this year, we significantly extended the part about virtualization, e.g. by adding discussion of nested virtualization on AMD-v and showing and analyzing the actual code for implementing this. Also we have used the New Blue Pill code with VT-x support (previously it worked only on AMD-V), making it possible to use both AMD and Intel machines for the class. This allowed us to offer this training in a "Bring Your Own Laptop" fashion, that we know is much preferred by attendees, who simply feel better when using their own, known, work environment.

At the upcoming Black Hat Vegas 2008 we are also going to offer this class. That would be our 3rd public edition. Again, we hope to improve it even more beyond what we have presented at BH Europe 2008. Similarly as last time, we will not provide the computers, but rather expect the attendees to bring their own systems. At the end of this article are the requirements that should be met by your machine, if you would like to use it during the training and be able to do all the exercises. Of course, you should back up all your important data before coming to the class, as the computer might become corrupt after doing some of the exercises (although this has never happened so far).

There will be only one class offered on August 4/5 (the weekday class). You can view the detailed training agenda that we used for the BH Europe class in March here. Please note that the exact shape of the Vegas class is subject to be a bit different, as we are planning to add new material again.

This might be the very last chance for you to attended this specific training, as it's quite possible that next year we will be offering some other class, focused on Virtualization security entirely. Don't worry, however, if you don't get a seat in the Vegas class, there is still a chance to have that class presented on-site in your town.

You can register for the Vegas training here.

See you in Vegas!

Hardware Requirements

1. 64-bit (x64) AMD or Intel processor with hardware virtualization support (AMD-v or VT-x)
2. DVD-ROM
3. 2GB RAM (for convenient work with VMWare)

Software Requirement

1. 64-bit Vista OS (primary OS, non virtualized)
2. Windows Driver Kit (WDK) 6000 or newer (available via MSDN subscription).
3. VMWare Workstation 6.x or VMWare Player 2.x (the latter is free)
4. Optionally: IDA Pro 5.x disassembler (for exercises that involve finding bugs in drivers)

AMD Processors
Most modern AMD mobile processors, like e.g. AMD Turion and Athlon, used in modern laptops support AMD-v technology. Unfortunately there is no single place on AMD website that would provide the complete description of all CPUs that support AMD-v technology or provide an answer whether a given model does support it. When in doubt use google and always verify with the CHKSVMX program described below.

Intel Processors
Most modern Intel processors used in notebooks support Intel VT-x virtualization technology, this include Core 2 Solo, Core 2 Duo (except T5500, T5550 and T5750 models) and Core 2 Extreme. You can check your own model starting at this website, then chose your processor family and chose "Specifications" tab. Make sure the processor supports "Intel® 64 architecture" and "Intel® Virtualization Technology".

Using Mac for the training
You can very easily use MacBook or MacBook Pro for this training. You can easily install Windows on a second partition using the Boot Camp program that ships with all the newer Macs. You simply start Boot Camp application when running Mac OS X and then it automatically shrinks your current Mac partition, creates a new one for Windows, and asks to insert the installation media and reboots the system and you then perform normal Windows setup (after installation is complete your Vista should find all the necessary drivers via Windows Update). You might also want to use the free AutoHotKey program for the right-click emulation on your newly installed Vista. Please don't worry that Boot Camp tells that you should install a 32-bit Vista - you can ignore this and insert a 64-bit Vista installation disk.

Testing your machine with CHKSVMX
We have prepared a special little program, CHKSVMX, to test whether a given machine indeed supports hardware virtualization technology. The CHKSVMX program can be downloaded from here

The program doesn't introduce any persistent changes to the OS and doesn't require any installation procedure. It checks for virtualization support (on both AMD and Intel processors) not only by reading the CPUID information but also by trying to actually enable virtualization mode and then disable it again. Although most of the laptops available these days support hardware virtualization, in many cases this feature is disabled or locked down in the BIOS. If the virtualization is reported as "locked", please try to enable it in the BIOS. Please note that in most cases you will have to fully power down your system for the BIOS changes to take effect (reboot is not enough)!

Additionally CHKSVMX checks whether a 64-bit edition of Windows is running, as such OS is required for the training.

DISCLAIMER: The test program is digitally signed with the Invisible Things Lab's certificate and we assure that the program does not perform any malicious actions. ITL is, however, not responsible for any accidental damage or system instability issues the test program might cause.

Research Obfuscated

Update 07-Sept-2008: Four months later after writing his open letter to me (see below), Christofer Hoff experienced on his own difficult it is for one to control the press, making sure it correctly reports what you say. In this blog entry he describes how he was terribly misquoted by a report after his Black Hat presentation and he also explicitly admits that "[I] was essentially correct in [my] assertion during our last debate that you cannot control the press, despite best efforts." and that "[he] humbly submit[s] to [me] on that point." :)

This article has been brought to my attention recently. It’s an “Open Letter to Joanna Rutkowska”, by Christofer Hoff over at the “Rational Survivability” blog. I decided to spend time reading and answering this piece as 1) technorati.com reported the blog’s authority as above 100 which suggests it has a reasonable number of readers, and also 2) because I believe this is a good example of the social engineering techniques used by my opponents and I couldn’t refrain myself from not commenting about this. Besides I felt a bit flattered that some individual decided to write an “Open Letter” to me, sort of like if I was a prime minister or some other important person ;)

Let me now analyze the letter, point by point:

1. Fire rules! The first thing that Hoff accuses me of in his letter is myself being an irresponsible individual, not caring about safety of my audience (not a joke!):
   

   “As the room filled to over capacity before your talk began, you were upset and couldn't seem to understand why the conference organizers would not let people spill over from seats and sit on the floor and in the aisles to hear you speak. The fact that fire and safety codes prohibit packing a room beyond capacity was something you attributed to people being "...crazy in America." Go figure.”

   Dear Christofer, if you only read my recent blog post about this very specific incident, read thoroughly shall I say, you would notice this paragraph undoubtedly:
   

   “Interestingly it was perfectly ok for the additional people to stay in the room, provided they arranged for additional chairs for themselves. In other words it was fine for people to sit and block the main aisle, provided they sit on chairs, but they couldn’t stay and sit on the same aisle without having a chair (maybe a "certificated" chair also), as that would be against the fire regulations!”

   Conclusion: I was not so much picking upon the fire regulations that forced people to leave the room, but rather on the idiotic rule, that allowed those same people to stay in this very same room, provided they also had additional chairs with them.
   


2. Type I vs. Type II hypervisors confusion. Hoff then switches to the actual content of the presentation and writes this:
   

   “When I spoke to you at the end of your presentation and made sure that I understood correctly that you were referring specifically to type-2 hosted virtualization on specific Intel and AMD chipsets, you conceded that this was the case.”

   This simply is an incorrect statement! On the contrary, when describing the security implications of nested virtualization (which was the actual new thing I was presenting at the RSA), I explicitly gave an example of how this could be used to compromise type I hypervisors. Kindly refer to slides 85-90 of my presentation that can be downloaded here.
   
   I said that the code we posted on bluepillproject.org indeed targets type II hypervisors and the only reason for that being that it has been built on top of our New Blue Pill code that was designed as a Windows kernel driver.


3. Shit not giving. Mr. Hoff goes even further:
   

   “When I attempted to suggest that while really interesting and intriguing, your presentation was not only confusing to many people but also excluded somewhere north of 80% of how most adopters have deployed virtualization (type-1 "bare-metal" vs. type-2 hosted) as well as excluding the market-leading virtualization platform, your response (and I quote directly) was: I don't give a shit, I'm a researcher.”

   Now that was a hard blow! I understand that the usage of such a slang expression by an Eastern European female during an informal conversation with a native speaker must have made an impression on him! However, I couldn’t give such an answer to this very question, simply because of the reasons given in point #2 (see above).
   
   If I remember correctly, I indeed used this very American expression to answer somebody’s concern (undoubtedly our Christofer Hoff’s) that most of the type I hypervisors out there are based on monolithic hypervisor architecture, and not on the micro-hypervisor architecture (and that I should not try to convince people to switch to micro-hypervisor architecture). In that context it makes it more logical for me to use the “I’m a researcher” as an excuse for not caring so much that most people use monolithic based hypervisors. Obviously, the usage of micro-hypervisors would allow to better secure the whole VMM infrastructure. And I also said, that I don’t care what people are using today, because I try to help to build a product that would be secure in the future (Phoenix’s HyperCore).


4. No obfuscation postulate. Hoff then comes up with some postulates that:
   

   “[I], as a researcher who is also actively courting publicity for commercial gain and speaking at conferences like RSA which are less technical and more "executive" in nature, you have a responsibility to clarify and not obfuscate (intentionally or otherwise) the facts surrounding your research.”

   This postulate is cleverly constructed because it also contains an embedded accusation of me being a commercially motivated researcher. Well, I never tried to hide that fact, and the reason for this is very simple: I consider security research as my job, and one of the primary goals of any job is to… bring commercial gain to the individual doing the job.
   
   Second, I really don’t understand what Hoff means by asking me to not obfuscate my research?! Maybe he was just disappointed that the presentation was too technical for an average CISSP to understand it? But, well, this presentation was classified as “Advanced Technical”, which was displayed in the conference program. I still did my best so that, say 70% of the material, was understandable to an average IT people, but, come on, there always must be some deep technical meat in any non-keynote-presentation, at least this is my idea for how a conference should look like.


5. Commercially motivated. Hoff accuses me of presenting commercial product, i.e. the Phoenix’s HyperSpace, during my speech:
   

   “No less than five times during your presentation, you highlighted marketing material in the form of graphics from Phoenix, positioned their upcoming products and announced/credited both Phoenix and AMD as funding your research.”

   Well, let me tell you this – this was one of the main reasons why I decided to speak at the RSA – just to announce this very product that I try to help to secure. Why would that be wrong?
   
   BTW, I have no idea how Mr. Hoff concluded that AMD was founding my research. I never said that, nor did I have it in my slides. Needles to say, AMD has not been founding my research. NOTE: interestingly I consider this particular mistake by Hoff to be accidental – at least I don’t see how this could be connected to any PR campaign, in contrast to all the other incorrectness he made use of.


6. Independence. Hoff, for some reason, apparently known only to him, tries to argue that I’m not an “independent researcher”:
   

   “I think it's only fair to point out that given your performance, you're not only an "independent researcher" but more so an "independent contractor." Using the "I'm a researcher" excuse doesn't cut it.”

   “I know it's subtle and lots of folks are funded by third parties, but they also do a much better job of drawing the line than you do.”

   Well, I found this one to be particularly amusing, as, for at least several years now, I have not claimed I have been an independent researcher.


7. Final hit. You might have been wondering by now – why this gentleman, nah, I think “the guy” would fit better here, so why the guy decided to spent so much time to write all those points, all those quasi-arguments and why he made so many “mistakes”? Well he seems to give an answer right in this paragraph:
   

   “I care very much that your research as presented to the press and at conferences like RSA isn't only built to be understood by highly skilled technicians or researchers because the continued thrashing that they generate without recourse is doing more harm than good, quite frankly.”

   Aha, now all is clear. May I ask then, which virtualization vendor you write PR for? ;)

So, then Hoff quotes the Forbes article that was written after my presentation and accuses me that the article (written by some Forbes reporter) was too sensationalist. I definitely agree the article was very sensationalist (but correct) and when I saw the article I even got angry and even wanted to write a blog about it (but as the article was actually correct, I had no good arguments to use against it). And you know why I was so angry? Because I actually spent over 40 minutes with this very Forbes reporter in the RSA’s speaker’s lounge just after my speech, I spent that time on clarifying to that guy what my presentation was about and what it was not about and what was the main message of the presentation. Still, the reporter had his own vision of how to write about it (i.e. make it into a sensation) and I hardly, as it turned out, could do anything about it…

So, what was the main massage of my presentation? Interestingly Mr. Hoff forgot to mention that… Let me then remind it here (a curious reader might want to have a look at the the slide #96 in my presentation):

• Virtualization technology could be used to improve security on desktop systems


• However there are non-trivial challenges in making this all working well...


• ... and not to introduce security problems instead...

Additionally, the message I was trying to pass during the whole presentation was:

“Keep hypervisors simple, do not put drivers there, as otherwise we would get to the same point where we are with current OSes these days, i.e. no kernel security at all!”

Now I wonder, maybe Christofer Hoff doesn’t do PR for any VMM vendor, maybe he just didn’t listen carefully to my presentation. Maybe he’s just one of those many guys who always know in advance what they want to hear and selectively pick up only those facts that match their state of mind? Otherwise, why would he not realize that my presentation was actually a pro-virtualization one and needed no (false) counter-arguments?

The Most Stupid Security News Ever

Seems like the BBC reporters have a shortage of subjects to write about these days… Maybe the next winter we will also be able to read about how many snowflakes fell during Christmas all over the world or something like that (which BTW, would still be way more interesting that the news quoted above).

I remember that some time ago, a group of researchers used automatic generators to create a few tens of thousands of variants of some malware, just to do some testing of A/V engines. And I remember how all the A/V people were complaining how irresponsible that was bla bla bla, as now they would have to work after hours to fight all this new malware. What a BS!

For any given class of a bug (think: exploits), or a file infection method (think: viruses), or a system compromise technique (think: rootkits, stealth malware), one can come up with pretty much infinite number of examples that would be exploiting the specific bug, the specific file infection method, or the specific system compromise technique. One virus would display you a “Hello, you’re being 0wned, sir.” Message, while the other one would just flash your keyboard leds. Sure, two different beings, but if exploiting the same mechanisms, also the protection against them is the same.

But, I know, it looks so cool in the news to read: “The number of viruses, worms and trojans in circulation has topped the one million mark”. It’s most definitely a good way to scare all the housewives and make them to rush to the computer shop at the coroner to buy the brand new A/V product that already can detect 99.9% out of all those scary things out there.

The RSA Absurd

Today I was giving a speech at the RSA Conference in San Francisco. The RSA is a really big conference and also seems to me like a very well organized one – e.g. they have all those computers at the registration hall where you put your name and then it immediately says to which check-in counter you should proceed and then when you get there they already have a badge waiting for you. Pretty cool stuff.

So my speech turned out to be scheduled in a very small room, say with seats for 100-200 people only (I haven't counted exactly). But then it turned out that there are more people interested in seeing the speech, so, as it usually happens on conferences, people started seating on the floor and also standing at the back of the room. I would say there was about 30% overflow, but still they could fit ok in the room. And then came this guy from the conference and said that all people who don’t have a seat should leave the room! It turned out that this is a fire regulation.

Interestingly it was perfectly ok for the additional people to stay in the room, provided they arranged for additional chairs for themselves. In other words it was fine for people to sit and block the main aisle, provided they sit on chairs, but they couldn’t stay and sit on the same aisle without having a chair (maybe a "certificated" chair also), as that would be against the fire regulations!

Yes, I know there are more examples of stupid pseudo-security rules (think airports), but, come on, this is on of the most well known security conference...

That situation annoyed me so much (because, of course, it turned out to be impossible to arrange for the additional chairs, so all those people had to leave) that I decided to submit this story to my blog using the totally unsecured public WiFi in my hotel. It was really unwise for me to do that, as Google’s Blogger uses HTTPS only for authentication (i.e. the login screen) but then it switches back to the good old plain text HTTP, making it possible for some evil guy sitting in the lobby to hijack my session. Is it that I miss something here or Google simple forgot that it is 2008 and not the 90’s anymore? Anyway, I'm just taking this risk bravely, hoping that the potential attacker, seeing my determination here, would refrain themselves from compromising this blog.

I know, I know, instead of complaining about Google, I should just move my blog to some other place. One day that’s gonna happen for sure :)