Reverse Engineering Malware (Part 1)

by Don Parker [Published on 18 Jan. 2007 / Last Updated on 18 Jan. 2007]

How to apply reverse engineering, the rapidly growing computer security field.

If you would like to read the other parts in this article series please go to:

To many of us the world of reverse engineering is a rather exotic one. Many people don’t even know how to go about doing it. In this article series we shall go over how to apply this rapidly growing computer security field.

Reverse engineering and you

I remember many years ago, well just a few years actually, when I first heard the term reverse engineering. The words meant nothing to me at that time. Once my knowledge of computer security began to expand from the networking protocols to other fields such as intrusion detection, and web application security, I decided to revisit reverse engineering. After having been active in computer security for several years I was able to understand why having the ability to reverse engineer an executable was of use. Furthermore, the skills necessary to undertake such a task were far clearer to me as well. Knowing that you want to do something is only half the battle. The other half is simply understanding how to approach the task that you have set for yourself.

Before we discuss what skills and knowledge are required to be able to practice reverse engineering, perhaps we should go over why it is important to you as a system administrator or security practitioner to know it. It really is inevitable that the computer network you work in will be subjected to a successful hack. After all, there are hundreds if not thousands of hackers, of varying skill levels, scanning and then attacking your network. It really is just a matter of when, and not if. Where an attacker only has to be lucky once, you on the other have to be at the top of your game every day.

We will then assume that an attacker has successfully breached your network. You have quickly identified the attack, and been able to contain it quickly, thereby minimizing the damage done. This is excellent as your IDS did the job it was supposed to, alert you to a possible attack, and you then swung into action. Hopefully with this attack has come a certain amount of curiosity. Was the executable used against your network fully contained? You did find the attack by verifying your IDS logs, and were then able to pull the affected computer offline. Nothing else in your system logs reveals anything further. That said, I would still be rather nervous as to what else may have happened once the attack successfully gained a toe-hold to the internal network. Whatever can you do?

The first of many steps

The first step that you will want to do after an attack is contain it, and then afterwards clean up the affected computer. After that I would try to identify what exploit it was that was used. Was it a 0 day exploit or were you simply lax in applying patches to a specific computer. If it is simply a case of unapplied patches then rebuild the computer and apply those patches! In either case what you should try to do is obtain a copy of the exploit in question, be it in source code form or compiled. Do I hear you saying “that is pretty much impossible to do!”. You would be correct in saying that, however, you can quite often find the exploit with some creative searches. There now exists a pretty neat way of looking for malware that we shall explore within the confines of this article series. It was created and put up by the very clever HDM of Metasploit fame.

With that piece of information in hand we shall go off and get ourselves a piece of malware to look at. At this point please understand that should you decide to download and study a piece of malware, that you do so at your own risk. I am in no way responsible for what may happen to your computer. That said, use standard practices when dealing with malware samples. Either work within a VMware image or use a dedicated computer which is separated from your network. This will allow you to study malware without fear of contaminating your home or corporate network.

Let's get some malware!

So with the above warning about downloading and using malware samples out of the way let's actually go and get ourselves a copy. Sadly, with FrSirt no longer hosting exploit code we need to find ourselves a different repository.  There are many other sites which host exploit code such as Securiteam for one. Dependant on your skill level you may not know how to compile source code into an actual PE executable. To be honest with you, if you do not know how to compile source code, then this article may be a bit advanced for you. Though on the other hand, I am a great proponent of learning things when one can.

So far this first part has been mostly contextual information about reverse engineering. There is an important reason for writing it as such. Reverse engineering is not an easy topic to come to grips with. Many readers will have varying skill levels as well. In order for you, the reader, to get the most out of this article series it is therefore very important to lay out the foundation of skills that one requires to do basic reverse engineering. Much as said above, if you do not know how to compile code, then you may be in for a bit of a learning curve here. Do your best to understand, and if need be, go back and learn the skills required.

What happened to downloading malware?!

Yes, yes, I remember saying that we were about to download some malware from the Metasploit website. Once again, I was sidetracked by trying to give you more background information that is necessary to this endeavor. To be able to do the much mentioned reverse engineering you need the following skills to varying degrees.

  1. First and foremost, you need to have some programming skills, or at the very least be able to understand and read source code. This will become crucial for when you use a disassembler such as Ollydbg or IDA Pro should you be able to afford it. The best of both worlds is the disassembler that I use in this article. It is very reasonably priced and quite functional.
  2. Have an understanding of the reverse engineering methodology. That would include both the static and dynamic portions.
  3. Know what tools to use during both parts of the static and dynamic phases.

Though the above three points may sound intimidating to some of you, believe me, they are not at all. All one has to do is gather the appropriate information, tools, and proper computer lab setup to begin reverse engineering in earnest. On that note I shall bring part one of this article series to a close. Though there was precious little actual hands on in this article, there will indeed be some in the following parts. It is very important to lay down the groundwork before getting into the guts of a project. Time spent on planning is rarely wasted. So, with that said, I look forward to seeing you in part two. Where we will definitely, I promise, download our malware sample and begin setting up for reverse engineering.

If you would like to read the other parts in this article series please go to:

See Also

Featured Links