Designing a minimal operating system to emulate 32/64bits x86 code snippets, shellcode or malware in Bochs

Reverse engineers almost invariably need to emulate some code while reverse engineering. Building an emulator from scratch can be time consuming and error prone. In this talk we present a few techniques on how to use an existing emulator (such as Bochs) and write a program that can construct a custom disk image containing a tailored operating system to run a specific piece of code or malware. This paper explains how we tackled the problem while covering the following aspects:

• disk structure: file system design and file format structure
• booting process: page table setup, GDT/IDT setup and exception dispatching
• the host: the role played by the host
• host/guest interaction: communication method used to exchange information between the host and the guest
• os environment: system structures and the memory layout of the main program and its dependencies
• API emulation: customization of APIs using scripts (Python, ...) or native code (asm or C code)
• debugging: debugging facilities using Bochs' built-in debugger
• use cases: malware and shellcode debugging