doc/jitter/jitter.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Miasm Emulation Engine\n",
    "\n",
    "This short document provides an introduction to the APIs of the Miasm emulation engine. This emulation engine is commonly referred to as **jitter** throughout the project's code (as it is based on [JiT](https://en.wikipedia.org/wiki/Just-in-time_compilation) methods)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first step is to obtain an instance of the jitter. To do this, we will go through an intermediate object, `Machine`. It will allow us to instantiate a set of elements linked to a given architecture supported by Miasm (x86, arm, mips, ...)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['arml',\n",
       " 'armb',\n",
       " 'armtl',\n",
       " 'armtb',\n",
       " 'sh4',\n",
       " 'x86_16',\n",
       " 'x86_32',\n",
       " 'x86_64',\n",
       " 'msp430',\n",
       " 'mips32b',\n",
       " 'mips32l',\n",
       " 'aarch64l',\n",
       " 'aarch64b',\n",
       " 'ppc32b',\n",
       " 'mepl',\n",
       " 'mepb']"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from miasm.analysis.machine import Machine\n",
    "Machine.available_machine()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x86_32\n"
     ]
    }
   ],
   "source": [
    "machine = Machine(\"x86_32\")\n",
    "print(machine.name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `Machine` class in Miasm provides various interfaces:\n",
    "\n",
    "* `jitter`: an emulation engine\n",
    "* `dis_engine`: a disassembly engine\n",
    "* `lifter`: a lifting engine, to lift assembly code to the corresponding Miasm internal representation (IR)\n",
    "* `lifter_model_call`: as `lifter`, but function calls are abstracted\n",
    "* `mn`: low level object to interact with an architecture (assembly, disassembly of only a few bytes, etc.)\n",
    "* `gdbserver`: a GDB remote debugging server (to link with a `jitter` instance)\n",
    "\n",
    "The IR related objects are already discussed in [this notebook](../ir/lift.ipynb)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To obtain an emulator instance, we first need to:\n",
    "\n",
    "1. instanciate a *symbol table* (`LocationDB` in Miasm, already explained in [this notebook](../locationdb/locationdb.ipynb)) ;\n",
    "1. (optionnaly) choose a backed for the emulation (more specifically, to JiT the code):\n",
    "\n",
    "    * \"python\": a backend entirely in Python, slow\n",
    "    * \"gcc\": a backend based on the compiler GCC (Assembly $\\rightarrow$ IR Miasm $\\rightarrow$ C $\\rightarrow$ GCC), fast\n",
    "    * \"llvm\": a backend based on the LLVM JiT engine (Assembly $\\rightarrow$ IR Miasm $\\rightarrow$ IR LLVM $\\rightarrow$ LLVM JiT), fast\n",
    "\n",
    "Note: using a JiT backend as `llvm` or `gcc` greatly improve the performance over the `python` one. As a result, without any argument, instanciating an emulator will automatically try to look for and choose the best suited backend. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<miasm.arch.x86.jit.jitter_x86_32 object at 0x7f195fe07d60>\n"
     ]
    }
   ],
   "source": [
    "from miasm.core.locationdb import LocationDB\n",
    "\n",
    "loc_db = LocationDB()\n",
    "jitter = machine.jitter(loc_db)\n",
    "print(jitter)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For now, our emulator is an empty box. It has:\n",
    "\n",
    "* registers, reachable from the `.cpu` attribute. These are initialized to 0.\n",
    "* a virtual memory, reachable from the `.vm` attribute. It starts empty."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### CPU state\n",
    "\n",
    "Let's manipulate a few registers:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n"
     ]
    }
   ],
   "source": [
    "# Read a register\n",
    "print(jitter.cpu.EAX)\n",
    "\n",
    "# Write a register\n",
    "jitter.cpu.EAX = 1\n",
    "print(jitter.cpu.EAX)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A few *helpers* are also available. For instance, one can get every registers using:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'RAX': 1, 'RBX': 0, 'RCX': 0, 'RDX': 0, 'RSI': 0, 'RDI': 0, 'RSP': 0, 'RBP': 0, 'R8': 0, 'R9': 0, 'R10': 0, 'R11': 0, 'R12': 0, 'R13': 0, 'R14': 0, 'R15': 0, 'RIP': 0, 'zf': 0, 'nf': 0, 'pf': 0, 'of': 0, 'cf': 0, 'af': 0, 'df': 0, 'ES': 0, 'CS': 0, 'SS': 0, 'DS': 0, 'FS': 0, 'GS': 0, 'MM0': 0, 'MM1': 0, 'MM2': 0, 'MM3': 0, 'MM4': 0, 'MM5': 0, 'MM6': 0, 'MM7': 0, 'XMM0': 0, 'XMM1': 0, 'XMM2': 0, 'XMM3': 0, 'XMM4': 0, 'XMM5': 0, 'XMM6': 0, 'XMM7': 0, 'XMM8': 0, 'XMM9': 0, 'XMM10': 0, 'XMM11': 0, 'XMM12': 0, 'XMM13': 0, 'XMM14': 0, 'XMM15': 0, 'tsc': 1234605616436508552}\n"
     ]
    }
   ],
   "source": [
    "# GPReg : General Purpose registers\n",
    "regs = jitter.cpu.get_gpreg()\n",
    "print(regs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note: in Miasm, `x86_32` implementation is merged with the `x86_64` one. As a result, the register names are the x64 ones. Still, the computation is made accordingly to the 32bits implementation, and registers can be accessed either using the x64 name or the 32bits name (ie `.EAX` and `.RAX`)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Virtual memory\n",
    "\n",
    "Initially, the virtual memory is empty. The *repr* output offers a summary of the availables pages: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Addr               Size               Access Comment"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "jitter.vm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Memory pages can then be added.\n",
    "The **memory address is arbitrary** (no alignment requirement). **The size is also arbitrary** (byte precision).\n",
    "\n",
    "Optionnaly, a comment associated to the page can be provided."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Addr               Size               Access Comment\n",
       "0x1000             0x1000             RWX    test page\n",
       "0x112233           0x666              RWX    no alignment, byte precision"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from miasm.jitter.csts import PAGE_READ, PAGE_WRITE, PAGE_EXEC\n",
    "jitter.vm.add_memory_page(0x1000, PAGE_READ | PAGE_WRITE | PAGE_EXEC, b\"\\x00\" * 0x1000, \"test page\")\n",
    "jitter.vm.add_memory_page(0x112233, PAGE_READ | PAGE_WRITE | PAGE_EXEC, b\"\\x00\" * 0x666, \"no alignment, byte precision\")\n",
    "jitter.vm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The memory can now be accessed, read and write:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "b'\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00'"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "jitter.vm.get_mem(0x1000, 0x10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "b'toto\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00'"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "jitter.vm.set_mem(0x1000, b\"toto\")\n",
    "jitter.vm.get_mem(0x1000, 0x10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note: having a byte-precision memory has some advantages. For instance:\n",
    "* when mapping a partially known structure, instead of defaulting unknown fields to 0, we can instead only map in memory the known field. As a result, we obtain a sparse memory layout and if the programm try to read an unknown field, the execution will stop right on the responsible instruction, helping the reversing work\n",
    "* allocation can be made byte-wise. Thus, it could be easier to detect overflows and underflows"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Emulation\n",
    "\n",
    "We will now run the emulator.\n",
    "First, actual instructions are needed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "jitter.vm.set_mem(0x1000, bytes.fromhex(\"B844332211C3\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will also enable debug logging.\n",
    "By default, the logger will logs executed instruction and register values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "jitter.set_trace_log()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "The emulation is initialized with `jitter.init_run(address)` and resumed with `jitter.continue_run()`.\n",
    "For convenience, `jitter.run(address)` is usually used, wrapping these two API."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[WARNING ]: [Errno cannot get mem ad] 0x1337beef\n",
      "WARNING: address 0x1337BEEF is not mapped in virtual memory:\n",
      "[WARNING ]: cannot disasm at 1337BEEF\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "00001000 MOV        EAX, 0x11223344\n",
      "EAX 11223344 EBX 00000000 ECX 00000000 EDX 00000000 ESI 00000000 EDI 00000000 ESP 88880FFC EBP 00000000 EIP 00001005 zf 0 nf 0 of 0 cf 0\n",
      "00001005 RET        \n",
      "EAX 11223344 EBX 00000000 ECX 00000000 EDX 00000000 ESI 00000000 EDI 00000000 ESP 88881000 EBP 00000000 EIP 1337BEEF zf 0 nf 0 of 0 cf 0\n"
     ]
    }
   ],
   "source": [
    "jitter.vm.add_memory_page(0x88880000, PAGE_READ | PAGE_WRITE | PAGE_EXEC, b\"\\x00\" * 0x1000, \"stack\")\n",
    "jitter.vm.set_mem(0x88880000 + 0x1000 - 4, b\"\\xef\\xbe\\x37\\x13\")\n",
    "jitter.cpu.ESP = 0x88880000 + 0x1000 - 4\n",
    "\n",
    "jitter.run(0x1000)\n",
    "\n",
    "# The execution ends with an error, which is expected.\n",
    "# Indeed, we RET on 0x1337beef, which is not mapped in memory,\n",
    "# hence the \"WARNING: address 0x1337BEEF is not mapped in virtual memory\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Breakpoints\n",
    "\n",
    "One can register breakpoints to be raised just before a given address will be executed.\n",
    "\n",
    "As we control the CPU, the breakpoint implementation is invisible from the emulated environnement and we can use an arbitrary number of them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Breakpoints are actually callbacks which receive the emulation instance in argument\n",
    "def hello_world(jitter):\n",
    "    print(\"Hello, world!\")\n",
    "    print(\"EAX value is %d\" % jitter.cpu.EAX)\n",
    "    # Stop execution right here\n",
    "    return False\n",
    "\n",
    "jitter.add_breakpoint(0x1005, hello_world)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If a breakpoint callback returns `True`, the emulation continues.\n",
    "Otherwise, the emulation will stop and the value returned by the `jitter.continue_run()` call is the callback return value.\n",
    "\n",
    "Note: if one need to add more arguments to the callback, there are many Pythonic way to do it.\n",
    "For instance, one can use `lambda` to capture an argument or object instance (ie. `jitter.add_breakpoint(address, obj.callback)` with a `def callback(self, jitter)` inside the `obj` definition).\n",
    "\n",
    "The breakpoint mechanism is used to implement several features. For instance:\n",
    "* it can be used to hook a function, by breakpointing on the function address, emulating a few side effects then returning to the return address. Indeed, modifying the programm counter (usually using `jitter.pc = XXX`) in a breakpoint will resume the execution on this new address\n",
    "* in Miasm examples, a breakpoint on a fake address is often used to properly stop the emulation. The callback is usually named `code_sentinelle` and allow to get back control after the emulation of a code snippet"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Exercices\n",
    "\n",
    "For the interested reader, this section introduce a few exercice to practice the API.\n",
    "\n",
    "#### Exercice 1\n",
    "\n",
    "Starting from a new instance:\n",
    "1. Add a memory page which will act as a stack page\n",
    "1. Update the stack pointer accordingly\n",
    "1. Manually push a fake return address\n",
    "1. Map a `RET` instruction (0xC3) in another page\n",
    "1. Run the emulation and ensure it finish on your fake address"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercice 1 - Spoiler\n",
    "\n",
    "A few helpers are actually availables to reduce the burden of these manual actions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0x1000\n"
     ]
    }
   ],
   "source": [
    "# Init a stack and set stack pointer accordingly (architecture agnostic way of writing)\n",
    "jitter.init_stack()\n",
    "\n",
    "# Push 0x1000 on the stack\n",
    "jitter.push_uint32_t(0x1000)\n",
    "\n",
    "# Pop 0x1000 from the stack\n",
    "print(hex(jitter.pop_uint32_t()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercice 2\n",
    "\n",
    "Same a Exercice 1, but instead of ending on an error, we will this time add a breakpoint on the fake return address to properly stop the execution."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "#### Exercice 2 - Spoiler\n",
    "\n",
    "The example `jitter/x86_32.py` is exactly doing this."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}