General updates and bugfixes

dod-cyber-crime-center · Aug 10, 2022 · 769c511 · 769c511
1 parent 263f960
commit 769c511
Show file tree

Hide file tree

Showing 61 changed files with 901 additions and 462 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -0,0 +1,67 @@
+# Changelog
+
+
+## [Unreleased]
+
+- Fixed getting non-user defined exports in Ghidra.
+- Fixed issue getting KeyError if Ghidra isn't setup.
+- Updated documentation.
+- Added `FunctionSignature.calling_convention` get/set property.
+- Added `FunctionSignature.return_type` get/set property.
+- Fixed issue with `ida_hexrays.DecompilationFailure` getting thrown. Switched to logging warning instead.
+- Fixed issue with incorrect immediate operand value being produced with IDA, sometimes causing an OverflowError.
+- Added `Instruction.rep` property for x86 instructions.
+- Fixed issue with incorrectly getting NotExistError in IDA when base address is zero.
+
+
+## [0.4.0] - 2022-06-28
+
+- Added `Symbol.references_to` to get references to imports or exports.
+- Added `Disassembler.get_import()` and `Disassembler.get_export()` functions.
+- Added `BACKEND_GHIDRA` and `BACKEND_IDA` constants.
+- Miscellaneous bugfixes for Ghidra support.
+
+
+## [0.3.0] - 2022-06-01
+
+- Fixed connection issues with running IDA disassembler in Linux.
+- Add auto detection of 64bit size for IDA.
+- Changed `Function.instructions()` implementation to use flowchart.
+- Added `Function.lines()` function.
+- Added `Disassembler.instructions()` function.
+- Added `Disassembler.find_bytes()` function.
+- Added ability to use dragodis locally in underlying disassembler.
+- Added `Disassembler.teleport()` function to run a function within the underlying disassembler.
+
+
+## [0.2.0] - 2022-02-03
+
+- Updated IDA disassembler to use [rpyc](https://rpyc.readthedocs.io/en/latest).
+- Updated support to IDA 7.7
+- Updated Ghidra disassembler to use [pyhidra](https://github.com/dod-cyber-crime-center/pyhidra).
+- Added proper handling when a disassembler isn't setup/installed.
+- Renamed `dragodis.open()` to `dragodis.open_program()`
+- Updated README
+- Interface has been completely refactored.
+- Added support for:
+  : - Flowcharts
+    - Function Signatures
+    - Insturctions
+    - Memory
+    - Operands
+    - Operand value types
+    - References
+    - Imports/Export symbols
+    - Stack/Global variables
+    - Segments
+
+
+## 0.1.0 - 2020-11-25
+
+- Initial release
+
+
+[Unreleased]: https://github.com/dod-cyber-crime-center/dragodis/compare/0.4.0...HEAD
+[0.4.0]: https://github.com/dod-cyber-crime-center/dragodis/compare/0.3.0...0.4.0
+[0.3.0]: https://github.com/dod-cyber-crime-center/dragodis/compare/0.2.0...0.3.0
+[0.2.0]: https://github.com/dod-cyber-crime-center/dragodis/compare/0.1.0...0.2.0
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
diff --git a/README.rst → README.md b/README.rst → README.md
@@ -1,6 +1,4 @@
-********
-Dragodis
-********
+# Dragodis
 
 Dragodis is a Python framework which allows for the creation of
 universal disassembler scripts.  Dragodis currently only supports
@@ -21,83 +19,79 @@ of scripts between users of different disassemblers. Dragodis also aims to provi
 a cleaner and easier to use API than those provided by other disassemblers.
 
 
-Usage
-=====
+## Usage
 
-To use Dragodis, simply pass in the path to your input binary file into either the ``IDA`` or ``Ghidra`` class.
+To use Dragodis, simply pass in the path to your input binary file into either the `IDA` or `Ghidra` class.
 This will create an instance of the disassembler with the given input file analyzed.
 
-.. code-block:: python
+```python
+import dragodis
 
-   import dragodis
+with dragodis.Ghidra(r"C:\strings.exe") as ghidra:
+    print(ghidra.get_dword(0x401000))
+```
 
-   with dragodis.Ghidra(r"C:\strings.exe") as ghidra:
-       print(ghidra.get_dword(0x401000))
-
-
-.. code-block:: python
-
-   import dragodis
-
-   with dragodis.IDA(r"C:\strings.exe") as ida:
-       print(ida.get_dword(0x401000))
+```python
+import dragodis
 
+with dragodis.IDA(r"C:\strings.exe") as ida:
+    print(ida.get_dword(0x401000))
+```
 
 A disassembler can also be run without using a context manager using the `start()` and `stop()` functions.
 
-.. code-block:: python
-
-   import dragodis
-
-   ghidra = dragodis.Ghidra(r"C:\strings.exe")
-   ghidra.start()
-   ghidra.get_dword(0x401000)
-   ghidra.stop()
+```python
+import dragodis
 
+ghidra = dragodis.Ghidra(r"C:\strings.exe")
+ghidra.start()
+ghidra.get_dword(0x401000)
+ghidra.stop()
+```
 
-Alternatively, you can use ``open_program()`` to choose the disassembler more dynamically by providing
-the disassembler name in the ``disassembler`` parameter or by setting the ``DRAGODIS_DISASSEMBLER``
+Alternatively, you can use `open_program()` to choose the disassembler more dynamically by providing
+the disassembler name in the `disassembler` parameter or by setting the `DRAGODIS_DISASSEMBLER`
 environment variable.
 
-.. code-block:: python
+```python
+import dragodis
 
-    import dragodis
+with dragodis.open_program(r"C:\strings.exe", disassembler="ida") as ida:
+    print(ida.get_dword(0x401000))
+```
 
-    with dragodis.open_program(r"C:\strings.exe", disassembler="ida") as ida:
-        print(ida.get_dword(0x401000))
-
-
-It is highly recommended to use the ``DRAGODIS_DISASSEMBLER`` environment variable to ensure your scripts
+It is highly recommended to use the `DRAGODIS_DISASSEMBLER` environment variable to ensure your scripts
 are cross compatible without any modification. As well, to give the user the power to choose
 which disassembler they would like to use.
 
-*NOTE: A ``dragodis.NotInstalledError`` will be thrown if the disassembler chosen is not properly installed.*
-
-.. code-block:: python
 
-    import os
-    os.environ["DRAGODIS_DISASSEMBLER"] = "ida"
+```{note} 
+A "NotInstalledError" will be thrown if the disassembler chosen is not properly installed.
+```
 
-    import dragodis
+```python
+import os
+os.environ["DRAGODIS_DISASSEMBLER"] = "ida"
 
-    with dragodis.open_program(r"C:\strings.exe") as dis:
-        print(f"Disassembler used: {dis.name}")
-        print(dis.get_dword(0x401000))
+import dragodis
 
+with dragodis.open_program(r"C:\strings.exe") as dis:
+    print(f"Disassembler used: {dis.name}")
+    print(dis.get_dword(0x401000))
+```
 
 If you are locally within the disassembler's interpreter (the output window for IDA or pyhidraw interpreter in Ghidra)
 then you can initialize a disassembler object by directly acccessing the object:
 
-.. code-block:: python
-
-    # If in IDA
-    import dragodis
-    dis = dragodis.IDA()
-
-    # If in Ghidra
-    import dragodis
-    dis = dragodis.Ghidra()
+```python
+# If in IDA
+import dragodis
+dis = dragodis.IDA()
 
+# If in Ghidra
+import dragodis
+dis = dragodis.Ghidra()
+```
 
-We can also directly call scripts that using the ``open_program()`` function locally in the disassembler.
+We can also directly call scripts that using the `open_program()` function locally in the disassembler.
 When this happens, the input file path provided must match the detected input file path by the disassembler.
diff --git a/docs/api.rst b/docs/api.rst
diff --git a/docs/changelog.md b/docs/changelog.md
@@ -0,0 +1,2 @@
+```{include} ../CHANGELOG.md
+```
diff --git a/docs/changelog.rst b/docs/changelog.rst
diff --git a/docs/conf.py b/docs/conf.py
@@ -4,6 +4,9 @@
 # list see the documentation:
 # https://www.sphinx-doc.org/en/master/usage/configuration.html
 
+# Pull dragodis version before we mess with the path.
+from dragodis import __version__
+
 # -- Path setup --------------------------------------------------------------
 
 # If extensions (or modules to document with autodoc) are in another directory,
@@ -18,19 +21,23 @@
 # -- Project information -----------------------------------------------------
 
 project = 'Dragodis'
-copyright = '2021, DC3'
+copyright = '2022, DC3'
 author = 'DC3'
 
 # The full version, including alpha/beta/rc tags
-release = '0.2.0'
+version = __version__
+release = version
 
 
 # -- General configuration ---------------------------------------------------
 
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
-extensions = ["sphinx.ext.autodoc", "sphinx_rtd_theme"
+extensions = [
+    "sphinx.ext.autodoc",
+    "sphinx_rtd_theme",
+    "myst_parser",
 ]
 
 # Add any paths that contain templates here, relative to this directory.

diff --git a/docs/differences.md b/docs/differences.md
@@ -0,0 +1,27 @@
+# Differences Between Disassemblers
+
+Given the nature of the project, there is bound to be some variance
+in the way that different disassemblers handle certain situations.
+Some of these situations may be simple to work around, while others may require
+more effort to work around.  Understanding these differences will aid in creating
+better scripts that won't stop working when switching from one disassembler to another.
+
+Some of the differences that have been found so far include: 
+- When it comes to alignment, IDA combines all of the bytes into one line, Ghidra separates each byte into its own line.
+- There may be differences between the min and max addresses of binaries between Ghidra and IDA.
+- The initial current address may be different in Ghidra and IDA.
+- IDA and Ghidra have different naming conventions for various components of the disassembly such as
+  functions.  IDA names functions `sub_XXXXXX` by default, while Ghidra names functions
+  `FUN_00XXXXXX` by default.
+
+If you **do** need to write disassembler specific code, you can check the `.name` attribute of the
+disassembler.
+
+```python
+if dis.name == "IDA":
+    # do IDA specific thing
+elif dis.name == "Ghidra":
+    # do Ghidra specific thing
+else:
+    raise ValueError(f"{dis.name} disassembler is not supported.")
+```
diff --git a/docs/differences.rst b/docs/differences.rst