Skip to content

HorstBaerbel/res2h

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

res2h - A flexible resource compiler similar to bin2h and qrc

License: MIT Build Tests Clang-format Clang-tidy

res2h can:

  • Convert binary data from arbitrary files to a raw hex C/C++ arrays for including them into your software (similar to bin2h with added functionality).
  • Compile the data from all the files into one binary archive file (like tar or qrc) which you can then load from disk or append to the application executable and access at runtime. No external libraries are needed for this.
  • It is currently tested under Linux only, but Windows and to some extent MacOS are plannend.

The main tools are res2h which can convert files to .h/.cpp files or pack them into binary archives and res2hdump which lets you view or unpack those binary archives.

If you find a bug or make an improvement your pull requests are appreciated.

License

All of this is under the MIT License.

Prequisites

  • A C++14-capable compiler with support for std::filesystem. For installing a new g++ version see this.
  • CMake 3.1 or higher for building and unit tests via CTest.

Building

  • Clone repo using git clone https://github.com/HorstBaerbel/res2h
  • Navigate to the res2h folder, then run cmake: cmake .
  • Then build using: make
  • You can run the unit test using: make tests

Usage

res2h

res2h has different modes. It can convert binary data from files to a raw hex arrays in .c/.cpp source files which you can then include in your project and compile them into the executable. It can also create a common header that lets you access all the converted arrays with one include. If you don't want your data to be loaded into memory res2h also provides the possiblility to create one binary archive containing all the files which you can then access via the "Res2h" class provided in seperate headers. You can also embed this archive in your executable, so you only have one file, and access it like you would with any other archive on disk. It should compile and work at least in Windows, Ubuntu and Raspbian.

res2h INFILE/INDIR OUTFILE/OUTDIR [OPTIONS]

Valid OPTIONS

-r: Recurse into subdirectories below indir.
-c: Use .c files and C-arrays for storing the data definitions, else uses .cpp files and std::vector / std::map.
-h HEADERFILE: Puts all declarations in the file "HEADERFILE" using "extern" and includes that header file in the source files.
-u SOURCEFILE: Create utility functions and arrays in a .c/.cpp file. Only makes sense in combination with -h.
-1: Combine all converted files into one big .c/.cpp file (use together with -u).
-b: Compile binary archive OUTFILE containing all infile(s). For reading in your software include res2hinterface.h/.c/.cpp (depending on -c) and consult the docs.
-a: Append INFILE to OUTFILE. Can be used to append an archive to an executable (only one embedded archive possible).
-v: Be verbose.

Examples

  • Convert a single file: res2h ./lenna.png ./resources/lenna_png.cpp
  • Convert all files in a directory, create a common header and utilities: res2h ./data ./resources -r -h resources.h -u resources.cpp
  • Convert all files in a directory, create a common header and utilities, combine all data in resources.cpp: res2h ./data ./resources -r -1 -h resources.h -u resources.cpp
  • Convert data to a binary archive: res2h ./data ./resources/data.bin -b
  • Append an archive to an executable: res2h ./resources/data.bin ./program.exe -a

Generating compilable / includable files

The command res2h a.x b_x.cpp -h bla.h would create those files

a_x.cpp:

// this file was auto-generated from a.x by res2h

#include "bla.h"

const uint32_t a_x_size = 123;
const uint8_t a_x_data[a_x_size] = {
    0x11,0x22,...
};

bla.h:

// this file was auto-generated by res2h

extern const uint32_t a_x_size;
extern const uint8_t a_x_data[];

The command res2h a.x b_x.cpp *-c* -h bla.h -u bla.cpp would create a_x.cpp too, and

bla.h:

// this file was auto-generated by res2h

extern const uint32_t a_x_size;
extern const uint8_t a_x_data[];

struct Res2hEntry {
    const char * relativeFileName;
    const uint32_t size;
    const uint8_t * data;
};

// this contains all the resources with their names and data
extern const uint32_t res2hNrOfFiles;
extern const Res2hEntry res2hFiles[];

bla.cpp:

// this file was auto-generated by res2h

#include "bla.h"

const uint32_t res2hNrOfFiles = 4;
const Res2hEntry res2hFiles[res2hNrOfFiles] = {
 {":/a.x", a_x_size, a_x_data}
};

The command res2h a.x b_x.cpp -h bla.h *-u bla.cpp* would create a_x.cpp too, and

bla.h:

// this file was auto-generated by res2h

extern const uint32_t a_x_size;
extern const uint8_t a_x_data[];

struct Res2hEntry {
    const std::string relativeFileName;
    const uint32_t size;
    const uint8_t * data;
};

// this contains all the resources with their names and data
extern const uint32_t res2hNrOfFiles;
extern const Res2hEntry res2hFiles[];

// this maps the relative file name of resource to its data
// usage e.g: Res2hEntry resource = res2hMap["a.x"];
typedef const std::map<const std::string, const Res2hEntry> res2hMapType;
extern res2hMapType res2hMap;

bla.cpp:

// this file was auto-generated by res2h

#include "bla.h"

const uint32_t res2hNrOfFiles = 4;
const Res2hEntry res2hFiles[res2hNrOfFiles] = {
 {":/a.x", a_x_size, a_x_data}
};

res2hMapType::value_type mapTemp[] = {
    std::make_pair(":/a.x", res2hFiles[0]),
};

res2hMapType res2hMap(mapTemp, mapTemp + sizeof mapTemp / sizeof mapTemp[0]);

Generating binary archives

The command res2h ./data archive.bin -r -b

would find all files in the directory ./data and pack them into the binary archive test.bin. For reading archive files or embedded archives in your application include the files "res2hinterface.h/.cpp" resp.# the class "Res2h". They provide all functions needed for reading resources from archives or from disk. You can find an example on how to use the functions in "res2hdump.cpp" / dumpArchive().

res2hdump

res2hdump is a tool that lets you dump information and/or files from a binary res2h archive or an archive embedded in another file, e.g. executable. It also serves as an example on how to use the "Res2h" class contained in the "res2hinterface" files.

res2hdump ARCHIVE [OUTDIR] [OPTIONS]

Valid OPTIONS

-f: Recreate path structure, creating directories as needed.
-i: Display information about the archive and files, but don't extract anything.
-v: Be verbose.

Examples

  • Display information about the archive: res2hdump ./resources/data.bin -i
  • Extract all files from an archive with subdirectories: res2hdump ./resources/data.bin ./resources -f
  • Extract files from embedded archive: res2hdump ./resources/program.exe ./resources

Binary archive format

Offset (decimal)TypeDescription
Startchar[8]magic number string "res2hbin"
08uint32_tfile format version number (currently 2)
12uint32_tformat flags(32/64 bit depth of archive)
16uint32_t/uint64_tsize of whole archive in bytes
20/24uint32_tnumber of directory and file entries following
Then follows the directory:
24/28 + 00uint16_tfile entry #0, size of internal name WITHOUT null-terminating character
24/28 + 02char[]file entry #0, internal name (NOT null-terminated)
24/28 + 02 + nameuint32_tfile entry #0, format flags for entry (currently 0)
24/28 + 06 + nameuint32_t/uint64_tfile entry #0, size of data
24/28 + 10/14 + nameuint32_t/uint64_tfile entry #0, absolute offset of data in file
24/28 + 14/22 + nameuint32_t/uint64_tfile entry #0, Fletcher32/64 checksum of data
Then follow the other directory entries
Directly after the directory the data blocks begin
End - 04/08uint32_t/uint64_tFletcher32/64 checksum of whole file up to this point
Obviously with a 32bit archive you're limited to ~4GB for the whole binary file and ~4GB per data entry. Res2h will automagically create a 32bit archive to save space if data permits it, or a 64bit archive if needed. This is all transparently handled by res2hinterface, so you don't really need to care about it.

Todo

  • Make CI build + test on Windows + MacOS too.
  • More unit tests.
  • More cleanup. More modern C++14.
  • Enable some more warnings in compiler and clang-tidy.
  • Use nested exceptions.
  • Use cxxopts for argument parsing (support long options).
  • Unicode support.
  • Save space on .c / .cpp files by outputting 32bit or even 64bit hex strings.
  • Re-use compile results of "Build" action in "Unit tests" and "Clang-tidy" action to save time.
  • Use CRC instead of Fletcher.
  • Add optional resource compression.
  • Parallel processing of input files.
  • Support updating archives.
  • Option to only save hash to archives to save space.
  • More compact binary format.

I found a bug or have suggestion

The best way to report a bug or suggest something is to post an issue on GitHub. Try to make it simple, but descriptive and add ALL the information needed to REPRODUCE the bug. "Does not work" is not enough! If you can not compile, please state your system, compiler version, etc! You can also contact me via email if you want to.