SignalBlocks is a C++17 framework to implement blocks of logic which can be tied together. The concept is inspired from the Simulink modeling in Matlab but this framework can be used for multiple applications. This framework aims to be fast, adaptable and free (GPLv3 license). Although I initially intended this for the student and research community, but this tool is generic enough to be used elsewhere.
This project provides a clean C++17 interface which is lean and thin, while retaining features to be generic at the same time. There are a lot of opensource projects which does a lot more than this project can ever do but the idea is to provide a simplified approach in both data interchange and software design.
This framework tries to achieve standardisation in data interchange and module writing, which allows plug-and-play in a unique way. The approach is simplistically raw data interface with pre-defined contracts between the connected modules (or blocks) before they start working on the data. There is no special encoding or decoding of data although the framework is generic enough to allow anything complex.
This is a GPLv3 licensed software, so anything which links or compiles with it must take note of the license. This may not look attractive while developing proprietary or closed source implementations, but then you can use it with outside your application by using the IPC (socket) blocks to send and receive data. In that case your application just need to open socket interface for receiving and sending data without being linked directly. Since the interface is raw data, so there is no data encoding/decoding involved other than data representation of various types in memory.
The framework has the potential to be used in multiple areas although not limited to the following. Additionally, this is just a framework so it basically sets way but do not show the complete path.
- Data Analytics: Realtime, streaming data analytics.
- Digital Signal Processing: Various DSP related simulations.
The author is busy (apart from other enhancements) in finalizing a front-end for creating a standalone application which dynamically creates network and runs simulations.
There are a number of javascript frameworks to get the job done, while contribution will be definitely of great help.
This framework supports Gnu/Linux operating system, but with little change it can be made to build on any of the POSIX compliant operating system. The following depencies are required to use this project; namely
- g++ v8.0+
- cmake
- GSL
- libsndfile
- python-dev
- numpy
Optional Dependencies:
The following dependencies are optional, but some of the blocks depend on them so they will build only when these are available.
Note: use https://github.com/neeraj9/librtlsdr because there is an additional API (rtlsdr_read_timed_sync) added in this fork and used in this project.
Note: It is convinient to use "cmake ../ -DINSTALL_UDEV_RULES=ON" while building librtlsdr (or rtl-sdr) with udev on so that non-root users can also use the device. The above will ensure that appropriate udev rules are installed in /etc/udev/rules.d/rtl-sdr.rules. In case you dont use udev or dont care then use "cmake ../" instead inside librtlsdr/build (as suggested here).
The standard compatibility for gcc cxx17 status gives details of the appropriate compiler versions to use.
Travis config sets up the ubuntu plaform for the build (hint: look at the apt-get commands).
The following base data types are largely supported within the framework, but this framework do not restrict you in any way. Other than being GPLv3 this framework relies on templates heavily, so that new data type can be easily added.
- char, unsigned char
- short, unsigned short
- int, unsigned int
- long, unsigned long
- float
- double, long double
- complex of double (preliminary support)
The framework supports scalar, vector and N-Dimensional matrix and each of the blocks must support any of them (unless intentionally not supported where assertions are to be made).
"The proof is in the pudding ..." and the following demo should get sufficient information (and possibly motivation) to decide on this project.
The following figure is the websocket demo after opening the websockets.html in browser and running websocket_demo with a sample audio file. Notice that the surrounding oscilloscope, which thanks to Soundbeam iPhone Project looks cool. The surrounded signal is streaming from the websocket demo app.
Figure: The sample webpage (htmlapp) for websocket demo with cool oscilloscope.
Lets take a sneak peak at a sample c++ source which reads audio from a file and then stream the output to a remote html5 app via a websocket server. The demo can be built via the "websocket_demo" target in the cmake build rule. Cool huh!
using BaseDataType = short;
using BlockSharedPointer = std::shared_ptr<IPort<BaseDataType> >;
void RunHttpServer(HttpTcpWebsocketServer* pHttpTcpWebsocketServer,
std::atomic<bool>* pKeepRunning) {
LOG_INFO("Running http tcp websocket server\n");
while (std::atomic_load<bool>(pKeepRunning)) {
pHttpTcpWebsocketServer->Run();
}
LOG_INFO("http server thread is terminating.\n");
}
void run_signal_blocks(const char *filename) {
// create the blocks and connect them together
std::unique_ptr<std::istream> audiofile(new std::ifstream(filename));
BlockSharedPointer source(new AudioSource<BaseDataType>(
"audio-source", block_size, std::move(audioStream)));
BlockSharedPointer block(new Splitter<2, BaseDataType>("extract-ch0", 0));
BlockSharedPointer nullport(new Terminator<BaseDataType>("nullport"));
BlockSharedPointer sink(
new JsonDataExtractableSink<BaseDataType>("json-data-extractor"));
connect(source, connect(connect(block, sink), nullport, 1, 0));
// all the blocks are now connected and ready to run
// create a websocket server thread
std::atomic<bool> keep_running(true);
std::unique_ptr<HttpTcpWebsocketServer> server(
new HttpTcpWebsocketServer(http_port));
// connect the json data block sink with websocket server
JsonDataExtractableSink<BaseDataType>* archive =
dynamic_cast<JsonDataExtractableSink<BaseDataType>*>(sink.get());
JsonDataCallbackFuncType cb = archive->GetDataCallback();
server->AddRoute("/1", std::move(cb));
std::thread http_server(&RunHttpServer, server.get(), &keep_running);
// now you can browse at http://<ip>:<port>/1 for websocket live
// streaming data
// run a demo simulation loop
TimeTick time_tick(1); // always start with non-zero value
for (int i = 0; i < 1000000000; ++i) {
source->ClockCycle(time_tick);
time_tick += 1;
usleep(200 * 1000); // sleep for 200 milliseconds
}Although the installation for various GNU/Linux distributions differ but the dependencies are easily available on any one of them.
The following commands were tested on Ubuntu 14.04 but things should be similar (if not same) on other releases and Debian.
The following commands needs to be as a system administrator or with sudo (as shown below) so that the relavent packages are installed in the system.
If your system is not updated then probably it is a good idea to do that before installing anything else.
sudo apt-get update
In case you prefer the GNU GCC compler then install g++
sudo apt-get install -y g++-8 gcc-8
The following other depencies are required as mentioned before.
sudo apt-get install -y libsndfile1-dev libgsl0-dev python-numpy
The following commands were tested on Centos 7 but things should be similar (if not same) on other releases and Fedora or Redhat.
If your system is not updated then probably it is a good idea to do that before installing anything else.
sudo yum update
In case you prefer the GNU GCC compler then install g++
sudo yum install -y gcc-c++
The following other depencies are required as mentioned before.
sudo yum install -y gsl-devel libsndfile-devel python-devel numpy
The versions gcc (g++) which are installed in your (rpm based) operating needs to meet the previously mentioned criteria. At the time of this writing CentOS 7, Fedora 23, RHEL 7 are the latest releases and works well. Anything earlier may not work for you unless you are willing to get under the hood and build things on your own.
If your system is not updated then probably it is a good idea to do that before installing anything else.
sudo pacman -Syu
In case you prefer the GNU GCC compler then install g++
sudo pacman -Sy gcc-multilib
The following other depencies are required as mentioned before.
sudo yum install -y gsl libsndfile python2 python2-numpy
I was always fascinated by Matlab's power and ease of use while in college and wanted to implement something similar, but in a language which can be compiled to native format for fast performance. This project is an attempt to give a basic framework which will (soon) encapsulate many of the open source software to construct a building block for such requirement.
After you are done setting up the development environment the build is pretty straight-forward (see below).
git clone https://github.com/neeraj9/SignalBlocks
cd SignalBlocks
mkdir build
cd build
../cmake.sh ..
make
Note that cmake.sh is shell script to set CXX and CC for gcc-8 or g++-8, since gnu c++ version 8 supports C++17 standard.
The tests are integrated into the ctest framework, so you can run it simply as follows:
make test
The testsuite is another target which is built and run as follows (in case you want to run that directly instead of via the ctest way):
make testsuite
./test/testsuite
Take a look at the example projects for using this framework. Lets build the websocket_demo (for example).
cd SignalBlocks/build
make websocket_demo
./examples/websocket_demo/websocket_demo somewavfile.wav
The framework has the following components:
- core - The core of the library which forms the basis for writing usable blocks.
- socket - The socket related wrapper code (over system socket api) which is used by some of the blocks.
- blocks - The base blocks which provide a lot of useful functionality (for example sources, sinks, etc).
- http - A primitive (yet working) websocket server which hosts data results for websocket clients to view. There is even a cool demo in the examples subfolder.
- examples - Example applications to use the framework.
- test - Unit test based on a great (single header) test framework Catch.
The blocks are categorized in as follows:
- basicop - These blocks are used for basic operations like buffer, demux, etc.
- converters - It converts input say to-ordinal, or to-matrix.
- filters - It hosts the delay and down sample blocks at present, but primarily these should contain mathematical filter blocks.
- math - All the blocks are basic mathematical operations like sum, product, cos, sin and others.
- python - It is supposed to (untested at present) allow generic filters to be created from python code. The python plugin works (and is unit-tested as well but the code block is not).
- sinks - The blocks provide various options for archiving, storing or looking at the signals.
- sources - These are some of the basic signal (or data) sources like audio, file, constant, step, linear, rtlsdr, gpio, io port, and others.
- text - The blocks provide functionality to operate on text and seldom care about the time tick (so take note).
The following list of blocks are available within the system (as on date) although not all of them are thoroughly tested.
| Block | Description |
|---|---|
| de-multiplexer | one to many |
| multiplexer | many to one |
| buffer | convert serial data to a buffer of given size |
| duplicator | duplicate the input into multiple outputs |
| splitter | split a N dimmention vector to scalar which goes out to N output blocks |
| transpose | compute 2D matrix transpose |
| Block | Description |
|---|---|
| ordinal converter | learn automatically and convert from string to integer, where the id are automatically generated based on inputs and continuously increases as new text strings are encountered. It takes simple c++ string as input and unsigned long as output (so its a MixedPort). |
| char ordinal converter | It does exactly same as "ordinal converter" but takes raw character as input instead of std::string. This is particularly useful when working with modules which output raw character as output (for example the word splitter). |
| basic-type converter | convert from one base type to another |
| to-matrix converter | convert scalar or vector input to matrix output but retaining the dimensions. This is particularly useful when a block needs an input in matrix format. Note that the data is not transformed, just that the dimensions are modified to be of the matrix type. |
| Block | Description |
|---|---|
| delay | with some initial condition which is the initial output till delay is reached for first output |
| downsample | filter data to down-sample input stream of data |
| Block | Description |
|---|---|
| running-average | an infinite running average of input (no over/underflow checking) |
| average | an average of inputs (no over/underflow checking) |
| gain | multiply input with a constant factor |
| product | multiply two input values |
| sum | add two input values |
| difference | difference between two input values |
| arccosh | inverse hyperbolic cosine |
| arcsine | inverse hyperbolic sine |
| arctanh | inverse hyperbolic tan |
| constant power | raise the input to a constant power |
| cos | cosine |
| expm1 | it computes \exp(x)-1 |
| log1p | it computes \log(1+x) |
| sin | sine |
| tan | tan |
| Block | Description |
|---|---|
| archive sink | archives all the data without dropping any one. It is important to note that there is no bound checking, so you will quickly run out of memory for a very large data set. |
| output stream sink | a sink which can be used to store to a file or any c++ ostream |
| socket sink | output to a socket and send it to remote system (udp or tcp) |
| standard output | dump the data (signal) on the standard output or the console |
| json data extractable sink | a special sink to convert input signal to a json (text) data stream, which can then be attached to a websocket server to stream them to connected clients. There is a cool demo also added which demonstrates its usefulness via an html5 app to monitor live stream of signal (inside a virtual oscilloscope). This block stores a limited history of data, while dropping older events beyond the configured threshold. |
| terminator | a null port (or sort of a ground) which is useful when you dont want to look at a signal but since all the input and output must be connected so this must be used in such cases. |
| Block | Description |
|---|---|
| rtlsdr source | Read samples from a RTLSDR device when rtlsdr library is available. This block is optional and will be available only when the optional dependency of rtlsdr is satisfied. |
| csv file source | source of csvfile and generates a vector output of string data type |
| constant | source of constant number |
| audio source | read audio file (multiple non-patented) formats and generate signal stream |
| complex stream source | reads {I, Q} complex data from a stream and pass on as-is |
| input stream source | a source which can read from any c++ istream source |
| linear | generates a linear signal based on an initial value and an increment per tick |
| pulse | generates a pulse signal of high and low values |
| random | generates a random signal source based on the c psuedo random number generator |
| selective csv file source | selectively read specific columns from a text csv file and generate appropriate signal streams |
| socket | reads packets from socket and sends them out as a source. This can be used to receive data from remote location and feed to your subsystem. At present it supports tcp and udp sockets. |
| step | generates a function which starts with an initial value and always increase by 1/td> |
| linux-raw-io | Allows reading from Linux raw IO port. |
| linux-gpio | Allows reading from Linux GPIO port (0 or 1 value). |
Note that unless mentioned otherwise the following blocks deal with ascii characters only (no unicode unless specifically mentioned).
| Block | Description |
|---|---|
| split-words | Take a single character (scalar) or multiple characters (vector) as input and spit vector output where each one is a separate ascii word. |
| split-lines | Take a single character (scalar) or multiple characters (vector) as input and spit vector output where each one is a separate line (excluding \n and \r which are the line terminators). |
| parse-csv-stream | Take a std::string scalar input and consider that as a stream while spliting at configured field delimiter. There is not validation regarding number of fields for every line. That is each row of csv record may have different number of fields, although this is not correct eventually but that validation is not performed in this block (but passed down as-is). |
Each of the block is written such that it can receive either scalar, vector or matrix data and mostly (unless there is a strong reason against it) it computes over it. Additionally, the same block instance can receive different type of data in its lifetime as well. There is no hard requirements against it and not configuration option is provided to enforce it as well. This design choice is made to allow complex scenarios (where changing dimension is required) and avoid configuration. Instead this design lets the system learn the data dimension automatically. The downside of this approach is that technically the block can receive data of different dimensions (scalar, vector or matrix) without any validation and needs to accept as it receives it. In case a specific block cannot work in this manner then appropriate errors or assertions should be enforced (but that is on a case-by-case basis).
At present there is no restriction on the dimensions of data passed each time to the same block, while this is a lot flexible but can be an issue for an incorrectly written block. (hint: see Buffer block)
There are number of items and which are in the pipeline and mostly I have it documented in my personal notes. The most important of them is to improve the code coverage of the unit tests and any new feature. Having said that there will be new features available from time to time.
There are a number of projects which are similar to SignalBlocks in one way or another although they are much bigger and older. The following set of projects a a few of the bigger ones.
- Gnu Radio
- Mathworks Simulink (commercial)
- Gnu Octave
- Scilab
Thanks for evaluating this project and hope you find it useful. Feel free to create issues for bugs or new features.
C++ code can be daunting for newbies, but I strive to make the framework API simple to understand and use. Having said that it is important to pick up any good book on c++ which covers c++11 to make the best use this framework. This will also allow you to avoid spending a lot of time debugging language usage issues. A plethora of open source frameworks allow an ever expanding functionality (on top of this framework) with practically limitless possibilities.
- Neeraj Sharma {github: neeraj9}