Distributed Motion Surveillance Security System (DMS3) is a Go-based application that integrates third-party open-source motion detection applications (e.g., the Motion motion detection software package, or OpenCV, the Open Source Computer Vision Library) into an automated distributed motion surveillance system that:
- Using a local network, wirelessly senses when someone is "at home" and when someone is "not at home" and automatically enables or disables the surveillance system
- Through the DMS3Server, the system coordinates video stream processing, reporting, and user notification to participating device clients (e.g., a Raspberry Pi or similar) running the DMS3Client component which:
- Greatly minimizes network congestion, particularly during high-bandwidth surveillance events of interest
- Better utilizes device client CPU/GPU processing power: keeping stream processing on-board and distributed around the network
- Optionally, DMS3Clients can generate email reports for events of interest containing images or video using the available DMS3Mail component
- Optionally, the DMS3Server can display the current state of all reporting DMS3Clients visually through the use of the DMS3Dashboard component
- Works cooperatively with "less smart" device clients such as IP cameras (wired or WiFi), webcams, and other USB camera devices
This release makes several minor security, component, and environment updates, including:
- Security update: bump to golang.org/x/crypto package from 0.1.0 to 0.17.0
- A revision to the DMS3 component codebase, moving from Go 1.17 to Go 1.22, bringing with this language release update numerous platform performance optimizations and security enhancements
- Minor refactor in the DMS3Dashboard component to dynamically show operating system (OS) and environment updates in DMS3Clients. Previously, DMS3Client OS changes were only updated in the dashboard when a DMS3Client first came online, or when the DMS3Dashboard component was restarted
- A refactor to permit better motion detector configuration when using the Motion (and more recent MotionPlus) applications. It's now easier to configure DMS3 components for use with either motion detector application across different operating systems
- Minor changes to the DMS3Mail component
- Changed the subject line in motion surveillance event emails to make it easier to identify the DMS3Client
- Changed the list order of information presented in event emails
Much has changed over the past 4+ years since the 1.3.1 release of DMS3, so this release has focused on upgrades and improvements to make the DMS3 surveillance security system that so many have relied upon even more efficient, stable, and secure.
- The DMS3Mail component gets a significant makeover in response to ongoing changes in the use of advanced progressive HTML5 email templates developed to work with myriad end-user email applications. Upgrades to DMS3Mail include:
- NEW! A much-anticipated and fully configurable HTML5 email template, based on the very popular Cerberus responsive email patterns. For use in DMS3, we integrated the Go HTML/template package into the Cerberus fluid template, very similar to what we did when creating the DMS3Dashboard component. This new responsive email template now presents a more complete email message to the end user, with the following functionality:
- NEW! Larger image attachments are now integrated directly into the email body (versus as an attachment)
- NEW! More complete metrics now presented in the email for each security event, including the hostname of the DMS3 Client component sourcing the event.
- NEW! The percentage of changes (in pixels) is now provided, thanks to a new GetImageDimensions() routine that provides image details as DMS3Mail processes the security event in real-time.
- As part of this new progressive email template, email "dark mode" is now handled automatically, making it easier to view email on disparate mobile platforms
- NEW! A much-anticipated and fully configurable HTML5 email template, based on the very popular Cerberus responsive email patterns. For use in DMS3, we integrated the Go HTML/template package into the Cerberus fluid template, very similar to what we did when creating the DMS3Dashboard component. This new responsive email template now presents a more complete email message to the end user, with the following functionality:
Ever wonder if your surveillance cameras are operational, in need of updates, or even a reboot? The DMS3Dashboard component can be enabled to run on a DMS3Server and provide regularly-updated information of all DMS3Client components with device metrics that include:
- Hostname
- Hardware platform and operating system
- Kernel version
- Current DMS3Server and all reporting DMS3Client components' uptime
- Count of DMS3Clients reporting to the DMS3Server
- Count of surveillance events generated by DMS3Clients components
- Date/time (ISO 8601) the component last reported to the DMS3Server
Additionally, DMS3Dashboard provides a quick visual health check of all DMS3Client components, using color-sensitive component icons, where:
- Green: a DMS3Client has reported to the server within an expected period of time (as configured)
- Yellow: a DMS3Client is late in reporting, exceeding its configured reporting interval
- Red: a DMS3Client has not reported to the server, and requires attention
The DMS3Dashboard component is written using Go's HTML templating package, making it very easy to integrate new or existing HTML template themes into the component. The template used by DMS3Dashboard is based largely on the following resources:
- Creative Tim's Paper Dashboard Theme
- Fonts provided by Icomoon
New for this release are the following additional configuration options for DMS3Dashboard:
- NEW! Independently configurable client icon status option timeouts (warning, danger, missing) visually provide a status of a DMS3Client health in real-time
- NEW! Option to make DMS3Server always first in the set of DMS3Client devices displayed in the dashboard
- NEW! Option to alphabetically sort DMS3 devices displayed in the dashboard
- NEW! Reporting a more comprehensive--and now dynamically updated--list of DMS3 device attributes, including:
- Operating system name and version release (e.g., Raspbian GNU/Linux 10)
- Hardware platform (e.g., Linux ARM7l)
- Kernel release (e.g., 5.10.63-v7+)
- NEW! Various additional upgrades to the dashboard HTML template, including revisions to the template display, and updates to use the new DMS3 logo in the template header
- Both of these DMS3 components received a significant upgrade that includes:
- A revision to the DMS3 component codebase, moving from Go 1.8 to Go 1.17, bringing with this language release update numerous new low-level packages, platform performance optimizations, and security enhancements
- The addition of the ARM8 platform type (great news for Raspberry Pi and related SBC users), automatically incorporated into our native DMS3Build process
- As part of the DMS3Build process, the remote installers have been rewritten to abstract away specific Linux dependencies
- Revised overall project structure to reflect idiomatic Go principles
- Commands now organized into a
cmdfolder, while configuration files are now managed in aconfigfolder - Project moved from use of the
gocodeprocess to usinggopls - Project migration over to the use of Go modules
- Commands now organized into a
- System-level service (daemon) calls are now abstracted away to work on across a broader array of Unix-like operating systems
- DMS3Server listening port moved from the previous registered port range into the more appropriate dynamic/private range
- All TOML configuration files revved from 0.4.0 and validated (tomlv) to 1.0.0
At its core, DMS3 sensing relies on the concept of a user proxy. In this context:
A user proxy is any device representing a user that can be sensed on a network
A smartphone is an excellent example of a user proxy, assuming that the user's smartphone is a participating device on a home network while the user is "at home," and that smartphone then drops from the network when the user is no longer "at home."
This concept can extend to multiple user proxies, making it possible for DMS3 to keep a surveillance system disabled until everyone in an entire family has left home: once the last registered user proxy is no longer sensed on the home network, DMS3 automatically enables the surveillance system.
The reverse is true as well: DMS3 will keep the surveillance system enabled only until the first user proxy is seen on the home network (e.g., someone pulling into the driveway), at which time DMS3 will automatically disable the surveillance system.
In addition to sensing user proxies, DMS3 can be configured to keep the surveillance system enabled for a specific periods of time. Called Always On, this DMS3 feature works as an override to sensing user proxies: regardless of whether DMS3 senses a user proxy on the network, and as long as the Always On time-of-day policy is met, DMS3 will enable the surveillance system.
This feature is particularly useful for nighttime surveillance, when users may be asleep and/or smartphones may be turned off. For example, DMS3 can be configured to turn a surveillance system on at 2300, and stay on until 0500 the next morning. During this time, DMS3 will remain operational and monitor (and report) surveillance events as they occur.
DMS3 is organized into the following application components:
| Component | Description | |
|---|---|---|
| DMS3Server | This component acts as the central coordinator for the DMS3 system. It determines whether to enable/disable the overall surveillance system and broadcasts this state to connected DMS3Client devices. DMS3Servers are typically deployed on headless servers or home desktop computers. | |
| DMS3Client | These components are client-side services that interact with the DMS3Server. They handle starting and stopping a locally installed motion detection application, such as Motion. Multiple DMS3Client devices can be part of a single DMS3 system. These clients are typically installed on IoT hardware like Raspberry PIs or other SBC devices. | |
| DMS3Dashboard (Optional) | This optional component provides a visual interface for displaying real-time status information from connected DMS3Client devices through a web browser. | |
| DMS3Libs | This set of shared libraries provides core functionality for managing DMS3 client-server services. DMS3Libs include tools for low-level system and network commands, system logging, and unit testing. |
Optional for DMS3Client devices configured to use the Motion motion detection application:
| Component | Description | |
|---|---|---|
| DMS3Mail | A separate, configurable DMS3 component for generating and sending an email in real-time whenever a client running Motion generates a significant security event |
-
Support for the DMS3Dashboard component, allowing for the easy, visual monitoring of all DMS3Client devices managed by a DMS3Server component
- Mobile first, responsive, web-based design
- Uses Go's HTML templating package to simplify Go/HTML integration
- Easily integrate 3rd-party configurable HTML website templates
-
Automated starting/stopping of any number of motion detection applications installed on smart device clients (e.g., the Motion motion detector software package) based on the presence/absence of user proxy devices
-
Always On feature starts/stops the motion detection application based on time-of-day (e.g., can enable video surveillance during nighttime or specific holiday hours)
-
Device clients can be custom-configured to process and respond to surveillance events independently and uniquely (e.g., an outdoor IR camera device only sends email during nighttime hours)
-
Optionally play separate audio file(s) on surveillance system enable and disable
-
Configurable event logging
- INFO, ERROR, FATAL, and DEBUG log levels
- Persist logs to file or send to STDOUT (terminal)
-
MAC (Layer 2) address sensing
- IPv4 protocol support
- IPv6 protocol support [planned]
-
Bluetooth user proxy sensing (using RSSI, L2CAP, or similar) [planned]
-
Developed for use exclusively with Motion, DMS3Mail is an automated, real-time email notification service triggered by Motion-generated detection events
- Fully configurable email message subject, body, etc. using the excellent Cerberus responsive HTML email template
- Optionally attach an event image or video to an email message
- SMTP-support for compatibility with most web-mail services (e.g., Gmail)
- Configurable event logging
- INFO, ERROR, FATAL, and DEBUG log levels
- Persist logs to file or send to STDOUT (terminal)
While DMS3 is primarily responsible for monitoring user proxies and determining when to enable or disable the surveillance system, it alone does not manage the processing of video stream data. That complex real-time task is left to motion detection libraries/applications which can be integrated directly into DMS3.
-
Support for the Motion motion detector software package
-
Support for the OpenCV Library [planned]
- OpenCV support is highly anticipated, but still experimental, though the current DMS3 codebase cleanly abstracts away any specific motion detection application dependencies so it is anticipated to be a very straightforward integration
DMS3 is designed to utilize intelligent IoT devices, called Smart Device Clients (SDCs), while still supporting less intelligent, single-purpose devices, called Less Smart Device Clients (LSDCs).
-
DMS3 Smart Device Clients (SDCs) are hardware devices capable of processing local video streams for motion detection on-board, with dedicated hardware. Most computers and smaller single board computers (IoT SBCs) would be classed as smart device clients, including:
- Raspberry PIs (DMS3 was tested with the RaspPi Model 2, Model 3, and Pi Zero W) with a configured on-board camera module
- Any IoT single board computer (SBC) capable of running a Unix-like operating system
- Personal computers with a camera and wired or wireless (WiFi) connectivity
-
DMS3 Less Smart Device Clients (LSDCs) are hardware devices--typically purpose-built--unable to process motion detection video streams locally. These devices usually generate raw real-time video data, which is then sent, consumed and processed by an external device(s), oftentimes wirelessly across a network. Some examples of LSDCs include:
- IP cameras (e.g., the Nest Cam), either wired or wireless
- Webcams, typically using USB connections and run from a laptop or desktop computer
DMS3 is patterned after a client-server model, where the DMS3Server component is centrally responsible for the logic of enabling/disabling the video surveillance system, while each participating smart device client (SDC), through the use of the DMS3Client component, is responsible for starting/stopping the locally-installed motion detection application. For "less smart" device clients (LSDCs), the processing of video stream data is passed over the wire to the server for processing and eventual system response and/or user notification.
If you appreciate isometric drawings, please check out our Isometric-Icons project, located here.
In the example above, one IP camera device, one IoT SBC device (a Raspberry Pi), and one webcam device are managed through the DMS3Server component (using the TCP protocol). The DMS3Server determines when to enable/disable the surveillance system, and notifies each participating device client running their own local instance of the DMS3Client component. Since the Raspberry Pi can be configured to run a local instance of a motion detection application, actual video stream processing, imaging, and eventual reporting is all done locally, greatly limiting network congestion.
The webcam device and the IP camera device--both less smart device clients (LSDCs), incapable of on-board stream processing--must pass raw stream data along to a device proxy running a DMS3Client component, which then applies motion detection processing on the incoming video streams.
The DMS3Server component is responsible for signaling the logic of enabling/disabling the video surveillance system to all device client endpoints. That is, the DMS3Server sends either a Start or a Stop message to all DMS3 device clients configured with a DMS3Client component, listening on the network.
DMS3Server does this by periodically scanning the network for the existence of one or more registered user proxies. These proxy devices can be anything that exposes their MAC address on the network (e.g., a mobile phone on a home LAN). If that device is found on the network, it's assumed that "someone is home" and so DMS3Server sends out a Stop message to all participating device clients, and their respective motion detection application is subsequently stopped (if currently running).
If that user proxy (or multiple proxies) is then no longer found on the network, it's assumed that "nobody is home", and the DMS3Server sends out a Start message to all participating device clients, and the motion detection application on that client is started (if currently stopped). Similar logic is used in the reverse case: when a user proxy is once again "back home," the motion detection application of each device client is signalled to Stop.
Alternatively, the Always On feature instead uses the time of day to enable/disable the surveillance system (effectively overriding the presence/absence of user proxies). The DMS3Server will look at the time range specified, and if the current time falls between the time range, the motion detection application of all client devices will be started. Once the current time falls outside of the specified time range, the motion detection application for each device client is then stopped.
The DMS3Client component runs on each configured smart device client endpoint, and is responsible for starting/stopping its locally installed motion detection application. The DMS3Client does this by periodically listening to the configured DMS3Server at the pre-configured IP address and port (network socket address). When the DMS3Client receives a change in motion detection application state, it either starts or stops its locally-installed motion detection application.
In instances where the device client is "less smart" and unable to process motion detection in local video streams, an LSDC instead passes motion detection processing to a DMS3Client proxy. Multiple LSDCs can be served by a single DMS3Client proxy. This proxy is then responsible for the operations of a typical DMS3Client.
Operationally, the DMS3Server and all DMS3Client device clients work together to establish a synchronized security surveillance state across all endpoints:
- DMS3Server: usually configured as a daemon running on a central server, walks a logic tree whenever a client connects (or re-connects) to the server. DMS3Server is responsible for answering the question "should the surveillance system be enabled or disabled right now?"
- DMS3Client: usually configured as a daemon that runs on each of the participating smart device clients, a DMS3Client regularly polls (at a configurable interval) the DMS3Server, and receives from the DMS3Server the current motion detection application state (called
MotionDetectorState), that is, whether the locally installed motion detection application should be started or stopped
The activity diagram below shows the work flow of these two components:
When using Motion, DMS3Mail is a feature written for DMS3 that allows for the creation and sending an email whenever a valid capture event is triggered in Motion. DMS3Mail is very tightly integrated into Motion, where image and video capture events are identified, analyzed, and processed. DMS3Mail is triggered by the on_picture_save and the on_movie_end commands in Motion.
Note: the optional DMS3Mail component is called by neither DMS3Client nor DMS3Server. Instead, DMS3Mail is called directly by the Motion motion detection application.
The syntax for these Motion commands are:
<on_picture_save|on_movie_end> <absolute path to dms3mail> -pixels=%D -filename=%f
Once configured, DMS3Mail will respond to these two Motion event hooks, and an email will be generated and sent out with an image file or video clip capturing the surveillance event of interest.
- In order to compile the DMS3 project components, an operational Go environment is required (this release of DMS3 was developed using Go 1.22.5)
- A Unix-like operating system installed on the server and smart device client (SDC) endpoints
- While DMS3 was written and tested under Linux (Ubuntu 17.04+, and various Debian and Raspian/Raspberry Pi OS releases), there should be no reason why DMS3 won't work under other Linux distributions
- A motion detection application, such as Motion, correctly installed and configured with appropriate video devices configured on all smart device clients
- Specific Unix and Unix-like commands and tools used by DMS3 components include:
- aplay: ALSA audio player (optional)
- bash: a Unix shell and command language
- cat: a standard Unix utility that reads files/input and writes them to standard output
- env: Unix shell command to run a program in an altered environment
- grep: globally search a regular expression and print
- ip: displays or manipulate routing, devices, policy routing, and tunnels
- pgrep: globally search a regular expression and print
- ping: ICMP network packet echo/response tool
- pkill: globally search a regular expression and send signals to a process
At its core, DMS3 sensing relies on the concept of a user proxy. In this context, a user proxy is any device representing a user that can be sensed on a home network. A smartphone is an excellent user proxy, assuming that a user's smartphone is active on the home network when the user is "at home," and drops from the network when the user leaves and is then "not at home." DMS3 performs this sensing by searching the end user's network for MAC addresses registered during the configuration of the DMS3Server component (in the dms3server.toml file).
Historically, MAC addresses have always represented, 1-for-1, the underlying hardware. However, more recently, and as a broader privacy policy, some device vendors now provide users the option to have their device generate MAC addresses randomly for over-the-air communications. This feature can disrupt the sensing services used by the DMS3Server component.
As a result, it's important to review smartphone (or other user proxies) privacy policies and feature options and configure them accordingly. A typical resolution available on smartphones permits users to set static MAC addresses for specific networks.
A separate installation document is available here.
The MIT License (MIT)
Copyright (c) Business Learning Incorporated
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.