Object Detection Getting Started

The purpose of this package is to enable object detection application on a STM32H7 board.

This project provides an STM32H7 microcontroller embedded real time environment to execute STEdgeAI generated models targeting object detection application.

Before You Start

Hardware and Software environment

In order to run this object detection application examples you need to have the following hardware:

• STM32H747I-DISCO discovery board
• B-CAMS-OMV camera bundle

Tools

• STM32CubeIDE (v1.17.0)
• STM32CubeProgrammer (v2.18.0)
• STEdgeAI (v3.0.0)

Deployment

Generate C code from tflite file

This repo provide an example of an AI C-model generated by stedgeai: st_ssd_mobilenet_v1_025_192_int8.tflite

This application is a C-based project required by the deployment service in the ModelZoo. The ModelZoo enables you to train, evaluate, and automatically deploy any supported model.

To deploy your model using the ModelZoo, refer to the Deployment README for STM32H7 for detailed instructions on deploying to either the STM32N6570-DK or the NUCLEO-N657X0-Q.

Build and deploy

The deployment script of the model zoo:

• Generate your model
• Build your application
• Deploy it on the target

You can launch as well the Application\STM32H747I-DISCO\STM32CubeIDE\.project with STM32CubeIDE. With the IDE you can modify, build and deploy on the target. The default model will be used in that case.

Getting started deep dive

The purpose of this package is to enable object detection application on a STM32H7 board.

This package also provides a feature-rich image processing library (STM32_ImageProcessing_Library software component).

Processing workflow

The software executes an object detection on each image captured by the camera. The framerate depends on the inference time

• Captured_image: Image From the camera

• Network_Preprocess: 3 steps:

  • ImageResize: rescale to the resolution needed by the network
  • PixelFormatConversion: Convert Image input (usually RGB565) to needed color channels (RGB888 or Grayscale)
  • PixelValueConversion: Convert to pixel types used by the network (uint8 or int8)

• HxWxC : Height, Width and Number of color channels: format defined by the given network

• Network_Inference: Call AI C-model network

• Network post process:

  • Call Output_Dequantize to convert the output to the right output type (only float32 for now)

  • In the context of Object detections model there is several filtering algorithms to apply at the output of the model in order to get the proper bounding boxes

  • For now we support ssd type of post processing, YoloV2 postprocessing as well as centernet networks

Memory Layout

The application software uses different buffers. The following diagram describes how there are used and which functions interact with it.

Model configuration

The '<getting-start-install-dir>/Application/STM32H747I-DISCO/Inc/CM7/ai_model_config.h' file contains configuration information.

This file is generated by the mode zool deployment script when if you use it.

The number of output class for the model:

#define NB_CLASSES          (2)

The dimension of the model input tensor:

#define INPUT_HEIGHT        (192)
#define INPUT_WIDTH         (192)
#define INPUT_CHANNELS      (3)

A table containing the list of the labels for the output classes:

#define CLASSES_TABLE const char* classes_table[NB_CLASSES] = {\
   "background", "person" }\

Concerning the type of resizing algorithm that is used by the preprocessing stage, only the nearest neighbor algorithm is supported.

Input frame aspect ratio algorithms:

#define ASPECT_RATIO_FIT            0
#define ASPECT_RATIO_CROP         1
#define ASPECT_RATIO_PADDING 2

#define ASPECT_RATIO_MODE ASPECT_RATIO_FIT

Post processing type to apply

#define POSTPROCESS_CENTER_NET (0)
#define POSTPROCESS_YOLO_V2    (1)
#define POSTPROCESS_ST_SSD     (2)

#define POSTPROCESS_TYPE POSTPROCESS_ST_SSD

The pixel color format that is expected by the neural network model:

#define RGB_FORMAT        (1)
#define BGR_FORMAT        (2)
#define GRAYSCALE_FORMAT  (3)
#define PP_COLOR_MODE    RGB_FORMAT

Data format supported for the input and/or the output of the neural network model:

#define UINT8_FORMAT     (1)
#define INT8_FORMAT      (2)
#define FLOAT32_FORMAT   (3)

Data format that is expected by the input layer of the quantized neural network model (only UINT8 and INT8 formats are supported in V1.0):

#define QUANT_INPUT_TYPE    INT8_FORMAT

Data format that is provided by the output layer of the quantized neural network model (only FLOAT32 format is supported in V1.0):

#define QUANT_OUTPUT_TYPE    FLOAT32_FORMAT

The rest of the model details will be embedded in the .c and .h files generated by the tool X-CUBE-AI.

Image preprocessing

The frame captured by the camera is in a standard video format. As the neural network needs to receive a square-shaped image in input, three solutions are provided to reshape the captured frame before running the inference

• ASPECT_RATIO_FIT: the frame is compacted to fit into a square with a side equal to the height of the captured frame. The aspect ratio is modified.
• ASPECT_RATIO_CROP: the frame is cropped to fit into a square with a side equal to the height of the captured frame. The aspect ratio remains but some data is lost on each side of the image.

-ASPECT_RATIO_PADDING: the frame is filled with black borders to fit into a square with a side equal to the width of the captured frame. The aspect ratio remains.

How to update my project with a new version of ST Edge AI

The neural network model files (network.c/h, etc.) included in this project were generated using STEdgeAI version 3.0.0.

If you use a different version of STEdgeAI to generate these model files, please follow the STEdgeAI instructions on How to update my project with a new version of ST Edge AI Core to update your project.

Limitations

• Supports only networks up to 240x240 input resolutions (SSD 256x256 is not yet supported by this code base)
• Supports only the STM32H747I-DISCO board with B-CAMS-OMV camera module.
• Supports only 8-bits quantized model
• Input layer of the quantized model supports only data in UINT8 or INT8 format
• Output layer of the quantized model provides data in only FLOAT32 format
• Limited to STM32CubeIDE / arm gcc toolchain.
• Manageable only through STM32CubeIDE (open, modification, debug)