Thermal Bridges on Building Rooftops Detection (TBBRDet)

This repository contains the corresponding code and training configurations for the paper Investigating deep learning approaches to building rooftop thermal bridge detection from aerial images and the dataset Thermal Bridges on Building Rooftops (TBBRv2). The raw, unannotated full set of collected drone images can be found here.

TBBRv2_annotations_flyover.mp4

Installation

Detectron2

To install you'll need a Python virtual environment with Detectron2 and PyTorch. (We used CUDA 11.1 for the paper)

# Set up virtual environment
python3 -m venv ~/venvs/tbbrdet_det2
. ~/venvs/tbbrdet_det2/bin/activate

# Install dependencies
pip install --upgrade pip
pip install torch==1.10.1+cu113 torchvision==0.11.2+cu113 torchaudio==0.10.1+cu113 -f https://download.pytorch.org/whl/torch_stable.html
python -m pip install detectron2 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu113/torch1.10/index.html

We have provided a requirements file, we recommend using that to set up the remainder of the environment correctly.

MMDetection)

Create a separate venv for this. In it install mmdetection according to the instructions on their repository (we used version 2.21.0), and also install tensorboard and mlflow.

pip install future tensorboard mlflow

Again, we provide a requirements file, though we recommend first setting up mmdetection and pytorch first and then installing the remaining requirements with the requirements file.

Using MMDetection pretrained models

To train with pretrained weights provided by MMDetection it's advised to download the corresponding weight files. You will find the name of the pretrained file used in our work within the corresponding config file in config/mmdet/. You can then find the file within the MMDetection github repository configs/ directory of that model.

TBBR Dataset

You can find the latest version of the TBBR dataset here on Zenodo: train: wget -r https://zenodo.org/record/4767772/files/Flug1_100-104Media_coco.json?download=1 test: wget -r https://zenodo.org/record/4767772/files/Flug1_105Media_coco.json?download=1

Usage

Paths to datasets/MLFlow/outputs/etc. need to be set within the config files. You can search for the string /path/to/ to know which to replace.

For library usage instructions, see either the MMDetection README or the Detectron2 README.

Structure

├── README.md                           <- This file
├── configs                             <- Directory for neural network and hyperparameter configurations
│   ├── detectron2                      <- Detectron2 experiment configs
│   └── mmdet                           <- MMDetection experiment configs
└── scripts                             <- Python and Slurm scripts for running experiments

License

This project is release under the BSD-3-Clause license

Acknowledgement

This work is supported by the Helmholtz Association Initiative and Networking Fund on the HAICORE@KIT partition. We thank Marinus Vogl and the Air Bavarian GmbH for their support with equipment and service for the recording of images. We also thank Tobias Beiersdörfer for support in the development of the TBBR dataset.