AMMOD: Automated Multisensor station for Monitoring Of species Diversity

Research project funded by BMBF

Team: Paul Bodesheim, Dimitri Korsch, Daphne Auer, Julia Böhlke

Time period: 2019 to 2022 (with a one-year extension to 2023)

Joint research project: building weather stations for biodiversity

Climate change is affecting ecosystems all over the world, causing irreversible loss of biodiversity. Since humans are the main contributors to the situation, it is our responsibility to closely monitor the effects of climate change in order to take action for the living beings in our environment in a targeted way. By combining multiple sensors at a single site, we will be able to monitor the whole diversity of flora and fauna in its surrounding, similar to weather stations that record multiple measurements for a local area.

The goal of all AMMOD project partners is therefore to monitor biological diversity in Germany using new technologies. To this end, prototypes for so-called AMMOD stations are to be set up in German forests within the 3-years project period. These are equipped with sensors for recording animal sounds and plant emissions, with animal cameras for birds, mammals and insects, and with automated insect and pollen sample collectors for monitoring by DNA barcoding. These recordings will be used to generate a solid data pool that will enable the analysis of change and possible trends in species richness and populations. For instance, we want to use the detected species to determine their population and record the change over a long period of time.

Find more information on the project homepage (offline as of 2024) of the joint research project. It has been funded by the Federal Ministry of Education and Research (Bundesministerium für Bildung und Forschung).

Our subprojects for visAMMOD to enable visual monitoring of moths and mammals

The visual monitoring subsystems of AMMOD (visAMMOD) consist of automated insect cameras for collecting images of nocturnal moths, which are called moth scanner, and conventional wildlife camera traps for recording observations of birds and mammals. Experimental stereo camera setups are tested by colleagues from Bonn University.

Our tasks in visAMMOD are the development of algorithms for insect detection and species identification in images provided by the moth scanner, as well as the continuous integration of new camera trap observations in a species identification model, including components for novelty detection, lifelong learning and taxonomic classification.

Moth Scanner Teaser
Moth Scanner

Dimitri Korsch

One camera, the so-called SpeciesMothCam, will take high-resolution images of insects that rest on an illuminated screen at regular intervals. The goal of monitoring is not to detect every species, but to describe trends in local insect populations. Seasonal activity differences of single species, the correlation with weather conditions and with vegetation phenology are important for ecological analyses. The SpeciesMothCam will deliver this type of data. The aim is to implement methods that enable an efficient detection of regions of interest (ROIs) on the screen and a classification of the species in this fine-grained setting. Taxonomic experts will help in the confirmation of the correct identification.

Moth Scanner Teaser
Novelty Detection, Life-long Learning and Classification

Daphne Auer, Julia Böhlke

Another camera, the so-called SpeciesSiteCam, will capture stereo images and videos mainly of terrestrial animals (mammals) using motion sensors. To enable image acquisition during night and twilight, infrared illumination is switched on. Our goal is to build robust classification models that can identify different species. Since there are only a few annotated training image databases, one goal is to compile a first workable system that exploits the image data present in the World Wide Web. Julia Böhlke has written her Bachelor thesis on this topic which is also summarized in a paper. Active learning will play an important role to integrate unlabeled sample sets. Furthermore, a core element of AMMOD is novelty detection, an alerting mechanism in case that a so far unknown species has been recorded by the monitoring devices.

Publications

2022

Dimitri Korsch, Paul Bodesheim, Gunnar Brehm, Joachim Denzler:
Automated Visual Monitoring of Nocturnal Insects with Light-based Camera Traps.
CVPR Workshop on Fine-grained Visual Classification (CVPR-WS). 2022.
[bibtex] [pdf] [web] [code] [abstract]

J. Wolfgang Wägele, Paul Bodesheim, Sarah J. Bourlat, Joachim Denzler, Michael Diepenbroek, Vera Fonseca, Karl-Heinz Frommolt, Matthias F. Geiger, Birgit Gemeinholzer, Frank Oliver Glöckner, Timm Haucke, Ameli Kirse, Alexander Kölpin, Ivaylo Kostadinov, Hjalmar S. Kühl, Frank Kurth, Mario Lasseck, Sascha Liedke, Florian Losch, Sandra Müller, Natalia Petrovskaya, Krzysztof Piotrowski, Bernd Radig, Christoph Scherber, Lukas Schoppmann, Jan Schulz, Volker Steinhage, Georg F. Tschan, Wolfgang Vautz, Domenico Velotto, Maximilian Weigend, Stefan Wildermann:
Towards a multisensor station for automated biodiversity monitoring.
Basic and Applied Ecology. 59 : pp. 105-138. 2022.
[bibtex] [pdf] [web] [doi] [abstract]

Rapid changes of the biosphere observed in recent years are caused by both small and large scale drivers, like shifts in temperature, transformations in land-use, or changes in the energy budget of systems. While the latter processes are easily quantifiable, documentation of the loss of biodiversity and community structure is more difficult. Changes in organismal abundance and diversity are barely documented. Censuses of species are usually fragmentary and inferred by often spatially, temporally and ecologically unsatisfactory simple species lists for individual study sites. Thus, detrimental global processes and their drivers often remain unrevealed. A major impediment to monitoring species diversity is the lack of human taxonomic expertise that is implicitly required for large-scale and fine-grained assessments. Another is the large amount of personnel and associated costs needed to cover large scales, or the inaccessibility of remote but nonetheless affected areas. To overcome these limitations we propose a network of Automated Multisensor stations for Monitoring of species Diversity (AMMODs) to pave the way for a new generation of biodiversity assessment centers. This network combines cutting-edge technologies with biodiversity informatics and expert systems that conserve expert knowledge. Each AMMOD station combines autonomous samplers for insects, pollen and spores, audio recorders for vocalizing animals, sensors for volatile organic compounds emitted by plants (pVOCs) and camera traps for mammals and small invertebrates. AMMODs are largely self-containing and have the ability to pre-process data (e.g. for noise filtering) prior to transmission to receiver stations for storage, integration and analyses. Installation on sites that are difficult to access require a sophisticated and challenging system design with optimum balance between power requirements, bandwidth for data transmission, required service, and operation under all environmental conditions for years. An important prerequisite for automated species identification are databases of DNA barcodes, animal sounds, for pVOCs, and images used as training data for automated species identification. AMMOD stations thus become a key component to advance the field of biodiversity monitoring for research and policy by delivering biodiversity data at an unprecedented spatial and temporal resolution.

2021

Bernd Radig, Paul Bodesheim, Dimitri Korsch, Joachim Denzler, Timm Haucke, Morris Klasen, Volker Steinhage:
Automated Visual Large Scale Monitoring of Faunal Biodiversity.
Pattern Recognition and Image Analysis. Advances in Mathematical Theory and Applications (PRIA). 31 (3) : pp. 477-488. 2021.
[bibtex] [pdf] [web] [doi] [abstract]

To observe biodiversity, the variety of plant and animal life in the world or in a particular habitat, human observers make the most common examinations, often assisted by technical equipment. Measuring objectively the number of different species of animals, plants, fungi, and microbes that make up the ecosystem can be difficult. In order to monitor changes in biodiversity, data have to be compared across space and time. Cameras are an essential sensor to determine the species range, abundance, and behavior of animals. The millions of recordings from camera traps set up in natural environments can no longer be analyzed by biologists. We started research on doing this analysis automatically without human interaction. The focus of our present sensor is on image capture of wildlife and moths. Special hardware elements for the detection of different species are designed, implemented, tested, and improved, as well as the algorithms for classification and counting of samples from images and image sequences, e.g., to calculate presence, absence, and abundance values or the duration of characteristic activities related to the spatial mobilities. For this purpose, we are developing stereo camera traps that allow spatial reconstruction of the observed animals. This allows three-dimensional coordinates to be recorded and the shape to be characterized. With this additional feature data, species identification and movement detection are facilitated. To classify and count moths, they are attracted to an illuminated screen, which is then photographed at intervals by a high-resolution color camera. To greatly reduce the volume of data, redundant elements and elements that are consistent from image to image are eliminated. All design decisions take into account that at remote sites and in fully autonomous operation, power supply on the one hand and possibilities for data exchange with central servers on the other hand are limited. Installation at hard-to-reach locations requires a sophisticated and demanding system design with an optimal balance between power requirements, bandwidth for data transmission, required service and operation in all environmental conditions for at least ten years.

Daphne Auer, Paul Bodesheim, Christian Fiderer, Marco Heurich, Joachim Denzler:
Minimizing the Annotation Effort for Detecting Wildlife in Camera Trap Images with Active Learning.
Computer Science for Biodiversity Workshop (CS4Biodiversity), INFORMATIK 2021. Pages 547-564. 2021.
[bibtex] [pdf] [doi] [abstract]

Dimitri Korsch, Paul Bodesheim, Joachim Denzler:
Deep Learning Pipeline for Automated Visual Moth Monitoring: Insect Localization and Species Classification.
INFORMATIK 2021, Computer Science for Biodiversity Workshop (CS4Biodiversity). Pages 443-460. 2021.
[bibtex] [pdf] [web] [doi] [code] [abstract]

Julia Böhlke, Dimitri Korsch, Paul Bodesheim, Joachim Denzler:
Exploiting Web Images for Moth Species Classification.
Computer Science for Biodiversity Workshop (CS4Biodiversity), INFORMATIK 2021. Pages 481-498. 2021.
[bibtex] [pdf] [web] [doi] [abstract]

Julia Böhlke, Dimitri Korsch, Paul Bodesheim, Joachim Denzler:
Lightweight Filtering of Noisy Web Data: Augmenting Fine-grained Datasets with Selected Internet Images.
International Conference on Computer Vision Theory and Applications (VISAPP). Pages 466-477. 2021.
[bibtex] [pdf] [web] [doi] [abstract]