Typical Workflows

This chapter documents all user-facing parameters for the executable scripts. Most of the CLI parameters have default values that work just fine in many cases. However, sometimes it can be beneficial to tune them by hand according to the used dataset.

Regarding any data format including detections and grund truth annotations see Data In/Out.

---

Simple Workflow for Detection Only

The bird detection script can be run on individual audio files or in batch mode across an entire directory. Both approaches highly depend on default values for CLI parameters. For full control see Complete Workflow Script.

Select the tab below that matches your operating system.

Single File Processing

Linux / macOSWindows (PowerShell)Windows (CMD)

python src/inference/detect_birds.py \
    --audio path/to/recording.wav \
    --model models/best.pt \
    --species-mapping species_mapping

python src/inference/detect_birds.py `
    --audio path/to/recording.wav `
    --model models/best.pt `
    --species-mapping species_mapping

python src/inference/detect_birds.py ^
    --audio path/to/recording.wav ^
    --model models/best.pt ^
    --species-mapping species_mapping

Batch Directory Processing

Linux / macOSWindows (PowerShell)Windows (CMD)

python src/inference/detect_birds.py \
    --audio path/to/audio/folder \
    --model models/best.pt \
    --species-mapping species_mapping

python src/inference/detect_birds.py `
    --audio path/to/audio/folder `
    --model models/best.pt `
    --species-mapping species_mapping

python src/inference/detect_birds.py ^
    --audio path/to/audio/folder ^
    --model models/best.pt ^
    --species-mapping species_mapping

Possible Species Mapping Values

To see the full list of selectable species mappings see Audio and Model Inputs — Species mapping.

---

Simple Workflow for Detection & Evaluation

The following scripts show how to run inference and evaluate the results on given ground truth annoation data. They highly depend on default values for the CLI parameters. For full controll see Complete Workflow Script.

Note: The F-beta score analysis has to be run before any merging because it handles merging itself. At each confidence threshold it computes the corresponding merging individually.

Steps 2–4 use default paths under results/. Step 1 with --no-merge writes results/raw_detections.json (or results/run_N/raw_detections.json if you re-run while results/ already has outputs). Later steps follow results/.active_run and resolve raw_detections.json / simplified.csv automatically.

Linux / macOSWindows (PowerShell)Windows (CMD)

# Step 1: Run inference with low confidence and --no-merge to get raw detections
python src/inference/detect_birds.py \
    --audio path/to/audio/folder \
    --model models/model_name.pt \
    --species-mapping mapping_name \
    --output-path results \
    --conf 0.001 \
    --no-merge \

# Step 2: Analyze F-beta scores to find optimal threshold
python src/evaluation/f_beta_score_analysis.py \
    --labels path/to/labels.csv \

# Step 3: Filter raw detections to optimal threshold and merge
python src/evaluation/filter_and_merge_detections.py

# Step 4: Generate confusion matrix
python src/evaluation/confusion_matrix_analysis.py \
    --labels path/to/labels.csv \

# Step 5: Examine results in results/ directory

# Step 1: Run inference with low confidence and --no-merge to get raw detections
python src/inference/detect_birds.py `
    --audio path/to/audio/folder `
    --model models/model_name.pt `
    --species-mapping mapping_name `
    --output-path results `
    --conf 0.001 `
    --no-merge `

# Step 2: Analyze F-beta scores to find optimal threshold
python src/evaluation/f_beta_score_analysis.py `
    --labels path/to/labels.csv `

# Step 3: Filter raw detections to optimal threshold and merge
python src/evaluation/filter_and_merge_detections.py

# Step 4: Generate confusion matrix
python src/evaluation/confusion_matrix_analysis.py `
    --labels path/to/labels.csv `

# Step 5: Examine results in results/ directory

rem Step 1: Run inference with low confidence and --no-merge to get raw detections
python src/inference/detect_birds.py ^
    --audio path/to/audio/folder ^
    --model models/model_name.pt ^
    --species-mapping mapping_name ^
    --output-path results ^
    --conf 0.001 ^
    --no-merge ^

rem Step 2: Analyze F-beta scores to find optimal threshold
python src/evaluation/f_beta_score_analysis.py ^
    --labels path/to/labels.csv ^

rem Step 3: Filter raw detections to optimal threshold and merge
python src/evaluation/filter_and_merge_detections.py

rem Step 4: Generate confusion matrix
python src/evaluation/confusion_matrix_analysis.py ^
    --labels path/to/labels.csv ^

rem Step 5: Examine results in results/ directory

Complete Workflow Script

The following scripts offer the most elaborate and precise way of interacting with BirdBox. They mirror the already present scripts at the repository root.

Feel free to adapt any parameter to your use-case. For a detailed description of each CLI parameter see the other sections in this chapter. There exists one for each Python file.

Linux / macOSWindows (PowerShell)Windows (CMD)

#!/bin/bash

# Exit immediately if a command fails
set -e

# Optional: activate local virtual environment if present.
# If .venv does not exist, the script uses the current Python on PATH.
if [ -f ".venv/bin/activate" ]; then
    echo "Activating .venv"
    # shellcheck source=/dev/null
    source ".venv/bin/activate"
fi


#### select the dataset on which inference shall be performed #####
# DATASET_NAME="All-In-One_testset"
# DATASET_NAME="Western-US"
# DATASET_NAME="Hawaii_testset"
DATASET_NAME="Northeastern-US_testset-subset"


#### select model #####
MODEL_PATH="models/${DATASET_NAME/_testset-subset/}.pt"
# MODEL_PATH="models/Just-Bird.pt"
# MODEL_PATH="models/All-In-One-Transfer.pt"


#### select the species mapping (according to dataset and model) #####
SPECIES_MAPPING="${DATASET_NAME/_testset-subset/}"
# SPECIES_MAPPING="Just-Bird"
# SPECIES_MAPPING="All-In-One"


#### toggle single class mode #####
# USE_SINGLE_CLS=true
USE_SINGLE_CLS=false


#### select output path #####
OUTPUT_PATH="results/${DATASET_NAME/_testset-subset/}"
# OUTPUT_PATH="results/Just-Bird"
# OUTPUT_PATH="results/All-In-One-Transfer"


#########################################################################
# The most important parameters are already set via the script variables.
# But details like IoU threshold, 
# song gap threshold, etc. can be changed below this heading.
# To skip entire steps (for instance the confusion matrix) 
# just uncomment the respective lines.
#########################################################################


SINGLE_CLS_FLAG=()
if [ "${USE_SINGLE_CLS}" = true ]; then
    SINGLE_CLS_FLAG+=(--single-cls)
fi

# Step 1: Run inference with low confidence and 
# --no-merge to get raw (unmerged) detections.
# This matches the filter-then-merge policy when 
# later filtering at each confidence threshold.
echo "Running inference (raw detections, no merge)..."
python src/inference/detect_birds.py \
    --audio "datasets/${DATASET_NAME}/soundscape_data" \
    --model "${MODEL_PATH}" \
    --species-mapping "${SPECIES_MAPPING}" \
    --output-path "${OUTPUT_PATH}" \
    --output-format json-with-algorithm-metadata \
    --conf 0.001 \
    --no-merge \
    --nms-iou 0.8 \
    --workers 4


# Step 2: F-beta analysis on raw detections: 
# at each confidence threshold we filter then merge.
echo "Running F-beta score analysis (filter-then-merge per threshold)..."
python src/evaluation/f_beta_score_analysis.py \
    --raw-detections "${OUTPUT_PATH}" \
    --labels "datasets/${DATASET_NAME}/annotations.csv" \
    --output-path "${OUTPUT_PATH}/f_1.0_score_analysis" \
    --beta 1.0 \
    --iou-threshold 0.25 \
    --song-gap 0.1 \
    --num-workers 8 \
    "${SINGLE_CLS_FLAG[@]}"


# Step 3: From raw detections, filter at conf=0.25 and merge.
echo "Filtering raw detections at conf=0.25 and merging for confusion matrix..."
python src/evaluation/filter_and_merge_detections.py \
    --raw-detections "${OUTPUT_PATH}" \
    --output-path "${OUTPUT_PATH}" \
    --output-format all \
    --conf 0.2 \
    --song-gap 0.1


# # Step 4: Run confusion matrix analysis.
echo "Running confusion matrix analysis..."
python src/evaluation/confusion_matrix_analysis.py \
    --detections "${OUTPUT_PATH}" \
    --labels "datasets/${DATASET_NAME}/annotations.csv" \
    --output-path "${OUTPUT_PATH}/confusion_matrix_analysis" \
    --iou-threshold 0.25 \
    "${SINGLE_CLS_FLAG[@]}"


# Step 5: Examine results in results/ directory
echo
echo "All tasks completed!"
echo "Results can now be examined in the ${OUTPUT_PATH} directory."

#Requires -Version 5.1
$ErrorActionPreference = "Stop"

# Optional: activate local virtual environment if present.
# If .venv is missing, script uses python from PATH.
if (Test-Path ".venv\Scripts\Activate.ps1") {
    Write-Host "Activating .venv"
    . ".venv\Scripts\Activate.ps1"
}


#### select the dataset on which inference shall be performed #####
# $DATASET_NAME = "All-In-One_testset"
# $DATASET_NAME = "Western-US"
# $DATASET_NAME = "Hawaii_testset"
$DATASET_NAME = "Northeastern-US_testset-subset"


#### select model #####
$DATASET_BASE = $DATASET_NAME -replace "_testset-subset", ""
$MODEL_PATH = "models\$DATASET_BASE.pt"
# $MODEL_PATH = "models\Just-Bird.pt"
# $MODEL_PATH = "models\All-In-One-Transfer.pt"


#### select the species mapping (according to dataset and model) #####
$SPECIES_MAPPING = $DATASET_BASE
# $SPECIES_MAPPING = "Just-Bird"
# $SPECIES_MAPPING = "All-In-One"


#### toggle single class mode #####
# $USE_SINGLE_CLS = $true
$USE_SINGLE_CLS = $false


#### select output path #####
$OUTPUT_PATH = "results\$DATASET_BASE"
# $OUTPUT_PATH = "results\Just-Bird"
# $OUTPUT_PATH = "results\All-In-One-Transfer"


#########################################################################
# The most important parameters are already set via the script variables.
# But details like IoU threshold,
# song gap threshold, etc. can be changed below this heading.
# To skip entire steps (for instance the confusion matrix)
# just uncomment the respective lines.
#########################################################################


$SINGLE_CLS_FLAG = @()
if ($USE_SINGLE_CLS) { $SINGLE_CLS_FLAG = @("--single-cls") }

# Step 1: Run inference with low confidence and
# --no-merge to get raw (unmerged) detections.
# This matches the filter-then-merge policy when
# later filtering at each confidence threshold.
Write-Host "Running inference (raw detections, no merge)..."
python src\inference\detect_birds.py `
    --audio "datasets\$DATASET_NAME\soundscape_data" `
    --model "$MODEL_PATH" `
    --species-mapping "$SPECIES_MAPPING" `
    --output-path "$OUTPUT_PATH" `
    --output-format json-with-algorithm-metadata `
    --conf 0.001 `
    --no-merge `
    --nms-iou 0.8 `
    --workers 4


# Step 2: F-beta analysis on raw detections:
# at each confidence threshold we filter then merge.
Write-Host "Running F-beta score analysis (filter-then-merge per threshold)..."
python src\evaluation\f_beta_score_analysis.py `
    --raw-detections "$OUTPUT_PATH" `
    --labels "datasets\$DATASET_NAME\annotations.csv" `
    --output-path "$OUTPUT_PATH\f_1.0_score_analysis" `
    --beta 1.0 `
    --iou-threshold 0.25 `
    --song-gap 0.1 `
    --num-workers 8 `
    @SINGLE_CLS_FLAG


# Step 3: From raw detections, filter at conf=0.25 and merge.
Write-Host "Filtering raw detections at conf=0.25 and merging for confusion matrix..."
python src\evaluation\filter_and_merge_detections.py `
    --raw-detections "$OUTPUT_PATH" `
    --output-path "$OUTPUT_PATH" `
    --output-format all `
    --conf 0.2 `
    --song-gap 0.1


# # Step 4: Run confusion matrix analysis.
Write-Host "Running confusion matrix analysis..."
python src\evaluation\confusion_matrix_analysis.py `
    --detections "$OUTPUT_PATH" `
    --labels "datasets\$DATASET_NAME\annotations.csv" `
    --output-path "$OUTPUT_PATH\confusion_matrix_analysis" `
    --iou-threshold 0.25 `
    @SINGLE_CLS_FLAG


# Step 5: Examine results in results\ directory
Write-Host ""
Write-Host "All tasks completed!"
Write-Host "Results can now be examined in the $OUTPUT_PATH directory."

@echo off
setlocal EnableExtensions EnableDelayedExpansion

REM Exit if any command fails
set "FAILED=0"

REM Optional: activate local virtual environment if present.
REM If .venv is missing, script uses python from PATH.
if exist ".venv\Scripts\activate.bat" (
    echo Activating .venv
    call ".venv\Scripts\activate.bat"
    if errorlevel 1 goto :fail
)


REM #### select the dataset on which inference shall be performed #####
REM set "DATASET_NAME=All-In-One_testset"
REM set "DATASET_NAME=Western-US"
REM set "DATASET_NAME=Hawaii_testset"
set "DATASET_NAME=Northeastern-US_testset-subset"


REM #### select model #####
set "DATASET_BASE=%DATASET_NAME:_testset-subset=%"
set "MODEL_PATH=models\%DATASET_BASE%.pt"
REM set "MODEL_PATH=models\Just-Bird.pt"
REM set "MODEL_PATH=models\All-In-One-Transfer.pt"


REM #### select the species mapping (according to dataset and model) #####
set "SPECIES_MAPPING=%DATASET_BASE%"
REM set "SPECIES_MAPPING=Just-Bird"
REM set "SPECIES_MAPPING=All-In-One"


REM #### toggle single class mode #####
REM set "USE_SINGLE_CLS=true"
set "USE_SINGLE_CLS=false"


REM #### select output path #####
set "OUTPUT_PATH=results\%DATASET_BASE%"
REM set "OUTPUT_PATH=results\Just-Bird"
REM set "OUTPUT_PATH=results\All-In-One-Transfer"


REM #########################################################################
REM # The most important parameters are already set via the script variables.
REM # But details like IoU threshold,
REM # song gap threshold, etc. can be changed below this heading.
REM # To skip entire steps (for instance the confusion matrix)
REM # just uncomment the respective lines.
REM #########################################################################


set "SINGLE_CLS_FLAG="
if /I "%USE_SINGLE_CLS%"=="true" set "SINGLE_CLS_FLAG=--single-cls"

REM Step 1: Run inference with low confidence and
REM --no-merge to get raw (unmerged) detections.
REM This matches the filter-then-merge policy when
REM later filtering at each confidence threshold.
echo Running inference (raw detections, no merge)...
python src\inference\detect_birds.py ^
    --audio "datasets\%DATASET_NAME%\soundscape_data" ^
    --model "%MODEL_PATH%" ^
    --species-mapping "%SPECIES_MAPPING%" ^
    --output-path "%OUTPUT_PATH%" ^
    --output-format json-with-algorithm-metadata ^
    --conf 0.001 ^
    --no-merge ^
    --nms-iou 0.8 ^
    --workers 4
if errorlevel 1 goto :fail


REM Step 2: F-beta analysis on raw detections:
REM at each confidence threshold we filter then merge.
echo Running F-beta score analysis (filter-then-merge per threshold)...
python src\evaluation\f_beta_score_analysis.py ^
    --raw-detections "%OUTPUT_PATH%" ^
    --labels "datasets\%DATASET_NAME%\annotations.csv" ^
    --output-path "%OUTPUT_PATH%\f_1.0_score_analysis" ^
    --beta 1.0 ^
    --iou-threshold 0.25 ^
    --song-gap 0.1 ^
    --num-workers 8 ^
    %SINGLE_CLS_FLAG%
if errorlevel 1 goto :fail


REM Step 3: From raw detections, filter at conf=0.25 and merge.
echo Filtering raw detections at conf=0.25 and merging for confusion matrix...
python src\evaluation\filter_and_merge_detections.py ^
    --raw-detections "%OUTPUT_PATH%" ^
    --output-path "%OUTPUT_PATH%" ^
    --output-format all ^
    --conf 0.2 ^
    --song-gap 0.1
if errorlevel 1 goto :fail


REM REM Step 4: Run confusion matrix analysis.
echo Running confusion matrix analysis...
python src\evaluation\confusion_matrix_analysis.py ^
    --detections "%OUTPUT_PATH%" ^
    --labels "datasets\%DATASET_NAME%\annotations.csv" ^
    --output-path "%OUTPUT_PATH%\confusion_matrix_analysis" ^
    --iou-threshold 0.25 ^
    %SINGLE_CLS_FLAG%
if errorlevel 1 goto :fail


REM Step 5: Examine results in results\ directory
echo.
echo All tasks completed!
echo Results can now be examined in the %OUTPUT_PATH% directory.
exit /b 0

:fail
echo.
echo Pipeline failed. See the error above.
exit /b 1