Just Bird Model - Zero Shot Transfer¶
The Just Bird model is unique in that testing was not conducted on the same dataset used for training and validation. We can therefore call this process zero-shot object detection [2].
The Just Bird model has been trained and evaluated on:
- Southwestern Amazon Basin
- Island of Hawai'i
- Northeastern United States
- Southern Sierra Nevada mountain range
The testing and therefore the computation of the final metrics was conducted on:
Precision, Recall and F1-Score¶
The graphs below show the performance of the model on the testset under different confidence thresholds. The measured metrics are precision, recall and the F1-score. If we want to optimize the F1-score for the model, one should thus pick a confidence-threshold-value around 0.3.
Rising Recall at Low Confidence
The rising recall from confidence values 0 to ~0.1 is unusual but by design. At very low confidence thresholds, the merging algorithm produces imprecise merged boxes that can accidentally overlap more ground truth labels. See How it works for details.
Training Results¶
The following figure illustrates various data that has been recorded during training. The metrics have been computed after each epoch with the evaluation split of the dataset.
Model Checkpoint
The weights that produced the highest metrics/mAP50-95 on the validation split were selected for the final model.
Species Distribution Across Splits¶
The following table shows the amount of annotations in total and for each species as described in Dataset Splits.
| Species | Train | Val | Test | Total | 70/15/15 Quality |
|---|---|---|---|---|---|
| bird | 310,271 | 137,864 | 79,629 | 527,764 | 58.8/26.1/15.1 |
References¶
[2] Bansal, A., Sikka, K., et al. (2018). "Zero-shot object detection." Proceedings of the European Conference on Computer Vision (ECCV), 384–400.