Adding Models to the Demo¶
The model menu lists the models in docs/models/manifest.json; the first one is the default. The instant sliders need the graph's per-window boxes (raw_all) and a JavaScript port of its merge: docs/javascripts/demo/fusion.js for window fusion (fusion, the export's default for Just-Bird) and docs/javascripts/demo/songs.js for song reconstruction (reconstruct, the default for Birds-With-Labels). The Just-Bird frequency gate (--merge reconstruct with Just-Bird) has no port; such entries are skipped.
-
Export the model.
For another model, add
--pt-model, for example--pt-model models/Birds-With-Labels_fp32.pt --output-path docs/models/Birds-With-Labels_fp16.onnx.--precision fp16stores the network weights in fp16, which halves the file (20 MB instead of 38 MB for Just-Bird, 52 MB instead of about 100 MB for Birds-With-Labels). Feature extraction and merging stay in fp32. On 13 test recordings (tests/onnx_graph_parity.py), the fp16 graphs reproduced the boxes ofdetect_birds.pywith the.ptmodel: 99.8% for Just-Bird (mean IoU 0.995) and 99.6% for Birds-With-Labels, with the same species (mean IoU 0.997). The export prints the merge it baked in (Merge (baked in)) and the suggested defaults (Default conf,Default song_gap), which it also stores in the ONNX metadata. -
Place the
.onnxfile underdocs/models/if the export path was elsewhere. -
Add an entry to
docs/models/manifest.json, first in the list if the demo should start with it.{ "models": [ { "file": "Just-Bird_fp16.onnx", "label": "Just-Bird_fp16.onnx (recommended)", "merge_method": "fusion", "conf": 0.15, "phrase_gap": 0.05, "species_mapping": "Just-Bird", "names": { "0": "bird" } }, { "file": "Birds-With-Labels_fp16.onnx", "label": "Birds-With-Labels_fp16.onnx", "merge_method": "reconstruct", "conf": 0.25, "song_gap": 0.1, "species_mapping": "Birds-With-Labels", "names": { "0": "????", "1": "amabaw1", "…": "…" }, "common_names": { "0": "Unknown", "1": "Amazonian Barred-Woodcreeper", "…": "…" } } ] }Field Meaning fileThe ONNX file, relative to docs/models/labelName in the model menu merge_methodfusionorreconstruct, as printed by the export. Other entries are skipped.confWhere the confidence slider starts: usually the printed Default conf(0.15 for Just-Bird, 0.2 for the other models). Birds-With-Labels starts at 0.25.phrase_gapfusiononly: where the phrase gap slider starts, the printedDefault song_gap(0.05 for Just-Bird, 0 for Birds-With-Labels)song_gapreconstructonly: where the song gap slider starts, the printedDefault song_gap(0.1)species_mappingWritten to model_configof the exported JSONnameseBird codes by class index, for the table and the exports (ONNX Runtime Web does not expose ONNX metadata) common_namesOptional: common names by class index, shown in the table instead of the codes For a species model,
namesandcommon_namescome fromid_to_ebirdandebird_to_nameof its mapping insrc/config.py. With more than one class, the table gets a species column. -
Check that the demo's merge still matches the graph:
The script compares
fusion.jsorsongs.js, whichever the manifest entry'smerge_methodnames, with the graph's own detections for 24 slider settings (16 for song reconstruction) ontests/test.wavand fails on any difference. Run it after exporting a new model or changing a merge insrc/inference/onnx_export.py.npm testruns all demo tests for every model in the manifest: the merge, the chunked and streamed runs, the boxes while recording, which boxes are dashed. They run in parallel processes, each on one ONNX Runtime thread like the demo, so the rows are the browser's. That takes about 10 minutes with 14 processes on a 20-core machine (--jobs=Nsets how many run at once), against about 45 minutes one after another (npm run test:serial).