Does audio recognition for siren, civil_defence_siren and explosions even work?

[denysdovhan]

I live in Kyiv, Ukraine and air alert siren happens like 5 times a day. Explosions happen regularly during missile attack as well.

I am using Frigate since July, and I've setup audio detection like this:

audio:
  enabled: true
  listen:
    - fire_alarm
    - scream
    - speech
    - siren
    - civil_defense_siren
    - explosion
    - gunshot

It didn't detect any of these types (besides speech) not even a single time.

My guess: maybe that's because I use indoor cameras and their mics are not sensitive enough. However, I hear siren on the recordings. Opening windows in the summer didn't help either.

Am I doing it wrong?

[inkeep-ai-assistant[bot]]

---

Your configuration looks correct, but there are several factors that could explain why you're not detecting sirens and explosions:

1. Volume Threshold

Frigate uses volume levels to determine when to run audio detection(1). The default min_volume is 500(2)(3). If your camera's microphone isn't picking up the sounds at sufficient volume, detection won't trigger even if the audio type matches your listen list(1).

You can adjust this threshold:

audio:
  enabled: true
  min_volume: 200  # Lower threshold for higher sensitivity
  listen:
    - siren
    - civil_defense_siren
    - explosion

(1)(2)

As a rule of thumb: 200 is high sensitivity, 500 is medium sensitivity, and 1000 is low sensitivity(2)(3).

2. Detection Confidence Threshold

Each audio label has a minimum score threshold (default 0.8) that must be met to trigger an event(2)(3). You can lower this per label:

audio:
  enabled: true
  listen:
    - siren
    - civil_defense_siren
    - explosion
  filters:
    siren:
      threshold: 0.6
    civil_defense_siren:
      threshold: 0.6
    explosion:
      threshold: 0.6

(2)(3)

3. Model Limitations

Frigate uses the YamNet audio detection model(4), which has over 500 different audio types available(1). However, the model's training data may not include the specific characteristics of Ukrainian civil defense sirens or the acoustic signatures of explosions in your environment. The model detects everything in one go and then checks against your listen list(5), but if the sounds don't match the model's learned patterns closely enough, they won't be detected.

4. Debugging Steps

• Check MQTT topics frigate/<camera_name>/audio/dBFS and frigate/<camera_name>/audio/rms to see actual volume levels during siren events(5)
• Verify your audio stream is working by confirming speech detection works consistently
• Try significantly lowering min_volume (to 200 or even lower) during a known siren event
• Lower the threshold values for your specific labels

If you continue having issues after adjusting these settings, the model may simply not recognize the specific acoustic patterns of your local sirens and explosions well enough, which is a limitation of the pre-trained model rather than your configuration.

📚 Sources:

1. Audio Detectors
2. Full Reference Config
3. Full Reference Config
4. Adding drone sound recognition functionality.
5. Audio detection question: how to trigger on any noise?

---

Have another question? Just tag @inkeep.

[hawkeye217]

Another point to add to the AI's response - IP camera microphones are typically not high quality, high fidelity devices, which also adds to the issue.

Nonetheless, I'd definitely try lowering your thresholds as suggested by the AI.

[denysdovhan]

Thanks! I'll try that and see if it helps.

[mattenz]

Are there examples of what the siren and civil_defense_siren are trained on? I'm wondering if it would detect the rural fire sirens we have here in New Zealand - most European's that visit mistake them for an air raid siren

[hawkeye217]

Here is the YAMNet audio detection model that Frigate uses: https://www.tensorflow.org/hub/tutorials/yamnet

It's trained on this data set: https://research.google.com/audioset/