Enhancing Bermuda tracking with the Bayesian integration

[jackjourneyman]

TL;DR

Room presence detection tracks devices, not people. You can improve results by using the Bayesian integration to identify the room an individual is most likely to be in.

The idea

Not many people have their phones with them every minute of the day. They're forgotten, left on charge, set aside when you need both hands free. This limits the effectiveness of any room presence integration and Bermuda is no exception. It tracks devices, not people.

The Bayesian integration monitors the state of multiple sensors and calculates the probability of something being true. If you set up a Bayesian sensor for each room, Home Assistant can estimate that while there is a chance that Ashley will be in the kitchen (because his phone is there), other sensors show that the chances of his being in the living room are greater.

It sounds flaky, but if you monitor enough sensors, it's remarkably accurate.

The Bayesian integration can:

• Monitor multiple Bermuda devices for each person - phone, watch, keys, wallet (anything with a Bluetooth tag) - the more the better. Devices do not have to be with the person all the time.
• It smooths out the "juddering" that sometimes occurs when a device is on the boundary between two rooms.
• It detects presence on different floors more reliably.
• It can take into account other factors. Time of day, for example. At 4am Ashley is likely to be in the bedroom. Movement in the Kitchen means the chances of his being there are increased. If the Xbox is on, he's probably playing Destiny - and therefore in the study.

Notes:

This post is not a Blueprint or a Tutorial. Both Bermuda and the Bayesian integrations need quite a lot of fine tuning, and every implementation is going to be different. Think of it as a rattlebag of suggestions.

• It assumes that Bermuda is set up and working, with plenty of proxies, properly calibrated (although the idea should work with any tracker integration).
• It assumes that you have looked through the Bayesian documentation.

Examples are based on the case of a single person living in a small house, with a single Bayesian sensor for each room. A larger houshold would be more complex, with a sensor for each person for each room, but the principle should be the same.

The Bayesian integration

The Bayesian integration tests a statement like "Ashley is in the living room" against a set of sensors and generates a binary sensor - true or false.

The binary entity is not particularly useful in this application, but it has an attribute probability which is assigned a number between 0 (false) and 1 (true). (I personally use a template to convert this to a percentage, which I find easier to read at a glance). So if you have a Bayesian sensor for each room, the one with the highest probability is the most likely room.

This not simply a matter of adding up the number of sensors - the value of each "observation" is weighted so that this...

...correctly produces this:

In this example there are four devices tracked by Bermuda:

• Keys, which have a Bluetooth tag attached. These are usually left on the kitchen counter when Ashley comes into the house, so they have a low value when calculating his probable location.
• Wallet, also with a Bluetooth tag attached. Ashley normally carries this in his pocket, so it has a high value.
• Android phone, identified by its IRK. Normally, but not always, carried from room to room, so relatively high value.
• Watch, identified by an iBeacon generated by the HA app. Ashley wears this nearly all the time, but it sleeps for 50% of the day to conserve battery, so often unavailable.

In addition there are data from other sensors:

• Movement, detected by motion sensors - high value in the calculations.
• TV on (the TV is in the living room) - sometimes left on, so medium value.
• Speakers in use (there are speakers in every room) - also sometimes left on.
• Away mode on (triggered by alarm being set) - high value in detecting that Ashley is out, otherwise low.

For each of these sensors, the Bayesian integration needs an "observation", true or false, with an estimate of its likelihood. So for the living room:

    # Do not just copy this code - it's incomplete (see Bayesian documentation) 
    # and the probability values will be completely different for you.

  observations:
    # TV on ------------------------------------------------------------------------
    - platform: "state"
      entity_id: "binary_sensor.tv_on"
      to_state: "on"
    # Room is occupied when TV is turned on, but it may be left on
      prob_given_true: 0.5
      prob_given_false: 0.25

    # Movement in living room ------------------------------------------------------
    - platform: "state"
      entity_id: "sensor.last_movement"
      to_state: "Living room"
    # Allow for dog!
      prob_given_true: 0.9
      prob_given_false: 0.01

    # Wallet in living room --------------------------------------------------------
    - platform: "state"
      entity_id: "sensor.wallet_tag_area"
      to_state: "Living room"
    # Wallet may not be carried
      prob_given_true: 0.8
      prob_given_false: 0.01

    # Phone in living room ---------------------------------------------------------
    - platform: "state"
      entity_id: "sensor.nokia_area"
      to_state: "Living room"
    # Phone sometimes gets left behind
      prob_given_true: 0.8
      prob_given_false: 0.25

    # Watch in living room ---------------------------------------------------------
    - platform: "state"
      entity_id: "sensor.watch_area"
      to_state: "Living room"
    # Watch usually on wrist, except when charging
      prob_given_true: 0.8
      prob_given_false: 0.25

    # Away mode on -----------------------------------------------------------------
    - platform: "state"
      entity_id: "input_boolean.away_mode"
      to_state: "on"
    # Home when off
      prob_given_true: 0.01
      prob_given_false: 0.16

    # Keys in living room ----------------------------------------------------------
    - platform: "state"
      entity_id: "sensor.keys_tag_area"
      to_state: "Living room"
    # Usually left in kitchen
      prob_given_true: 0.17
      prob_given_false: 0.01

    # Speakers on ------------------------------------------------------------------
    - platform: "state"
      entity_id: "media_player.living_room"
      to_state: "playing"
    # Only occasionally on
      prob_given_true: 0.17
      prob_given_false: 0.01

Fortunately you don't have to work all these probabilities out. There is an interactive spreadsheet to help.

Bayesian Tester

When estimating times for this spreadsheet it is very important to use real data. Don't guess. Use HA history to check.

Useful templates

Not my strong point - these can certainly be improved on.

With a Bayesian sensor binary_sensor.probably_in_living_room...

To extract a probability (expressed as a percentage for ease of reading):

  - sensor:
      - name: "Living room probability"
        unit_of_measurement: "%"
        icon: mdi:progress-question
        state: '{{ state_attr("binary_sensor.probably_in_living_room", "probability") | float(0) * 100 }}'

To extract the highest probability from several rooms...

  - sensor:
      - name: "Highest probability"
        state: >
          {% set values = [
            states('sensor.bedroom_probability') | int,
            states('sensor.study_probability') | int,
            states('sensor.kitchen_probability') | int,
            states('sensor.living_room_probability') | int,
            states('sensor.landing_probability') | int,
            states('sensor.hallway_probability') | int
          ] %}
          {{ values | max }}

To extract the name of the most likely room...

  - sensor:
      - name: "Most likely room"
        icon: mdi:map-marker-radius
        state: >
          {% if states('binary_sensor.probably_occupied') == 'off' %}
              Away 
          {% elif states('sensor.highest_probability') | float(0) == 0 %}
              Unknown
          {% elif states('sensor.living_room_probability') | float(0) == states('sensor.highest_probability') | float(0) %}
              Living room
          {% elif states('sensor.bedroom_probability') | float(0) == states('sensor.highest_probability') | float(0) %}
              Bedroom
          {% elif states('sensor.study_probability') | float(0) == states('sensor.highest_probability') | float(0) %}
              Study
          {% elif states('sensor.kitchen_probability') | float(0) == states('sensor.highest_probability') | float(0) %}
              Kitchen
          {% elif states('sensor.landing_probability') | float(0) == states('sensor.highest_probability') | float(0) %}
              Landing
          {% elif states('sensor.hallway_probability') | float(0) == states('sensor.highest_probability') | float(0) %}
              Hallway
          {% else %}
              Unknown
          {% endif %}

[agittins]

This is a great write-up, thanks! I can assure you though that I'm far more likely to be in the kitchen than the bedroom at 4am! 🤣

[oulianxian]

good

[donutsoft]

Relating to this, I've been working on integrating other machine learning models with Home Assistant specifically to improve Bermuda, and give it the ability to detect my phone in rooms that do not currently have BLE sensors.
https://github.com/donutsoft/ml2mqtt

Currently it supports Random Forest and KNN, but I'll be adding a few more algorithms soon. No mathematics or in-depth knowledge of machine learning algorithms required. It's pretty much entirely just point and click.

I'd appreciate any feedback :-)

[jackjourneyman]

Thanks for the link.

For anyone coming to fresh to this, it may be useful to understand how Bayesian probability is different from machine learning or AI. In the HA integration at least, it's just a calculation that's repeated every time a sensor value changes. There's no database and system doesn't "learn" anything. In statistical terms its output is based on reasonable predictions of what will be true or false (the "observations") rather than historical data.

It's been around for a long time (Thomas Bayes was born in 1702).

https://en.wikipedia.org/wiki/Bayesian_probability

[donutsoft]

Yup, there are many different ways of skinning a cat, and there are definitely going to be cases where manually specifying these probabilities may result in a better outcome.