Technical architecture of my digital home

This article describes the components that build up the architecture of my (digital) home.

Logical configuration (ArangoDB): the logical configuration includes the areas (rooms) in the house, the sensors and actors located in those areas, and the “scenes” that can be applied to actors and areas.
Use-case controllers (custom .NET Core worker services): I’ve build a separate micro service for each use-case, for example, controlling lights, planning washer, dishwasher, and dryer, and planning the generation of domestic hot water.
Digital Twin (Eclipse Ditto): the Digital Twin stores the state of all connected sensors and is used by the use-case controllers to consume sensor states or push commands back down to the sensors.
Messaging (Slack): I’m using Slack as a messaging service to interact with my home. Slack informs me on specific state changes in the Digital Twin and I can feed commands in Slack to influence the behavior of my home. I try to minimize this as most decisions should be fully automated.
Sensor services (custom .NET Core worker services): the sensor services read sensor states via open or proprietary protocols. They are also responsible for pushing commands down to actors.
Sensor history (InfuxDB): InfluxDB stores the (relevant) history of the Digital Twin as well as the history from the different energy services that feed directly into InfluxDB.
Sensor configuration (ArangoDB): ArangoDB stores the information needed to communicate with local and cloud-based sensors.
Visualisation (Grafana): I’m using Grafana as a visualisation front-end.

Visualization of the architecture of my digital home.

Minimizing grid consumption

Context

Roughly 20% of Dutch homes is equipped with solar panels. In Germany the adoption rate of solar panels is roughly 11%. This means that these homes produce power locally, but they hardly use it at the same time when it’s being produced. Only 30% of the produced energy is consumed directly. That means that 70% of the energy consumption is still consumed from the grid.

Metrics

Considering my own installation the numbers are even worse. Only ~20% of the locally produced (10mWh) energy is directly consumed (2Mwh). The real problem is that almost 80% of the consumed energy is consumed from the grid.

Vision

In 2026, three years from now, I want to reduce my grid consumption to 50% (on average, per year).

I have no idea if this is achievable, but it’s good to have a concrete goal. If I figure out I can achieve this in one year, I will increase my ambition.

In order to achieve this vision, I need to increase the direct consumption which will result in a decrease of overall grid consumption.

Steps along the way

To reach this vision, I need to understand at which times energy is already directly consumed and which consumers are causing this consumption. Additionally, I need to understand when energy is consumed from the grid and which consumers are causing this consumption. I’m hoping that, by influencing the consumption patterns of the largest consumers, I can make the first large step.

Year	Grid consumption	Direct consumption
2023 (today)	80%	20%
2024	60%	40%
2025	53%	47%
2026	50%	50%

Overview of yearly goals to reach my vision in 2027.

I expect that the first major improvements will already start paying off in 2024. I’m aiming for a 20% increase in 2024 over 2023. Then the real challenge will probably start.

First analysis of available data

In the first half of the year my system was still in development. After that is has become more stable, which resulted in a stable data collection since June.

When is direct consumption low?

At this moment I considered one interesting data points per week:

Direct consumption (%) = direct consumption / total production (solar);

The direct consumption (%) in red per week. It’s clearly visible that when the solar production goes down in the winter months, the direct consumption goes up.

Considering the available data I should start trying to focus on the summer months when solar production is high.

Which are the largest consumers?

Let’s have a look at the usual suspects fist: the car, heatpump, washer, and dishwasher.

The car takes roughly 35% of the yearly consumption. The (combined) heatpump follows with 28%.

Next steps

Considering my vision and the available data I should focus on moving grid consumption for the car and heatpump to direct consumption, especially in the summer months. I will share concrete objectives and key-results for this in a next post.

Calculating energy (in)dependence

One of the goals of building a digital twin of our house is to reduce our dependence on external energy sources. To achieve this goal, and to validate if my measures are having a positive effect, I need to compute our dependence on external energy sources. Our house will be fully electric, so that makes things a bit easier as I don’t have to take gas into account.

Basically, we will have two sources of energy (the grid and the PV (solar) panels) and one consumer: the house itself, including all the appliances consuming energy.

External energy, in this context, is energy consumed from the grid. I calculate the total amount of energy we consume using:

total_consumption = pv_production + grid_consumption - pv_overproduction

When the PV panels are not producing energy, there will not be any overproduction, and the total_consumption will be equal to the grid_consumption. When there is no grid_consumption and the PV panels are producing sufficient energy to meet the demand, the total_consumption is equal to the pv_production minus the pv_overproduction.

I’m interested in our dependence from the grid. This is then an easy next step:

dependence = grid_consumption / total_consumption

This gives me a number that gives me the amount of energy consumed from the grid related to the total consumption. Initially, I will calculate our dependence based on 30 minute intervals.

The good news is that I already have the data, but it’s spread over multiple time-series. I’m building a small service that consumes the required data from the time-series, computes the dependence, and writes it back into a new time-series for historic analysis.

Grafana showing pv_overproduction (green) and grid_consumption(yellow).

Determine washer program start-time based on predicted PV energy production.

When we decided we wanted to build a new house I wanted to invest in both passive and active technology to reduce our energy consumption as much as possible. My goal is to reduce our dependency from external energy sources, without installing batteries. This means I need to match our energy consumption with its (local) availability (or simply said: solar-powered production).

Use case

The washer, dishwasher, and dryer are energy-consuming devices. We are used to start these devices at night: the energy was cheaper, there was no noise pollution in the living room, and it made sense at the end of the day.

At the same time, it doesn’t really matter when these devices finish their work. Typically you want them finished within the next 12 hours or so. Therefore, a smart system could nicely plan their consumption based on the next available solar-power production peak, which typically happens around lunch-time anyway. This would increase the energy we consume directly from the solar panels and reduce the energy we need to consume from the grid.

To make things a bit easier I decided to buy devices that support the B/S/H Home Connect system. As we are still building the house I don’t have a dishwasher and at the moment we don’t use a dryer, so I started with the washer.

Challenges

I need a lot of information to make this work. Luckily there are some public (free-of-charge) cloud-services available that helped me a bit here and there. All of the selected services have well documented APIs that I could implement with ease. My biggest challenge was to get the OAuth 2.0 Device Authorization flow up and running for Home Connect.

Appliance status to determine if there is a need to start a program.
Solar-panel production prediction, at least 24 hours ahead.
Solar-power production past, to validate quality of the prediction.
Appliance program consumption pattern to match it against the prediction.
Some user interface to notify me what is happening.
[optional] Day-ahead energy pricing, in case the production is not sufficient to select an optimal (cost-efficient and grid-optimised) timeslot to consume energy from the grid.

In our current apartment I don’t have solar-panels. Therefore I’m using data from another (live) solar production site to simulate the behaviour of the concept.

System architecture

To manage expectations: I’m not a professional software engineer. I’m not planning to productise this, and just want to be able to maintain everything myself. I might publish some of the related projects on Github, but don’t expect a lot of documentation on it.

The image below shows a high-level system architecture. I’m using Ditto as a local digital twin for capturing the current state of all the entities in the system. Additionally, I’m using InfluxDB to store the historical states of the digital twins and Grafana to visualise the historical states. The green services are “sensors”: they retrieve data from their sources and update the digital twin. The blue services are “controllers”: they control the devices based on the status of the digital twin. The orange services are “processors”: they transform data or decide to start actions. The red services are “communicators”: they communicate with the user of the specific use-case.

How to use it?

Well, that’s the good news. Instead of pressing the “start” button on the washer, we press the “remote start enabled” (or “app”) button. The HomeConnect.Sensor captures this event and updates the digital twin. The HomeConnect.Planner is subscribed to this update and starts calculating the best possible timeslot in the next 16 hours to start the program. It considers the predicted solar-panel production, the history washer program consumption registered by the Shelly Plug, and the Epex spot pricing. Once it has calculated the ideal start time updates the “scheduledprogram” digital twin. The HomeConnect.Controller is subscribed on this update and sends the start command to the washer based on the time HomeConnect.Planner has defined. Slack (using Slack.Messenger) keeps the user up-to-date on what is happening, for example, when the program is planned, started, finished, or cancelled.