Learning to Control Home Batteries in the Smart Grid

Abstract

Modern residential buildings are complex cyber-physical systems housing energy systems with numerous sensors and actuators. In recent years, the falling costs of battery storage and photovoltaic systems have substantially increased the number of “solar-plus-battery” installations in these buildings. The solar-plus-battery system enables homeowners to protect their homes during a power outage and save on their electricity bills by stacking multiple value streams that battery storage can provide (e.g., energy arbitrage, and maximizing the self-use of solar power). However, controlling a solar-plus-battery system is quite challenging, mainly due to the wide range of variability and uncertainty associated with the building energy demand, electricity price, and meteorological factors affecting solar generation. Physical constraints, such as energy and power ratings of the lithium-ion battery and solar micro-inverter, only exacerbate this problem. This thesis aims to investigate how to model the solar-plus-battery system and the stochastic environment, and how to design learning-based control policies for operating batteries in the smart grid to cut the monthly electricity bills for customers. We seek to develop control policies that are adaptive, optimal, and suitable for real-world applications. We model different components of the solar-plus-battery system based on first principles, and the surrounding environment utilizing historical data about building energy demand, electricity price, and weather condition in Chapter 3. In particular, we develop and evaluate various supervised learning models for predicting the available solar energy and household demand over the next 24 hours. We propose four learning-based methods for the optimal control of the solar-plus-battery system, under various operating conditions in Chapter 4, and study their effectiveness in terms of maximizing the revenue of homeowners. The control methods developed and discussed in this thesis are Model Predictive Control (MPC), Advantage Actor-Critic (A2C), Proximal Policy Optimization (PPO), and Direct Learning-based Control (DLC) using a neural network. The battery control is optimal in the sense that it minimizes the monthly electricity bills for customers. We implement these algorithms and integrate them into EnergyBoost, a Python program that runs on a Raspberry Pi and controls the battery. This allows us to compare their performance with specific baselines under various pricing schemes. Experiments presented in Chapter 5 are based on real traces of solar irradiance and power consumption of 70 homes located in the same jurisdiction. We investigate how these sophisticated control policies compare with simple policies that are being used today to control battery storage systems. We further study whether it makes sense economically to install a battery controlled by the proposed algorithms in different jurisdictions with distinct tariff structures.