Merge pull request #1 from davidusb-geek/dev/forecast_csv

Dev/forecast csv
davidusb-geek · Dec 22, 2021 · 0085797 · 0085797
2 parents 464b049 + 0449280
commit 0085797
Show file tree

Hide file tree

Showing 12 changed files with 416 additions and 141 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -4,16 +4,24 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## [0.2.0]
+## [0.2.1] - Unreleased
+### Fixed
+- Cleaned unittest implementation
+### Added
+- Added support to choose the methods for weather and load forecast from configuration file.
+- Added new load cost and power production price forecasts, mainly allowing the user to load a CSV file with their own forecast.
+
+## [0.2.0] - 2021-10-14
 ### Fixed
 - Fixed tests to pass with latest changes on passing path and logger arguments.
 - Updated requirements for PVLib and Protobuf.
 - Removed unnecessary use of abstract classes.
 - Fixed test_optimization bad setup.
-- Implemented typing for compatibility with Python4
 - Fixed logger integration in classes
-- Implemented different types of cost functions
 - Updated documentation
+### Added
+- Implemented typing for compatibility with Python4
+- Implemented different types of cost functions
 
 ## [0.1.5] - 2021-09-22
 ### Fixed
@@ -51,3 +59,4 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 [0.1.3]: https://github.com/davidusb-geek/emhass/releases/tag/v0.1.3
 [0.1.4]: https://github.com/davidusb-geek/emhass/releases/tag/v0.1.4
 [0.1.5]: https://github.com/davidusb-geek/emhass/releases/tag/v0.1.5
+[0.2.0]: https://github.com/davidusb-geek/emhass/releases/tag/v0.2.0
diff --git a/README.md b/README.md
@@ -184,15 +184,16 @@ make html
 - [x] Create a plotting script to visualize the optimization results.
 - [x] Propose multiple types of cost functions: profit maximization, self-consumption maximization, etc.
 - [ ] Implement an energy management with a Model Predictive Control approach. Consider implementing the receiding horizon approach.
-- [ ] Introduce the modeling of constraints during optimization for a thermal energy storage
+- [ ] Introduce the modeling of constraints during optimization for a thermal energy storage.
 - [ ] Integrate the possibility of variable tariffs, for purshasing and selling energy to the grid.
 - [ ] Add elasticity to LP formulation in case on infeasible solution.
 ### Related to forecasting improvement
-- [x] Define the type of forecast that should be used from the configuration file
-- [ ] Move get_load_unit_cost from optimization to forecast class: define forecast methods for load and PV production prices.
+- [x] Define the type of forecast that should be used from the configuration file.
+- [x] Move get_load_unit_cost from optimization to forecast class: define forecast methods for load and PV production prices.
 - [ ] Improve load forecasting using a time series forecast algorithm. Some tests were made with fbprophet but results are not completly satisfactory. The model needs some regressors for more accuracy.
-- [ ] Test with LTSM with or without Autoencoders
-### Packaging and HA integration
+- [ ] Test with LTSM with or without Autoencoders.
+### Packaging, HA integration, testing
 - [ ] EMHASS hass been tested in Home Assistan Core. It need to be tested on Home Assistant Operating System and Home Assistant Container. 
-- [ ] Create an EMHASS add-on for even easier installation on Home Assistant Operating System and Home Assistant Supervised
-- [ ] Package everything in a docker container
+- [ ] Create an EMHASS add-on for even easier installation on Home Assistant Operating System and Home Assistant Supervised.
+- [ ] Package everything in a docker container.
+- [ ] Improve testing to be used with no running hass instance.
diff --git a/config.yaml b/config.yaml
@@ -14,6 +14,8 @@ retrieve_hass_conf:
  - 'sensor.power_load_no_var_loads'
 
 optim_conf:
+ - set_use_battery: False # consider a battery storage
+ - delta_forecast: 1 # days
  - num_def_loads: 2
  - P_deferrable_nom: # Watts
  - 3000
@@ -27,20 +29,20 @@ optim_conf:
  - set_def_constant: # set as a constant fixed value variable with just one startup for each 24h
  - False
  - False
- - list_hp_periods: # list of different tariff periods
+ - weather_forecast_method: 'scrapper' # options are 'scrapper' and 'pvlib' (but pvlib remain unstable)
+ - load_forecast_method: 'naive' # options are 'csv' to load a custom load forecast from a CSV file or 'naive' for a persistance model
+ - load_cost_forecast_method: 'hp_hc_periods' # options are 'hp_hc_periods' for peak and non-peak hours contracts and 'csv' to load custom cost from CSV file 
+ - list_hp_periods: # list of different tariff periods (only needed if load_cost_forecast_method='hp_hc_periods')
  - period_hp_1:
  - start: '02:54'
  - end: '15:24'
  - period_hp_2:
  - start: '17:24'
  - end: '20:24'
- - load_cost_hp: 0.1907 # €/kWh
- - load_cost_hc: 0.1419 # €/kWh
- - prod_sell_price: 0.065 # €/kWh
- - set_use_battery: False # consider a battery storage
- - delta_forecast: 1 # days
- - weather_forecast_method: 'scrapper' # options are 'scrapper' and 'pvlib' (but pvlib remain unstable)
- - load_forecast_method: 'naive' # options are 'csv' to load a CSV file or 'naive' for a persistance model
+ - load_cost_hp: 0.1907 # peak hours load cost in €/kWh (only needed if load_cost_forecast_method='hp_hc_periods')
+ - load_cost_hc: 0.1419 # non-peak hours load cost in €/kWh (only needed if load_cost_forecast_method='hp_hc_periods')
+ - prod_price_forecast_method: 'constant' # options are 'constant' for constant fixed value or 'csv' to load custom price forecast from a CSV file
+ - prod_sell_price: 0.065 # power production selling price in €/kWh (only needed if prod_price_forecast_method='constant')
 
 plant_conf:
  - P_grid_max: 9000 # Watts

diff --git a/data/data_load_cost_forecast.csv b/data/data_load_cost_forecast.csv
@@ -0,0 +1,48 @@
+2021-04-29 00:00:00+00:00,0.1419
+2021-04-29 00:30:00+00:00,0.1419
+2021-04-29 01:00:00+00:00,0.1419
+2021-04-29 01:30:00+00:00,0.1419
+2021-04-29 02:00:00+00:00,0.1419
+2021-04-29 02:30:00+00:00,0.1419
+2021-04-29 03:00:00+00:00,0.1907
+2021-04-29 03:30:00+00:00,0.1907
+2021-04-29 04:00:00+00:00,0.1907
+2021-04-29 04:30:00+00:00,0.1907
+2021-04-29 05:00:00+00:00,0.1907
+2021-04-29 05:30:00+00:00,0.1907
+2021-04-29 06:00:00+00:00,0.1907
+2021-04-29 06:30:00+00:00,0.1907
+2021-04-29 07:00:00+00:00,0.1907
+2021-04-29 07:30:00+00:00,0.1907
+2021-04-29 08:00:00+00:00,0.1907
+2021-04-29 08:30:00+00:00,0.1907
+2021-04-29 09:00:00+00:00,0.1907
+2021-04-29 09:30:00+00:00,0.1907
+2021-04-29 10:00:00+00:00,0.1907
+2021-04-29 10:30:00+00:00,0.1907
+2021-04-29 11:00:00+00:00,0.1907
+2021-04-29 11:30:00+00:00,0.1907
+2021-04-29 12:00:00+00:00,0.1907
+2021-04-29 12:30:00+00:00,0.1907
+2021-04-29 13:00:00+00:00,0.1907
+2021-04-29 13:30:00+00:00,0.1907
+2021-04-29 14:00:00+00:00,0.1907
+2021-04-29 14:30:00+00:00,0.1907
+2021-04-29 15:00:00+00:00,0.1907
+2021-04-29 15:30:00+00:00,0.1419
+2021-04-29 16:00:00+00:00,0.1419
+2021-04-29 16:30:00+00:00,0.1419
+2021-04-29 17:00:00+00:00,0.1419
+2021-04-29 17:30:00+00:00,0.1907
+2021-04-29 18:00:00+00:00,0.1907
+2021-04-29 18:30:00+00:00,0.1907
+2021-04-29 19:00:00+00:00,0.1907
+2021-04-29 19:30:00+00:00,0.1907
+2021-04-29 20:00:00+00:00,0.1907
+2021-04-29 20:30:00+00:00,0.1907
+2021-04-29 21:00:00+00:00,0.1419
+2021-04-29 21:30:00+00:00,0.1419
+2021-04-29 22:00:00+00:00,0.1419
+2021-04-29 22:30:00+00:00,0.1419
+2021-04-29 23:00:00+00:00,0.1419
+2021-04-29 23:30:00+00:00,0.1419
diff --git a/data/data_prod_price_forecast.csv b/data/data_prod_price_forecast.csv
@@ -0,0 +1,48 @@
+2021-04-29 00:00:00+00:00,0.065
+2021-04-29 00:30:00+00:00,0.065
+2021-04-29 01:00:00+00:00,0.065
+2021-04-29 01:30:00+00:00,0.065
+2021-04-29 02:00:00+00:00,0.065
+2021-04-29 02:30:00+00:00,0.065
+2021-04-29 03:00:00+00:00,0.065
+2021-04-29 03:30:00+00:00,0.065
+2021-04-29 04:00:00+00:00,0.065
+2021-04-29 04:30:00+00:00,0.065
+2021-04-29 05:00:00+00:00,0.065
+2021-04-29 05:30:00+00:00,0.065
+2021-04-29 06:00:00+00:00,0.065
+2021-04-29 06:30:00+00:00,0.065
+2021-04-29 07:00:00+00:00,0.065
+2021-04-29 07:30:00+00:00,0.065
+2021-04-29 08:00:00+00:00,0.065
+2021-04-29 08:30:00+00:00,0.065
+2021-04-29 09:00:00+00:00,0.065
+2021-04-29 09:30:00+00:00,0.065
+2021-04-29 10:00:00+00:00,0.065
+2021-04-29 10:30:00+00:00,0.065
+2021-04-29 11:00:00+00:00,0.065
+2021-04-29 11:30:00+00:00,0.065
+2021-04-29 12:00:00+00:00,0.065
+2021-04-29 12:30:00+00:00,0.065
+2021-04-29 13:00:00+00:00,0.065
+2021-04-29 13:30:00+00:00,0.065
+2021-04-29 14:00:00+00:00,0.065
+2021-04-29 14:30:00+00:00,0.065
+2021-04-29 15:00:00+00:00,0.065
+2021-04-29 15:30:00+00:00,0.065
+2021-04-29 16:00:00+00:00,0.065
+2021-04-29 16:30:00+00:00,0.065
+2021-04-29 17:00:00+00:00,0.065
+2021-04-29 17:30:00+00:00,0.065
+2021-04-29 18:00:00+00:00,0.065
+2021-04-29 18:30:00+00:00,0.065
+2021-04-29 19:00:00+00:00,0.065
+2021-04-29 19:30:00+00:00,0.065
+2021-04-29 20:00:00+00:00,0.065
+2021-04-29 20:30:00+00:00,0.065
+2021-04-29 21:00:00+00:00,0.065
+2021-04-29 21:30:00+00:00,0.065
+2021-04-29 22:00:00+00:00,0.065
+2021-04-29 22:30:00+00:00,0.065
+2021-04-29 23:00:00+00:00,0.065
+2021-04-29 23:30:00+00:00,0.065
diff --git a/docs/config.md b/docs/config.md
@@ -39,6 +39,8 @@ The parameters in the `secrets.yaml` file are:
 
 These are the parameters needed to properly define the optimization problem.
 
+- set_use_battery: Set to True if we should consider an energy storage device such as a Li-Ion battery. Defaults to False.
+- delta_forecast: The number of days for forecasted data. Defaults to 1.
 - num_def_loads: Define the number of deferrable loads to consider. Defaults to 2.
 - P_deferrable_nom: The nominal power for each deferrable load in Watts. This is a list with a number of elements consistent with the number of deferrable loads defined before. For example:
  - 3000
@@ -52,20 +54,22 @@ These are the parameters needed to properly define the optimization problem.
 - set_def_constant: Define if we should set each deferrable load as a constant fixed value variable with just one startup for each optimization task. For example:
  - False
  - False
-- list_hp_periods: Define a list of peak hour periods for load consumption from the grid. This is useful if you have a contract with peak and non-peak hours. For example for two peak hour periods: 
- - period_hp_1:
-  - start: '02:54'
-  - end: '15:24'
- - period_hp_2:
-  - start: '17:24'
-  - end: '20:24'
-- load_cost_hp: The cost of the electrical energy from the grid during peak hours in €/kWh. Defaults to 0.1464.
-- load_cost_hc: The cost of the electrical energy from the grid during non-peak hours in €/kWh. Defaults to 0.096359.
-- prod_sell_price: The paid price for energy injected to the grid from excedent PV production in €/kWh. Defaults to 0.065.
-- set_use_battery: Set to True if we should consider an energy storage device such as a Li-Ion battery. Defaults to False.
-- delta_forecast: The number of days for forecasted data. Defaults to 1.
-- weather_forecast_method: This will define the weather forecast method that will be used, the options are 'scrapper' and 'pvlib' (but pvlib method remains unstable).
-- load_forecast_method: The load forecast method that will be used, the options are 'csv' to load a CSV file or 'naive' for a simple 1-day persistance model.
+- weather_forecast_method: This will define the weather forecast method that will be used. The options are 'scrapper' for a scrapping method for weather forecast from clearoutside.com and 'pvlib' to use the forecast utilities from the PVLib package (but beware that 'pvlib' method remains unstable, it should be tested before use). Defaults to 'scrapper'.
+- load_forecast_method: The load forecast method that will be used. The options are 'csv' to load a CSV file or 'naive' for a simple 1-day persistance model. Defaults to 'naive'.
+- load_cost_forecast_method: Define the method that will be used for load cost forecast. The options are 'hp_hc_periods' for peak and non-peak hours contracts and 'csv' to load custom cost from CSV file. The default CSV file path that will be used is '/data/data_load_cost_forecast.csv'.
+The following parameters and definitions are only needed if load_cost_forecast_method='hp_hc_periods':
+ - list_hp_periods: Define a list of peak hour periods for load consumption from the grid. This is useful if you have a contract with peak and non-peak hours. For example for two peak hour periods: 
+ - period_hp_1:
+ - start: '02:54'
+ - end: '15:24'
+ - period_hp_2:
+ - start: '17:24'
+ - end: '20:24'
+ - load_cost_hp: The cost of the electrical energy from the grid during peak hours in €/kWh. Defaults to 0.1907.
+ - load_cost_hc: The cost of the electrical energy from the grid during non-peak hours in €/kWh. Defaults to 0.1419.
+
+- prod_price_forecast_method: Define the method that will be used for PV power production price forecast. This is the price that is payed by the utility for energy injected to the grid. The options are 'constant' for a constant fixed value or 'csv' to load custom price forecast from a CSV file. The default CSV file path that will be used is '/data/data_prod_price_forecast.csv'.
+- prod_sell_price: The paid price for energy injected to the grid from excedent PV production in €/kWh. Defaults to 0.065. This parameter is only needed if prod_price_forecast_method='constant'.
 
 ## System configuration parameters
 

diff --git a/src/emhass/command_line.py b/src/emhass/command_line.py
@@ -49,7 +49,8 @@ def setUp(config_path, costfun, logger):
  P_load_forecast = fcst.get_load_forecast(method=optim_conf['load_forecast_method'])
  df_input_data_dayahead = pd.concat([P_PV_forecast, P_load_forecast], axis=1)
  df_input_data_dayahead.columns = ['P_PV_forecast', 'P_load_forecast']
- opt = optimization(retrieve_hass_conf, optim_conf, plant_conf, days_list, 
+ opt = optimization(retrieve_hass_conf, optim_conf, plant_conf, 
+ fcst.var_load_cost, fcst.var_prod_price, days_list, 
  costfun, config_path, logger)
  # The input data dictionnary to return
  input_data_dict = {
@@ -78,8 +79,12 @@ def perfect_forecast_optim(input_data_dict, logger):
  :rtype: pd.DataFrame
 
  """
- logger.info("Performing perfect forecast optimiaztion")
- df_input_data = input_data_dict['opt'].get_load_unit_cost(input_data_dict['df_input_data'])
+ logger.info("Performing perfect forecast optimization")
+ df_input_data = input_data_dict['fcst'].get_load_cost_forecast(
+ input_data_dict['df_input_data'], 
+ method=input_data_dict['fcst'].optim_conf['load_cost_forecast_method'])
+ df_input_data = input_data_dict['fcst'].get_prod_price_forecast(
+ df_input_data, method=input_data_dict['fcst'].optim_conf['prod_price_forecast_method'])
  opt_res = input_data_dict['opt'].perform_perfect_forecast_optim(df_input_data)
  # Save CSV file for analysis
  filename = 'opt_res_perfect_optim_'+input_data_dict['costfun']
@@ -99,7 +104,11 @@ def dayahead_forecast_optim(input_data_dict, logger):
 
  """
  logger.info("Performing day-ahead forecast optimization")
- df_input_data_dayahead = input_data_dict['opt'].get_load_unit_cost(input_data_dict['df_input_data_dayahead'])
+ df_input_data_dayahead = input_data_dict['fcst'].get_load_cost_forecast(
+ input_data_dict['df_input_data_dayahead'],
+ method=input_data_dict['fcst'].optim_conf['load_cost_forecast_method'])
+ df_input_data_dayahead = input_data_dict['fcst'].get_prod_price_forecast(
+ df_input_data_dayahead, method=input_data_dict['fcst'].optim_conf['prod_price_forecast_method'])
  opt_res_dayahead = input_data_dict['opt'].perform_dayahead_forecast_optim(
  df_input_data_dayahead, input_data_dict['P_PV_forecast'], input_data_dict['P_load_forecast'])
  # Save CSV file for publish_data