Merge pull request #70 from tum-ewk/develop

v0.1

.gitignore

@@ -1,5 +1,9 @@
 .idea
 
+# Folders
+04_scenarios/*
+05_results/*
+
 # C extensions
 *.so
 
@@ -66,7 +70,7 @@ instance/
 .scrapy
 
 # Sphinx documentation
-00 - docs/_build/
+00_docs/_build/
 
 # PyBuilder
 target/
@@ -124,8 +128,10 @@ dmypy.json
 
 # Pyre type checker
 .pyre/
-lem_analysis/monte_carlo/simulations/
 
-lem_analysis/
+**/__pycache__/
+scripts
 
-scripts_case_study_1/
+# for macbook
+**/.DS_Store
+.Rhistory

00_docs/00_General_information.rst

@@ -0,0 +1,42 @@
+General information
+===================
+
+:Authors: `Sebastian D. Lumpp`_, `Michel Zadé`_, `Markus Doepfert`_
+:Organization: `Chair of Energy Economy and Application Technology`_, Technical University of Munich
+:Version: 1.0
+:Date: 01.06.2021
+:Copyright: The model code is licensed under the `GNU General Public License 3.0`_.
+            This documentation is licensed under a `Creative Commons Attribution 4.0 International`_ license.
+
+Description
+--------
+lemlab is an open-source tool for the agent-based development and testing of local energy market applications offering:
+
+* a fully open-source, agent-based local energy market modelling toolbox
+* a modular and extendable design for easy adaptation to your own research questions
+* real-time capabilities for the development and testing of hard- and software
+* a database-agnostic approach that enables the integration of multiple database technologies
+* integrated time-series data for several plant types (household loads, pv, wind, heat pumps, electric vehicles etc...)
+* template functionality for load and generation forecasting, trading strategies, cutting-edge market clearing
+  algorithms designed specifically for LEMs [paper under review] and much more...
+
+Changes
+-------
+01.06.2021 - first release
+
+Dependencies
+------------
+* `Python`_ please see lemlab-env.yaml for virtual environment configuration
+* Any solver supported by pyomo. We suggest `gurobi`_ or `cplex`_
+* PostgreSQL
+
+
+.. _Sebastian D. Lumpp: [email protected]
+.. _Michel Zadé: [email protected]
+.. _Markus Doepfert: [email protected]
+.. _Chair of Energy Economy and Application Technology: https://www.ei.tum.de/en/ewk/
+.. _GNU General Public License 3.0: https://www.gnu.org/licenses/gpl-3.0
+.. _Creative Commons Attribution 4.0 International: https://creativecommons.org/licenses/by/4.0/
+.. _Python: https://www.python.org/
+.. _gurobi: https://www.gurobi.com/
+.. _cplex: https://www.ibm.com/analytics/cplex-optimizer

00_docs/01_Introduction_to_LEMs.rst

@@ -0,0 +1,121 @@
+Introduction to LEMs
+====================
+This chapter's purpose is to familiarize the user with what local energy markets (LEMs) actually are, how they work,
+who the participants are and basic terminology used in this research area.
+
+The why and what of LEMs
+------------------------
+
+The fundamental goal of markets is to efficiently match supply and the demand. While the definition of "efficiency" may
+vary, energy markets share this goal. Traditional energy markets are usually implemented on national levels. The
+transmission of electricity is treated as an entirely separate task in European liberalized markets. These traditional
+systems are facing challenges in the face of the rapid expansion of distributed energy resources (often in the form of
+fluctuating renewables) as well as the anticipated electrification of the mobility and heat sectors in the form of
+electric vehicles and heat pumps.
+
+The European Union has included new rules in its "Directive on common rules for the internal electricity market" ((EU)
+2019/944) "that enable active consumer participation, individually or through citizen energy communities, in all
+markets, either by generating, consuming, sharing or selling  electricity, or by providing flexibility services through
+demand-response and storage.  The directive aims to improve the uptake of energy communities and make it easier for
+citizens to integrate efficiently in the electricity system, as active participants."
+
+Specifically, the aim of these energy communities is to enable decentralized producers and consumers (prosumers) to
+participate "on equal footing with large participants".
+
+There is no doubt that developing local energy markets and effectively integrating these into existing structures
+presents a major engineering challenge. In addition to questions of market design, social welfare, and regulation,
+technical questions regarding infrastructure, organization, agent strategies, market gaming, price formation and many
+more threaten to overwhelm designers.
+
+In order to tackle the challenges faced by engineers, designers, and regulators, we present the local energy market
+laboratory, **lemlab**, an open-source tool for the agent-based development and testing of local energy market
+applications.
+
+What does a LEM look like?
+--------------------------
+There is as yet no consensus on what the best structure for a LEM is. For this reason we have attempted to keep our
+architecture as generic as possible in order to allow the user maximal flexibility in the system designs they wish to
+implement.
+
+We begin by explaining the basic structure of a LEM according to *lemlab*:
+
+.. image:: images/lem_structure.png
+    :width: 400px
+    :align: center
+    :height: 263px
+    :alt: Architecture of a generalized LEM
+
+At the heart of the *lemlab* is the **prosumer**. In modern power systems, the end user is no longer just a consumer.
+The prosumer is the generalized market participant, combining production, consumption, and flexibility with a
+preference for local and sustainable energy. Of course a prosumer could be a pure consumer, a simple producer, they
+might be entirely inflexible or they might prefer their electrical energy sources to produce as much carbon dioxide as
+possible. In any case, we will still refer to them as a prosumer.
+
+Prosumers place bids on a **LEM platform**. This platform combines all market functionality, beginning with user
+management, bid/offer collection and clearing, and ending with meter reading collection, market settlement and levy
+collection. **Aggregators** trade on a LEM on behalf of a group of prosumers. **Retailers** take care of coupling the
+LEM to the wholesale market.
+
+That's pretty much it. Of course we can also model grid operators, transmission grids and wholesale markets as well
+as multiple LEMs. These are possible in *lemlab* and some are covered in this documentation. All are simply a matter of
+rearranging the building blocks *lemlab* supplies to construct the desired system.
+
+When does what happen in a LEM?
+-------------------------------
+Similarly to LEM structure, there is no general consensus on the timeline of events in a LEM. We present here a, in our
+opinion, sensible timeline for a local energy market, based loosely on the general principles underlying European
+electricity markets.
+
+.. image:: images/lem_timeline_basic.png
+    :width: 600px
+    :align: center
+    :height: 200px
+    :alt: Basic timeline for a generalized LEM
+
+We divide our market timeline into four fundamental time periods. The first is **registration**, which takes place
+before trading commences. We then enter the **rolling horizon market sequence**, which repeats for each delivery period.
+Each day is divided into an arbitrary number of periods of energy exchange, referred to as **timestep of delivery**
+(ts_delivery) labelled as the period between td and td + T. We usually use 15 minute windows (T=900s), as the German
+electricity market is divided into 15 minute periods. During this time, the physical flow of energy takes place. All
+other activities either take place between market opening, td - tO, and the beginning of the ts_delivery in question,
+td, and are known as **pre-exchange-activities**, or they take place after delivery, in which case they are known as
+**settlement** activities. Examples of pre-exchange activities are forecasting of demand and energy trading, while
+settlement activites include meter value logging, market settlement and balancing energy settlement.
+
+.. image:: images/lem_timeline_detailed.png
+    :width: 600px
+    :align: center
+    :height: 268px
+    :alt: Detailed timeline for a generalized LEM
+
+The above figure shows a slightly more detailed timeline. We see pre-clearing activities taking place during each of *n*
+clearing periods before the timestep of energy delivery. Market clearing can take place during some or all of these
+periods. Post-clearing activities take place after clearing but before delivery and typically include the checking of
+clearing results. Metering and settlement activities are performed after the period of energy delivery is complete.
+
+For a more detailed look into the construction of lemlab and the individual agents, please see chapters 3 and 4.
+
+Basic terminology
+-----------------
+
++--------------------------+----------------------------------------------------------------------------------------------------------+
+| agent                    | someone participating in the LEM                                                                         |
++--------------------------+----------------------------------------------------------------------------------------------------------+
+| aggregator               | an agent trading on behalf of several other agents                                                       |
++--------------------------+----------------------------------------------------------------------------------------------------------+
+| LEM                      | local energy market                                                                                      |
++--------------------------+----------------------------------------------------------------------------------------------------------+
+| pre-clearing activities  | activities taking place before market clearing, e.g. forecasting and trading                             |
++--------------------------+----------------------------------------------------------------------------------------------------------+
+| pre-exchange activities  | activities taking place before energy exchange, e.g. forecasting, trading, and checking clearing results |
++--------------------------+----------------------------------------------------------------------------------------------------------+
+| post-clearing activities | activities taking place before energy exchange but after market clearing, e.g. checking clearing results |
++--------------------------+----------------------------------------------------------------------------------------------------------+
+| prosumer                 | an agent with production and/or consumption, participating the in the LEM                                |
++--------------------------+----------------------------------------------------------------------------------------------------------+
+| registration             | the first step in participating in a LEM                                                                 |
++--------------------------+----------------------------------------------------------------------------------------------------------+
+| retailer                 | an agent who couples the LEM with the wholesale markets                                                  |
++--------------------------+----------------------------------------------------------------------------------------------------------+
+| settlement               | activities taking place after market clearing, e.g. market settlement and balancing settlement           |
++--------------------------+----------------------------------------------------------------------------------------------------------+

00_docs/02_Getting_started_with_lemlab.rst

@@ -0,0 +1,134 @@
+===========================
+Getting started with lemlab
+===========================
+
+General
+=======
+In order to further enable the integration of ever-increasing shares of decentralized electricity generation and
+flexible prosumers, local/regional energy markets (LEM) are being investigated as a potential solution to maximize
+local matching of generation of consumption in order to relieve pressure on electrical grids by leveraging the
+inherent temporal flexibility of the smart prosumer.
+
+Investigating the unique challenges involved in constructing stable and efficient LEMs usually involves the setting up
+of complex and extensive simulation environments, the collection of sufficient data and deep technical knowledge of all
+components involved. Only then can the desired solution be implemented and tested. Developing software or hardware tools
+for deployment in the field typically requires the development of hardware/software-in-the-loop (XiL) development
+environments in order to test prototypes and before field testing can commence.
+
+lemlab was designed specifically with these use cases in mind. lemlab allows the user to simulate a LEM using a full
+agent-based modelling (ABM) in either simulation (SIM) or real-time (RTS) modes. This allows the rapid testing of
+algorithms as well as the real-time integration of hardware and software components.
+
+Installation guide
+==================
+
+lemlab is maintained using a combination of PyCharm, PostgreSQL, Gurobi and Anaconda. This installation guide will
+explain the procedure for this software combination. It is recommended that beginners follow this guide.
+
+**Install the following software**
+
+    - PyCharm (Community or Professional)
+    - Anaconda Individual Edition
+    - PostgreSQL*
+    - Gurobi* or CPLEX. GLPK can be used although this is non-ideal.
+
+**Clone repository**
+You can download or clone the repository to a local directory of your choice. You can use version control tools such as
+GitHub Desktop, Sourcetree, GitKraken or pure Git. The link for pure Git is:
+
+``git clone https://github.com/tum-ewk/lemlab.git``
+
+If using PyCharm, clone the repository to `./PyCharmProjects/lemlab/`
+
+**Create a virtual python environment**
+    - Open the AnacondaPrompt.
+    - Type `conda env create -f ./PycharmProjects/lemlab/lemlab-env.yaml`
+    - Take care to set the correct (absolute) path to your cloned repository.
+
+**Activate the environment**
+    - Open PyCharm
+    - Go to 'File->Open'
+    - Navigate to PyCharmProjects and open lemlab
+    - When the project has opened, go to
+        `File->Settings->Project->Python Interpreter->Show all->Add->Conda Environment->Existing environment->Select folder->OK``
+
+**Install a solver (we recommend Gurobi)**
+    - Go to gurobi.com
+    - Create an account with your university email
+    - When the account has been activated, log in and download the newest Gurobi solver.
+    - Go to Academia->Academic Program and Licenses
+    - Follow the installation instructions under "Individual Academic Licenses" to activate your copy of Gurobi
+
+**Install and configure PostgreSQL**
+    - Install PostgreSQL from https://www.postgresql.org/
+    - if possible, select port 5432. If this is not possible, you will need to edit the configuration file
+      before executing any simulations
+    - When your installation has been completed, launch pgAdmin 4
+    - Select your local server (PostgreSQL 13)
+    - Open Login/Group Roles and create two new login roles as follows
+        1. name: admin_lem
+            password: admin
+            privileges: can login
+                        superuser
+        2. name: market_participant
+            password: user
+            privileges: can login
+
+Test your installation
+======================
+    - navigate to ./PycharmProjects/lemlab/code_examples
+    - execute sim_1_create_scenario.py, followed by sim_3_run.py
+    - When the simulation has completed (this may take some time, depending on your system),
+      analyze the results by executing sim_4_plot_results.py
+    - Look at the output plots under lemlab/simulation_results/test_sim/analyzer/
+
+Your installation was successful if you see plots similar to the following:
+
+.. image:: images/virtual_feeder_flow.png
+    :width: 400px
+    :align: center
+    :height: 300px
+    :alt: Virtual feeder flow of LEM
+
+.. image:: images/mcp_ex_ante_da.png
+    :width: 400px
+    :align: center
+    :height: 300px
+    :alt: Market clearing price of the ex-ante auction
+
+.. image:: images/balance_0.png
+    :width: 400px
+    :align: center
+    :height: 300px
+    :alt: Prosumer account balances at the end of the simulation
+
+.. image:: images/price_type_ex_ante_da.png
+    :width: 400px
+    :align: center
+    :height: 300px
+    :alt: Market clearing price vs. energy quality share
+
+.. image:: images/household_power_(20).png
+    :width: 400px
+    :align: center
+    :height: 300px
+    :alt: Prosumer household power flow
+
+.. image:: images/household_finance_(20).png
+    :width: 400px
+    :align: center
+    :height: 300px
+    :alt: Prosumer finances
+
+Structure
+=========
+
+Workflow
+========
+
+Input Data
+==========
+
+
+.. _OPEN - An open-source platform for developing smart local energy system applications: https://www.researchgate.net/publication/342849801_OPEN_An_open-source_platform_for_developing_smart_local_energy_system_applications
+.. _tool: https://www.smartdraw.com/workflow-diagram/examples/ecommerce-workflow-diagram/