Reinforcement Learning Applied to Multi Agent Modelling, the Case of the Iranian Power Market | ||
| پژوهشنامه اقتصاد انرژی ایران | ||
| Article 1, Volume 7, Issue 25 - Serial Number 4, January 2018, Pages 1-40 PDF (1.44 M) | ||
| Document Type: Research Paper | ||
| DOI: 10.22054/jiee.2018.9993 | ||
| Authors | ||
| Mohammadreza Asghari Oskoei* 1; Farhad Fallahi2; Meysam Doostizadeh3; saeed Moshiri4 | ||
| 1Computer Science Dept, Allameh Tabataba'i University | ||
| 2Research Fellow in Niro Research Inst., | ||
| 3Assistant Prof., Lorestan University, | ||
| 4Associate Prof., Allameh Tabataba’i University | ||
| Abstract | ||
| With increasing competition in the wholesale Electricity markets and advances in behavioral economics in recent years, the multi-agent modeling approach has been applied widely to simulate the outcome of the markets. The electricity market consists of power generating agents that compete over production in daily auction conducted by an independent system operator (ISO). The market clearing mechanism can be seen as a static game that repeats every hour. In this game, an agent proposes her price for the next day and the ISO chooses the best proposals that minimizes the total costs given the demand and the technical constraints. Agents are also assumed to learn from the outcomes and adjust their biding strategy accordingly. In this paper, we develop an agent-based model for the day-ahead and pay-as-bid electricity market in Iran. The objective is to compare the outcome of the market measured by the agents profit and the time to converge using three different strategies: greedy, random and reinforcement learning. The simulation results indicate that the reinforcement learning leads to higher profits with a faster convergence rate than the other two strategies. | ||
| Keywords | ||
| Agent Based Model; Electricity Market; Reinforcement Learning; Game Theory; Iran | ||
| References | ||
|
Rashedi, N., Tajeddini, M. A., and Kebriae, H. (2016), Markov Game Approach for Multi-Agent Competitive Bidding Strategies in Electricity Market, IET Generation, Transmission & Distribution, Vol.10, No.15, pp. 3756-3763.
Rashedi, N., and Kebriaei, H. (2014), Cooperative and Non-Cooperative Nash Solution for Linear Supply Function Equilibrium Game. Applied Mathematics and Computation, Vol. 244, pp.794-808.
Rahimiyan, M., and Rajabi Mashhadi, H. (2010), An Adaptive Q-Learning Algorithm Developed for Agent-Based Computational Modeling of Electricity Market. IEEE in Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), Vol.40, Issue.5, pp. 547-556.
Krause, Th., Vdovina Beck, E., Cherkaoui, R., Germond, A., Andersson, G. and Ernst, D. (2006), A Comparison of Nash Equilibria Analysis and Agent-Based Modeling for Power Markets. International Journal of Electrical Power & Energy Systems, Vol.28, Issue.9, pp.599-607.
Busoniu, L., Babuska, R. and De Schutter B.(2008), A Comprehensive Survey of Multiagent Reinforcement Learning. IEEE in Transactions on Systems, Man, And Cybernetics-Part C: Applications and Reviews, Vol.38, Issue.2, pp.1-18.
Lauer, M., and Riedmiller, M. (2000), an Algorithm for Distributed Reinforcement Learning in Cooperative Multi-Agent Systems. In Proceedings of the Seventeenth International Conference on Machine Learning.
Toman, M. and Barbora, J. (2003), Energy and Economic Development: an Assessment of the State of Knowledge, Discussion Paper 03–13, Resources for the Future, Washington. | ||
|
Statistics Article View: 1,407 PDF Download: 1,539 |
||