A theoretical demonstration for reinforcement learning of PI control dynamics for optimal speed control of DC motors by using Twin Delay Deep Deterministic Policy Gradient Algorithm

Simple item page
dc.contributor.authorTufenkci, Sevilaycs
dc.contributor.authorAlagoz, Baris Baykantcs
dc.contributor.authorKavuran, Gürkancs
dc.contributor.authorYeroglu, Celaleddincs
dc.contributor.authorHerencsár, Norbertcs
dc.contributor.authorMahata, Shibenducs
dc.coverage.issueMarch 2023cs
dc.coverage.volume213,Part Ccs
dc.date.issued2023-03-01cs
dc.description.abstractTo benefit from the advantages of Reinforcement Learning (RL) in industrial control applications, RL methods can be used for optimal tuning of the classical controllers based on the simulation scenarios of operating con-ditions. In this study, the Twin Delay Deep Deterministic (TD3) policy gradient method, which is an effective actor-critic RL strategy, is implemented to learn optimal Proportional Integral (PI) controller dynamics from a Direct Current (DC) motor speed control simulation environment. For this purpose, the PI controller dynamics are introduced to the actor-network by using the PI-based observer states from the control simulation envi-ronment. A suitable Simulink simulation environment is adapted to perform the training process of the TD3 algorithm. The actor-network learns the optimal PI controller dynamics by using the reward mechanism that implements the minimization of the optimal control objective function. A setpoint filter is used to describe the desired setpoint response, and step disturbance signals with random amplitude are incorporated in the simu-lation environment to improve disturbance rejection control skills with the help of experience based learning in the designed control simulation environment. When the training task is completed, the optimal PI controller coefficients are obtained from the weight coefficients of the actor-network. The performance of the optimal PI dynamics, which were learned by using the TD3 algorithm and Deep Deterministic Policy Gradient algorithm, are compared. Moreover, control performance improvement of this RL based PI controller tuning method (RL-PI) is demonstrated relative to performances of both integer and fractional order PI controllers that were tuned by using several popular metaheuristic optimization algorithms such as Genetic Algorithm, Particle Swarm Opti-mization, Grey Wolf Optimization and Differential Evolution.en
dc.formattextcs
dc.format.extent1-16cs
dc.format.mimetypeapplication/pdfcs
dc.identifier.citationEXPERT SYSTEMS WITH APPLICATIONS. 2023, vol. 213,Part C, issue March 2023, p. 1-16.en
dc.identifier.doi10.1016/j.eswa.2022.119192cs
dc.identifier.issn0957-4174cs
dc.identifier.orcid0000-0002-9504-2275cs
dc.identifier.other179598cs
dc.identifier.researcheridA-6539-2009cs
dc.identifier.scopus23012051100cs
dc.identifier.urihttp://hdl.handle.net/11012/208567
dc.language.isoencs
dc.publisherPERGAMON-ELSEVIER SCIENCE LTDcs
dc.relation.ispartofEXPERT SYSTEMS WITH APPLICATIONScs
dc.relation.urihttps://www.sciencedirect.com/science/article/pii/S0957417422022102cs
dc.rightsCreative Commons Attribution-NonCommercial-NoDerivatives 4.0 Internationalcs
dc.rights.accessopenAccesscs
dc.rights.sherpahttp://www.sherpa.ac.uk/romeo/issn/0957-4174/cs
dc.rights.urihttp://creativecommons.org/licenses/by-nc-nd/4.0/cs
dc.subjectDeep reinforcement learningen
dc.subjectDC motoren
dc.subjectPI controlleren
dc.subjectTwin -delayed deep deterministic policyen
dc.subjectgradienten
dc.subjectMetaheuristic optimizationen
dc.titleA theoretical demonstration for reinforcement learning of PI control dynamics for optimal speed control of DC motors by using Twin Delay Deep Deterministic Policy Gradient Algorithmen
dc.type.driverarticleen
dc.type.statusPeer-revieweden
dc.type.versionacceptedVersionen
sync.item.dbidVAV-179598en
sync.item.dbtypeVAVen
sync.item.insts2025.02.03 15:42:29en
sync.item.modts2025.01.17 18:48:53en
thesis.grantorVysoké učení technické v Brně. Fakulta elektrotechniky a komunikačních technologií. Ústav telekomunikacícs
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
j.eswa.2022.119192_accepted.pdf
Size:
7.5 MB
Format:
Adobe Portable Document Format
Description:
file j.eswa.2022.119192_accepted.pdf
Download
Collections
Ústav telekomunikací