Jump to content

Machine learning control: Difference between revisions

From Wikipedia, the free encyclopedia
Browse history interactively
← Previous editNext edit →
Content deleted Content added
VisualWikitext
Add See also
Tags: Mobile edit Mobile web edit
General improvements
Tags: Mobile edit Mobile web edit
Line 1: Line 1:
{{short description|Subfield of machine learning, intelligent control and control theory}}
{{short description|Subfield of machine learning, intelligent control, and control theory}}
'''Machine learning control''' ('''MLC''') is a subfield of [[machine learning]], [[intelligent control]], and [[control theory]] which aims to solve [[optimal control]] problems with machine learning methods. Key applications are complex nonlinear systems for which [[linear control theory]] methods are not applicable.
'''Machine learning control''' ('''MLC''') is a subfield of [[machine learning]], [[intelligent control]], and [[control theory]] which aims to solve [[optimal control]] problems with machine learning methods. Key applications are complex nonlinear systems for which [[linear control theory]] methods are not applicable.


== Types of problems and tasks ==
== Types of problems and tasks ==
Four types of problems are commonly encountered.
Four types of problems are commonly encountered:

* Control parameter identification: MLC translates to a parameter identification<ref name=Baeck1993>Thomas Bäck & Hans-Paul Schwefel (Spring 1993) [http://doi.org/10.1162/evco.1993.1.1.1 "An overview of evolutionary algorithms for parameter optimization"], [[Evolutionary Computation (journal)|Journal of Evolutionary Computation (MIT Press)]], vol. 1, no. 1, pp. 1-23</ref> if the structure of the control law is given but the parameters are unknown. One example is the [[genetic algorithm]] for optimizing coefficients of a [[PID controller]]<ref name=Benard2015aiaa>N. Benard, J. Pons-Prats, J. Periaux, G. Bugeda, J.-P. Bonnet & E. Moreau, (2015) [https://arc.aiaa.org/doi/abs/10.2514/6.2015-2957 "Multi-Input Genetic Algorithm for Experimental Optimization of the Reattachment Downstream of a Backward-Facing Step with Surface Plasma Actuator"], Paper AIAA 2015-2957 at 46th AIAA Plasmadynamics and Lasers Conference, Dallas, TX, USA, pp. 1-23.</ref> or discrete-time optimal control.<ref>Zbigniew Michalewicz, Cezary Z. Janikow & Jacek B. Krawczyk (July 1992) [https://doi.org/10.1016/0898-1221(92)90094-X "A modified genetic algorithm for optimal control problems"], [Computers & Mathematics with Applications], vol. 23, no 12, pp. 83-94.</ref>
* Control parameter identification: MLC translates to a parameter identification<ref name=Baeck1993>Thomas Bäck & Hans-Paul Schwefel (Spring 1993) [http://doi.org/10.1162/evco.1993.1.1.1 "An overview of evolutionary algorithms for parameter optimization"], [[Evolutionary Computation (journal)|Journal of Evolutionary Computation (MIT Press)]], vol. 1, no. 1, pp. 1-23</ref> if the structure of the control law is given but the parameters are unknown. One example is the [[genetic algorithm]] for optimizing coefficients of a [[PID controller]]<ref name=Benard2015aiaa>N. Benard, J. Pons-Prats, J. Periaux, G. Bugeda, J.-P. Bonnet & E. Moreau, (2015) [https://arc.aiaa.org/doi/abs/10.2514/6.2015-2957 "Multi-Input Genetic Algorithm for Experimental Optimization of the Reattachment Downstream of a Backward-Facing Step with Surface Plasma Actuator"], Paper AIAA 2015-2957 at 46th AIAA Plasmadynamics and Lasers Conference, Dallas, TX, USA, pp. 1-23.</ref> or discrete-time optimal control.<ref>Zbigniew Michalewicz, Cezary Z. Janikow & Jacek B. Krawczyk (July 1992) [https://doi.org/10.1016/0898-1221(92)90094-X "A modified genetic algorithm for optimal control problems"], [Computers & Mathematics with Applications], vol. 23, no 12, pp. 83-94.</ref>
* Control design as regression problem of the first kind: MLC approximates a general nonlinear mapping from sensor signals to actuation commands, if the sensor signals and the optimal actuation command are known for every state. One example is the computation of sensor feedback from a known [[full state feedback]]. A [[neural network]] is commonly used technique for this task.<ref>C. Lee, J. Kim, D. Babcock & R. Goodman (1997) [https://dx.doi.org/10.1063/1.869290 "Application of neural networks to turbulence control for drag reduction"], [[Physics of Fluids]], vol. 6, no. 9, pp. 1740-1747</ref>
* Control design as [[Regression analysis|regression]] problem of the first kind: MLC approximates a general nonlinear mapping from sensor signals to actuation commands, if the sensor signals and the optimal actuation command are known for every state. One example is the computation of sensor feedback from a known [[full state feedback]]. [[Neural network (machine learning)|Neural networks]] are commonly used for such tasks.<ref>C. Lee, J. Kim, D. Babcock & R. Goodman (1997) [https://dx.doi.org/10.1063/1.869290 "Application of neural networks to turbulence control for drag reduction"], [[Physics of Fluids]], vol. 6, no. 9, pp. 1740-1747</ref>
* Control design as regression problem of the second kind: MLC may also identify arbitrary nonlinear control laws which minimize the cost function of the plant. In this case, neither a model, nor the control law structure, nor the optimizing actuation command needs to be known. The optimization is only based on the control performance (cost function) as measured in the plant. [[Genetic programming]] is a powerful regression technique for this purpose.<ref>D. C. Dracopoulos & S. Kent (December 1997) [http://doi.org/10.1007/BF01501508 "Genetic programming for prediction and control"], Neural Computing & Applications (Springer), vol. 6, no. 4, pp. 214-228.</ref>
* Control design as regression problem of the second kind: MLC may also identify arbitrary nonlinear control laws which minimize the [[Loss function|cost function]] of the plant. In this case, neither a model, the control law structure, nor the optimizing actuation command needs to be known. The optimization is only based on the control performance (cost function) as measured in the plant. [[Genetic programming]] is a powerful regression technique for this purpose.<ref>D. C. Dracopoulos & S. Kent (December 1997) [http://doi.org/10.1007/BF01501508 "Genetic programming for prediction and control"], Neural Computing & Applications (Springer), vol. 6, no. 4, pp. 214-228.</ref>
* Reinforcement learning control: The control law may be continually updated over measured performance changes (rewards) using [[reinforcement learning]].<ref>Andrew G. Barto (December 1994) [http://doi.org/10.1016/0959-4388(94)90138-4 "Reinforcement learning control"], [[Current Opinion in Neurobiology]], vol. 6, no. 4, pp. 888–893</ref>
* Reinforcement learning control: The control law may be continually updated over measured performance changes (rewards) using [[reinforcement learning]].<ref>Andrew G. Barto (December 1994) [http://doi.org/10.1016/0959-4388(94)90138-4 "Reinforcement learning control"], [[Current Opinion in Neurobiology]], vol. 6, no. 4, pp. 888–893</ref>

MLC comprises, for instance, neural network control,
genetic algorithm based control,
genetic programming control,
reinforcement learning control,
and has methodological overlaps with other data-driven control,
like [[artificial intelligence]] and [[robot control]].


== Applications ==
== Applications ==
MLC has been successfully applied
MLC has been successfully applied
to many nonlinear control problems,
to many nonlinear control problems,
exploring unknown and often unexpected actuation mechanisms.
exploring unknown and often unexpected actuation mechanisms. Example applications include:
Example applications include


* Attitude control of satellites.<ref>Dimitris. C. Dracopoulos & [[Antonia J. Jones|Antonia. J. Jones]] (1994)
* [[spacecraft attitude control]],<ref>Dimitris. C. Dracopoulos & [[Antonia J. Jones|Antonia. J. Jones]] (1994)
[[doi:10.1007/BF01414807|Neuro-genetic adaptive attitude control]], Neural Computing & Applications (Springer), vol. 2, no. 4, pp. 183-204.</ref>
[[doi:10.1007/BF01414807|Neuro-genetic adaptive attitude control]], Neural Computing & Applications (Springer), vol. 2, no. 4, pp. 183-204.</ref>
* Building thermal control.<ref>Jonathan A. Wright, Heather A. Loosemore & Raziyeh Farmani (2002) [http://doi.org/10.1016/S0378-7788(02)00071-3 "Optimization of building thermal design and control by multi-criterion genetic algorithm], [Energy and Buildings], vol. 34, no. 9, pp. 959-972.</ref>
* thermal control of buildings,<ref>Jonathan A. Wright, Heather A. Loosemore & Raziyeh Farmani (2002) [http://doi.org/10.1016/S0378-7788(02)00071-3 "Optimization of building thermal design and control by multi-criterion genetic algorithm], [Energy and Buildings], vol. 34, no. 9, pp. 959-972.</ref>
* Feedback turbulence control.<ref name=Benard2015aiaa /><ref>Steven J. Brunton & Bernd R. Noack (2015) [http://doi.org/10.1115/1.4031175 Closed-loop turbulence control: Progress and challenges], [[Applied Mechanics Reviews]], vol. 67, no. 5, article 050801, pp. 1-48.</ref>
* feedback control of [[turbulence]],<ref name=Benard2015aiaa /><ref>Steven J. Brunton & Bernd R. Noack (2015) [http://doi.org/10.1115/1.4031175 Closed-loop turbulence control: Progress and challenges], [[Applied Mechanics Reviews]], vol. 67, no. 5, article 050801, pp. 1-48.</ref>
* [[Remotely operated underwater vehicle]]s.<ref>J. Javadi-Moghaddam, & A. Bagheri (2010 [http://doi.org/10.1016/j.eswa.2009.06.015 "An adaptive neuro-fuzzy sliding mode based genetic algorithm control system for under water remotely operated vehicle"], [https://www.journals.elsevier.com/expert-systems-with-applications/ Expert Systems with Applications], vol. 37 no. 1, pp. 647-660.</ref>
* and [[remotely operated underwater vehicle]]s.<ref>J. Javadi-Moghaddam, & A. Bagheri (2010 [http://doi.org/10.1016/j.eswa.2009.06.015 "An adaptive neuro-fuzzy sliding mode based genetic algorithm control system for under water remotely operated vehicle"], [https://www.journals.elsevier.com/expert-systems-with-applications/ Expert Systems with Applications], vol. 37 no. 1, pp. 647-660.</ref>

* Many more engineering MLC application are summarized in the review article of PJ Fleming & RC Purshouse (2002).<ref>Peter J. Fleming, R. C. Purshouse (2002 [http://doi.org/10.1016/S0967-0661(02)00081-3 "Evolutionary algorithms in control systems engineering: a survey"]
Many more engineering MLC application are summarized in the review article of PJ Fleming & RC Purshouse (2002).<ref>Peter J. Fleming, R. C. Purshouse (2002 [http://doi.org/10.1016/S0967-0661(02)00081-3 "Evolutionary algorithms in control systems engineering: a survey"]
[[:nl:Control Engineering Practice|Control Engineering Practice]], vol. 10, no. 11, pp. 1223-1241</ref>
[[:nl:Control Engineering Practice|Control Engineering Practice]], vol. 10, no. 11, pp. 1223-1241</ref>


As for all general nonlinear methods,
As is the case for all general nonlinear methods,
MLC comes with no guaranteed convergence,
MLC does not guarantee convergence,
optimality or robustness for a range of operating conditions.
[[Algorithmic efficiency|optimality]], or [[Robustness (computer science)|robustness]] for a range of operating conditions.


== See also ==
== See also ==

Revision as of 08:34, 29 October 2024

Machine learning control (MLC) is a subfield of machine learning, intelligent control, and control theory which aims to solve optimal control problems with machine learning methods. Key applications are complex nonlinear systems for which linear control theory methods are not applicable.

Types of problems and tasks

Four types of problems are commonly encountered:

  • Control parameter identification: MLC translates to a parameter identification[1] if the structure of the control law is given but the parameters are unknown. One example is the genetic algorithm for optimizing coefficients of a PID controller[2] or discrete-time optimal control.[3]
  • Control design as regression problem of the first kind: MLC approximates a general nonlinear mapping from sensor signals to actuation commands, if the sensor signals and the optimal actuation command are known for every state. One example is the computation of sensor feedback from a known full state feedback. Neural networks are commonly used for such tasks.[4]
  • Control design as regression problem of the second kind: MLC may also identify arbitrary nonlinear control laws which minimize the cost function of the plant. In this case, neither a model, the control law structure, nor the optimizing actuation command needs to be known. The optimization is only based on the control performance (cost function) as measured in the plant. Genetic programming is a powerful regression technique for this purpose.[5]
  • Reinforcement learning control: The control law may be continually updated over measured performance changes (rewards) using reinforcement learning.[6]

Applications

MLC has been successfully applied to many nonlinear control problems, exploring unknown and often unexpected actuation mechanisms. Example applications include:

Many more engineering MLC application are summarized in the review article of PJ Fleming & RC Purshouse (2002).[11]

As is the case for all general nonlinear methods, MLC does not guarantee convergence, optimality, or robustness for a range of operating conditions.

See also

References

  1. ^ Thomas Bäck & Hans-Paul Schwefel (Spring 1993) "An overview of evolutionary algorithms for parameter optimization", Journal of Evolutionary Computation (MIT Press), vol. 1, no. 1, pp. 1-23
  2. ^ a b N. Benard, J. Pons-Prats, J. Periaux, G. Bugeda, J.-P. Bonnet & E. Moreau, (2015) "Multi-Input Genetic Algorithm for Experimental Optimization of the Reattachment Downstream of a Backward-Facing Step with Surface Plasma Actuator", Paper AIAA 2015-2957 at 46th AIAA Plasmadynamics and Lasers Conference, Dallas, TX, USA, pp. 1-23.
  3. ^ Zbigniew Michalewicz, Cezary Z. Janikow & Jacek B. Krawczyk (July 1992) "A modified genetic algorithm for optimal control problems", [Computers & Mathematics with Applications], vol. 23, no 12, pp. 83-94.
  4. ^ C. Lee, J. Kim, D. Babcock & R. Goodman (1997) "Application of neural networks to turbulence control for drag reduction", Physics of Fluids, vol. 6, no. 9, pp. 1740-1747
  5. ^ D. C. Dracopoulos & S. Kent (December 1997) "Genetic programming for prediction and control", Neural Computing & Applications (Springer), vol. 6, no. 4, pp. 214-228.
  6. ^ Andrew G. Barto (December 1994) "Reinforcement learning control", Current Opinion in Neurobiology, vol. 6, no. 4, pp. 888–893
  7. ^ Dimitris. C. Dracopoulos & Antonia. J. Jones (1994) Neuro-genetic adaptive attitude control, Neural Computing & Applications (Springer), vol. 2, no. 4, pp. 183-204.
  8. ^ Jonathan A. Wright, Heather A. Loosemore & Raziyeh Farmani (2002) "Optimization of building thermal design and control by multi-criterion genetic algorithm, [Energy and Buildings], vol. 34, no. 9, pp. 959-972.
  9. ^ Steven J. Brunton & Bernd R. Noack (2015) Closed-loop turbulence control: Progress and challenges, Applied Mechanics Reviews, vol. 67, no. 5, article 050801, pp. 1-48.
  10. ^ J. Javadi-Moghaddam, & A. Bagheri (2010 "An adaptive neuro-fuzzy sliding mode based genetic algorithm control system for under water remotely operated vehicle", Expert Systems with Applications, vol. 37 no. 1, pp. 647-660.
  11. ^ Peter J. Fleming, R. C. Purshouse (2002 "Evolutionary algorithms in control systems engineering: a survey" Control Engineering Practice, vol. 10, no. 11, pp. 1223-1241

Further reading

  • Dimitris C Dracopoulos (August 1997) "Evolutionary Learning Algorithms for Neural Adaptive Control", Springer. ISBN 978-3-540-76161-7.
  • Thomas Duriez, Steven L. Brunton & Bernd R. Noack (November 2016) "Machine Learning Control - Taming Nonlinear Dynamics and Turbulence", Springer. ISBN 978-3-319-40624-4.
Retrieved from "https://en.wikipedia.org/w/index.php?title=Machine_learning_control&oldid=1254071392"