Professor Jan Křetínský, who joined the Faculty of Informatics at Masaryk University less than two years ago, will use his ERC Consolidator grant from the European Research Council to find out how to ensure the reliability of “cyber physical systems” with the help of artificial intelligence.
“In this project, we will try to verify the reliability of cyber physical systems and design reliable controllers for them to meet not only traditional challenges, such as system complexity, but also modern challenges related to the use of artificial intelligence. To look for solutions, we will integrate artificial intelligence into traditional verification methods,” says Křetínský.
Cyber physical systems are computer controlled devices and machines including production lines and even self driving cars. These systems integrate cybernetics with the physical environment and a key requirement is that they are set up and operating as expected and safely with respect to their surroundings. Verifying their reliability is challenging even now and the challenges increase with the potential involvement of artificial intelligence in the decision making mechanisms of these systems. “The problem is that the subsystems controlled by artificial intelligence are unreliable and they have a complex structure that we often do not understand, which makes it really difficult to analyse and verify its operation,” says Křetínský about the research topic that won him the grant.
His goal is to develop verification tools that will enable a safe deployment of artificial intelligence in cyber physical systems. In principle, traditional human designed verification algorithms run in a similar way regardless of the system they are used for. However, what Křetínský wants to achieve is for AI to look at a particular system, try to understand its structure, and then adapt an existing algorithm or design a new algorithm tailor made for that system to allow it to operate more efficiently while maintaining a specified level of reliability and security.
“It will be very challenging because the current AI is inherently unreliable and we want to use it in situations where reliability is our primary concern. We will try to combine the currently slow and well tested algorithms created by people with artificial intelligence, which is fast and good at coming up with new ideas but also erratic.”
Křetínský adds that in order to use artificial intelligence more safely, we have to either use it for less complex systems, or to set limits for its operation. “For example, we can use artificial intelligence to optimise fuel consumption. If we give it free rein, it will drive more economically than any human, but then we cannot guarantee that the car will drive safely. So, to be able to guarantee that the car will behave on the road the way we want it to, we have to limit it in some way, which will also reduce its efficiency in terms of fuel economy,” says Křetínský.
The project will involve Křetínský’s team of students and PhD students as well as experts from all over the world and will include documenting how synthesized controllers behave and how they make decisions. The current European legislation requires systems that use AI to be documented well enough that, if necessary, it can be clearly established why the AI made a particular decision in a particular situation and whether its decision was the right one. “Going back to the car example, this documentation is necessary so that in case of an accident we can say why the car decided the way it did and that it was the right decision to make.”
Křetínský has been working on AI assisted verification of computer systems for ten years. Before arriving at Masaryk University, he worked at the Technical University of Munich. The unique MUNI Award in Science and Humanities (MASH) grant, which Masaryk University uses to attract top international and Czech scientists, brought him back to the Faculty of Computer Science in Brno, where he originally received his PhD in parallel with the University of Munich.
“Working at the Technical University of Munich, which is in the top ten universities in computer science worldwide, gave me the opportunity to learn from the best. This has given me a huge boost and I was able to gain experience that would have been difficult to access in the Czech Republic. I was motivated to return by my family and also by the fact that the MASH grant offered conditions that allowed me to move a part of my German team under reasonable conditions and seamlessly continue building on my previous research,” says Křetínský, adding that part of his team remains in Munich, bringing the two universities closer together.