Learn more about the use case:

FZI Forschungszentrum Informatik

Research Project: Software Engineering for AI, Robotics/Autonomous Systems

27.04.2020

CyberProtect - Safeguarding AI components

The CyberProtect project, which is funded by the Baden-Württemberg Ministry of Economics, Labor and Housing, aims to strengthen SMEs in Baden-Württemberg by improving the security of complex software systems. All three areas of security (security, safety and privacy) are considered, with the focus being on the sub-area of security.

Within the framework of the project, methods are being developed to make the behavior and decisions of complex software systems, e.g. AI systems, visible and thus enable statements to be made about the security status of the systems. SMEs are involved in the project by means of a wide range of services such as quick checks, training courses and open lab days in order to offer them the opportunity to have their complex software examined for security and, if necessary, improved.

The following describes one aspect of the project:

What was the assignment for the collaboration/development of an AI-based use case/solution/service?
The assignment was to look at current methods and technologies used in connection with AI. Based on this, various attack possibilities were researched and evaluated so that measures for improving the functional safety of manufacturing systems with AI components could be derived.

What technologies and AI methods were used and why?
No specific AI methods were used, but software-technical means were considered to additionally secure AI methods. Self-adaptive systems were used to observe additional environmental variables. The goal of self-adaptive Systems is to predict potential fault conditions and proactively intervene in the functional execution of the actual system.

With which partners was the project realized with which respective parts?
In addition to the FZI Research Center for Information Technology, the project's consortium partners include the Fraunhofer Institute of Optronics, System Technologies and Image Exploitation (IOSB) and the Fraunhofer Institute for Manufacturing Engineering and Automation.

How did you contribute to the transfer of knowledge and technology to the client?
As part of the CyberProtect project, we developed a showcase that prototypes a self-adaptive system and provides additional protection for an AI-based vision component.

In what timeframe did they implement the AI solution in your company?
Because it was only a showcase, the development time was about 2 months.

A conclusion to your AI solution:
The showcase illustrated that even slight influences of external factors (such as changes in the brightness of images) force an AI-based vision component to make incorrect calculations and thus lead to potential errors. Such potential errors must be identified and prevented at an early stage.

One final piece of advice/tip to other entrepreneurs looking to apply AI:
Depending on the complexity of the AI method used and considering the context of use (safety-critical systems), additional risk analyses are recommended. Miscalculations of AI components should be planned for and compensated for by additional mechanisms (e.g., software-technical).

More info on the project as a whole at:
https://www.fzi.de/forschung/angewandte-ki/