- The Friendly Hackers team from Thales, a world leader in data protection and cybersecurity, wins CAID[1] Challenge organized by the French Ministry of Defense2 During the 5th European Cyber Week (21-23 November 2023) in France.
- The challenge is the first of its kind, sponsored by the French Ministry of Defense, and aims to assess the extent to which hacker teams are able to exploit certain inherent vulnerabilities in AI models.
- Thales’ commitment to AI security and trust aligns with the requirements of both the defense community and civilian organizations such as critical infrastructure providers. All of these organizations face the same challenge of protecting training datasets and intellectual property and ensuring that results produced by AI are securely available for use. Trusted in making important decisions.
French Ministry of Defense’s AI security challenges
Participants in the CAID Challenge had to perform two tasks:
- For a given set of images, determine which images were used to train an AI algorithm and which images were used for testing.
AI-based image recognition applications learn from large amounts of training images. By studying the inner workings of the AI model, his Friendly Hackers team at Thales has managed to identify some of the images used to create the application, providing valuable insight into the training method used and the quality of the model. I got some information.
2. Find images of the aircraft used by the AI algorithm that were protected using “unlearning” techniques.
“Unlearning” techniques consist of removing data, such as images, used to train the model to maintain confidentiality. This technology can be used, for example, to protect the sovereignty of algorithms in the event of their export, theft, or loss. For example, an AI-equipped drone must be able to recognize any enemy aircraft as a potential threat. Models of friendly aircraft, on the other hand, must be trained to identify as friendly and then erased through a technique known as unlearning. This way, even if the drone is stolen or lost, the sensitive aircraft data contained in the AI model cannot be extracted for malicious purposes. However, Thales’ team of friendly hackers managed to succeed.Electronically identify data that should have been erased from the modelwhich overrides the unlearning process.
Such exercises can help you assess vulnerabilities in your training data and trained models. These are valuable tools and can not only provide great performance but also new attack vector For the military. Attacks on training data or trained models can have serious consequences in the military field, and this type of information can give an advantage to adversaries. Risks include theft of models, theft of data used to recognize military hardware and other capabilities within a theater of operations, and backdoors that compromise the operation of AI-powered systems. While AI in general, and generative AI in particular, offers significant operational benefits, providing military personnel with centrally trained decision support tools and reducing cognitive burden, the defense community is seeing new developments in this technology as a priority. threats must be addressed.
Thales BattleBox approach to addressing AI vulnerabilities
In the defense sector, protecting training data and trained models is critical. AI Cyber Security is becoming increasingly important and must be autonomous to thwart the many new opportunities that the world of AI opens up to malicious actors. In response to the risks and threats associated with the use of artificial intelligence, Thales has developed a series of countermeasures, including: battle box Increase protection against potential breaches.
- battle box training Protect against training data poisoning and prevent hackers from introducing backdoors.
- battle box ip Digitally watermark your AI models to ensure authenticity and trustworthiness.
- Battle box avoidance It is intended to protect the model from prompt injection attacks. Prompt injection attacks use Large-Scale Language Models (LLMs) to manipulate prompts to bypass chatbot security measures, including adding patches to images to fool the detection process of classification models. Can counter hostile attacks. .
- Battlebox privacy Provides a framework for training machine learning algorithms that uses advanced cryptography and secure secret sharing protocols to ensure a high level of confidentiality.
In the case of CAID challenge tasks, one possible solution to prevent AI from being hacked is to encrypt the AI model.
“AI offers significant operational benefits, but requires high levels of security and cybersecurity protection to prevent data breaches and misuse. We are implementing a wide range of AI-based solutions, aiming to be explainable and embeddable”. We have significant expertise. This is to significantly improve operational capabilities. ” Said David Sadek, Thales Vice President, Research, Technology and Innovation, Artificial Intelligence;
Thales and AI
Thales has developed an ethical and scientific framework for security development, as the Group’s defense and security businesses address critical requirements that often concern the safety of human life. Trustworthy AI Based on the four strategic pillars of Effectiveness, Safety, Accountability, Responsibility. Thales’ solutions combine the above know-how. 300 senior AI experts and over 4,500 cybersecurity experts It leverages the operational expertise of the Group’s aerospace, land defense, naval defense, space and other defense and security businesses.
Thales has developed the technical capabilities necessary to test the security of AI algorithms and neural network architectures, detect vulnerabilities, and recommend effective countermeasures. Thales’ friendly hacker team On the basis of ThereSIS Institute in Palaiseau They were one of about 12 teams to take part in the AI Challenge and won first place in both tasks.
Thales ITSEF (Information Technology Security Assessment Institute) is accredited by the French National Cyber Security Agency (ANSSI) to conduct pre-certification security assessments. During European Cyber Week, the ITSEF team also gave a presentation. The world’s first project of its kind aimed at Exploiting processor electromagnetic emissions to compromise embedded AI decisions.
Mr. Thales’ Cybersecurity consulting and auditing The team makes these tools and methodologies available to customers who want to develop their own AI models or establish a framework for the use and training of commercial models.
About Thales
Thales (Euronext Paris: HO) is a global leader in advanced technologies in three areas: defense and security, aerospace and digital identity and security. We develop products and solutions that make the world safer, greener and more inclusive.
The Group invests nearly €4 billion a year in research and development in key areas such as quantum technology, edge computing, 6G and cybersecurity, among others.
Thales has 77,000 employees in 68 countries. In 2022, the Group’s turnover amounted to 17.6 billion euros.