+ | Name | Company | Country | Contribution |
|---|
+ | Adelin Travers | Trail of Bits | | |
+ | Alon Tron | Stealth | Israel | Improved supply chain management |
+ | Angie Qarry | QDeepTech | Austria | several elaborations and references on datascience defence mechanisms |
+ | Annegrit Seyerlein-Klug | TH Brandenburg | Germany | mapping with misc. standards |
+ | Anthony Glynn | CapitalOne | US | many textual improvements & link to LLM top 10 |
+ | Behnaz Karimi | Accenture | Germany | misc. contributions including model obfuscation and explanation |
+ | Disesdi Susanna Cox | BobiHealth | US | Federative learning |
+ | Feiyang Tang | Software Improvement Group (SIG) | | |
+ | John Sotiropoulos | Kainos | | |
+ | Marko Lihter | SplxAI | Croatia | step-by-step guide for organizations, website creation, various textual improvements |
+ | Niklas Bunzel | Fraunhofer institute | Germany | datascience discussion and references around evasion attacks |
+ | Rob van der Veer | Software Improvement Group (SIG) | Netherlands | Project leader |
+ | Roger Sanz | Universidad Isabel | Spain | |
+ | Sandy Dunn | Boise State University, AI Cyber Advisors | US | |
+ | Sean Oesch | Oak Ridge National Laboratory | US | BLUF, Adversarial Training, OOD detection, NISTIR 8269, Guide Usability/Structure |
+ | Srajan Gupta | Dave | | |
+ | Steve Francolla | Workforce Tech LLC | | |
+ | Wei Wei | IBM | Germany | mapping with ISO/IEC 42001 |
+ | Yiannis Kanellopoulos and team | Code4thought | Greece | evasion robustness |
+ | Zoe Braiterman | Mutual Knowledge Systems | US | Many markdown improvements |
+
+
+{{< /html-tab >}}
{{< tab >}}
-
-
-- Adelin Travers - Trail of Bits
-- Alon Tron
-- Anthony Glynn - Capital One
-- Behnaz Karimi - Accenture
-- Feiyang Tang - Software Improvement Group (SIG)
-- John Sotiropoulos - Kainos
-- Marko Lihter - SplxAI
-- Niklas Bunzel - Fraunhofer SIT
-- Rob van - der Veer|Software Improvement Group (SIG)
-- Roger Sanz - SIA Group
-- Sandy Dunn - Boise State University, AI Cyber Advisors
-- Srajan Gupta - Dave
-- Steve Francolla - Workforce Tech LLC
-
-{{< /tab >}}
-{{< tab >}}
-
-
-- Rob van der Veer (SIG, Netherlands) - results from AI security research at SIG: threat model, risks, navigator, matrix, risk approach, controls, gap analysis with ISO
-- Yiannis Kanellopoulos and team (Code4thought, Greece) - evasion robustness
-- Annegrit Seyerlein-Klug (TH Brandenburg, Germany) - mapping with misc. standards
-- Wei Wei (IBM, Germany) - mapping with ISO/IEC 42001
-- Roger Sanz (Universidad Isabel, Spain)
-- Angie Qarry (QDeepTech, Austria) - some elaborations and references on datascience defence mechanisms
-- Behnaz Karimi (Accenture, Germany)- misc. contributions including model obfuscation and explanation
-- Sean Oesch (Oak Ridge National Laboratory, US) - BLUF, Adversarial Training, OOD detection, NISTIR 8269, Guide Usability/Structure
-- Anthony Glynn (CapitalOne, US) - many textual improvements & link to LLM top 10
-- Zoe Braiterman (Mutual Knowledge Systems, US) - Many markdown improvements
-- Niklas Bunzel (Fraunhofer institute, Germany) - datascience discussion and references around evasion attacks
-- Marko Lihter (SplxAI, Croatia) - various textual improvements & the Exchange website
-- Milad Masoodi (SIG, Netherlands) - restructured document to put controls in sections, visible in the TOC
-
-
-{{< /tab >}}
-{{< tab >}}
-
-
-
{{< /tab >}}
{{< tab >}}
@@ -116,6 +103,7 @@ If you're unsure about anything, feel free to [reach out to us](/connect) with y
- Under INPUTDISTORTION: See ENISA Annex C to add data randomisation, input transformation and input denoising.
- Under INPUTDISTORTION: add Gradient masking - Annex C ENISA 2021
- Cover integrity checks in development pipeline (build, deploy, supply chain) - under supplychainmanage and/or secdevprogram
+- Create an overall community outreach marketing plan, and regional outreach plans.
## TODOs requiring access to ISO/IEC documents
diff --git a/content/ai_exchange/content/docs/1_general_controls.md b/content/ai_exchange/content/docs/1_general_controls.md
index 41b6c826..e4172aec 100644
--- a/content/ai_exchange/content/docs/1_general_controls.md
+++ b/content/ai_exchange/content/docs/1_general_controls.md
@@ -19,21 +19,21 @@ See Risk management under SECPROGRAM for security-specific risk analysis.
Note that an AI program is not just about risk TO AI, such as security risks - it is also about risks BY AI, such as threats to fairness, safety, etc.
Links to standards:
- - ISO/IEC 42001 AI management system (under development). Gap: covers this control fully.
+ - ISO/IEC 42001 AI management system. Gap: covers this control fully.
-42001 is about extending your risk management system - it focuses on governance. 5338 (see #DEVPROGRAM below) is about extending your software lifecycle practices - it focuses on engineering and everything around it. The 42001 can be seen as a management system for the governance of responsible AI in an organization, similar to how 27001 is a management system for information security. The 42001 doesnāt go into the lifecycle processes. It for example does not discuss how to train models, how to do data lineage, continuous validation, versioning of AI models, project planning challenges, and how and when exactly sensitive data is used in engineering.
+42001 is about extending your risk management system - it focuses on governance. 5338 (see [#DEVPROGRAM](#devprogram) below) is about extending your software lifecycle practices - it focuses on engineering and everything around it. The 42001 can be seen as a management system for the governance of responsible AI in an organization, similar to how 27001 is a management system for information security. The 42001 doesnāt go into the lifecycle processes. It for example does not discuss how to train models, how to do data lineage, continuous validation, versioning of AI models, project planning challenges, and how and when exactly sensitive data is used in engineering.
#### #SECPROGRAM
(management). Having a security program. Include the whole AI lifecycle and AI particularities in the organization's security program (also referred to as _information security management system_).
-Make sure to include AI-specific threats and assets (e.g. assets the development environment includign AI Ops / ML Ops).
+Make sure to include AI-specific threats and assets (e.g. assets the development environment including AI Ops / ML Ops).
Purpose: reduces probability of AI initiatives being overlooked for information security management, vastly decreasing security risk as the security program takes responsibility for the AI-specific threats and corresponding controls in this document. For more details on using this document in risk analysis, see the Introduction section.
Particularity: the AI lifecycle and its specific assets and security threats need to be part of the organization's information security governance.
-Because AI has specific assets (e.g. training data), **AI-speific honeypots** are a partiularly interesting control. These are fake parts of the data/model/datascience infrastucture that are exposed on purpose, in order to detect or capture attackers, before they succeed to access the real assets. Examples:
+Because AI has specific assets (e.g. training data), **AI-specific honeypots** are a partiularly interesting control. These are fake parts of the data/model/datascience infrastucture that are exposed on purpose, in order to detect or capture attackers, before they succeed to access the real assets. Examples:
- Hardened data services, but with an unpatched vulnerability (e.g. Elasticsearch)
- Exposed data lakes, not revealing details of the actual assets
@@ -92,7 +92,7 @@ Links to standards:
- See [OpenCRE on secure software development processes](https://www.opencre.org/cre/616-305) with notable links to NIST SSDF and OWASP SAMM. Gap: covers this control fully, with said particularity
#### #DEVPROGRAM
-(management). Having a development program for AI. Apply general (not just security-oriented) software engineering best practices to AI development.
+(management). Having a development lifecycle program for AI. Apply general (not just security-oriented) software engineering best practices to AI development.
Data scientists are focused on creating working models, not on creating future-proof software per se. Often, organizations already have software practices and processes in place. It is important to extend these to AI development, instead of treating AI as something that requires a separate approach. Do not isolate AI engineering. This includes automated testing, code quality, documentation, and versioning. ISO/IEC 5338 explains how to make these practices work for AI.
@@ -104,9 +104,12 @@ Another best practice is to continuously measure quality aspects of data science
Apart from conventional software best practices, there are important AI-specific engineering practices, including for example data provenance & lineage, model traceability and AI-specific testing such as continuous validation, testing for model staleness and concept drift. ISO/IEC 5338 discussess these AI engineering practices.
+The below interpretation diagram of ISO/IEC 5338 provides a good overview to get an idea of the topics involved.
+
+
Links to standards:
- - [ISO/IEC 5338 - AI lifecycle](https://www.iso.org/standard/81118.html) Gap: covers this control fully - the 5338 covers the complete software development lifecycle for AI, by extending the existing 12207 standard on software lifecycle: defining several new processes and discussing AI-specific particularities for existing processes.
+ - [ISO/IEC 5338 - AI lifecycle](https://www.iso.org/standard/81118.html) Gap: covers this control fully - the 5338 covers the complete software development lifecycle for AI, by extending the existing 12207 standard on software lifecycle: defining several new processes and discussing AI-specific particularities for existing processes. See also [this blog](https://www.softwareimprovementgroup.com/iso-5338-get-to-know-the-global-standard-on-ai-systems/).
- 27002 control 5.37 Documented operating procedures. Gap: covers this control minimally - this covers only a very small part of the control
- [OpenCRE on documentation of function](https://www.opencre.org/cre/162-655) Gap: covers this control minimally
@@ -175,12 +178,20 @@ Adding sufficient noise to training data can make it effectively uncrecognizable
- Objective function perturbation
-In privacy-preserving machine learning, objective function perturbation is a technique to enhance training data privacy. It introduces noise or modifications to the objective function, adding controlled randomness to make it difficult for adversaries to extract specific details about individual data points. This method contributes to achieving differential privacy, preventing the undue influence of a single data point on the model's behavior.
+Objective function perturbation is a differential privacy technique used to train machine learning models while maintaining data privacy. It involves the intentional introduction of a controlled amount of noise into the learning algorithmās objective function, which is a measure of the discrepancy between a modelās predictions and the actual results. The perturbation, or slight modification, involves adding noise to the objective function, resulting in a final model that doesnāt exactly fit the original data, thereby preserving privacy. The added noise is typically calibrated to the objective functionās sensitivity to individual data points and the desired privacy level, as quantified by parameters like epsilon in differential privacy. This ensures that the trained model doesnāt reveal sensitive information about any individual data point in the training dataset. The main challenge in objective function perturbation is balancing data privacy with the accuracy of the resulting model. Increasing the noise enhances privacy but can degrade the modelās accuracy. The goal is to strike an optimal balance where the model remains useful while individual data points stay private.
+
+References:
+
+- [Differentially Private Objective Perturbation: Beyond Smoothness and Convexity](https://arxiv.org/abs/1909.01783v1)
- Masking
-
-Masking encompasses the intentional concealment or modification of sensitive information within training datasets to enhance privacy during the development of machine learning models. This is achieved by introducing a level of obfuscation through techniques like data masking or feature masking, where certain elements are replaced, encrypted, or obscured, preventing unauthorized access to specific details. This approach strikes a balance between extracting valuable data insights and safeguarding individual privacy, contributing to a more secure and privacy-preserving data science process.
-
+
+Masking involves the alteration or replacement of sensitive features within datasets with alternative representations that retain the essential information required for training while obscuring sensitive details. Various methods can be employed for masking, including tokenization, perturbation, generalization, and feature engineering. Tokenization replaces sensitive text data with unique identifiers, while perturbation adds random noise to numerical data to obscure individual values. Generalization involves grouping individuals into broader categories, and feature engineering creates derived features that convey relevant information without revealing sensitive details. Once the sensitive features are masked or transformed, machine learning models can be trained on the modified dataset, ensuring that they learn useful patterns without exposing sensitive information about individuals. However, achieving a balance between preserving privacy and maintaining model utility is crucial, as more aggressive masking techniques may lead to reduced model performance.
+
+References:
+
+- [Data Masking with Privacy Guarantees]([https://arxiv.org/abs/1909.01783v1](https://arxiv.org/abs/1901.02185))
+
- Encryption
Encryption is a fundamental technique for pseudonymization and data protection. It underscores the need for careful implementation of encryption techniques, particularly asymmetric encryption, to achieve robust pseudonymization. Emphasis is placed on the importance of employing randomized encryption schemes, such as Paillier and Elgamal, to ensure unpredictable pseudonyms. Furthermore, homomorphic encryption, which allows computations on ciphertexts without the decryption key, presents potential advantages for cryptographic operations but poses challenges in pseudonymization. The use of asymmetric encryption for outsourcing pseudonymization and the introduction of cryptographic primitives like ring signatures and group pseudonyms in advanced pseudonymization schemes are important.
@@ -209,7 +220,6 @@ In the healthcare sector with personally identifiable information (PII), there a
These use cases demonstrate the practical relevance and applicability of pseudonymization techniques in real-world scenarios, offering valuable insights for stakeholders involved in data pseudonymization and data protection.
-
Links to standards:
@@ -262,9 +272,19 @@ Example: LLMs (GenAI), just like most AI models, induce their results based on t
**Controls to limit the effects of unwanted model behaviour:**
#### #OVERSIGHT
-(runtime). Oversight of model behaviour by humans or business logic (guardrails).
+(runtime). Oversight of model behaviour by humans or business logic in the form of rules (guardrails).
-Purpose: Detect unwanted model behavior and correct or halt the execution of a model's decision. Note: Unwanted model behavior often cannot be entirely specified, limiting the effectiveness of guardrails.
+Purpose: Detect unwanted model behavior and correct or halt the execution of a model's decision.
+
+**Limitations of guardrails:**
+The properties of wanted or unwanted model behavior often cannot be entirely specified, limiting the effectiveness of guardrails.
+
+**Limitations of human oversight:**
+The alternative to guardrails is to apply human oversight. This is of course more costly and slower, but allows for more intelligent validation given the involved common sense and human domain knowledge - provided that the person performing the oversight actually has that knowledge.
+For human operators or drivers of automated systems like self-driving cars, staying actively involved or having a role in the control loop helps maintain situational awareness. This involvement can prevent complacency and ensure that the human operator is ready to take over control if the automated system fails or encounters a scenario it cannot handle. However, maintaining situational awareness can be challenging with high levels of automation due to the "out-of-the-loop" phenomenon, where the human operator may become disengaged from the task at hand, leading to slower response times or decreased effectiveness in managing unexpected situations.
+In other words: If you as a user are not involved actively in performing a task, then you lose understanding of whether it is correct or what the impact can be. If you then only need to confirm something by saying 'go ahead' or 'cancel', a badly informed 'go ahead' is easy to pick.
+
+Designing automated systems that require some level of human engagement or regularly update the human operator on the system's status can help maintain situational awareness and ensure safer operations.
Examples:
diff --git a/content/ai_exchange/content/docs/2_threats_through_use.md b/content/ai_exchange/content/docs/2_threats_through_use.md
index f45032f0..0ed6bde7 100644
--- a/content/ai_exchange/content/docs/2_threats_through_use.md
+++ b/content/ai_exchange/content/docs/2_threats_through_use.md
@@ -81,32 +81,32 @@ Detect odd input: implement tools to detect whether input is odd: significantly
Purpose: Odd input can result in unwanted model behaviour because the model by definition has not seen this data before and will likely produce false results, whether the input is malicious or not. When detected, the input can be logged for analysis and optionally discarded. It is important to note that not all odd input will be malicious and not all malicious input will be odd. There are examples of adversarial input specifically crafted to bypass detection of odd input. Nevertheless, detecting odd input is critical to maintaining model integrity, addressing potential concept drift, and preventing adversarial attacks that may take advantage of model behaviors on out of distribution data.
-**Types of detecting odd input**
+**Types of detecting odd input**
Out-of-Distribution Detection (OOD), Novelty Detection (ND), Outlier Detection (OD), Anomaly Detection (AD), and Open Set Recognition (OSR) are all related and sometimes overlapping tasks that deal with unexpected or unseen data. However, each of these tasks has its own specific focus and methodology. In practical applications, the techniques used to solve the problems may be similar or the same. Which task or problem should be addressed and which solution is most appropriate also depends on the definition of in-distribution and out-of-distribution. We use an example of a machine learning system designed for a self-driving car to illustrate all these concepts.
-##### Out-of-Distribution Detection (OOD) - the broad category of detecting odd input:
+**Out-of-Distribution Detection (OOD)** - the broad category of detecting odd input:
Identifying data points that differ significantly from the distribution of the training data. OOD is a broader concept that can include aspects of novelty, anomaly, and outlier detection, depending on the context.
Example: The system is trained on vehicles, pedestrians, and common animals like dogs and cats. One day, however, it encounters a horse on the street. The system needs to recognize that the horse is an out-of-distribution object.
Methods for detecting out-of-distribution (OOD) inputs incorporate approaches from outlier detection, anomaly detection, novelty detection, and open set recognition, using techniques like similarity measures between training and test data, model introspection for activated neurons, and OOD sample generation and retraining. Approaches such as thresholding the output confidence vector help classify inputs as in or out-of-distribution, assuming higher confidence for in-distribution examples. Techniques like supervised contrastive learning, where a deep neural network learns to group similar classes together while separating different ones, and various clustering methods, also enhance the ability to distinguish between in-distribution and OOD inputs. For more details, one can refer to the survey by [Yang et al.](https://arxiv.org/pdf/2110.11334.pdf) and other resources on the learnability of OOD: [here](https://arxiv.org/abs/2210.14707).
-##### Outlier Detection (OD) - a form of OOD:
+**Outlier Detection (OD)** - a form of OOD:
Identifying data points that are significantly different from the majority of the data. Outliers can be a form of anomalies or novel instances, but not all outliers are necessarily out-of-distribution.
Example: Suppose the system is trained on cars and trucks moving at typical city speeds. One day, it detects a car moving significantly faster than all the others. This car is an outlier in the context of normal traffic behavior.
-##### Anomaly Detection (AD) - a form of OOD:
+**Anomaly Detection (AD)** - a form of OOD:
Identifying abnormal or irregular instances that raise suspicions by differing significantly from the majority of the data. Anomalies can be outliers, and they might also be out-of-distribution, but the key aspect is their significance in terms of indicating a problem or rare event.
Example: The system might flag a vehicle going the wrong way on a one-way street as an anomaly. It's not just an outlier; it's an anomaly that indicates a potentially dangerous situation.
-##### Open Set Recognition (OSR - a way to perform Anomaly Detection):
+**Open Set Recognition (OSR)** - a way to perform Anomaly Detection):
Classifying known classes while identifying and rejecting unknown classes during testing. OSR is a way to perform anomaly detection, as it involves recognizing when an instance does not belong to any of the learned categories. This recognition makes use of the decision boundaries of the model.
Example: During operation, the system identifies various known objects such as cars, trucks, pedestrians, and bicycles. However, when it encounters an unrecognized object, such as a fallen tree, it must classify it as "unknown. Open set recognition is critical because the system must be able to recognize that this object doesn't fit into any of its known categories.
-##### Novelty Detection (ND) - OOD input that is recognized as not malicious:
+**Novelty Detection (ND)** - OOD input that is recognized as not malicious:
OOD input data can sometimes be recognized as not malicious and relevant or of interest. The system can decide how to respond: perhaps trigger another use case, or log is specifically, or let the model process the input if the expectation is that it can generalize to produce a sufficiently accurate result.
Example: The system has been trained on various car models. However, it has never seen a newly released model. When it encounters a new model on the road, novelty detection recognizes it as a new car type it hasn't seen, but understands it's still a car, a novel instance within a known category.
diff --git a/content/ai_exchange/content/docs/3_development_time_threats.md b/content/ai_exchange/content/docs/3_development_time_threats.md
index 4cb945b0..131b461b 100644
--- a/content/ai_exchange/content/docs/3_development_time_threats.md
+++ b/content/ai_exchange/content/docs/3_development_time_threats.md
@@ -78,20 +78,43 @@ Links to standards:
#### #FEDERATIVELEARNING
(development-time datascience). Federative learning can be applied when a training set is distributed over different organizations, preventing that the data needs to be collected in a central place - increasing the risk of leaking.
+Federated Learning is a decentralized Machine Learning architecture wherein a number of edge clients (e.g., sensor or mobile devices) participate in collaborative, decentralized, asynchronous training, which is orchestrated and aggregated by a controlling central server. Advantages of Federated Learning include reduced central compute, and the potential for preservation of privacy, since training data may remain local to the edge/client device.
+
+Broadly, Federated Learning generally consists of four high-level steps: First, there is a server-to-client broadcast; next, local models are updated on the client; once trained, local models are then returned to the central server; and finally, the central server updates via model aggregation.
+
+Challenges in Federated Learning include managing device and model heterogeneity, latency in broadcast phases, and preservation of privacy. Security concerns also include backdoor attacks via data/model poisoning; with federated systems additionally introducing a vast network of edge clients, some of which may be malicious.
+
+Device Heterogeneity. User- or other edge devices may vary widely in their computational, storage, transmission, or other capabilities, presenting challenges for federated deployments. These may additionally introduce device-specific security concerns, which practitioners should take into consideration in design phases. While designing for constraints including connectivity, battery life, and compute, it is also critical to consider edge device security.
+
+Broadcast Latency & Security. Efficient communication across a federated network introduces additional challenges. While strategies exist to minimize broadcast phase latency, they must also take into consideration potential data security risks. Because models are vulnerable during transmission phases, any communication optimizations must account for data security in transit.
+
+Preservation of Privacy. Because data remain local to the client device, Federated Learning architectures are sometimes assumed to be fully privacy-preserving. However this is not the case, as sensitive data may still be extracted from the transmitted models themselves. Data may be revealed from models intercepted in transmission, or during the process of model aggregation by the central server. For this reason, additional privacy-enhancing measures may be applied to Federated Learning settings.
+
+Privacy-preserving mitigations for Federated Learning include cryptographic and information-theoretic strategies, such as Secure Function Evaluation (SFE), also known as Secure Multi-Party Computation (SMC/SMPC); and Differential Privacy. However, all approaches entail tradeoffs between privacy and utility.
+
+Backdoor Attacks. Federated Learning systems have shown particular vulnerability to data- and model-poisoning attacks. Two key aspects of federated systems may exacerbate this vulnerability in production. First, Federated Learning introduces a potentially vast number of edge clients, with few guarantees against malicious actors at the device level. Second, data localizationāintended to be privacy-preservingācircumvents typical centralized data curation strategies.
+
+Recent advances in Federated (on-device) Analytics may provide tools to address some of these issues. Practitioners should implement the fullest suite of MLSecOps tools possible to detect and mitigate backdoor attacks against Federated Learning systems.
+
+
Links to standards:
- Not covered yet in ISO/IEC standards
#### #SUPPLYCHAINMANAGE
-(development-time infosec) Supply chain management: Managing the supply chain to to minimize the security risk from externally obtained elements. In regular software engineering these elements are source code or software components (e.g. open source). The particularity for AI is that this also includes obtained data and obtained models.
+(development-time infosec) Supply chain management: Managing the supply chain to minimize the security risk from externally obtained elements. In regular software engineering these elements are source code or software components (e.g. open source). The particularities for AI are:
+1. supplied elements can include data and models,
+2. many of the software components are executed development-time instead of just in production (the runtime of the application),
+3. as explained in the development-time threats, there are new vulnerable assets during AI development: training data and model parameters.
-Security risks in obtained elements can arise from accidental mistakes or from manipulations - just like with obtained source code or software components.
+Security risks in obtained data or models can arise from accidental mistakes or from manipulations - just like with obtained source code or software components.
-Just like with obtained source code or software components, data or models may involve multiple suppliers. For example: a model is trained by one vendor and then fine-tuned by another vendor. Or: an AI system contains multiple models, one is a model that has been fine-tuned with data from source X, using a base model from vendor A that claims data is used from sources Y and Z, where the data from source Z was labeled by vendor B.
+The AI supply chain can be complex. Just like with obtained source code or software components, data or models may involve multiple suppliers. For example: a model is trained by one vendor and then fine-tuned by another vendor. Or: an AI system contains multiple models, one is a model that has been fine-tuned with data from source X, using a base model from vendor A that claims data is used from sources Y and Z, where the data from source Z was labeled by vendor B.
+Because of hise supply chain complexity, data and model provenance is a helpful activity. The Software Bill Of Materials (SBOM) becomes the AIBOM (AI Bill Of Materials) or MBOM (Model Bill of Material).
-Data provenance is a helpful activity to support supply chain management for obtained data. The Software Bill Of Materials (SBOM) becomes the AIBOM (AI Bill Of Materials) or MBOM (Model Bill of Material). AI systems often have a variation of supply chains, including the data supply chain, the labeling supply chain, and the model supply chain.
+Standard supply chain management includes provenance & pedigree, verifying signatures, using package repositories, frequent patching, and using dependency verification tools.
-Standard supply chain management includes provenance & pedigree, verifying signatures, using package repositories, frequent patching, and using dependency verification tools.
+As said, in AI many of the software components are executed development-time, instead of just in production. Data engineering and model engineering involve operations on data and models for which often external components are used (e.g. tools such as Notebooks, or other MLOps applications). Because AI development has new assets such as the data and model parameters, these components pose a new threat. To make matters worse, data scientists also install dependencies on the Notebooks which makes the data and model engineering environment a dangerous attack vector and the classic supply chain guardrails typically donāt scan it.
See [MITRE ATLAS - ML Supply chain compromise](https://atlas.mitre.org/techniques/AML.T0010).
@@ -154,7 +177,7 @@ Links to standards:
#### #DATAQUALITYCONTROL
(development-time datascience). Data quality control: Perform quality control on data including detecting poisoned samples through statistical deviation or pattern recognition. For important data and scenarios this may involve human verification.
-Particularity: standard quality control needs to take into account that data may have maliciously been changed.
+Particularity for AI and security: standard quality control needs to take into account that data may have maliciously been changed. This means that extra checks can be placed to detect changes that would normally not happen by themselves. For example: safely storing hash codes of data elements, such as images, and regularly checking to see if the images have been manipulated.
A method to detect statistical deviation is to train models on random selections of the training dataset and then feed each training sample to those models and compare results.
@@ -184,7 +207,12 @@ Link to standards:
- Not covered yet in ISO/IEC standards
#### #POISONROBUSTMODEL
-(development-time datascience). Poison robus model: select model types that are less sensitive to poisoned training data.
+(development-time datascience). Poison robust model: select a model type and approach to reduce sensitivity to poisoned training data.
+
+Example: Reducing sensitivity to backdoor poisoning attacks with **fine-pruning** (See [paper on fine-pruning](https://arxiv.org/pdf/1805.12185.pdf)). Fine-pruning consists of two complementary approaches: Pruning and fine-tuning:
+**Pruning** in essence reduces the size of the model so it does not have the capacity to trigger on backdoor-examples while remaining sufficient accuracy for the intended use case. The approach removes neurons in a neural network that have been identified as non-essential for sufficient accuracy.
+**Fine tuning** retrains a model on a clean dataset(without poisoning) with the intention to remove memorisation of any backdoor triggers.
+
Links to standards:
- Not covered yet in ISO/IEC standards
diff --git a/content/ai_exchange/content/docs/4_runtime_application_security_threats.md b/content/ai_exchange/content/docs/4_runtime_application_security_threats.md
index 518b783b..d4f0844f 100644
--- a/content/ai_exchange/content/docs/4_runtime_application_security_threats.md
+++ b/content/ai_exchange/content/docs/4_runtime_application_security_threats.md
@@ -128,9 +128,10 @@ Impact: Confidentiality breach of sensitive input data.
Input data can be sensitive (e.g. GenAI prompts) and can either leak through a failure or through an attack, such as a man-in-the-middle attack.
-GenAI models mostly live in the cloud - often managed by an external party, which may increase the risk of leaking training data and leaking prompts. This issue is not limited to GenAI, but GenAI has 2 particular risks here: 1) model use involves user interaction through prompts, adding user data and corresponding privacy/sensitivity issues, and 2) GenAI model input (prompts) can contain rich context information with sensitive data (e.g. company secrets). The latter issue occurs with *in context learning* or *Retrieval Augmented Generation(RAG)* (adding background information to a prompt): for example data from all reports ever written at a consultancy firm. First of all, this information will travel with the prompt to the cloud, and second: the system will likely not respect the original access rights to the information.
+GenAI models mostly live in the cloud - often managed by an external party, which may increase the risk of leaking training data and leaking prompts. This issue is not limited to GenAI, but GenAI has 2 particular risks here: 1) model use involves user interaction through prompts, adding user data and corresponding privacy/sensitivity issues, and 2) GenAI model input (prompts) can contain rich context information with sensitive data (e.g. company secrets). The latter issue occurs with *in context learning* or *Retrieval Augmented Generation(RAG)* (adding background information to a prompt): for example data from all reports ever written at a consultancy firm. First of all, this information will travel with the prompt to the cloud, and second: the system will likely not respect the original access rights to the information. Also see [Risk analysis](https://owaspai.org/docs/ai_security_overview/#how-to-select-relevant-threats-and-controls-risk-analysis) on the responsbility aspect.
**Controls:**
+- See General controls, in particular Minimizing data
#### #MODELINPUTCONFIDENTIALITY
-(runtime appsec). Model input confidentiality: see SECDEVPROGRAM to attain application security, with the focus on protecting the transport and storage of model parameters (e.g. access control, encryption, minimize retention)
+(runtime appsec). Model input confidentiality: see SECDEVPROGRAM to attain application security, with the focus on protecting the transport and storage of model input (e.g. access control, encryption, minimize retention)
diff --git a/content/ai_exchange/content/docs/_index.md b/content/ai_exchange/content/docs/_index.md
index e57475c7..7c32f296 100644
--- a/content/ai_exchange/content/docs/_index.md
+++ b/content/ai_exchange/content/docs/_index.md
@@ -3,9 +3,9 @@ title: Content
---
{{< cards >}}
- {{< card link="/docs/ai_security_overview/" title="AI Security Overview">}}
- {{< card link="/docs/1_general_controls/" title="1. General controls">}}
- {{< card link="/docs/2_threats_through_use/" title="2. Threats through use">}}
- {{< card link="/docs/3_development_time_threats/" title="3. Development-time threats">}}
- {{< card link="/docs/4_runtime_application_security_threats/" title="4. Runtime application security threats">}}
+ {{< small-card link="/docs/ai_security_overview/" title="AI Security Overview">}}
+ {{< small-card link="/docs/1_general_controls/" title="1. General controls">}}
+ {{< small-card link="/docs/2_threats_through_use/" title="2. Threats through use">}}
+ {{< small-card link="/docs/3_development_time_threats/" title="3. Development-time threats">}}
+ {{< small-card link="/docs/4_runtime_application_security_threats/" title="4. Runtime application security threats">}}
{{< /cards >}}
\ No newline at end of file
diff --git a/content/ai_exchange/content/docs/ai_security_overview.md b/content/ai_exchange/content/docs/ai_security_overview.md
index e20cec81..af16f66d 100644
--- a/content/ai_exchange/content/docs/ai_security_overview.md
+++ b/content/ai_exchange/content/docs/ai_security_overview.md
@@ -2,13 +2,10 @@
title: AI Security Overview
weight: 1
---
-See [home](https://owaspai.org/) for more information about this initiative, how to contribute or connect.
+## Summary - How to address AI Security?
+>See [home](https://owaspai.org/) for more information about this initiative, how to contribute or connect.
This page contains an overview of AI security and the next pages provide the main content: details on security threats to AI and controls against them. You can navigate through pages at the bottom of every page, or in the left sidebar. The right sidebar shows the different sections on a page. On small screens you can navigate through the menu.
-## Summary and visualisations
-
-### Short summary: How to address AI Security?
-
While AI offers powerful perfomance boosts, it also increases the attack surface available to bad actors. It is therefore imperative to approach AI applications with a clear understanding of potential threats and which of those threats to prioritize for each use case. Standards and governance help guide this process for individual entities leveraging AI capabilities.
- Implement **AI governance**
@@ -17,44 +14,65 @@ While AI offers powerful perfomance boosts, it also increases the attack surface
- **Limit the impact** of AI by minimizing privileges and adding oversight, e.g. guardrails, human oversight.
- **Countermeasures in data science** through understanding of model attacks, e.g. data quality assurance, larger training sets, detecting common perturbation attacks, input filtering.
-
+## Threats overview
-### Navigator diagram
-The navigator diagram below shows all threats, controls and how they relate, including risks and the types of controls.
-{{< callout type="info" >}}
- Click on the image to get a PDF with clickable links.
-{{< /callout >}}
-[](https://github.com/OWASP/www-project-ai-security-and-privacy-guide/raw/main/assets/images/owaspaioverviewpdfv3.pdf)
+### Threat model
+We distinguish three types of threats: during development-time (when data is obtained and prepared, and the model is trained/obtained), through using the model (providing input and reading the output), and by attacking the system during runtime (in production).
+The diagram shows the threats in these three groups as arrows. Each threat has a specific impact, indicated by letters referring to the Impact legend. The control overview section contains this diagram with groups of controls added.
+
### AI Security Matrix
-The AI security matrix below shows all threats and risks, ordered by attack surface and lifecycle.
+The AI security matrix below shows all threats and risks, ordered by type and impact.
[](https://github.com/OWASP/www-project-ai-security-and-privacy-guide/blob/main/assets/images/OwaspAIsecuritymatix.png)
-### Summary with controls
-
-How to address AI security, including all controls (in capitals - and discussed further on in the document):
+## Controls overview
-1. Implement governance processes for AI risk, and if not already there: governance of information security and software lifecycle:
+### Threat model with controls - general
+The below diagram puts the controls in the AI Exchange into groups and places these groups in the right lifecycle with the corresponding threats.
+
+The groups of controls form a summary of how to address AI security (controls are in capitals):
+1. **AI Governance**: implement governance processes for AI risk, and include AI into your processes for information security and software lifecycle:
>(AIPROGRAM, SECPROGRAM, DEVPROGRAM, SECDEVPROGRAM, CHECKCOMPLIANCE, SECEDUCATE)
-2. Apply technical IT security controls risk-based:
- - 2a Apply **standard** conventional IT security controls (e.g. 15408, ASVS, OpenCRE, ISO 27001 Annex A, NIST SP800-53) to the complete AI system and don't forget the new AI-specific parts :
+2. Apply conventional **technical IT security controls** risk-based, since an AI system is an IT system:
+ - 2a Apply **standard** conventional IT security controls (e.g. 15408, ASVS, OpenCRE, ISO 27001 Annex A, NIST SP800-53) to the complete AI system and don't forget the new AI-specific assets :
- Development-time: model & data storage, model & data supply chain, data science documentation:
>(DEVDATAPROTECT, DEVSECURITY, SEGREGATEDATA, SUPPLYCHAINMANAGE, DISCRETE)
- - Runtime: model storage, model use and model input/output:
+ - Runtime: model storage, model use, plug-ins, and model input/output:
>(RUNTIMEMODELINTEGRITY, RUNTIMEMODELIOINTEGRITY, RUNTIMEMODELCONFIDENTIALITY, MODELINPUTCONFIDENTIALITY, ENCODEMODELOUTPUT, LIMITRESOURCES)
- - 2b **Adapt** conventional IT security controls to make them more suitable for AI:
+ - 2b **Adapt** conventional IT security controls to make them more suitable for AI (e.g. which usage patterns to monitor for):
>(MONITORUSE, MODELACCESSCONTROL, RATELIMIT)
- 2c Adopt **new** IT security controls:
>(CONFCOMPUTE, MODELOBFUSCATION, PROMPTINPUTVALIDATION, INPUTSEGREGATION)
-3. Apply datascience security controls risk-based:
+3. Data scientists apply **datascience security controls** risk-based :
- 3a Development-timeĀ controls when developing the model:
>(FEDERATIVELEARNING, CONTINUOUSVALIDATION, UNWANTEDBIASTESTING, EVASIONROBUSTMODEL, POISONROBUSTMODEL, TRAINADVERSARIAL, TRAINDATADISTORTION, ADVERSARIALROBUSTDISTILLATION, FILERSENSITIVETRAINDATA, MODELENSEMBLE, MORETRAINDATA, SMALLMODEL, DATAQUALITYCONTROL)
- - 3b RuntimeĀ controls when running the model:
- >(CONTINUOUSVALIDATION, UNWANTEDBIASTESTING, DETECTODDINPUT, DETECTADVERSARIALINPUT, DOSINPUTVALIDATION, INPUTDISTORTION, FILTERSENSITIVEMODELOUTPUT, OBSCURECONFIDENCE)
-4. Limit the amount of data and the time it is stored:
+ - 3b RuntimeĀ controls to filter and detect attacks:
+ >(DETECTODDINPUT, DETECTADVERSARIALINPUT, DOSINPUTVALIDATION, INPUTDISTORTION, FILTERSENSITIVEMODELOUTPUT, OBSCURECONFIDENCE)
+4. **Minimize data:** Limit the amount of data in rest and in transit, and the time it is stored, development-time and runtime:
>(DATAMINIMIZE, ALLOWEDDATA, SHORTRETAIN, OBFUSCATETRAININGDATA)
-5. Limit the effectĀ of unwanted model behaviour:
- >(OVERSIGHT, LEASTMODELPRIVILEGE, AITRAINSPARENCY, EXPLAINABILITY, CONTINUOUSVALIDATION)
+5. **Control behaviour impact** as the model can behave in unwanted ways - by mistake or by manipulation:
+ >(OVERSIGHT, LEASTMODELPRIVILEGE, AITRAINSPARENCY, EXPLAINABILITY, CONTINUOUSVALIDATION, UNWANTEDBIASTESTING)
+
+
+All threats and controls are discussed in the further content of the AI Exchange.
+
+### Threat model with controls - GenAI trained/finetuned
+Below diagram restricts the threats and controls to Generative AI only, for situations in which **training or finetuning** is done by the organization (note: this is not very common given the high cost and required expertise).
+
+
+
+### Threat model with controls - GenAI as-is
+Below diagram restricts the threats and controls to Generative AI only where the model is used **as-is** by the organization. The provider (e.g. OpenAI) has done the training/finetuning. Therefore, some threats are the responsibility of the model provider (sensitive/copyrighted data, manipulation at the provider). Nevertheless, the organization that uses the model should take these risks into account and gain assurance about them from the provider.
+
+
+
+
+### Navigator diagram
+The navigator diagram below shows all threats, controls and how they relate, including risks and the types of controls.
+{{< callout type="info" >}}
+ Click on the image to get a PDF with clickable links.
+{{< /callout >}}
+[](https://github.com/OWASP/www-project-ai-security-and-privacy-guide/raw/main/assets/images/owaspaioverviewpdfv3.pdf)
## About this Document
@@ -98,14 +116,14 @@ There are many threats and controls described in this document. Your situation d
These are the responsbilities of the model maker, but be aware you may be effected by the unwanted results. The maker may take the blame for any issue, which would take care of confidentiality issues, but you would suffer effectively from any manipulated model behaviour.
- If your train data is not sensitive: ignore the confidentiality of train data threats
+ If your train data is not sensitive: ignore the confidentiality of train data threats. A special case is the threat of _membership inference_: this threat only applies when the **fact** that a person was part of the training set is harmful information about the person, for example when the trainset consists of criminals and their history to predict criminal careers: membership of that set gives away the person is a convicted or aledged criminal.
If your model is a GenAI model, ignore the following threats: evasion, model inversion. Also ignore prompt injection and insecure output handling if your GenAI model is NOT an LLM
If your model is not a GenAI model, ignore (direct) prompt injection, and insecure output handling
If your input data is not sensitive, ignore āleaking input dataā. If you use RAG, consider data you retrieve also as input data.
-2. **Arranging responsibility**: For each selected threat, determine who is responsible to address it. By default, the organization that builds and deploys the AI system is responsible, but building and deploying may be done by different organizations, and some parts of the building and deployment may be deferred to other organizations, e.g. hosting the model, or providing a cloud environment for the application to run. Some aspects are shared responsibilities.
+3. **Arranging responsibility**: For each selected threat, determine who is responsible to address it. By default, the organization that builds and deploys the AI system is responsible, but building and deploying may be done by different organizations, and some parts of the building and deployment may be deferred to other organizations, e.g. hosting the model, or providing a cloud environment for the application to run. Some aspects are shared responsibilities.
If components of your AI system are hosted, then you share responsibility regarding all controls for the relevant threats with the hosting provider. This needs to be arranged with the provider, using for example a responsibility matrix. Components can be the model, model extensions, your application, or your infrastructure.
@@ -152,7 +170,7 @@ Responsible or trustworthy AI include security, but not the other way around: th
AI Privacy can be divided into two parts:
1. The AI security threats and controls in this document that are about confidentiality and integrity of (personal) data (e.g. model inversion, leaking training data), plus the integrity of the model behaviour
-2. Threats and controls with respect to rights of the individual, as covered by privacy regulations such as the GDPR, including use limitation, consent, fairness, transparency, data accuracy, right of correction/objection/reasure/access. For an overview, see the [Privacy part of the OWASP AI guide](/docs/privacy/#how-to-deal-with-ai-privacy)
+2. Threats and controls with respect to rights of the individual, as covered by privacy regulations such as the GDPR, including use limitation, consent, fairness, transparency, data accuracy, right of correction/objection/reasure/access. For an overview, see the [Privacy part of the OWASP AI guide](https://owasp.org/www-project-ai-security-and-privacy-guide/)
### How about Generative AI (e.g. LLM)?
@@ -180,6 +198,7 @@ GenAI security particularities are:
GenAI References:
- [OWASP LLM top 10](https://llmtop10.com/)
+- [Demystifying the LLM top 10](https://blog.kloudzone.co.in/demystifying-the-owasp-top-10-for-large-language-model-applications/)
- [Impacts and risks of GenAI](https://arxiv.org/pdf/2306.13033.pdf)
### How about the NCSC/CISA guidelines?
diff --git a/content/ai_exchange/content/media.md b/content/ai_exchange/content/media.md
index a341e6c5..21088425 100644
--- a/content/ai_exchange/content/media.md
+++ b/content/ai_exchange/content/media.md
@@ -11,4 +11,4 @@ excludeSearch: true
| 6 Sep 2023 | The MLSecOps Podcast | A Holistic Approach to Understanding the AI Lifecycle and Securing ML Systems: Protecting AI Through People, Processes & Technology | [Podcast](https://mlsecops.com/podcast/a-holistic-approach-to-understanding-the-ai-lifecycle-and-securing-ml-systems-protecting-ai-through-people-processes-technology) |
| 4 Jul 2023 | Software Improvement Group Podcast | A.I. Security: A guide to implementing security and risk controls in AI | [Podcast](https://www.brighttalk.com/webcast/19697/586526) |
| 23 Feb 2023 | The Application Security Podcast w/ Chris Romeo and Robert Hurlbut | OWASP AI Security & Privacy Guide w/ Rob van der Veer | [YouTube](https://www.youtube.com/watch?v=SLdn3AwlCAk&) [Podcast](https://www.buzzsprout.com/1730684/12313155-rob-van-der-veer-owasp-ai-security-privacy-guide) |
-| 15 Feb 2023 | OWASP Conference Dublin | Attacking And Protecting Artificial Intelligence w/ Rob Van Der Veer | [YouTube](https://www.youtube.com/watch?v=ABmWHnFrMqI) |
+| 15 Feb 2023 | OWASP Conference Dublin | Attacking And Protecting Artificial Intelligence w/ Rob Van Der Veer | [YouTube](https://www.youtube.com/watch?v=ABmWHnFrMqI) |
\ No newline at end of file
diff --git a/content/ai_exchange/content/meetings.md b/content/ai_exchange/content/meetings.md
new file mode 100644
index 00000000..433ddc22
--- /dev/null
+++ b/content/ai_exchange/content/meetings.md
@@ -0,0 +1,12 @@
+---
+title: 'AI Exchange Team Meetings'
+---
+- Bi-weekly Sync on **Thursdays at 5 PM (GMT), 8 AM (PST)**
+- Previous Meetings can be viewed on the **[YouTube channel](https://youtube.com/@RobvanderVeer-ex3gj?si=s2-gDFrRCazNge_c)**
+
+### Previous Meetings (last meetings first in the list)
+
+| Date | Title | Video |
+| - | - | - |
+| 8 Feb 2024 | Second call | [YouTube](https://www.youtube.com/watch?v=Qfo1Mjp1tJ0) |
+| 25 Jan 2024 | 2024 Kick Off | [YouTube](https://youtu.be/rwqv2m4-0vA?si=ZSB5-DfntaUjxF8I) |
diff --git a/content/ai_exchange/hugo.yaml b/content/ai_exchange/hugo.yaml
index a72c96f5..1274be9b 100644
--- a/content/ai_exchange/hugo.yaml
+++ b/content/ai_exchange/hugo.yaml
@@ -6,7 +6,11 @@ module:
imports:
- path: github.com/imfing/hextra
+
+
params:
+ og_image: https://owaspai.org/images/aix-og-logo.jpg
+ og_description: Comprehensive guidance and alignment on how to protect AI against security threats - by professionals, for professionals.
navbar:
displayTitle: true
displayLogo: true
@@ -38,18 +42,21 @@ menu:
- name: Media
pageRef: /media
weight: 3
+ - name: Meetings
+ pageRef: /meetings
+ weight: 4
- name: Contribute
pageRef: /contribute
- weight: 4
+ weight: 5
- name: Connect
pageRef: /connect
- weight: 5
- - name: Search
weight: 6
+ - name: Search
+ weight: 7
params:
type: search
- name: GitHub
- weight: 7
+ weight: 8
url: "https://github.com/OWASP/www-project-ai-security-and-privacy-guide"
params:
icon: github
@@ -71,9 +78,17 @@ menu:
- name: Media
pageRef: /media
weight: 5
+ - name: Meetings
+ pageRef: /meetings
+ weight: 6
- name: Register
url: https://forms.gle/XwEEK52y4iZQChuJ6
- weight: 6
+ weight: 7
- name: Navigator
url: https://github.com/OWASP/www-project-ai-security-and-privacy-guide/raw/main/assets/images/owaspaioverviewpdfv3.pdf
- weight: 7
\ No newline at end of file
+ weight: 8
+
+services:
+ googleAnalytics:
+ # This might not be correct format?
+ ID: G-QPGVTTDD3R
\ No newline at end of file
diff --git a/content/ai_exchange/layouts/partials/opengraph.html b/content/ai_exchange/layouts/partials/opengraph.html
new file mode 100644
index 00000000..14187b81
--- /dev/null
+++ b/content/ai_exchange/layouts/partials/opengraph.html
@@ -0,0 +1,21 @@
+{{/* From https://github.com/gohugoio/gohugoioTheme/blob/master/layouts/partials/opengraph/opengraph.html */}}
+
+
+ {{- with $image -}}
+ 
+ {{- end -}}
+
+
+ {{- with $icon }}{{ partial "utils/icon.html" (dict "name" $icon) -}}{{- end -}}
+ {{- $title -}}
+
+ {{- with $subtitle -}}
+ {{- $subtitle | markdownify -}}
+ {{- end -}}
+
+{{- /* Strip trailing newline. */ -}}
\ No newline at end of file
diff --git a/content/ai_exchange/static/images/5338.png b/content/ai_exchange/static/images/5338.png
new file mode 100644
index 00000000..b3eb94a7
Binary files /dev/null and b/content/ai_exchange/static/images/5338.png differ
diff --git a/content/ai_exchange/static/images/aisecthreats2.png b/content/ai_exchange/static/images/aisecthreats2.png
new file mode 100644
index 00000000..cf7256c4
Binary files /dev/null and b/content/ai_exchange/static/images/aisecthreats2.png differ
diff --git a/content/ai_exchange/static/images/aisecthreatscontrols2.png b/content/ai_exchange/static/images/aisecthreatscontrols2.png
new file mode 100644
index 00000000..30c40e2c
Binary files /dev/null and b/content/ai_exchange/static/images/aisecthreatscontrols2.png differ
diff --git a/content/ai_exchange/static/images/aix-og-logo.jpg b/content/ai_exchange/static/images/aix-og-logo.jpg
new file mode 100644
index 00000000..c2924ed9
Binary files /dev/null and b/content/ai_exchange/static/images/aix-og-logo.jpg differ
diff --git a/content/ai_exchange/static/images/threats.png b/content/ai_exchange/static/images/threats.png
new file mode 100644
index 00000000..ff5823cc
Binary files /dev/null and b/content/ai_exchange/static/images/threats.png differ
diff --git a/content/ai_exchange/static/images/threatscontrols-genainotready.png b/content/ai_exchange/static/images/threatscontrols-genainotready.png
new file mode 100644
index 00000000..4cda0b5e
Binary files /dev/null and b/content/ai_exchange/static/images/threatscontrols-genainotready.png differ
diff --git a/content/ai_exchange/static/images/threatscontrols-readymodel.png b/content/ai_exchange/static/images/threatscontrols-readymodel.png
new file mode 100644
index 00000000..8239861f
Binary files /dev/null and b/content/ai_exchange/static/images/threatscontrols-readymodel.png differ
diff --git a/content/ai_exchange/static/images/threatscontrols.png b/content/ai_exchange/static/images/threatscontrols.png
new file mode 100644
index 00000000..e3b6b86a
Binary files /dev/null and b/content/ai_exchange/static/images/threatscontrols.png differ
diff --git a/index.md b/index.md
index e89a6068..611196e2 100644
--- a/index.md
+++ b/index.md
@@ -56,6 +56,8 @@ GDPR's Article 5 refers to "fair processing" and EDPS' [guideline](https://edpb.
In the [literature](http://fairware.cs.umass.edu/papers/Verma.pdf), there are different fairness metrics that you can use. These range from group fairness, false positive error rate, unawareness, and counterfactual fairness. There is no industry standard yet on which metric to use, but you should assess fairness especially if your algorithm is making significant decisions about the individuals (e.g. banning access to the platform, financial implications, denial of services/opportunities, etc.). There are also efforts to test algorithms using different metrics. For example, NIST's [FRVT project](https://pages.nist.gov/frvt/html/frvt11.html) tests different face recognition algorithms on fairness using different metrics.
+The elephant in the room for fairness across groups (protected attributes) is that in situations a model is more accurate if it DOES discriminate protected attributes. Certain groups have in practice a lower success rate in areas because of all kinds of societal aspects rooted in culture and history. We want to get rid of that. Some of these aspects can be regarded as institutional discrimination. Others have more practical background, like for example that for language reasons we see that new immigrants statistically tend to be hindered in getting higher education.
+Therefore, if we want to be completely fair across groups, we need to accept that in many cases this will be balancing accuracy with discrimination. In the case that sufficient accuracy cannot be attained while staying within discrimination boundaries, there is no other option than to abandon the algorithm idea. For fraud detection cases, this could for example mean that transactions need to be selected randomly instead of by using an algorithm.
## 3. Data Minimization and Storage Limitation
diff --git a/owaspaiexchange.md b/owaspaiexchange.md
index 0ef18d6a..56ad9585 100644
--- a/owaspaiexchange.md
+++ b/owaspaiexchange.md
@@ -1,3 +1,4 @@
+
The content of the OWASP AI Exchange has moved to a new website with more advanced navigation at [owaspai.org](https://owaspai.org).