Skip to content

Commit

Permalink
Merge pull request #1 from OWASP/main
Browse files Browse the repository at this point in the history
Risk Management Changes
  • Loading branch information
rkopaee authored Feb 16, 2024
2 parents 42ff456 + 64f74cb commit 3b2e0a3
Show file tree
Hide file tree
Showing 29 changed files with 389 additions and 143 deletions.
19 changes: 8 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@
Welcome to the GitHub repository for two initiatives:

1. The OWASP AI Exchange, to be found at [owaspai.org](http://owaspai.org/): the living set of documents that collect AI threats and controls from collaboration between experts worldwide.
2. The OWASP AI Security and Privacy Guide project, also to be found at [owaspai.org](http://owaspai.org/), which is published automatically at [owasp.org/www-project-ai-security-and-privacy-guide/#](https://owasp.org/www-project-ai-security-and-privacy-guide/#). It holds a stable version of the AI Exchange, adds some illustrations and explanations for a wider audience, plus it has an additional section on the topic of AI privacy.
2. The OWASP AI Security and Privacy Guide project, which is published automatically at [owasp.org/www-project-ai-security-and-privacy-guide/#](https://owasp.org/www-project-ai-security-and-privacy-guide/#). It has a security part that links directly to the AI Exchange, and a privacy part.

The goal of these initiatives is to collect and present the state of the art on these topics through community collaboration.
The goal of these initiatives is to collect and clearly present the state of the art on these topics through community collaboration.

## Project Lead

Expand All @@ -15,19 +15,16 @@ The goal of these initiatives is to collect and present the state of the art on

The OWASP projects are an open source effort, and we enthusiastically welcome all forms of contributions and feedback.

To edit content on the website, you have two options

- click the "Edit on GitHub" button located in the right-hand navigation bar
- manually locate and edit the files in the directory.

### Participate in Content Development

- 📥 Send your suggestion to the [project leader](https://owaspai.org/connect/#owasp-ai-project-leader).
- 👋 Join `#project-ai` in our [Slack](https://owasp.slack.com/join/shared_invite/zt-g398htpy-AZ40HOM1WUOZguJKbblqkw#) workspace.
- 🗣️ Discuss with the [project leader](https://owaspai.org/connect/#owasp-ai-project-leader) how to become part of the writing group.
- 🗣️ Discuss with the [project leader](https://owaspai.org/connect/#owasp-ai-project-leader) how to become part of the author group.
- 💡Propose your [concepts](https://github.com/OWASP/www-project-ai-security-and-privacy-guide/discussions/categories/ideas), or submit an [issue](https://github.com/OWASP/www-project-ai-security-and-privacy-guide/issues).
- 📄 Fork our repo and submit a [Pull Request](https://github.com/OWASP/www-project-ai-security-and-privacy-guide/pulls) for concrete fixes (e.g. grammar/typos) or content already approved by the core team.
- 🙌 Showcase your [contributions](https://github.com/OWASP/www-project-ai-security-and-privacy-guide/discussions/categories/show-and-tell).
- 🐞 Identify an [issue](https://github.com/OWASP/www-project-ai-security-and-privacy-guide/issues) or fix it on a [Pull Request](https://github.com/OWASP/www-project-ai-security-and-privacy-guide/pulls).
- 💬 Provide your insights in [GitHub Discussions](https://github.com/OWASP/www-project-ai-security-and-privacy-guide/discussions/categories/general).
- 🙏 Pose your [questions](https://github.com/OWASP/www-project-ai-security-and-privacy-guide/discussions/categories/q-a).
- 🙏 Pose your [questions](https://github.com/OWASP/www-project-ai-security-and-privacy-guide/discussions/categories/q-a).

If you are part of the author group:
- At the [AI Exchange website](https://owaspai.org) click the "Edit on GitHub" button located in the right-hand navigation bar
- Or manually locate and edit the files in the [github repository](https://github.com/OWASP/www-project-ai-security-and-privacy-guide/tree/main)
Binary file added assets/images/aisecthreat.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/images/aisecthreatcontrols.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
29 changes: 15 additions & 14 deletions content/ai_exchange/content/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,21 +8,22 @@
{{< spacer height="40" >}}

{{< cards >}}
{{< card link="/connect" title="Connect with us!" icon="chat" >}}
{{< card link="/contribute" title="Contribute" icon="star" >}}
{{< card link="https://forms.gle/XwEEK52y4iZQChuJ6" title="Register" icon="login" >}}
{{< card link="/media" title="Media" icon="speakerphone" >}}
{{< card link="https://github.com/OWASP/www-project-ai-security-and-privacy-guide/raw/main/assets/images/owaspaioverviewpdfv3.pdf" title="Navigator" icon="document-download">}}
{{< small-card link="/charter" title="Charter" icon="document-text" >}}
{{< small-card link="/connect" title="Connect with us!" icon="chat" >}}
{{< small-card link="/contribute" title="Contribute" icon="star" >}}
{{< small-card link="https://forms.gle/XwEEK52y4iZQChuJ6" title="Register" icon="login" >}}
{{< small-card link="/media" title="Media" icon="speakerphone" >}}
{{< small-card link="https://github.com/OWASP/www-project-ai-security-and-privacy-guide/raw/main/assets/images/owaspaioverviewpdfv3.pdf" title="Navigator" icon="document-download">}}
{{< /cards >}}

## Our Content

{{< cards >}}
{{< card link="/docs/ai_security_overview/" title="AI Security Overview">}}
{{< card link="/docs/1_general_controls/" title="1. General controls">}}
{{< card link="/docs/2_threats_through_use/" title="2. Threats through use">}}
{{< card link="/docs/3_development_time_threats/" title="3. Development-time threats">}}
{{< card link="/docs/4_runtime_application_security_threats/" title="4. Runtime application security threats">}}
{{< small-card link="/docs/ai_security_overview/" title="AI Security Overview">}}
{{< small-card link="/docs/1_general_controls/" title="1. General controls">}}
{{< small-card link="/docs/2_threats_through_use/" title="2. Threats through use">}}
{{< small-card link="/docs/3_development_time_threats/" title="3. Development-time threats">}}
{{< small-card link="/docs/4_runtime_application_security_threats/" title="4. Runtime application security threats">}}
{{< /cards >}}

## Purpose
Expand All @@ -31,12 +32,12 @@ The OWASP AI Exchange has open sourced the global discussion on the security of

Our **mission** is to be the authoritative source for consensus, foster alignment, and drive collaboration among initiatives - NOT to set a standard, but to drive standards. By doing so, we provide a safe, open, and independent place to find and share insights for everyone. See [AI Exchange LinkedIn page](https://www.linkedin.com/company/owasp-ai-exchange/).

The AI Exchange is displayed here at [owaspai.org](https://owaspai.org) and edited using a [GitHub repository](https://github.com/OWASP/www-project-ai-security-and-privacy-guide/tree/main/content/ai_exchange/content) (see the links _Edit ont Github_). It is is an **open-source set of living documents** for the worldwide exchange of AI security expertise, and part of the [OWASP AI security & privacy guide](https://owasp.org/www-project-ai-security-and-privacy-guide/) project.
The AI Exchange is displayed here at [owaspai.org](https://owaspai.org) and edited using a [GitHub repository](https://github.com/OWASP/www-project-ai-security-and-privacy-guide/tree/main/content/ai_exchange/content) (see the links _Edit on Github_). It is is an **open-source set of living documents** for the worldwide exchange of AI security expertise, and part of the [OWASP AI security & privacy guide](https://owasp.org/www-project-ai-security-and-privacy-guide/) project.

## Other OWASP AI Initiatives

{{< cards >}}
{{< card link="https://owasp.org/www-project-ai-security-and-privacy-guide/" title="AI security & privacy guide" icon="lock-closed" >}}
{{< card link="https://llmtop10.com/" title="LLM Top 10" icon="brain" >}}
{{< card link="https://mltop10.info/" title="ML Top 10" icon="machinelearning" >}}
{{< small-card link="https://owasp.org/www-project-ai-security-and-privacy-guide/" title="AI security & privacy guide" icon="lock-closed" >}}
{{< small-card link="https://llmtop10.com/" title="LLM Top 10" icon="brain" >}}
{{< small-card link="https://mltop10.info/" title="ML Top 10" icon="machinelearning" >}}
{{< /cards >}}
53 changes: 53 additions & 0 deletions content/ai_exchange/content/charter.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
---
title: 'AI Exchange Charter'
---
## Purpose
>Comprehensive guidance and alignment on how to protect AI against security threats - by professionals, for professionals.
The goal of the OWASP AI Exchange is to protect society from AI security issues by independently harnessing the collective wisdom of global experts across various disciplines. This initiative focuses on advancing AI security understanding, supporting the development of global AI security guidelines, standards and regulations, and simplifying the AI security domain for professionals and organizations. Its goal is to provide a comprehensive overview of AI threats, risks, mitigations, and controls, aligning with global standardization initiatives such as the EU AI Act, ISO/IEC 27090 (AI Security), the OWASP ML Top 10, the OWASP LLM Top 10, and OpenCRE. This alignment, achieved through open source, is crucial to prevent confusion and ignorance, leading to harm from AI security incidents.

## Target Audience
This charter primarily addresses the needs of cybersecurity experts, privacy/regulatory/ legal professionals, AI leaders, developers, and data scientists. It offers accessible guidance and resources to these groups, enabling them to build and maintain secure AI systems effectively.

## Mission / Goals
Our mission is to establish the OWASP AI Exchange as the place to go for professionals who want to understand AI security, and to be the authoritative source for consensus, alignment, and collaboration among various AI initiatives. We aim to foster a unified approach to addressing AI security challenges.

## Scope & Responsibilities
- Develop a comprehensive framework for AI threats, risks, mitigations, and controls.
- Create a map integrating AI regulatory and privacy regulations.
- Establish a common taxonomy and glossary for AI security.
- Provide guidance on testing tools with outcome assessments.
- Formulate a shared responsibility model for third-party AI model usage.
- Offer supply chain guidance and an incident response plan.

## Relation to other OWASP or other organization initiatives
These are the other OWASP AI initiatives and the relation with the AI Exchange;
- The OWASP AI security and privacy guide is the official OWASP project under which the AI Exchange was established. The deliverable of this project consists of the AI Exchange content plus guidance on AI privacy.
- The OWASP LLM top 10 provides a list of the most important LLM security issues, plus deliverables that focus on LLM security, such as the LLM AI Security & Governance Checklist.
- The OWASP ML top 10 provides a list of the most important machine learning security issues.
- OpenCRE.org has been established under the OWASP Integration standards project and holds a catalog of common requirements across various security standards inside and outside of OWASP. The plan is to let OpenCRE contain new AI security controls as well.

## Roadmap
- Purpose and mission defined for OWASP AI Exchange Project Working Group
- Working group charter to 1.0
- Project Plan
- Working group established

## Implementation
- Create a roadmap, share documents, and establish a meeting cadence.
- Record meetings and take notes for transparency and accessibility.
- Communicate developments through newsletters.

## Next milestone for content
- Bring content to 1.0 draft.
- Address all outstanding tasks in the ‘Contribute’ section.
- Make sure all topics are sufficiently covered regarding depth and width, including references to relevant work.
- Ensure clarity of all content.
- Align content as good as possible, with other initiatives like Mitre Atlas, NIST, the LLM Top 10, ENISA’s work, and the AIAPP International Privacy Group.
- Review 1.0 draft.
- Conduct internal reviews.
- Get public comment from other communities / peer review.
- Release the final version 1.0, alongside a communication strategy and feedback process.

## Copyright
The AI security community is marked with CC0 1.0 meaning you can use any part freely, without attribution. If possible, it would be nice if the OWASP AI Exchange is credited and/or linked to, for readers to find more information.
12 changes: 6 additions & 6 deletions content/ai_exchange/content/connect.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,12 @@ excludeSearch: true
## Platforms

{{< cards >}}
{{< card link="https://forms.gle/XwEEK52y4iZQChuJ6" title="Register" icon="login" >}}
{{< card link="https://owasp.slack.com/join/shared_invite/zt-g398htpy-AZ40HOM1WUOZguJKbblqkw#" title="Slack" icon="slack-big" >}}
{{< card link="https://www.linkedin.com/company/owasp-ai-exchange/" title="LinkedIn" icon="linkedin" >}}
{{< card link="mailto:[email protected]" title="E-mail" icon="mail">}}
{{< card link="https://twitter.com/owasp" title="Twitter" icon="x-twitter" >}}
{{< card link="https://github.com/OWASP/www-project-ai-security-and-privacy-guide/discussions" title="GitHub" icon="github" >}}
{{< small-card link="https://forms.gle/XwEEK52y4iZQChuJ6" title="Register" icon="login" >}}
{{< small-card link="https://owasp.slack.com/join/shared_invite/zt-g398htpy-AZ40HOM1WUOZguJKbblqkw#" title="Slack" icon="slack-big" >}}
{{< small-card link="https://www.linkedin.com/company/owasp-ai-exchange/" title="LinkedIn" icon="linkedin" >}}
{{< small-card link="mailto:[email protected]" title="E-mail" icon="mail">}}
{{< small-card link="https://twitter.com/owasp" title="Twitter" icon="x-twitter" >}}
{{< small-card link="https://github.com/OWASP/www-project-ai-security-and-privacy-guide/discussions" title="GitHub" icon="github" >}}
{{< /cards >}}

Engage with the OWASP AI team through various platforms.
Expand Down
72 changes: 30 additions & 42 deletions content/ai_exchange/content/contribute.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,12 @@ excludeSearch: true
---

{{< cards >}}
{{< card link="https://github.com/OWASP/www-project-ai-security-and-privacy-guide" title="GitHub Repo" icon="github" >}}
{{< small-card link="https://github.com/OWASP/www-project-ai-security-and-privacy-guide" title="GitHub Repo" icon="github" >}}
{{< /cards >}}

&nbsp;{{< github-stars user="OWASP" repo="www-project-ai-security-and-privacy-guide" repo_url="https://github.com/OWASP/www-project-ai-security-and-privacy-guide" >}}

{{< tabs items="Guidelines,Team of Experts,Contributors,Organizations,TODOs" >}}
{{< tabs items="Guidelines,Authors,Organizations,TODOs" >}}

{{< tab >}}

Expand All @@ -36,48 +36,35 @@ We value every contribution to our project, but it's important to be aware of ce

If you're unsure about anything, feel free to [reach out to us](/connect) with your questions.
{{< /tab >}}
{{< html-tab >}}

<table border='1'>
<tr><th>Name</th><th>Company</th><th>Country</th><th>Contribution</th></tr>
<tr><td>Adelin Travers</td><td>Trail of Bits</td><td></td><td></td></tr>
<tr><td>Alon Tron</td><td>Stealth</td><td>Israel</td><td>Improved supply chain management</td></tr>
<tr><td>Angie Qarry</td><td>QDeepTech</td><td>Austria</td><td>several elaborations and references on datascience defence mechanisms</td></tr>
<tr><td>Annegrit Seyerlein-Klug</td><td>TH Brandenburg</td><td>Germany</td><td>mapping with misc. standards</td></tr>
<tr><td>Anthony Glynn</td><td>CapitalOne</td><td>US</td><td>many textual improvements & link to LLM top 10</td></tr>
<tr><td>Behnaz Karimi</td><td>Accenture</td><td>Germany</td><td>misc. contributions including model obfuscation and explanation</td></tr>
<tr><td>Disesdi Susanna Cox</td><td>BobiHealth</td><td>US</td><td>Federative learning</td></tr>
<tr><td>Feiyang Tang</td><td>Software Improvement Group (SIG)</td><td></td><td></td></tr>
<tr><td>John Sotiropoulos</td><td>Kainos</td><td></td><td></td></tr>
<tr><td>Marko Lihter</td><td>SplxAI</td><td>Croatia</td><td>step-by-step guide for organizations, website creation, various textual improvements</td></tr>
<tr><td>Niklas Bunzel</td><td>Fraunhofer institute</td><td>Germany</td><td>datascience discussion and references around evasion attacks</td></tr>
<tr><td><b>Rob van der Veer</b></td><td>Software Improvement Group (SIG)</td><td>Netherlands</td><td><b>Project leader</b></td></tr>
<tr><td>Roger Sanz</td><td>Universidad Isabel</td><td>Spain</td><td></td></tr>
<tr><td><b>Sandy Dunn</b></td><td>Boise State University, AI Cyber Advisors</td><td>US</td><td></td></tr>
<tr><td>Sean Oesch</td><td>Oak Ridge National Laboratory</td><td>US</td><td>BLUF, Adversarial Training, OOD detection, NISTIR 8269, Guide Usability/Structure</td></tr>
<tr><td>Srajan Gupta</td><td>Dave</td><td></td><td></td></tr>
<tr><td>Steve Francolla</td><td>Workforce Tech LLC</td><td></td><td></td></tr>
<tr><td>Wei Wei</td><td>IBM</td><td>Germany</td><td>mapping with ISO/IEC 42001</td></tr>
<tr><td>Yiannis Kanellopoulos and team</td><td>Code4thought</td><td>Greece</td><td>evasion robustness</td></tr>
<tr><td>Zoe Braiterman</td><td>Mutual Knowledge Systems</td><td>US</td><td>Many markdown improvements</td></tr>
</table>

{{< /html-tab >}}

{{< tab >}}

<!-- TODO: Transform to table -->
- Adelin Travers - Trail of Bits
- Alon Tron
- Anthony Glynn - Capital One
- Behnaz Karimi - Accenture
- Feiyang Tang - Software Improvement Group (SIG)
- John Sotiropoulos - Kainos
- Marko Lihter - SplxAI
- Niklas Bunzel - Fraunhofer SIT
- Rob van - der Veer|Software Improvement Group (SIG)
- Roger Sanz - SIA Group
- Sandy Dunn - Boise State University, AI Cyber Advisors
- Srajan Gupta - Dave
- Steve Francolla - Workforce Tech LLC

{{< /tab >}}
{{< tab >}}

<!-- TODO: Transform to table -->
- Rob van der Veer (SIG, Netherlands) - results from AI security research at SIG: threat model, risks, navigator, matrix, risk approach, controls, gap analysis with ISO
- Yiannis Kanellopoulos and team (Code4thought, Greece) - evasion robustness
- Annegrit Seyerlein-Klug (TH Brandenburg, Germany) - mapping with misc. standards
- Wei Wei (IBM, Germany) - mapping with ISO/IEC 42001
- Roger Sanz (Universidad Isabel, Spain)
- Angie Qarry (QDeepTech, Austria) - some elaborations and references on datascience defence mechanisms
- Behnaz Karimi (Accenture, Germany)- misc. contributions including model obfuscation and explanation
- Sean Oesch (Oak Ridge National Laboratory, US) - BLUF, Adversarial Training, OOD detection, NISTIR 8269, Guide Usability/Structure
- Anthony Glynn (CapitalOne, US) - many textual improvements & link to LLM top 10
- Zoe Braiterman (Mutual Knowledge Systems, US) - Many markdown improvements
- Niklas Bunzel (Fraunhofer institute, Germany) - datascience discussion and references around evasion attacks
- Marko Lihter (SplxAI, Croatia) - various textual improvements & the Exchange website
- Milad Masoodi (SIG, Netherlands) - restructured document to put controls in sections, visible in the TOC


{{< /tab >}}
{{< tab >}}

<!-- Add a grid of copmany logos-->

{{< /tab >}}

{{< tab >}}
Expand Down Expand Up @@ -116,6 +103,7 @@ If you're unsure about anything, feel free to [reach out to us](/connect) with y
- Under INPUTDISTORTION: See ENISA Annex C to add data randomisation, input transformation and input denoising.
- Under INPUTDISTORTION: add Gradient masking - Annex C ENISA 2021
- Cover integrity checks in development pipeline (build, deploy, supply chain) - under supplychainmanage and/or secdevprogram
- Create an overall community outreach marketing plan, and regional outreach plans.

## TODOs requiring access to ISO/IEC documents

Expand Down
Loading

0 comments on commit 3b2e0a3

Please sign in to comment.