Skip to content

Commit

Permalink
Made the word "initiative" more visible.
Browse files Browse the repository at this point in the history
Signed-off-by: Dean Wampler <[email protected]>
  • Loading branch information
deanwampler committed Dec 5, 2024
1 parent 356275b commit 2a1590f
Show file tree
Hide file tree
Showing 7 changed files with 22 additions and 22 deletions.
4 changes: 2 additions & 2 deletions GITHUB_PAGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ We discuss these steps in more depth [below](#setup-jekyll), but the following s

Install a recent version of Ruby 3. Note that on MacOS, the default Ruby installation is old, version 2.6.10. Installing Ruby will also install the `gem` dependency tool.

This project's `Makefile` will attempt to install the remaining dependencies, including `jekyll`, when you run `make all` or `make view-local`.
This repo's `Makefile` will attempt to install the remaining dependencies, including `jekyll`, when you run `make all` or `make view-local`.

> [!WARNING]
> The automatic setup of `jekyll` in the `Makefile` has only been tested on MacOS and it most likely doesn't work on Windows, unless you use the Linux subsystem. If you encounter problems on other platforms, please [post an issue](https://github.com/The-AI-Alliance/trust-safety-evals/issues) to get help, or if you can fix the issue, a [pull request](https://github.com/The-AI-Alliance/trust-safety-evals/pulls) (PR) is always welcome :nerd_face:. (More details on PRs below.)
Expand Down Expand Up @@ -59,7 +59,7 @@ Here are some things you should know.

### Using the Correct Branch

As for most Git projects, issue PRs to the `main` branch. However, the repo is actually configured to publish the docs from the `latest` branch, so we can accept PRs quickly, then decide when to publish a new version. (We will also tag `latest` for each release with a version number, for historical tracking.)
As for most Git repos, issue PRs to the `main` branch. However, the repo is actually configured to publish the docs from the `latest` branch, so we can accept PRs quickly, then decide when to publish a new version. (We will also tag `latest` for each release with a version number, for historical tracking.)

> [!NOTE]
> If you are curious, the details of how this publication branch is configured are discussed [below](#configuring-github-pages-in-the-repo-settings).
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,15 @@

[Published Documentation](https://the-ai-alliance.github.io/trust-safety-evals/)

This repo contains the code and documentation for the AI Alliance _Trust and Safety Evaluations_ project, which defines a reference stack for AI model and system evaluation, with evaluations, benchmarks, and leaderboards.
This repo contains the code and documentation for the AI Alliance _Trust and Safety Evaluations Initiative_, which defines a reference stack for AI model and system evaluation, with evaluations, benchmarks, and leaderboards.

See the [documentation](https://the-ai-alliance.github.io/trust-safety-evals/) for a more detailed description of the project.
See the [website](https://the-ai-alliance.github.io/trust-safety-evals/) for a more detailed description of this initiative.

At this time, only the "skeleton" documentation is provided. We will be making initial commits of code, etc. soon.

## Getting Involved

We welcome contributions as PRs. Please see our [Alliance community repo](https://github.com/The-AI-Alliance/community/) for general information about contributing to any of our projects. This section provides some specific details you need to know.
We welcome contributions as PRs. Please see our [Alliance community repo](https://github.com/The-AI-Alliance/community/) for general information about contributing to any of our projects and initiatives. This section provides some specific details you need to know.

In particular, see the AI Alliance [CONTRIBUTING](https://github.com/The-AI-Alliance/community/blob/main/CONTRIBUTING.md) instructions. You will need to agree with the AI Alliance [Code of Conduct](https://github.com/The-AI-Alliance/community/blob/main/CODE_OF_CONDUCT.md).

Expand Down
6 changes: 3 additions & 3 deletions docs/_config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,10 @@
# you will see them accessed via {{ site.title }}, {{ site.github_repo }}, and so on.
# You can create any custom variable you would like, and they will be accessible
# in the templates via {{ site.myvariable }}.
title: Trust and Safety Evaluations
title: Trust and Safety Evaluations Initiative
email: [email protected]
description: >- # this means to ignore newlines until "baseurl:"
The documentation for the AI Alliance project, Trust and Safety Evaluations.
The documentation for the AI Alliance Trust and Safety Evaluations Initiative.
baseurl: "/trust-safety-evals" # the subpath of your site, e.g. /blog
url: "https://the-ai-alliance.github.io" # the base hostname & protocol for your site, e.g. http://example.com

Expand Down Expand Up @@ -70,7 +70,7 @@ aux_links:
- "https://thealliance.ai"

nav_external_links:
- title: The AI Alliance repo for Trust and Safety Evaluations"
- title: The AI Alliance repo for Trust and Safety Evaluations Initiative"
url: "https://github.com/The-AI-Alliance/trust-safety-evals"
hide_icon: false

Expand Down
6 changes: 3 additions & 3 deletions docs/about.markdown
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,13 @@ show_contribute_dataset_button: true

# About The AI Alliance

**Trust and Safety Evaluations** is a core project of the [Trust and Safety Work Group](https://thealliance.ai/focus-areas/trust-and-safety){:target="tands"} in [The AI Alliance](https://thealliance.ai){:target="ai-alliance"}.
**Trust and Safety Evaluations Initiative** is a core AI Alliance initiative, developed by the [Trust and Safety Work Group](https://thealliance.ai/focus-areas/trust-and-safety){:target="tands"} in [The AI Alliance](https://thealliance.ai){:target="ai-alliance"}.

The AI Alliance is a global collaboration of startups, enterprises, academic, and other research institutions interested in advancing the state of the art, the availability, and the safety of AI technology and uses. The AI Alliance's core projects seek to address substantial cross-community challenges and are an opportunity for contributors to collaborate, build, and make an impact on the future of AI. Core Projects are managed directly by the AI Alliance and governed as described in our [community GitHub repository](https://github.com/The-AI-Alliance/community){:target="community"}. You can find a list of all AI Alliance projects on [our website](https://thealliance.ai/our-work){:target="our-work"}.
The AI Alliance is a global collaboration of startups, enterprises, academic, and other research institutions interested in advancing the state of the art, the availability, and the safety of AI technology and uses. The AI Alliance's core projects and initiatives seek to address substantial cross-community challenges and are an opportunity for contributors to collaborate, build, and make an impact on the future of AI. Core projects and initiatives are managed directly by the AI Alliance and governed as described in our [community GitHub repository](https://github.com/The-AI-Alliance/community){:target="community"}. You can find a list of all AI Alliance projects and initiatives on [our website](https://thealliance.ai/our-work){:target="our-work"}.

If you have any questions or concerns about this initiative, please contact us at [[email protected]](mailto:[email protected]).

## Maintainers of This Project
## Maintainers of This Initiative

* Jonathan Bnayahu and Dean Wampler - [AI Alliance](https://thealliance.ai){:target="maintainers"} and [IBM Research](https://research.ibm.com/){:target="maintainers"}
* More coming soon...
Expand Down
8 changes: 4 additions & 4 deletions docs/contributing.markdown
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,17 @@ title: Contributing
nav_order: 90
---

# How to Contribute to this Project
# How to Contribute to this Initiative

We welcome your contributions to our project for trust and safety tools, evaluators, and benchmarks. Visit the project's [GitHub repo](https://github.com/The-AI-Alliance/trust-safety-evals/){:target="repo"} to see what tools and leaderboards are already implemented. See the [issues](https://github.com/The-AI-Alliance/trust-safety-evals/issues){:target="repo"} and the work tracking projects:
We welcome your contributions for defining the full evaluation taxonomy, for building corresponding evaluators and benchmarks, and supporting tools. Visit the initiative's [GitHub repo](https://github.com/The-AI-Alliance/trust-safety-evals/){:target="repo"} to see what tools and leaderboards are already implemented. See the [issues](https://github.com/The-AI-Alliance/trust-safety-evals/issues){:target="repo"} and the work tracking projects:

* [Evaluation Platform Reference Stack](https://github.com/orgs/The-AI-Alliance/projects/23/views/1){:target="project23"}
* [Evaluation Taxonomy](https://github.com/orgs/The-AI-Alliance/projects/24/views/1){:target="project24"}
* [Evaluation Leaderboard](https://github.com/orgs/The-AI-Alliance/projects/25/views/1){:target="project25"}

# Joining the Work Group

For people interested in more active participation, we are structuring the work group to support individuals with any level of time and expertise they can contribute. If you want to join our regular meetings or otherwise contribute, use the form on the [Become a collaborator](https://thealliance.ai/become-a-collaborator){:target="aia-website"} page to let us know your interests. Check the box for _AI Trust and Safety WG_ (WG stands for &ldquo;work group&rdquo;) and mention the &ldquo;trust and safety evals project&rdquo; in the _Message_ box.
For people interested in more active participation, we are structuring the work group to support individuals with any level of time and expertise they can contribute. If you want to join our regular meetings or otherwise contribute, use the form on the [Become a collaborator](https://thealliance.ai/become-a-collaborator){:target="aia-website"} page to let us know your interests. Check the box for _AI Trust and Safety WG_ (WG stands for &ldquo;work group&rdquo;) and mention the &ldquo;Trust and Safety Evaluatoins Initiative&rdquo; in the _Message_ box.

# Making Smaller Contributions

Expand All @@ -26,4 +26,4 @@ However, notice that every page on this website has _Edit this page on GitHub_ l

The [README](https://github.com/The-AI-Alliance/trust-safety-evals){:target="repo"} for the source repo explains more details for testing this website locally and for creating more extensive changes as PRs, such as adding new content. The README also describes our "DCO" requirements, which you should understand first.

However, these ways of contributing require you to have a GitHub account. If you don't have one and you want to provide feedback for improvements, then [send us email](mailto:[email protected]) with your suggestions. Be sure to mention this particular website, as we have many AI Alliance project websites that look similar!
However, these ways of contributing require you to have a GitHub account. If you don't have one and you want to provide feedback for improvements, then [send us email](mailto:[email protected]) with your suggestions. Be sure to mention this particular initiative, as we have many AI Alliance projects and initiatives!
12 changes: 6 additions & 6 deletions docs/index.markdown
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ nav_order: 10
has_children: false
---

# Trust and Safety Evaluations
# Trust and Safety Evaluations Initiative


[GitHub Repo](https://github.com/The-AI-Alliance/trust-safety-evals){:target="repo" .btn .btn-primary .fs-5 .mb-4 .mb-md-0 .mr-2 .no-glyph}
Expand All @@ -14,7 +14,7 @@ has_children: false
| **Authors** | [The AI Alliance Trust and Safety Work Group](https://thealliance.ai/focus-areas/trust-and-safety){:target="ai-alliance-tns"} |
| **Last Update** | V0.3.0, 2024-12-05 |

Welcome to the **The AI Alliance** project for **Trust and Safety Evaluations**.
Welcome to the **The AI Alliance** initiative for **Trust and Safety Evaluations**.

> **Tips:**
>
Expand All @@ -27,14 +27,14 @@ Much like other software, generative AI (&ldquo;GenAI&rdquo;) [_Models_]({{site.

There are many organizations working on evaluations for safety, broadly defined, and other kinds of measurements, as well as [_Benchmarks_]({{site.baseurl}}/glossary/#benchmark) that aggregate some evaluations and [_Leaderboards_]({{site.baseurl}}/glossary/#leaderboard) that let you see how some models and systems do against benchmarks, without having to execute these benchmarks yourself.

The **Trust and Safety Evaluations** project addresses several under-served needs:
The **Trust and Safety Evaluations Initiative** addresses several under-served needs:

1. While very good [Taxonomies]({{site.baseurl}}/glossary/#taxonomy) of evaluation in the areas of risk and harms have [emerged]({{site.baseurl}}/taxonomy/taxonomy/#why-build-a-taxonomy), there are other areas of interest where a standard taxonomy, with corresponding evaluations, would be useful. See [taxonomy]({{site.baseurl}}/taxonomy/taxonomy).
2. [_Evaluators_]({{site.baseurl}}/glossary/#evaluator) that implement evaluations in the taxonomy are needed. Some areas are well-covered, while others have no available evaluators. These evaluators can be aggregated into benchmarks. See [evaluators]({{site.baseurl}}/evaluators/evaluators).
3. Leaderboards are needed that provide unique, user-configurable views on different benchmark combinations, which help users focus on the benchmarks most relevant to their needs. See [leaderboards]({{site.baseurl}}/leaderboards/leaderboards).
4. Users need a _reference stack_ of industry-standard OSS tools for evaluation, especially at [_Inference_]({{site.baseurl}}/glossary/#inference) time. See [Evaluation Platform Reference Stack]({{site.baseurl}}/ref-stack/ref-stack).

This website provides the documentation for this project, with links to other resources, including code and leaderboards, as they become available.
This website provides the documentation for this initiative, with links to other resources, including code and leaderboards, as they become available.

Are you interested in contributing? If so, please see the [contributing]({{site.baseurl}}/contributing) page for information on how you can get involved.

Expand All @@ -50,9 +50,9 @@ This site is organized into the following sections:
Additional links:

* [Contributing]({{site.baseurl}}/contributing): We welcome your contributions! Here's how you can contribute.
* [About Us]({{site.baseurl}}/about): More about the AI Alliance and this project.
* [About Us]({{site.baseurl}}/about): More about the AI Alliance and this initiative.
* [GitHub Repo](https://github.com/The-AI-Alliance/trust-safety-evals){:target="repo"}
* [The AI Alliance](https://thealliance.ai){:target="ai-alliance"}: The AI Alliance website.
* [Project GitHub Repo](https://github.com/The-AI-Alliance/trust-safety-evals){:target="repo"}

### Version History

Expand Down
2 changes: 1 addition & 1 deletion docs/user-personae/user-personae.markdown
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ has_children: true

## User Personae for Evaluation

There are a wide range of stakeholders in the AI space who can benefit from this project:
There are a wide range of stakeholders in the AI space who can benefit from this initiative:

* **Model Builders:** Who need to evaluate their models against desired criteria.
* **Independent Software Vendors:** Companies providing AI-as-a-Service, including evaluations for safety.
Expand Down

0 comments on commit 2a1590f

Please sign in to comment.