Add Robust Intelligence AI Firewall integration tile #2122

MayankR · 2023-10-02T23:26:56Z

What does this PR do?

Add Robust Intelligence AI Firewall integration tile

Motivation

This integration monitors the AI Firewall results through the Datadog Agent and allows users to visualize the firewall's performance using a dashboard.

Review checklist

PR has a meaningful title or PR has the no-changelog label attached
Feature or bugfix has tests
Git history is clean
If PR impacts documentation, docs team has been notified or an issue has been opened on the documentation repo

Additional Notes

Anything else we should know when reviewing?

…#1) * initial commit for check * check with correct metrics * add dashboard * update dashboard and readme * update dashboard title * fix readme links and add logos * fix support email link * update homepage and support email

* update section on metrics collected * update firewall link and dashboard

* readme fixes * update risk name * fix metrics section content * add image assets

* lint fixes * add sales email

github-actions · 2023-10-03T17:52:06Z

Test Results

2 files 2 suites 1s ⏱️
1 tests 1 ✔️ 0 💤 0 ❌
2 runs 2 ✔️ 0 💤 0 ❌

Results for commit 1bfd558.

♻️ This comment has been updated with latest results.

emarsha94

Thank you for submitting your integration! Please let me know if you have any questions on my comments and suggestions.

emarsha94 · 2023-10-04T17:41:54Z

robust_intelligence_ai_firewall/README.md

+## Overview
+
+[Robust Intelligence’s AI Firewall][1] wraps a protective layer around your AI models. It helps:
+1. **Block malicious inputs in real time.** Attacks on AI systems are increasing in frequency and sophistication. The nature of large language models (LLMs) make them a disproportionally high target, but all model types are at risk. AI Firewall inspects every input and automatically blocks malicious payloads before they can do damage to your model. Risks includes prompt injection, prompt extraction, and PII exfiltration.


We like to avoid marketing material. I think this can be shortened to explain what the product is and the value but we recommend the overview include:

What does your product do (in 1-2 sentences)?

What will customers get from this integration, and why is it valuable to them?

What types of data will your integration send in or pull out, and what's the value of that data?

Sure, updated the overview!

emarsha94 · 2023-10-04T17:42:23Z

robust_intelligence_ai_firewall/README.md

+1. Run the following command to install the Agent integration:
+
+   ```shell
+   datadog-agent integration install -t datadog-robust-intelligence-ai-firewall==<INTEGRATION_VERSION>


please include the integration version in the command the customer will run

robust_intelligence_ai_firewall/assets/dashboards/robust_intelligence_ai_firewall_overview.json

robust_intelligence_ai_firewall/manifest.json

* update overview and description * add integrtaion version * add classifier tags * add dashboard description

drichards-87

Left a couple of very minor suggestions from the Docs Team and approved the PR.

robust_intelligence_ai_firewall/README.md

* add details to configure section * Add suggestion Co-authored-by: Arley Lewis <[email protected]> --------- Co-authored-by: Arley Lewis <[email protected]>

emarsha94

Thank you for submitting the PR! Requested some changes/left comments. please let us know if you have any questions!

robust_intelligence_ai_firewall/README.md

robust_intelligence_ai_firewall/images/IMAGES_README.md

robust_intelligence_ai_firewall/assets/dashboards/robust_intelligence_ai_firewall_overview.json

* update readme and remove unnecessary file * upgrade dashboard

* add dashboard banner * update banner url

anawishnoff

A few more comments from the product side. Thanks!

robust_intelligence_ai_firewall/README.md

robust_intelligence_ai_firewall/assets/dashboards/robust_intelligence_ai_firewall_overview.json

robust_intelligence_ai_firewall/manifest.json

emarsha94 · 2023-10-27T14:24:11Z

Thank you for making the changes! Once all approvals are done, would you like to test this in your sandbox account prior to publishing live?

* add scope variable * fix description

MayankR · 2023-11-06T17:57:06Z

Hi @emarsha94 , I recall from our Friday meeting that you mentioned the possibility of including extra comments before we proceed with merging this PR. I wanted to follow up on it and check if there are any comments you'd like to add.

emarsha94 · 2023-11-06T22:12:14Z

@MayankR I am looking into the validation errors on my end to provide next best steps to resolve them. I will update you here once I learn more!

emarsha94 · 2023-11-07T20:40:32Z

Hi @MayankR I learned that you will need to delete that config_models folder from your integration to resolve the validation errors. This is an optional feature that adds more visibility into errors but this requires customers to be on Agent V7.48+. Please let me know if you have any questions!

MayankR · 2023-11-08T17:30:24Z

@emarsha94 I still see the errors after removing that folder. It seems that folder is auto generated when I run ddev -x validate models -s robust_intelligence_ai_firewal locally

MayankR · 2023-11-08T18:06:19Z

@emarsha94 I was able to fix a few more errors, any recommendations about the last few ones?

emarsha94 · 2023-11-08T21:38:52Z

Hey @MayankR - Have you run the tests locally and if so, have they been successful? If not, can you please run the tests locally and let me know? Thanks!

MayankR · 2023-11-10T17:40:40Z

@emarsha94 they were not running successfully on localhost. I've added the fixes you suggested and they are running successfully now!

MayankR added 3 commits September 29, 2023 11:33

[END-2821] Update README (#2)

103583f

* update section on metrics collected * update firewall link and dashboard

[PINT-2462] readme fixes (#3)

57cb90c

* readme fixes * update risk name * fix metrics section content * add image assets

MayankR requested review from a team as code owners October 2, 2023 23:26

alai97 added the editorial review Waiting on a more in-depth review from a docs team editor label Oct 2, 2023

[PINT-2485] PR build issues (#4)

fa9c655

* lint fixes * add sales email

emarsha94 added the kind/technology-partner label Oct 4, 2023

datadog-assets bot added agent/review-requested docs/review-requested ecosystems/review-requested product/review-requested labels Oct 4, 2023

emarsha94 requested changes Oct 4, 2023

View reviewed changes

datadog-assets bot added product/requested-changes and removed product/review-requested labels Oct 4, 2023

MayankR added 2 commits October 5, 2023 14:05

[PINT-2485] Address PR comments (#5)

a5d55e6

* update overview and description * add integrtaion version * add classifier tags * add dashboard description

add dashboard to manifest (#6)

4648403

drichards-87 approved these changes Oct 6, 2023

View reviewed changes

robust_intelligence_ai_firewall/README.md Outdated Show resolved Hide resolved

robust_intelligence_ai_firewall/README.md Outdated Show resolved Hide resolved

datadog-assets bot added docs/approved and removed docs/review-requested labels Oct 6, 2023

MayankR mentioned this pull request Oct 9, 2023

[PINT-2541] readme review RobustIntelligence/datadog-integrations-extras#7

Merged

4 tasks

[PINT-2541] readme review (#7)

c74f41d

* add details to configure section * Add suggestion Co-authored-by: Arley Lewis <[email protected]> --------- Co-authored-by: Arley Lewis <[email protected]>

MayankR requested a review from emarsha94 October 9, 2023 23:26

emarsha94 requested changes Oct 16, 2023

View reviewed changes

robust_intelligence_ai_firewall/README.md Outdated Show resolved Hide resolved

robust_intelligence_ai_firewall/images/IMAGES_README.md Outdated Show resolved Hide resolved

robust_intelligence_ai_firewall/assets/dashboards/robust_intelligence_ai_firewall_overview.json Show resolved Hide resolved

anawishnoff added the product/logos-requested label Oct 16, 2023

[PINT-2618] PR feedback and dashboard update (#8)

3e5fa71

* update readme and remove unnecessary file * upgrade dashboard

MayankR requested a review from emarsha94 October 18, 2023 16:46

[PINT-2651] add dashboard banner image (#9)

79fad00

* add dashboard banner * update banner url

anawishnoff suggested changes Oct 20, 2023

View reviewed changes

MayankR requested a review from anawishnoff October 26, 2023 05:30

emarsha94 approved these changes Oct 27, 2023

View reviewed changes

datadog-assets bot added product/approved and removed product/requested-changes labels Oct 27, 2023

anawishnoff assigned james-eichelbaum Oct 31, 2023

ofek approved these changes Oct 31, 2023

View reviewed changes

datadog-assets bot added agent/approved and removed agent/review-requested labels Oct 31, 2023

[PINT-2891] add scope variable to dashboard (#14)

3b7ce35

* add scope variable * fix description

MayankR added 2 commits November 7, 2023 22:26

remove folder (#15)

870feba

fix import (#16)

8ba05a1

MayankR added 7 commits November 8, 2023 09:33

run ddev validate ci

356b3c3

fix codecov

005d664

remove assets that are not required

7373933

add metrics check

c8cb9fb

add codeowners

20576a4

fix format

b92f025

remove non ascii character

4602e5f

MayankR added 2 commits November 10, 2023 09:06

fix test

053fc0d

add test

1bfd558

anawishnoff approved these changes Nov 13, 2023

View reviewed changes

james-eichelbaum merged commit 6a4223f into DataDog:master Nov 13, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Robust Intelligence AI Firewall integration tile #2122

Add Robust Intelligence AI Firewall integration tile #2122

MayankR commented Oct 2, 2023

github-actions bot commented Oct 3, 2023 •

edited

Loading

emarsha94 left a comment

emarsha94 Oct 4, 2023

MayankR Oct 5, 2023

emarsha94 Oct 4, 2023

MayankR Oct 5, 2023

drichards-87 left a comment

emarsha94 left a comment

anawishnoff left a comment

emarsha94 commented Oct 27, 2023

MayankR commented Nov 6, 2023 •

edited

Loading

emarsha94 commented Nov 6, 2023

emarsha94 commented Nov 7, 2023

MayankR commented Nov 8, 2023

MayankR commented Nov 8, 2023

emarsha94 commented Nov 8, 2023

MayankR commented Nov 10, 2023

Add Robust Intelligence AI Firewall integration tile #2122

Add Robust Intelligence AI Firewall integration tile #2122

Conversation

MayankR commented Oct 2, 2023

What does this PR do?

Motivation

Review checklist

Additional Notes

github-actions bot commented Oct 3, 2023 • edited Loading

Test Results

emarsha94 left a comment

Choose a reason for hiding this comment

emarsha94 Oct 4, 2023

Choose a reason for hiding this comment

MayankR Oct 5, 2023

Choose a reason for hiding this comment

emarsha94 Oct 4, 2023

Choose a reason for hiding this comment

MayankR Oct 5, 2023

Choose a reason for hiding this comment

drichards-87 left a comment

Choose a reason for hiding this comment

emarsha94 left a comment

Choose a reason for hiding this comment

anawishnoff left a comment

Choose a reason for hiding this comment

emarsha94 commented Oct 27, 2023

MayankR commented Nov 6, 2023 • edited Loading

emarsha94 commented Nov 6, 2023

emarsha94 commented Nov 7, 2023

MayankR commented Nov 8, 2023

MayankR commented Nov 8, 2023

emarsha94 commented Nov 8, 2023

MayankR commented Nov 10, 2023

github-actions bot commented Oct 3, 2023 •

edited

Loading

MayankR commented Nov 6, 2023 •

edited

Loading