With IAMY, you can configure your preferred parameters, and IAMY will generate LLM pipelines tailored to meet your specified target metrics based on your labeled data.
For instance, consider two automatically generated checks compared to a baseline request (which includes only the description of the exercise) along with their corresponding precision and recall parameters:
Course 1
| Requests | Precision | Recall |
|---|---|---|
| Baseline | 0.17 | 1 |
| IAMY | 0.98 | 0.84 |
Course 2
| Requests | Precision | Recall |
|---|---|---|
| Baseline | 0.23 | 1 |
| IAMY | 0.95 | 0.41 |
You require a small labeled dataset consisting of no more than a few hundred items.
