RAG example using financebench #738
Conversation
y27choi
changed the title
examples/dataset/retrieval_augmented_generation/retrieval_augmented_generation/__init__.py
Outdated
Show resolved
Hide resolved
| using the `--help` flag for more information: | ||
|
|
||
| ```shell | ||
| $ uv run python3 retrieval_augmented_generation/upload_results.py --help |
There was a problem hiding this comment.
The current default behaviour with no model specified only uploads the baseline. I think the default should run all 3 models. We may want to briefly define what baseline, qme, and query_decomp are here.
There was a problem hiding this comment.
I want this script to be used to learn how to upload one model's results which is why it's designed to run only one. Is there a reason why you want to run all three by default?
There was a problem hiding this comment.
Not a big deal either way imo, but in a live demo it'd be nice to do all 3 at once
There was a problem hiding this comment.
Ok I won't make this change but I did add more information about the model results and how to format raw model results file in the README if you could take a look at it.
|
For the speaker diarization CI error, you'll probably have to rebase of latest trunk -- there was a dep I had to pin to get the CI passing again. |
Linked issue(s)
N/A
What change does this PR introduce and why?
Introduces RAG example using financebench
Please check if the PR fulfills these requirements