swaps the order

castorini · Jan 30, 2025 · 9dadc6e · 9dadc6e
1 parent 8ff9bd2
commit 9dadc6e
Show file tree

Hide file tree

Showing 2 changed files with 26 additions and 26 deletions.
diff --git a/docs/regressions/regressions-rag24-doc-segmented-test.md b/docs/regressions/regressions-rag24-doc-segmented-test.md
@@ -68,27 +68,27 @@ bin/run.sh io.anserini.search.SearchCollection \
 Evaluation can be performed using `trec_eval`:
 
 ```
-bin/trec_eval -c -m recall.100 tools/topics-and-qrels/qrels.rag24.test-umbrela-all.txt runs/run.msmarco-v2.1-doc-segmented.bm25-default.topics.rag24.test.txt
-bin/trec_eval -c -m ndcg_cut.100 tools/topics-and-qrels/qrels.rag24.test-umbrela-all.txt runs/run.msmarco-v2.1-doc-segmented.bm25-default.topics.rag24.test.txt
 bin/trec_eval -c -m ndcg_cut.20 tools/topics-and-qrels/qrels.rag24.test-umbrela-all.txt runs/run.msmarco-v2.1-doc-segmented.bm25-default.topics.rag24.test.txt
+bin/trec_eval -c -m ndcg_cut.100 tools/topics-and-qrels/qrels.rag24.test-umbrela-all.txt runs/run.msmarco-v2.1-doc-segmented.bm25-default.topics.rag24.test.txt
+bin/trec_eval -c -m recall.100 tools/topics-and-qrels/qrels.rag24.test-umbrela-all.txt runs/run.msmarco-v2.1-doc-segmented.bm25-default.topics.rag24.test.txt
 
-bin/trec_eval -c -m recall.100 tools/topics-and-qrels/qrels.rag24.test-umbrela-all.txt runs/run.msmarco-v2.1-doc-segmented.bm25-default+rm3.topics.rag24.test.txt
-bin/trec_eval -c -m ndcg_cut.100 tools/topics-and-qrels/qrels.rag24.test-umbrela-all.txt runs/run.msmarco-v2.1-doc-segmented.bm25-default+rm3.topics.rag24.test.txt
 bin/trec_eval -c -m ndcg_cut.20 tools/topics-and-qrels/qrels.rag24.test-umbrela-all.txt runs/run.msmarco-v2.1-doc-segmented.bm25-default+rm3.topics.rag24.test.txt
+bin/trec_eval -c -m ndcg_cut.100 tools/topics-and-qrels/qrels.rag24.test-umbrela-all.txt runs/run.msmarco-v2.1-doc-segmented.bm25-default+rm3.topics.rag24.test.txt
+bin/trec_eval -c -m recall.100 tools/topics-and-qrels/qrels.rag24.test-umbrela-all.txt runs/run.msmarco-v2.1-doc-segmented.bm25-default+rm3.topics.rag24.test.txt
 
-bin/trec_eval -c -m recall.100 tools/topics-and-qrels/qrels.rag24.test-umbrela-all.txt runs/run.msmarco-v2.1-doc-segmented.bm25-default+rocchio.topics.rag24.test.txt
-bin/trec_eval -c -m ndcg_cut.100 tools/topics-and-qrels/qrels.rag24.test-umbrela-all.txt runs/run.msmarco-v2.1-doc-segmented.bm25-default+rocchio.topics.rag24.test.txt
 bin/trec_eval -c -m ndcg_cut.20 tools/topics-and-qrels/qrels.rag24.test-umbrela-all.txt runs/run.msmarco-v2.1-doc-segmented.bm25-default+rocchio.topics.rag24.test.txt
+bin/trec_eval -c -m ndcg_cut.100 tools/topics-and-qrels/qrels.rag24.test-umbrela-all.txt runs/run.msmarco-v2.1-doc-segmented.bm25-default+rocchio.topics.rag24.test.txt
+bin/trec_eval -c -m recall.100 tools/topics-and-qrels/qrels.rag24.test-umbrela-all.txt runs/run.msmarco-v2.1-doc-segmented.bm25-default+rocchio.topics.rag24.test.txt
 ```
 
 ## Effectiveness
 
 With the above commands, you should be able to reproduce the following results:
 
-| **R@100**                                                                                                    | **BM25 (default)**| **+RM3**  | **+Rocchio**|
+| **nDCG@20**                                                                                                  | **BM25 (default)**| **+RM3**  | **+Rocchio**|
 |:-------------------------------------------------------------------------------------------------------------|-----------|-----------|-----------|
-| RAG 24: Test queries                                                                                         | 0.1395    | 0.1318    | 0.1384    |
+| RAG 24: Test queries                                                                                         | 0.3198    | 0.3063    | 0.3196    |
 | **nDCG@100**                                                                                                 | **BM25 (default)**| **+RM3**  | **+Rocchio**|
 | RAG 24: Test queries                                                                                         | 0.2563    | 0.2410    | 0.2527    |
-| **nDCG@20**                                                                                                  | **BM25 (default)**| **+RM3**  | **+Rocchio**|
-| RAG 24: Test queries                                                                                         | 0.3198    | 0.3063    | 0.3196    |
+| **R@100**                                                                                                    | **BM25 (default)**| **+RM3**  | **+Rocchio**|
+| RAG 24: Test queries                                                                                         | 0.1395    | 0.1318    | 0.1384    |
diff --git a/src/main/resources/regression/rag24-doc-segmented-test.yaml b/src/main/resources/regression/rag24-doc-segmented-test.yaml
@@ -13,9 +13,9 @@ index_stats:
   total terms: 22707699649
 
 metrics:
-  - metric: R@100
+  - metric: nDCG@20
     command: bin/trec_eval
-    params: -c -m recall.100
+    params: -c -m ndcg_cut.20
     separator: "\t"
     parse_index: 2
     metric_precision: 4
@@ -27,9 +27,9 @@ metrics:
     parse_index: 2
     metric_precision: 4
     can_combine: false
-  - metric: nDCG@20
+  - metric: R@100
     command: bin/trec_eval
-    params: -c -m ndcg_cut.20
+    params: -c -m recall.100
     separator: "\t"
     parse_index: 2
     metric_precision: 4
@@ -47,29 +47,29 @@ models:
     display: BM25 (default)
     params: -bm25
     results:
-      R@100:
-        - 0.1395
-      nDCG@100:
-        - 0.2563
       nDCG@20:
         - 0.3198
+      nDCG@100:
+        - 0.2563
+      R@100:
+        - 0.1395
   - name: bm25-default+rm3
     display: +RM3
     params: -bm25 -rm3 -collection MsMarcoV2DocCollection
     results:
-      R@100:
-        - 0.1318
-      nDCG@100:
-        - 0.2410
       nDCG@20:
         - 0.3063
+      nDCG@100:
+        - 0.2410
+      R@100:
+        - 0.1318
   - name: bm25-default+rocchio
     display: +Rocchio
     params: -bm25 -rocchio -collection MsMarcoV2DocCollection
     results:
-      R@100:
-        - 0.1384
-      nDCG@100:
-        - 0.2527
       nDCG@20:
         - 0.3196
+      nDCG@100:
+        - 0.2527
+      R@100:
+        - 0.1384