Code

BoolQ Benchmark#802

Code

Merged

martindevans merged 4 commits intoSciSharp:masterSciSharp/LLamaSharp:masterfrom martindevans:boolq_benchmartindevans/LLamaSharp:boolq_benchCopy head branch name to clipboardJul 2, 2024

+12,962-8

Commits

Commits on Jun 29, 2024

• Added an example which runs the BoolQ benchmark against a model, outputting total accuracy. This can be used to quickly evaluate a model, and also demonstrates the use of grammars to constrain samp…

  Show description for 3151016
  martindevans

  committed
  3151016

  Copy full SHA for 3151016

• - Providing hints to model, converting BoolQ into a reading comprehension test rather than a model test.

  Show description for 99d918a
  martindevans

  committed
  99d918a

  Copy full SHA for 99d918a

• commented out debug printing

  martindevans

  committed
  defb953

  Copy full SHA for defb953

• Checking decode return value an exiting

  martindevans

  committed
  a9d4790

  Copy full SHA for a9d4790