How to Run Get Another Label

Running the program

Build the jar using the mvn package command. You will find the jar get-another-label-2.2.0.jar under the target folder.

To run the program, type (on Unix):

$ java -cp target/get-another-label-2.2.0.jar com.ipeirotis.gal.Main [--iterations <iterations>] [--epsilon <epsilon>] [--verbose] [--gold <goldfile>] [--cost <costfile>] [--eval <evaluationfile>] --categories <categoriesfile> --input <inputfile>

For example, you want to use the example data that we distribute with the code, you can type:

$ java -cp target/get-another-label-2.2.0.jar com.ipeirotis.gal.Main --categories data/BarzanMozafari/categories-prior.txt --input data/BarzanMozafari/labels.txt --gold data/BarzanMozafari/gold-30items.txt --cost data/BarzanMozafari/costs.txt --eval data/BarzanMozafari/evaluation.txt 

Special thanks to Barzan Mozafari for providing the data set, which contains 5000 labels, and 1000 data points, each together with its correct classification. This allows to run the code with the --eval function and report back the accuracy of the estimation both in terms of data quality but also in terms of worker quality.

Input parameters

See Input Files for a detailed description of the parameters.

Interpreting the output

The outcome of the algorithm is saved in the results folder. See Output Files for a detailed description of the files in there.