Skip to content

Commit

Permalink
remove compile options label from gbench (facebookresearch#3834)
Browse files Browse the repository at this point in the history
Summary:
Pull Request resolved: facebookresearch#3834

Looks like ServiceLab does not handle any metric that is not integer: https://fburl.com/code/chqi5hcr The current experiments are erroring with the message https://fburl.com/servicelab/99s69hbf:
```
ERROR:windtunnel.benchmarks.gbench_benchmark_runner.gbench_benchmark_runner:exception occurred while processing benchmark (this usually means a benchmark is misconfigured) faiss/perf_tests/scalar_quantizer_distance_test1/bench_scalar_quantizer_distance:QT_bf16_128d_2000n/iterations invalid literal for int() with base 10: 'OPTIMIZE AVX2 '
Traceback (most recent call last):
  File "windtunnel/benchmarks/gbench_benchmark_runner/gbench_benchmark_runner.py", line 116, in parse_gbench_results
    int(entry[metric_name]),
ValueError: invalid literal for int() with base 10: 'OPTIMIZE AVX2 '
```

Removing the label that's causing the failure. We can track the optimization mode via experiment names, or a dummy counter name in the future as I roll out multi-platform experiments.

Reviewed By: junjieqi

Differential Revision: D62166280

fbshipit-source-id: 6de70c945cf5058feb479e6dd501d7e84d08ef83
  • Loading branch information
mengdilin authored and facebook-github-bot committed Sep 4, 2024
1 parent d296b2c commit 1cafc71
Show file tree
Hide file tree
Showing 4 changed files with 0 additions and 7 deletions.
2 changes: 0 additions & 2 deletions faiss/perf_tests/bench_scalar_quantizer_accuracy.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,6 @@ static void bench_reconstruction_error(
ScalarQuantizer::QuantizerType type,
int d,
int n) {
state.SetLabel(faiss::get_compile_options());
std::vector<float> x(d * n);

float_rand(x.data(), d * n, 12345);
Expand Down Expand Up @@ -64,7 +63,6 @@ static void bench_reconstruction_error(
state.counters["ndiff_for_idempotence"] = ndiff;

state.counters["code_size_two"] = codes.size();
state.SetLabel(faiss::get_compile_options());
}

int main(int argc, char** argv) {
Expand Down
1 change: 0 additions & 1 deletion faiss/perf_tests/bench_scalar_quantizer_decode.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,6 @@ static void bench_decode(
ScalarQuantizer::QuantizerType type,
int d,
int n) {
state.SetLabel(faiss::get_compile_options());
std::vector<float> x(d * n);

float_rand(x.data(), d * n, 12345);
Expand Down
3 changes: 0 additions & 3 deletions faiss/perf_tests/bench_scalar_quantizer_distance.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,6 @@ static void bench_distance(
ScalarQuantizer::QuantizerType type,
int n,
int d) {
state.SetLabel(faiss::get_compile_options());
std::vector<float> x(d * n);

float_rand(x.data(), d * n, 12345);
Expand All @@ -46,8 +45,6 @@ static void bench_distance(
std::vector<uint8_t> codes(code_size * n);
sq.compute_codes(x.data(), codes.data(), n);

state.SetLabel(faiss::get_compile_options());

std::unique_ptr<ScalarQuantizer::SQDistanceComputer> dc(
sq.get_distance_computer());
dc->codes = codes.data();
Expand Down
1 change: 0 additions & 1 deletion faiss/perf_tests/bench_scalar_quantizer_encode.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@ static void bench_encode(
ScalarQuantizer::QuantizerType type,
int d,
int n) {
state.SetLabel(faiss::get_compile_options());
std::vector<float> x(d * n);

float_rand(x.data(), d * n, 12345);
Expand Down

0 comments on commit 1cafc71

Please sign in to comment.