Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

3753: WER and CER qualification of donations #56

Merged
merged 9 commits into from
Feb 19, 2025
Merged

Conversation

jekuaitk
Copy link
Contributor

@jekuaitk jekuaitk commented Feb 6, 2025

https://leantime.itkdev.dk/#/tickets/showTicket/3753

  • Introduces WER and CER
  • Modifies qualify command to only transcribe donations.
  • Makes commands calculating each of the metrics (WER, CER and similar_text)

Note that previous commands have changed name.

@jekuaitk jekuaitk force-pushed the feature/wer-and-cer branch from 51a7ab2 to a4bd5ec Compare February 7, 2025 08:55
@jekuaitk jekuaitk requested a review from cableman February 10, 2025 08:19
}

$gds->save();
$donationIds = $query->accessCheck()->execute();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need the access check, will commands not alway be runned as user 1?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Commands are actually ran as the anonymous user. The access check however is still needed cf. https://www.drupal.org/node/3201242.

@@ -95,7 +104,9 @@ public function buildRow(EntityInterface $entity) {
}

$row['whisper_guess'] = $entity->getWhisperGuess() ?? '-';
$row['similar_text_score'] = $entity->getWhisperGuessSimilarTextScore() ? round($entity->getWhisperGuessSimilarTextScore(), 2) . '%' : '-';
$row['similar_text_score'] = $entity->getWhisperGuessSimilarTextScore() ? (100 - round($entity->getWhisperGuessSimilarTextScore(), 2)) / 100 : '-';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the math here with /100 when the other do not have changes to the value from the database. Please comment in the code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated README and added comment explaining why.

To align the similar_text score with WER and CER we report the dissimilarity score as a decimal,
such that all three metrics have 0 being good and 1 (or more) being bad.

@jekuaitk jekuaitk force-pushed the feature/wer-and-cer branch from f227f84 to 850ed38 Compare February 10, 2025 13:25
@jekuaitk jekuaitk requested a review from cableman February 10, 2025 13:26
@cableman cableman merged commit 17f183f into develop Feb 19, 2025
8 checks passed
@cableman cableman deleted the feature/wer-and-cer branch February 19, 2025 10:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants