Add support for distil-large-v3

[sanchit-gandhi]

The latest Distil-Whisper model, distil-large-v3, is intrinsically designed to work with the OpenAI sequential algorithm.

This PR adds support for this checkpoint.

[BBC-Esq]

Nice to see some comradare.

[sanchit-gandhi]

cc @trungkienbkhn - this one's ready for review!

[sanchit-gandhi]

Hey @trungkienbkhn! The model is now live under https://huggingface.co/distil-whisper/distil-large-v3-ct2

Feel free to merge this PR at your convenience to enable Faster-Whisper support!

Also cc @nguyendc-systran @metame-none @Purfview

[trungkienbkhn]

@sanchit-gandhi , thanks for your contribution.
After testing with the example provided, I found that the transcription time for the fw-distil-large-v3 model has been significantly reduced while maintaining almost the same quality compared with fw-large-v3 model.

However, when I added the option word_timestamps=True, I encountered an error:

segments, info = model.transcribe("audio.mp3", condition_on_previous_text=False, language="en", word_timestamps=True)

Processing segment at 00:00.000
[1]    7829 segmentation fault (core dumped)

This is caused by wrong alignment_heads field (a list of [layer, head]) in the config.json file of fw-distil-large-v3 model.
Since the distill-whisper only has 2 decode layers, fw is unable to extract word-level timestamps from layer index > 1.

So we should use the default alignment_heads for fw-distil-large-v3 model, same as fw-distil-large-v2 model. For more details on this logic, you can refer to the implementation in ctranslate2.
Could you re-verify and update this field? Tks.

[Purfview]

@trungkienbkhn
Could you make usual conversion to float16 and place it at https://huggingface.co/Systran/faster-distil-whisper-large-v3

[BBC-Esq]

Oh yeah, almost forgot, wouldn't the be necessary because the faster-whisper library automatically points to the Systran repository unless another one is explicitly specified?

[Vaibhavs10]

Could you make usual conversion to float16 and place it at https://huggingface.co/Systran/faster-distil-whisper-large-v3

Brilliant idea! That way all the faster-whisper checkpoints remain in one org.
We can remove the preliminary ct2 checkpoint in distil-whisper org and point the README and documentation to SYSTRAN.

[sanchit-gandhi]

Alignment heads updated according to those in distil-large-v2: https://huggingface.co/distil-whisper/distil-large-v3-ct2/discussions/1

Note that we'd get a better alignment by inspecting the alignment from the DTW algorithm, e.g. as done here.

However, the heuristic to use the last half layers for alignment by default should suffice for a first version.

[trungkienbkhn]

Could you make usual conversion to float16 and place it at https://huggingface.co/Systran/faster-distil-whisper-large-v3

FYI, we have released a new ct2 conversion model (using float16) for distil-large-v3: https://huggingface.co/Systran/faster-distil-whisper-large-v3

[sanchit-gandhi]

Awesome, thanks @trungkienbkhn. I've updated this PR to use these fp16 weights.

[sanchit-gandhi]

Feel free to merge this PR at your convenience - it would be awesome to unblock faster-whisper for the distil-whisper community.