Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Uzbek lang processors #71

Closed
wants to merge 22 commits into from
Closed

Uzbek lang processors #71

wants to merge 22 commits into from

Conversation

rimashahbazyan
Copy link
Collaborator

No description provided.

@rimashahbazyan rimashahbazyan requested a review from karpnv June 27, 2024 15:37
@rimashahbazyan rimashahbazyan self-assigned this Jun 27, 2024
i-vainn and others added 4 commits July 26, 2024 17:10
* added setup file

Signed-off-by: i-vainn <[email protected]>

* added init files

Signed-off-by: i-vainn <[email protected]>

* init

Signed-off-by: i-vainn <[email protected]>

* updated description

Signed-off-by: i-vainn <[email protected]>

---------

Signed-off-by: i-vainn <[email protected]>
Co-authored-by: i-vainn <[email protected]>
Signed-off-by: Rima <[email protected]>
rimashahbazyan and others added 15 commits July 26, 2024 17:12
Signed-off-by: Rima Shahbazyan [email protected]
* fix: gitignore

Signed-off-by: lilithgrigoryan <[email protected]>

* fix: add nemo_text_processing to requirements

Signed-off-by: lilithgrigoryan <[email protected]>

* add: text normalization processor

Signed-off-by: lilithgrigoryan <[email protected]>

* add: unit tests to text normalization processor

Signed-off-by: lilithgrigoryan <[email protected]>

* add: load_manifest, inner join with tests

Signed-off-by: lilithgrigoryan <[email protected]>

* add: logging for total hours

Signed-off-by: lilithgrigoryan <[email protected]>

* add: processor working time logging

Signed-off-by: lilithgrigoryan <[email protected]>

* merge: merge with changes in main main repo

Signed-off-by: lilithgrigoryan <[email protected]>

* add: docs

Signed-off-by: lilithgrigoryan <[email protected]>

* fix: inner join docs

Signed-off-by: lilithgrigoryan <[email protected]>

* add: inverse text normalization processor

Signed-off-by: lilithgrigoryan <[email protected]>

* fix: removed duplicate code

Signed-off-by: lilithgrigoryan <[email protected]>

* add: inverse text normalization docs

Signed-off-by: lilithgrigoryan <[email protected]>

* fix: inverse normalization processor comment

Signed-off-by: lilithgrigoryan <[email protected]>

* fix: made requested changes in docs, comments, variable names

Signed-off-by: lilithgrigoryan <[email protected]>

* fix: tests

Signed-off-by: lilithgrigoryan <[email protected]>

* fix: text normalization tests

Signed-off-by: lilithgrigoryan <[email protected]>

* add: changed github actions config file

Signed-off-by: lilithgrigoryan <[email protected]>

* add: changed github actions config file

Signed-off-by: lilithgrigoryan <[email protected]>

* removed nemo_text_processing from requirement

Signed-off-by: lilithgrigoryan <[email protected]>

* added nemo_text_processing back to reqs

Signed-off-by: lilithgrigoryan <[email protected]>

* fix tests

Signed-off-by: lilithgrigoryan <[email protected]>

* add: torchaudio version to reqs

Signed-off-by: lilithgrigoryan <[email protected]>

* add: versions to torch torch audio

Signed-off-by: lilithgrigoryan <[email protected]>

* removed whiper from reqs

Signed-off-by: lilithgrigoryan <[email protected]>

* restored

Signed-off-by: lilithgrigoryan <[email protected]>

* cleanup tests

Signed-off-by: lilithgrigoryan <[email protected]>

* cleanup tests

Signed-off-by: lilithgrigoryan <[email protected]>

* deleted openai-whisper dependency

Signed-off-by: lilithgrigoryan <[email protected]>

* deleted whisper from docs

Signed-off-by: lilithgrigoryan <[email protected]>

* clean up after whisper

Signed-off-by: lilithgrigoryan <[email protected]>

* update transcribe_speech

Signed-off-by: lilithgrigoryan <[email protected]>

* kazakh e2e test, arm confis, transformers processor

Signed-off-by: lilithgrigoryan <[email protected]>

* add pytorch_lightning to reqs

Signed-off-by: lilithgrigoryan <[email protected]>

* add nemo to no-nemo-tests

Signed-off-by: lilithgrigoryan <[email protected]>

* add cython to tests.yml

Signed-off-by: lilithgrigoryan <[email protected]>

* restored ASR processor

Signed-off-by: lilithgrigoryan <[email protected]>

* testing docs

Signed-off-by: lilithgrigoryan <[email protected]>

* finalize

Signed-off-by: lilithgrigoryan <[email protected]>

* rm: clean up

Signed-off-by: lilithgrigoryan <[email protected]>

---------

Signed-off-by: lilithgrigoryan <[email protected]>
Co-authored-by: lilithgrigoryan <[email protected]>
* disabled kazakh and portuguese tests

Signed-off-by: lilithgrigoryan <[email protected]>

* disabled all end2end tests

Signed-off-by: lilithgrigoryan <[email protected]>

* restored armenian tests

Signed-off-by: lilithgrigoryan <[email protected]>

* restored spanish tests

Signed-off-by: lilithgrigoryan <[email protected]>

* muted armenian_audio books test+spanish tests

Signed-off-by: lilithgrigoryan <[email protected]>

* remove: italian mls

Signed-off-by: lilithgrigoryan <[email protected]>

* restored transcribe from nemo

Signed-off-by: lilithgrigoryan <[email protected]>

* restored transcribe file

Signed-off-by: lilithgrigoryan <[email protected]>

* added pip cache purging

Signed-off-by: lilithgrigoryan <[email protected]>

* restored armenaian test

Signed-off-by: lilithgrigoryan <[email protected]>

* clean up

Signed-off-by: lilithgrigoryan <[email protected]>

---------

Signed-off-by: lilithgrigoryan <[email protected]>
Signed-off-by: lilithgrigoryan <[email protected]>
Co-authored-by: lilithgrigoryan <[email protected]>
* Added georgian mcv dataset config file

Signed-off-by: Ssofja <[email protected]>

* add georgian documentation

Signed-off-by: Ssofja <[email protected]>

* add DropRepeatedFields processors

Signed-off-by: Ssofja <[email protected]>

* fix docs

Signed-off-by: Ssofja <[email protected]>

* fix config

Signed-off-by: Ssofja <[email protected]>

* added needed changes

Signed-off-by: Ssofja <[email protected]>

* deleted not needed spaces

Signed-off-by: Ssofja <[email protected]>

---------

Signed-off-by: Ssofja <[email protected]>
* added setup file

Signed-off-by: i-vainn <[email protected]>

* added init files

Signed-off-by: i-vainn <[email protected]>

* init

Signed-off-by: i-vainn <[email protected]>

* updated description

Signed-off-by: i-vainn <[email protected]>

---------

Signed-off-by: i-vainn <[email protected]>
Co-authored-by: i-vainn <[email protected]>
Signed-off-by: Rima <[email protected]>
* fix: gitignore

Signed-off-by: lilithgrigoryan <[email protected]>

* fix: add nemo_text_processing to requirements

Signed-off-by: lilithgrigoryan <[email protected]>

* add: text normalization processor

Signed-off-by: lilithgrigoryan <[email protected]>

* add: unit tests to text normalization processor

Signed-off-by: lilithgrigoryan <[email protected]>

* add: load_manifest, inner join with tests

Signed-off-by: lilithgrigoryan <[email protected]>

* add: logging for total hours

Signed-off-by: lilithgrigoryan <[email protected]>

* add: processor working time logging

Signed-off-by: lilithgrigoryan <[email protected]>

* merge: merge with changes in main main repo

Signed-off-by: lilithgrigoryan <[email protected]>

* add: docs

Signed-off-by: lilithgrigoryan <[email protected]>

* fix: inner join docs

Signed-off-by: lilithgrigoryan <[email protected]>

* add: inverse text normalization processor

Signed-off-by: lilithgrigoryan <[email protected]>

* fix: removed duplicate code

Signed-off-by: lilithgrigoryan <[email protected]>

* add: inverse text normalization docs

Signed-off-by: lilithgrigoryan <[email protected]>

* fix: inverse normalization processor comment

Signed-off-by: lilithgrigoryan <[email protected]>

* fix: made requested changes in docs, comments, variable names

Signed-off-by: lilithgrigoryan <[email protected]>

* fix: tests

Signed-off-by: lilithgrigoryan <[email protected]>

* fix: text normalization tests

Signed-off-by: lilithgrigoryan <[email protected]>

* add: changed github actions config file

Signed-off-by: lilithgrigoryan <[email protected]>

* add: changed github actions config file

Signed-off-by: lilithgrigoryan <[email protected]>

* removed nemo_text_processing from requirement

Signed-off-by: lilithgrigoryan <[email protected]>

* added nemo_text_processing back to reqs

Signed-off-by: lilithgrigoryan <[email protected]>

* fix tests

Signed-off-by: lilithgrigoryan <[email protected]>

* add: torchaudio version to reqs

Signed-off-by: lilithgrigoryan <[email protected]>

* add: versions to torch torch audio

Signed-off-by: lilithgrigoryan <[email protected]>

* removed whiper from reqs

Signed-off-by: lilithgrigoryan <[email protected]>

* restored

Signed-off-by: lilithgrigoryan <[email protected]>

* cleanup tests

Signed-off-by: lilithgrigoryan <[email protected]>

* cleanup tests

Signed-off-by: lilithgrigoryan <[email protected]>

* deleted openai-whisper dependency

Signed-off-by: lilithgrigoryan <[email protected]>

* deleted whisper from docs

Signed-off-by: lilithgrigoryan <[email protected]>

* clean up after whisper

Signed-off-by: lilithgrigoryan <[email protected]>

* update transcribe_speech

Signed-off-by: lilithgrigoryan <[email protected]>

* kazakh e2e test, arm confis, transformers processor

Signed-off-by: lilithgrigoryan <[email protected]>

* add pytorch_lightning to reqs

Signed-off-by: lilithgrigoryan <[email protected]>

* add nemo to no-nemo-tests

Signed-off-by: lilithgrigoryan <[email protected]>

* add cython to tests.yml

Signed-off-by: lilithgrigoryan <[email protected]>

* restored ASR processor

Signed-off-by: lilithgrigoryan <[email protected]>

* testing docs

Signed-off-by: lilithgrigoryan <[email protected]>

* finalize

Signed-off-by: lilithgrigoryan <[email protected]>

* rm: clean up

Signed-off-by: lilithgrigoryan <[email protected]>

---------

Signed-off-by: lilithgrigoryan <[email protected]>
Co-authored-by: lilithgrigoryan <[email protected]>
* fix: gitignore

Signed-off-by: lilithgrigoryan <[email protected]>

* fix: add nemo_text_processing to requirements

Signed-off-by: lilithgrigoryan <[email protected]>

* add: text normalization processor

Signed-off-by: lilithgrigoryan <[email protected]>

* add: unit tests to text normalization processor

Signed-off-by: lilithgrigoryan <[email protected]>

* add: load_manifest, inner join with tests

Signed-off-by: lilithgrigoryan <[email protected]>

* add: logging for total hours

Signed-off-by: lilithgrigoryan <[email protected]>

* add: processor working time logging

Signed-off-by: lilithgrigoryan <[email protected]>

* merge: merge with changes in main main repo

Signed-off-by: lilithgrigoryan <[email protected]>

* add: docs

Signed-off-by: lilithgrigoryan <[email protected]>

* fix: inner join docs

Signed-off-by: lilithgrigoryan <[email protected]>

* add: inverse text normalization processor

Signed-off-by: lilithgrigoryan <[email protected]>

* fix: removed duplicate code

Signed-off-by: lilithgrigoryan <[email protected]>

* add: inverse text normalization docs

Signed-off-by: lilithgrigoryan <[email protected]>

* fix: inverse normalization processor comment

Signed-off-by: lilithgrigoryan <[email protected]>

* fix: made requested changes in docs, comments, variable names

Signed-off-by: lilithgrigoryan <[email protected]>

* fix: tests

Signed-off-by: lilithgrigoryan <[email protected]>

* fix: text normalization tests

Signed-off-by: lilithgrigoryan <[email protected]>

* add: changed github actions config file

Signed-off-by: lilithgrigoryan <[email protected]>

* add: changed github actions config file

Signed-off-by: lilithgrigoryan <[email protected]>

* removed nemo_text_processing from requirement

Signed-off-by: lilithgrigoryan <[email protected]>

* added nemo_text_processing back to reqs

Signed-off-by: lilithgrigoryan <[email protected]>

* fix tests

Signed-off-by: lilithgrigoryan <[email protected]>

* add: torchaudio version to reqs

Signed-off-by: lilithgrigoryan <[email protected]>

* add: versions to torch torch audio

Signed-off-by: lilithgrigoryan <[email protected]>

* removed whiper from reqs

Signed-off-by: lilithgrigoryan <[email protected]>

* restored

Signed-off-by: lilithgrigoryan <[email protected]>

* cleanup tests

Signed-off-by: lilithgrigoryan <[email protected]>

* cleanup tests

Signed-off-by: lilithgrigoryan <[email protected]>

* deleted openai-whisper dependency

Signed-off-by: lilithgrigoryan <[email protected]>

* deleted whisper from docs

Signed-off-by: lilithgrigoryan <[email protected]>

* clean up after whisper

Signed-off-by: lilithgrigoryan <[email protected]>

* update transcribe_speech

Signed-off-by: lilithgrigoryan <[email protected]>

* kazakh e2e test, arm confis, transformers processor

Signed-off-by: lilithgrigoryan <[email protected]>

* add pytorch_lightning to reqs

Signed-off-by: lilithgrigoryan <[email protected]>

* add nemo to no-nemo-tests

Signed-off-by: lilithgrigoryan <[email protected]>

* add cython to tests.yml

Signed-off-by: lilithgrigoryan <[email protected]>

* restored ASR processor

Signed-off-by: lilithgrigoryan <[email protected]>

* testing docs

Signed-off-by: lilithgrigoryan <[email protected]>

* finalize

Signed-off-by: lilithgrigoryan <[email protected]>

* rm: clean up

Signed-off-by: lilithgrigoryan <[email protected]>

---------

Signed-off-by: lilithgrigoryan <[email protected]>
Co-authored-by: lilithgrigoryan <[email protected]>
@rimashahbazyan rimashahbazyan requested review from erastorgueva-nv and removed request for karpnv September 24, 2024 06:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants