GitHub - acharyabi/Text-Data-Cleaning: Pre-processing Text Data In Multiple Language(tested in Nepali).

acharyabi / Text-Data-Cleaning Public

Notifications You must be signed in to change notification settings
Fork 0
Star 0

Pre-processing Text Data In Multiple Language(tested in Nepali).

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 3 Commits
Cleaning_Datase.ipynb		Cleaning_Datase.ipynb
Readme		Readme

Repository files navigation

This Code Can Be Used For Filtering Text Data Using Regex and Basic Data Frame Techniques, In Our Project Nepali Paraphrasing Using Trasnfomer We Have Used It For Following Things In Our Custom Dataset:
-Metrics Based Filtering
-Removal Of English Text And Numeric[Rows Removal] #Can Use Regex For Removal From Sentences Without Removing The Row
-Removal Of Hyperlinks and Unwanted Characters

Prerequisites:
#Basic Knowledge of Python, Data Frames and Regex.

Hope This Helped In Your Project.

Regards,
Abinash.

Contact: [email protected]