Skip to content

A voice based email system for the visually impaired. This is a python based project that allows users to send email, read email, search email in bin, favourite, etc all with the help of voice commands. It also summarises the email. The project uses whisper model for voice to text conversion, python for backend and flask for UI.

Notifications You must be signed in to change notification settings

RishikaC06/Voice-based-Email-System

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

8 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Voice based Email System

This project focuses on the development of a user-friendly application tailored specifically for visually impaired individuals. The application leverages voice commands to enable users to send emails, check email status, search through messages, and read inbox content, all without the need for visual interaction. This project uses Whisper model for speech to text conversion and GTTS library to vocalize text response, thus facilitating a voice based interaction with users.

Technology used

  1. Whisper Model
  2. gTTs library
  3. Python
  4. Flask
  5. HTML/ CSS/ JS

Workflow

image

  1. User Registration Enable 2-step verification in gmail and generate app password then use your email id and generated app password along with username and password of your choice for registration.
  2. Login with voice Users can login by speaking their id and password which is matched with hashed password for authentication.
  3. Voice based response gTTs python library is used to convert text to speech in order to provide voice based response.
  4. Audio Recording
    • The app uses PyAudio to record audio in WAV format which continues till user press space bar.
    • Audio parameters: 16-bit PCM, mono channel, 44.1 kHz sample rate.
  5. Speech to Text Conversion The app uses the Whisper tiny.en model for English transcription (ie. converting voice to text)
  6. Email functionalities
    • Compose and Send Email: Users can dictate emails, which are confirmed and sent securely using Gmail's SMTP server.
    • Mailbox Status: The app retrieves the status of email folders via IMAP, announcing the number of messages in each.
    • Search Email: Users can search emails by criteria like sender or subject, with results read aloud or skipped as desired.
    • Retrieve Latest Emails: Users can fetch and summarize the latest emails from selected folders. Summaries are created using NLTK python library.

Steps for installation

  1. Install the necessary packages
pip install playsound
pip install PyAudio
pip install gTTS
pip install nltk
  1. Installing whisper

    • Installing ffmpeg: To install whisper you will first need to download and install ffmpeg. You can follow this tutorial for its installation Install ffmpeg.
    • Install whisper: pip install -U openai-whisper You can also refer to this whisper installation guide Whisper reference
  2. Installing flask for frontend Use the file app.py

    pip install Flask
    
  3. To run the app use

    python -u "app.py"
    

    Open http://127.0.0.1:8000 to view the app.

Acknowledgements

Ronik22

About

A voice based email system for the visually impaired. This is a python based project that allows users to send email, read email, search email in bin, favourite, etc all with the help of voice commands. It also summarises the email. The project uses whisper model for voice to text conversion, python for backend and flask for UI.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published