No input from microphone & ollama fails to summarize & long conversation fail

[photosartd]

Hey, tried out the repo and encountered 3 issues:

1. The app does not transcribe (or use) the audio from my laptop's microphone. It only transcribes system audio (e.g. an audio from videos, podcasts, telegram conversations (other speakers only)). Checked it, the input device was Macbook Microphone.
2. I have ollama installed, it works with other extensions perfectly (e.g. continue for coding), but it fails to summarize here. In the logs in terminal I see it processes some inputs sequentially, then dumps the result json (I think I can even see the result there) and tries to repeat the process several times more until the error pop ups in the frontend. (Models I tried - llama3.1 8b and mistral-instruct).
3. Had a long Telegram call (about 1.5 hours), decided to try it out at least for an external speaker. The whole discussion I could see the transcriptions, but once I finished recording it just stuck, I was not able to see the older messages (some UI unresponsiveness) and was able only to rerun the frontend, after which I already could not find the conversation.

My laptop: MacBook Air M1, 16Gb

Overall happy to see such an app exists, just needs some more testing. If you need some more help in testing or maybe even coding (python/C++ primarily), let me know.

[sujithatzackriya]

Hey @photosartd ,

Thanks for offering support and sharing feedback. Let me address all issues

1. The app does not transcribe (or use) the audio from my laptop's microphone.

I had the same issue initially. then I checked the mic settings on my mac. my input volume level was initially very low and that resulted in lack of clarity while recording the Microphone. Could you please increase this and try? If the issue persists, please let us know.

2. I have ollama installed, it works with other extensions perfectly (e.g. continue for coding), but it fails to summarize here. - From the observed outputs, the ollama models with lower parameters (Below 32B parameters) is very bad at function calling, which gets them into a loop where it tries to call the function, gets the output and then hallucinate. this is a problem that we are trying to solve by fine tuning a smaller model for this purpose. Could you please share the details of models that you used? If the model does not support function / tool calling, this actually fails now.

3. once I finished recording it just stuck, I was not able to see the older messages (some UI unresponsiveness) and was able only to rerun the frontend, after which I already could not find the conversation. - The db connection has a bug in it and as of now, the transcript is not getting saved. This is a serious issue that we are trying to solve. This will be fixed in next update and history of meetings shall be added.

When recoding is stopped, the app tries to save the final recording locally. this is why the UI is getting stuck. We'll work on a thread based approach to make this process run in parallel.

Thank you for the feedbacks and the product is still small and new (Only 6 weeks since we started developing this.). We have a backlog as of now and happy to see that people actually find the core idea useful.

We'll be working on our next update and will try to fix 3 in that update. 1 and 2, please let me know your observations.

[photosartd]

Hey, sorry for the late answer:

1. I tried increasing this parameter, however, it did not help at all.
2. The models I tried to use are: llama3.1:8b, mistral-instruct:7b. Bot did not work.

[sujithatzackriya]

Hey @photosartd

Issue 1 - No input from microphone

We were fixing the Microphone issue. It was a bug and we created an issue to address it here

I've added the latest .dmg installer file to the release. We are still testing this version for microphone related issues.

Please find the release here - https://github.com/Zackriya-Solutions/meeting-minutes/releases/tag/v0.0.2

Instructions

Removing existing installation & Permissions

1. Delete the existing application from the Applications directory
2. Remove the meeting-minutes-frontend.app from the Screen & System Audio recording permissions table

Once selected, press the `-` button to remove

Downloading & Running the new dmg

1. Download the dmg.zip file from https://github.com/Zackriya-Solutions/meeting-minutes/releases/tag/v0.0.2
2. Extract the .zip file
3. double click meeting-minutes-frontend_0.1.0_aarch64.dmg file inside the dmg folder
4. Move the meeting-minutes-frontend.app to Applications folder
5. Execute the following command in terminal

xattr -c /Applications/meeting-minutes-frontend.app

6. Start the application, it should ask for microphone permission now.

If you find any issues, please share your thoughts here or under the PR

#38 (comment)

Issue 2 - Ollama fails to summarize

Smaller models are continuously failing in generating good results. it hallucinates and gets in a loop rather than doing the actual tool calling. We are finding ways to fix this behaviour.

Issue 3 - Long conversation fail

We found the issue. It is an audio processing issue and we are working on solving it. Also, we are adding meeting history to the tool so that you'll be able to look at the summary and minutes later. for now, we might remove the recorded audio playback option as it is causing some trouble.

[sujithatzackriya]

Issues 1 and 3 mentioned in this are solved.

[photosartd]

@sujithatzackriya there is a new small ollama model phi4-mini which is listed as being able to do function calling. Haven't tried it yet, but if it works it makes sense to put this somewhere to readme so that more people without API subscriptions could try out your app

https://ollama.com/library/phi4-mini

[photosartd]

Although it's for 0.5.13 Ollama version only which is still in pre-release

[photosartd]

Btw just tried with this phi4 model but it still fails in the same way:(

[photosartd]

Like if the ui does not break after a long conversation anymore it's still usable to e.g. copy-paste the transcript manually to some LLM, but haven't tried it yet

[sujithatzackriya]

@sujithatzackriya there is a new small ollama model phi4-mini which is listed as being able to do function calling. Haven't tried it yet, but if it works it makes sense to put this somewhere to readme so that more people without API subscriptions could try out your app

https://ollama.com/library/phi4-mini

Installed the ollama pre release version and tested the model. Following are my obsetvations

1. The model itself is hallucinating a lot and is Q4 quantised.

2. The model failed to do the tool calling. Maybe will work on another approach for generating summaries with smaller models

Like if the ui does not break after a long conversation anymore it's still usable to e.g. copy-paste the transcript manually to some LLM, but haven't tried it yet

The UI issue is fixed and the UI can handle long calls now. This has been tested. Copy Transcript option is available in the UI as well.