Integrating with Dart/Flutter #571

kunle-e · 2022-10-21T05:14:10Z

kunle-e
Oct 21, 2022

I've built the example, simple_capture into a flutter app using Dart:FFI to see if I can get the library to record from the microphone on a google pixel 3a. However I only get the failed to initialize output file error. The app is not prompting for the microphone access either. I feel like I'm missing something. Is there more that I need to do than simply copying the code from the the example? The example does not mention doing anything more.

mackron · 2022-10-21T07:14:55Z

mackron
Oct 21, 2022
Maintainer

I don't have experience with Flutter nor Dart so not sure I'll be of much help, but does Flutter run on Web technologies under the hood?miniaudio supports web, but it's done through Emscripten so I don't think the FFI system would pick that up properly.

The way miniaudio's build system works is that it only compiles in the backends that are relevant for the compiler you're using. For example, if you're compiling using a Linux compiler, it'll only include the ALSA and PulseAudio backends in the build. Could this perhaps be the problem? How are you supplying Dart with the C code? I'm guessing you need to compile it into a library and then give that to Dart with some kind of description of the relevant functions?

2 replies

kunle-e Oct 21, 2022
Author

Flutter has a couple of ways to interface with native C and C++ using Dart FFI and Platform Channels. The code is being compiled because I can call C functions from Flutter and log the output. My intention was to get miniaudio to compile targeting android. I have a CMakeLists file to build the miniaudio capture demo. The Flutter tooling takes care of the compile target when I run the build command, I believe. I'm just wondering if I should be adding some configuration or if using the plain miniaudio capture example is supposed to work on android as it is.
Dart FFI: https://docs.flutter.dev/development/platform-integration/android/c-interop

mackron Oct 21, 2022
Maintainer

Is it possible to compile miniaudio with the MA_DEBUG_OUTPUT define and somehow show the log (it uses printf() - not sure how that's translated in Flutter)? It would be useful to see what backends have been included in the build. My suspicion is that it's not detecting a supported backend and is falling back to the null backend (silence). The Android backends will only be included in the build if it's compiled with an Android toolchain (I also just fixed an issue with Android detection in the dev branch FYI).

kunle-e · 2023-01-10T02:08:56Z

kunle-e
Jan 10, 2023
Author

Just getting back to this. I was able to integrate miniaudio with flutter and call my native Android C++ function via Dart FFI that runs the miniaudio code. I'm using a slightly modified [Hello World example]
(https://miniaud.io/docs/examples/engine_hello_world.html) (below) because there is no CLI on mobile. I don't hear any audio and there are no errors. Maybe someone can make sense of it? [Update: I forgot about the previous comment re. compile options. I need to lookup how to do that in CMakeLists.txt and report back.]

Output:*

D/ple.weather_ff(21751): PlayerBase::PlayerBase()
D/AudioStreamInternal_Client(21751): open() original HW burst = 96, minMicros = 2000 => SW burst = 96
D/AAudioStream(21751): setState(s#1) from 0 to 2
I/AAudio  (21751): AAudioStreamBuilder_openStream() returns 0 = AAUDIO_OK for s#1 ----------------
D/AAudio  (21751): AAudioStream_requestStart(s#1) called --------------
D/AAudioStream(21751): setState(s#1) from 2 to 3
D/ple.weather_ff(21751): PlayerBase::start() from IPlayer
D/AAudio  (21751): AAudioStream_requestStart(s#1) returned 0 ---------
D/AudioStreamInternalPlay_Client(21751): callbackLoop() entering >>>>>>>>>>>>>>>
D/AudioStreamInternal_Client(21751): onEventFromServer - got AAUDIO_SERVICE_EVENT_STARTED
D/AAudioStream(21751): setState(s#1) from 3 to 4
D/AAudio  (21751): AAudioStream_requestStop(s#1) called
D/AudioStreamInternalPlay_Client(21751): callbackLoop() exiting, result = 480, isActive() = 1 <<<<<<<<<<<<<<
D/IsochronousClockModel(21751): stop(nanos = 14488029576920) max lateness = 0 micros
D/AAudioStream(21751): setState(s#1) from 4 to 9
D/ple.weather_ff(21751): PlayerBase::stop() from IPlayer
D/AudioStreamInternal_Client(21751): onEventFromServer - got AAUDIO_SERVICE_EVENT_STOPPED
D/AAudioStream(21751): setState(s#1) from 9 to 10
D/AAudio  (21751): AAudioStream_close(s#1) called ---------------
D/AAudioStream(21751): setState(s#1) from 10 to 11
D/AAudioStream(21751): setState(s#1) from 11 to 11
E/AAudioStream(21751): setState(1) tried to set to 11 but already CLOSING
D/AAudioStream(21751): setState(s#1) from 11 to 12
D/AAudioStream(21751): ~AudioStream(s#1) mPlayerBase strongCount = 2
D/AAudio  (21751): AAudioStream_close(s#1) returned 0 ---------

Modified Hello World Example

int testMiniaudio(){
    
    ma_result result;
    ma_engine engine;
    
    char * filePath = "/data/user/0/com.example.weather_ffi/app_flutter/testing123.wav";

    result = ma_engine_init(NULL, &engine);
    if (result != MA_SUCCESS) {
        printf("Failed to initialize audio engine.");
  __android_log_print(ANDROID_LOG_DEBUG, "loglevel: flutter", "Failed to initialize audio engine.. %s", "");

        return -1;
    }

    ma_engine_play_sound(&engine, filePath, NULL);

    sleep(5);

    ma_engine_uninit(&engine);

    return 0;
}

0 replies

kunle-e · 2023-01-10T02:46:41Z

kunle-e
Jan 10, 2023
Author

Added the #define MA_DEBUG_OUTPUT macro instead of the compile options flag. This is the output. It looks like it tried to initiate AAudio...:

D/loglevel: flutter(25504): StartAudio CAlled
D/miniaudio(25504): DEBUG: Failed to initialize WASAPI backend.
D/miniaudio(25504): DEBUG: Failed to initialize DirectSound backend.
D/miniaudio(25504): DEBUG: Failed to initialize WinMM backend.
D/miniaudio(25504): DEBUG: Failed to initialize Core Audio backend.
D/miniaudio(25504): DEBUG: Failed to initialize sndio backend.
D/miniaudio(25504): DEBUG: Failed to initialize audio(4) backend.
D/miniaudio(25504): DEBUG: Failed to initialize OSS backend.
D/miniaudio(25504): DEBUG: Failed to initialize PulseAudio backend.
D/miniaudio(25504): DEBUG: Failed to initialize ALSA backend.
D/miniaudio(25504): DEBUG: Failed to initialize JACK backend.
D/miniaudio(25504): DEBUG: Attempting to initialize AAudio backend...
D/miniaudio(25504): DEBUG: Loading library: libaaudio.so
D/miniaudio(25504): DEBUG: Loading symbol: AAudio_createStreamBuilder
D/miniaudio(25504): DEBUG: Loading symbol: AAudioStreamBuilder_delete
D/miniaudio(25504): DEBUG: Loading symbol: AAudioStreamBuilder_setDeviceId
D/miniaudio(25504): DEBUG: Loading symbol: AAudioStreamBuilder_setDirection
D/miniaudio(25504): DEBUG: Loading symbol: AAudioStreamBuilder_setSharingMode
D/miniaudio(25504): DEBUG: Loading symbol: AAudioStreamBuilder_setFormat
D/miniaudio(25504): DEBUG: Loading symbol: AAudioStreamBuilder_setChannelCount
D/miniaudio(25504): DEBUG: Loading symbol: AAudioStreamBuilder_setSampleRate
D/miniaudio(25504): DEBUG: Loading symbol: AAudioStreamBuilder_setBufferCapacityInFrames
D/miniaudio(25504): DEBUG: Loading symbol: AAudioStreamBuilder_setFramesPerDataCallback
D/miniaudio(25504): DEBUG: Loading symbol: AAudioStreamBuilder_setDataCallback
D/miniaudio(25504): DEBUG: Loading symbol: AAudioStreamBuilder_setErrorCallback
D/miniaudio(25504): DEBUG: Loading symbol: AAudioStreamBuilder_setPerformanceMode
D/miniaudio(25504): DEBUG: Loading symbol: AAudioStreamBuilder_setUsage
D/miniaudio(25504): DEBUG: Loading symbol: AAudioStreamBuilder_setContentType
D/miniaudio(25504): DEBUG: Loading symbol: AAudioStreamBuilder_setInputPreset
D/miniaudio(25504): DEBUG: Loading symbol: AAudioStreamBuilder_openStream
D/miniaudio(25504): DEBUG: Loading symbol: AAudioStream_close
D/miniaudio(25504): DEBUG: Loading symbol: AAudioStream_getState
D/miniaudio(25504): DEBUG: Loading symbol: AAudioStream_waitForStateChange
D/miniaudio(25504): DEBUG: Loading symbol: AAudioStream_getFormat
D/miniaudio(25504): DEBUG: Loading symbol: AAudioStream_getChannelCount
D/miniaudio(25504): DEBUG: Loading symbol: AAudioStream_getSampleRate
D/miniaudio(25504): DEBUG: Loading symbol: AAudioStream_getBufferCapacityInFrames
D/miniaudio(25504): DEBUG: Loading symbol: AAudioStream_getFramesPerDataCallback
D/miniaudio(25504): DEBUG: Loading symbol: AAudioStream_getFramesPerBurst
D/miniaudio(25504): DEBUG: Loading symbol: AAudioStream_requestStart
D/miniaudio(25504): DEBUG: Loading symbol: AAudioStream_requestStop
D/miniaudio(25504): DEBUG: System Architecture:
D/miniaudio(25504): DEBUG:   Endian: LE
D/miniaudio(25504): DEBUG:   SSE2:   NO
D/miniaudio(25504): DEBUG:   AVX2:   NO
D/miniaudio(25504): DEBUG:   NEON:   YES
I/AAudio  (25504): AAudioStreamBuilder_openStream() called ----------------------------------------
I/AudioStreamBuilder(25504): rate   =      0, channels  = 0, format   = 5, sharing = SH, dir = OUTPUT
I/AudioStreamBuilder(25504): device =      0, sessionId = -1, perfMode = 12, callback: ON with frames = 480
I/AudioStreamBuilder(25504): usage  =      0, contentType = 0, inputPreset = 0, allowedCapturePolicy = 0
I/AudioStreamBuilder(25504): privacy sensitive = false
D/ple.weather_ff(25504): PlayerBase::PlayerBase()
D/AudioStreamInternal_Client(25504): open() original HW burst = 96, minMicros = 2000 => SW burst = 96
D/AAudioStream(25504): setState(s#1) from 0 to 2
I/AAudio  (25504): AAudioStreamBuilder_openStream() returns 0 = AAUDIO_OK for s#1 ----------------
D/miniaudio(25504): INFO: [AAudio]
D/miniaudio(25504): INFO:    (Playback)
D/miniaudio(25504): INFO:     Format:      32-bit IEEE Floating Point -> 32-bit IEEE Floating Point
D/miniaudio(25504): INFO:     Channels:    2 -> 2
D/miniaudio(25504): INFO:     Sample Rate: 48000 -> 48000
D/miniaudio(25504): INFO:     Buffer Size: 480*3 (1440)
D/miniaudio(25504): INFO:     Conversion:
D/miniaudio(25504): INFO:       Pre Format Conversion:  NO
D/miniaudio(25504): INFO:       Post Format Conversion: NO
D/miniaudio(25504): INFO:       Channel Routing:        NO
D/miniaudio(25504): INFO:       Resampling:             NO
D/miniaudio(25504): INFO:       Passthrough:            YES
D/miniaudio(25504): INFO:       Channel Map In:         {CHANNEL_FRONT_LEFT CHANNEL_FRONT_RIGHT}
D/miniaudio(25504): INFO:       Channel Map Out:        {CHANNEL_FRONT_LEFT CHANNEL_FRONT_RIGHT}
D/AAudio  (25504): AAudioStream_requestStart(s#1) called --------------
D/AAudioStream(25504): setState(s#1) from 2 to 3
D/ple.weather_ff(25504): PlayerBase::start() from IPlayer
D/AAudio  (25504): AAudioStream_requestStart(s#1) returned 0 ---------
D/AudioStreamInternalPlay_Client(25504): callbackLoop() entering >>>>>>>>>>>>>>>
D/AudioStreamInternal_Client(25504): onEventFromServer - got AAUDIO_SERVICE_EVENT_STARTED
D/AAudioStream(25504): setState(s#1) from 3 to 4
D/AAudio  (25504): AAudioStream_requestStop(s#1) called
D/AudioStreamInternalPlay_Client(25504): callbackLoop() exiting, result = 480, isActive() = 1 <<<<<<<<<<<<<<
D/IsochronousClockModel(25504): stop(nanos = 17001455576106) max lateness = 0 micros
D/AAudioStream(25504): setState(s#1) from 4 to 9
D/ple.weather_ff(25504): PlayerBase::stop() from IPlayer
D/AudioStreamInternal_Client(25504): onEventFromServer - got AAUDIO_SERVICE_EVENT_STOPPED
D/AAudioStream(25504): setState(s#1) from 9 to 10
D/AAudio  (25504): AAudioStream_close(s#1) called ---------------
D/AAudioStream(25504): setState(s#1) from 10 to 11
D/AAudioStream(25504): setState(s#1) from 11 to 11
E/AAudioStream(25504): setState(1) tried to set to 11 but already CLOSING
D/AAudioStream(25504): setState(s#1) from 11 to 12
D/AAudioStream(25504): ~AudioStream(s#1) mPlayerBase strongCount = 2
D/AAudio  (25504): AAudioStream_close(s#1) returned 0 ---------

0 replies

mackron · 2023-01-10T03:09:50Z

mackron
Jan 10, 2023
Maintainer

The code you posted is not going to work. That is initialising the engine then stopping it straight away after 5 milliseconds. You need to keep the engine alive for the life of your program.

3 replies

kunle-e Jan 10, 2023
Author

[Update: Nevermind this comment. I just needed to add more time to the sleep. I can remove the uninit, but I can't let the function end until the audio is finished, so adding getchar() or a sleep(5000) works on coreaudio. Will try it on AAudio next]

I noticed that on coreaudio if I remove

getchar();
ma_engine_uninit(&engine);

then I get no audio. That's why I added it thinking that it would be enough time to play a short audio file. I guess the bigger question is why there is no audio when ma_engine_uninit(&engine); is not present. If I remove the sleep(), then ma_engine_uninit(&engine); will immediately be call resulting in no audio, so I have to have both for it to work. This is how it is behaving on coreaudio, so that's why I added it for the Android.

mackron Jan 10, 2023
Maintainer

Yeah it's because your function is returning which is resulting in your ma_engine object getting destroyed because you've declared it on the stack. miniaudio is unique in that you need to manage the memory of your objects yourself. My suggestion is to declare it globally, or allocate it on the heap and then pass a pointer around via a parameter.

More info in the documentation: https://miniaud.io/docs/manual/

kunle-e Jan 10, 2023
Author

Ok. I got some noise out of this thing on Android via Flutter. Thanks for that tip about ensuring that the ma_engine object is not destroyed prematurely. I need to revisit the documentation. At least I know it can work with Flutter/Dart.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Integrating with Dart/Flutter #571

{{title}}

Replies: 4 comments 5 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

Select a reply

Integrating with Dart/Flutter #571

kunle-e Oct 21, 2022

Replies: 4 comments · 5 replies

mackron Oct 21, 2022 Maintainer

kunle-e Oct 21, 2022 Author

mackron Oct 21, 2022 Maintainer

kunle-e Jan 10, 2023 Author

kunle-e Jan 10, 2023 Author

mackron Jan 10, 2023 Maintainer

kunle-e Jan 10, 2023 Author

mackron Jan 10, 2023 Maintainer

kunle-e Jan 10, 2023 Author

kunle-e
Oct 21, 2022

Replies: 4 comments 5 replies

mackron
Oct 21, 2022
Maintainer

kunle-e Oct 21, 2022
Author

mackron Oct 21, 2022
Maintainer

kunle-e
Jan 10, 2023
Author

kunle-e
Jan 10, 2023
Author

mackron
Jan 10, 2023
Maintainer

kunle-e Jan 10, 2023
Author

mackron Jan 10, 2023
Maintainer

kunle-e Jan 10, 2023
Author