ConcatenatingAudioSource with lots of children takes a very long time to load #294
Comments
|
Hi @smkhalsa unfortunately I will need you to fill in the sections as instructed. In the section called "User steps, not code", you actually copied and pasted code. This should be in the previous section "Minimal reproduction project" which you have left empty, but which should contain a link to a git repository that I can clone and test. |
|
This issue affects both iOS and macOS. I have a reproduction case on #206 which has helped me to reproduce this issue. The solution is to rewrite the This is a tricky area of the code to work on since it can easily break things, but it is an important feature to add, and as long as it undergoes sufficient testing, it should be worth doing. I've marked this issue as "fixing" which means it is on my priority list. That said, there are some other things that were already on my priority list, such as the visualizer, and the null-safe release of audio_service (which will be based on the one-isolate model). |
|
@ryanheise The I was noticing similar behavior with 43 items in the playlist. My UI freezes while they are loading on macOS and iOS. On the iOS simulator the freeze lasted 30 seconds. Web and Android are fine. Can you think of any temporary workarounds I could pursue on my side? |
|
That's correct. The solution involves improving the As for a workaround, I can think of 2 things to try:
|
|
I forgot about the possibility of adding items to an existing playlist. That's a good idea. Thank you! |
|
Just to add onto this, I have only noticed this issue on MacOS release/debug and only on iOS Simulator. This issue is not present, at least for me, on Physical iOS device whether release or debug. Android is Fine. |
|
As far as I understand, the question that I have is heavily related to this issue (though not exactly relevant, I use Android and it isn't a "bug" per say, it's just that retrieving 40+ URLs is time-costly with the API I use) I am not sure if I can inject a function to lazy load the track URL in |
|
|
|
As far as I understood, |
|
This issue is about iOS/macOS. If you have found an Android bug or have an Android feature request, it will be appreciated to open a separate issue for that. |
|
Any progress about the issue? I'm developing an app for windows/mac and player is freezing for 4-5 seconds when i add 60 song. |
|
This bug should have high priority in my opinion. Would solving it require switching to an |
|
The current AVQueuePlayer will work fine, but the enqueue logic should be rewritten to enqueue a configurable number of items, and at the same time, the gapless looping approach should be adapted to this. It's going to take a while to develop and test. However, you can work around this by doing the same sort of queue management in dart. That is, only add to the concatenating audio source as many children as you want to actually load, then lazily add more children just before they're needed. |
|
Also I'll quote my earlier comment from above which I think may do a better job of explaining how you could achieve your use case:
Incidentally, @ddfreiling , I would also recommend clicking the 👍 on this issue to vote for it. I do try to fix critical bugs quickly regardless of votes, but since this issue is not a critical bug (you can take resource management into your own hands to some extent with the above workaround), and since I'm dealing with a great number of requests and have to prioritise what I work on next based on helping the greatest number of people, the votes do count. The alternative is that if you can't wait for me personally to work on this, you can consider becoming a contributor, since those who need it will often have the strongest motivation to help work on it. I can't say it's an easy one to work on, though. First, it requires Objective C knowledge, and then second it also requires studying and comprehending what is probably the most complex part of the iOS implementation, and how it fits together with the rest of the implementation. |
|
For those subscribed to this issue, there is a related issue that can cause the same symptom of blocking the UI thread, but has a simpler solution. Basically, if you are constructing a very large list of children to pass into for (var i = 0; i < 100000; i++) {
children.add(...);
}Then that code itself will block the UI thread because it does not yield to co-routines. You can fix that for example by inserting I mention this just in case anyone experiencing the UI blocking issue actually has this other easier-to-fix issue. |
|
Any update on this issue? |
|
This will always be problematic to some degree because of Dart's single threaded model. Fortunately there are ways to deal with this which I have already mentioned (in my edit at the bottom of the very first post). |
|
Hi. I have tried a lot to play with your workaround. I was programatically adding the track in the list. But unfortunately it didn't work for me. It's a humble request to you. Please work for a fix. We love your work, You made it amazing. <3 Hope this will be fixed soon. |
|
I know that people are using that workaround successfully (lazily adding items to the playlist just-in-time so that you don't try to load them all at once). I suggest you keep trying to implement that workaround because it is going to be much harder to build multi-threaded behaviour into the plugin and you'll be waiting much longer. |
|
@ ryanheise : Can you provide us a code example that how can I lazily load items? |
|
For me lazy loading didn't work. To be honest - it looks ugly. But when it's covered with the tests - somehow you can trust it |
|
I think lazy loading will work. Because if it listen to the list that I am adding new item then it will work. But in my case the problem is when it loads, it loads with initial items. Then I keep adding the file by a for loop. But it don't see the changes (new files). After playing the initial files it stops. I think if we get an example code from @ryanheise then it will be easy to implement the feature. Let's wait with a hope from @ryanheise . |
|
In the case of If you do have a very long playlist, yes, you do have to just avoid passing very long lists into The long term future direction of the plugin should be to shift the internal implementation somewhat in this direction. There are complications to that, so there is no short term quick fix, apart from using |
|
@ryanheise Is there any example code available that I may refer that follows your suggested approach to fill ConcatenatingAudioSource dynamically by 3 items only? Thanks in advance! |
|
@ryanheise thanks for that quick response! That makes sense. Someday I'll be knowledgeable enough on the topic to be able to assist with the amazing work you do! @jagpreetsethi Once I finish my implemention of what Ryan is talking about, I'll post it here. It will include shuffling. |
|
Hi @ryanheise I've been trying to include the just_audio package in my Flutter project using its Git repository as the source. However, I've been encountering an issue when specifying the Git reference in the pubspec.yaml file. Here's how I'm adding the package to my pubspec.yaml file: However, every time I attempt to fetch the package, I get an error indicating that the pubspec.yaml file cannot be found in the specified Git reference:
The reference I'm using is: feature/treadmill, and I've verified that it exists in the repository: https://github.com/ryanheise/just_audio/tree/feature/treadmill. I've tried cleaning the cache, updating Flutter packages, and even cloning the repository locally to test, but the issue persists. Is there something I might be missing or any suggestions you can provide to help me resolve this? I greatly appreciate your assistance in this matter. Thank you! |
|
It's because just_audio isn't in the root directory of this repo, it's in the just_audio:
git:
url: https://github.com/ryanheise/just_audio.git
ref: feature/treadmill
path: just_audio |
|
For me this |
|
@jagpreetsethi I implemented what @ryanheise was suggesting. However, I wasn't able to figure out shuffling while preloading without having any visual glitches or refreshes when switching between shuffle mode and non-shuffle mode. So I left it out. If anyone can modify it to add it, I would be very grateful. The Implementation:import 'dart:async';
import 'dart:math' show max, min;
import 'package:audio_service/audio_service.dart';
import 'package:just_audio/just_audio.dart';
import 'package:quiver/iterables.dart' show range;
import 'package:rxdart/rxdart.dart';
Future<AudioHandler> initAudioService() async {
return await AudioService.init(
builder: () => MyAudioHandler(),
config: const AudioServiceConfig(
androidNotificationChannelId: 'com.example.app.audio',
androidNotificationChannelName: 'Example App',
androidNotificationOngoing: true,
androidStopForegroundOnPause: true,
),
);
}
class PlaylistManager {
final _player = AudioPlayer();
final _subPlaylist = ConcatenatingAudioSource(
children: [],
useLazyPreparation: false, // I want it to pre-cache everything I put in this playlist.
);
/// A copy of the media items so we can access them later
final List<MediaItem> _subPlaylistItems = [];
/// The full list of media items to pull from
final List<MediaItem> _fullPlaylist = [];
/// Used for keeping track of how many tracks are in front of the current index of the player.
int _playlistFrontLoadingCount = 0;
/// Change this if you want the index to not start at the beginning
static const kDefaultIndex = 0;
/// Keeps track of the current index of the _fullPlaylist
int _currentFullPlaylistIndex = kDefaultIndex;
/// Used for cleaning up the subscriptions when disposed
final List<StreamSubscription> _streamSubscriptions = [];
/// Used for preventing race conditions in the _lazyLoadPlaylist
bool _updatePlayerRunning = false;
/// Used for preventing race conditions in the _lazyLoadPlaylist
bool _updatePlayerWasCalledWhileRunning = false;
/// Used for not skipping too many times if the [_subPlaylist] isn't done loading yet.
int _directionOfSkipTo = 0;
// PUBLIC VARS / GETTERS
/// Notifies the system of the current media item
final BehaviorSubject<MediaItem> mediaItem = BehaviorSubject();
/// Use this to get the live data of what the state is of the player
///
/// Don't perform actions like skip on this object because there is extra logic
/// attached to the functions in this class. For example, use [seekToNext] instead of [player.seekToNext]
AudioPlayer get player => _player;
/// The index of the next item to be played
int get nextIndex => _getRelativeIndex(1);
/// The index of the previous item in play order
int get previousIndex => _getRelativeIndex(-1);
/// The index of the current index.
int get currentIndex => _currentFullPlaylistIndex;
/// How many tracks next and previous are loaded into ConcatenatingAudioSource
/// * Ex. 3 would look like [`prev3`, `prev2`, `prev1`, `current`, `next1`, `next2`, `next3`]
final int preloadPaddingCount;
PlaylistManager({this.preloadPaddingCount = 3}) {
_attachAudioSourceToPlayer();
_listenForCurrentSongIndexChanges();
}
/// Used for loading the tracks. This will reset the current index and position.
setQueue(List<MediaItem> mediaItems) async {
_fullPlaylist.clear();
_fullPlaylist.addAll(mediaItems);
_subPlaylist.clear();
_subPlaylistItems.clear();
_lazyLoadPlaylist();
}
/// Use this instead of [player.seekToNext]
Future<void> seekToNext() async {
_directionOfSkipTo = 1;
await _player.seekToNext();
}
/// Use this instead of [player.seekToPrevious]
Future<void> seekToPrevious() async {
_directionOfSkipTo = -1;
await _player.seekToPrevious();
}
Future<void> _attachAudioSourceToPlayer() async {
try {
await _player.setAudioSource(_subPlaylist);
} catch (e) {
print("Error: $e");
}
}
void _listenForCurrentSongIndexChanges() {
int? previousIndex = _player.currentIndex;
_streamSubscriptions.add(_player.currentIndexStream.listen((index) {
_updateMediaItem();
_lazyLoadPlaylist();
final previousIndex_ = previousIndex;
previousIndex = index;
if (previousIndex_ == null || index == null) return;
final delta = index - previousIndex_;
if (delta.sign == _directionOfSkipTo.sign) {
_currentFullPlaylistIndex += delta;
_playlistFrontLoadingCount += delta;
}
}));
}
int _getRelativeIndex(int offset) {
return max(0, min(_currentFullPlaylistIndex + offset, _fullPlaylist.length - 1));
}
_updateMediaItem() {
if (_subPlaylistItems.isEmpty) return;
final playerIndex = _player.currentIndex;
if (playerIndex == null) return;
final newMediaItem = _subPlaylistItems[playerIndex];
mediaItem.add(newMediaItem);
}
_lazyLoadPlaylist() async {
// prevent race conditions
if (_updatePlayerRunning) {
_updatePlayerWasCalledWhileRunning = true;
return;
}
_updatePlayerRunning = true;
final currentIndex_ = _currentFullPlaylistIndex;
final playerIndex = _player.currentIndex ?? 0;
// Pad/pre-cache the ending of the playlist
final currentNextPadding = max(0, _subPlaylist.length - playerIndex);
var nextCountToAdd = preloadPaddingCount - currentNextPadding + 1;
if (nextCountToAdd > 0 && _fullPlaylist.isNotEmpty) {
for (final iNum in range(nextCountToAdd)) {
var mediaItem = _fullPlaylist[iNum.toInt() + currentIndex_];
await _subPlaylist.add(_createAudioSource(mediaItem));
_subPlaylistItems.add(mediaItem);
await Future.microtask(() {});
}
}
// Pad/pre-cache the beginning of the playlist
final currentPreviousPadding = _player.currentIndex ?? 0;
final previousCountToAdd = preloadPaddingCount - currentPreviousPadding;
if (previousCountToAdd > 0) {
for (int i = 1; i <= previousCountToAdd; i++) {
var index = currentIndex_ - currentPreviousPadding - _playlistFrontLoadingCount - i;
if (index < 0 || _fullPlaylist.length <= index) continue;
var mediaItem = _fullPlaylist[index];
final future = _subPlaylist.insert(0, _createAudioSource(mediaItem));
_subPlaylistItems.insert(0, mediaItem);
_playlistFrontLoadingCount++;
await future;
await Future.microtask(() {});
}
}
_updateMediaItem();
// prevent race conditions
_updatePlayerRunning = false;
if (_updatePlayerWasCalledWhileRunning) {
_updatePlayerWasCalledWhileRunning = false;
Future.microtask(() {
_lazyLoadPlaylist();
});
}
}
UriAudioSource _createAudioSource(MediaItem mediaItem) {
return AudioSource.uri(
Uri.parse(mediaItem.extras!['url'] as String),
tag: mediaItem,
);
}
Future<void> dispose() async {
for (final subscription in _streamSubscriptions) {
await subscription.cancel();
}
_streamSubscriptions.clear();
return _player.dispose();
}
}
class MyAudioHandler extends BaseAudioHandler {
final _playlistManager = PlaylistManager();
final List<StreamSubscription> _streamSubscriptions = [];
MyAudioHandler() {
_notifyAudioHandlerAboutPlaybackEvents();
_listenForDurationChanges();
_listenForCurrentSongIndexChanges();
}
void _notifyAudioHandlerAboutPlaybackEvents() {
_streamSubscriptions.add(_playlistManager.player.playbackEventStream.listen((PlaybackEvent event) {
final playing = _playlistManager.player.playing;
playbackState.add(playbackState.value.copyWith(
controls: [
MediaControl.skipToPrevious,
if (playing) MediaControl.pause else MediaControl.play,
MediaControl.stop,
MediaControl.skipToNext,
],
systemActions: const {
MediaAction.seek,
},
androidCompactActionIndices: const [0, 1, 3],
processingState: const {
ProcessingState.idle: AudioProcessingState.idle,
ProcessingState.loading: AudioProcessingState.loading,
ProcessingState.buffering: AudioProcessingState.buffering,
ProcessingState.ready: AudioProcessingState.ready,
ProcessingState.completed: AudioProcessingState.completed,
}[_playlistManager.player.processingState]!,
repeatMode: const {
LoopMode.off: AudioServiceRepeatMode.none,
LoopMode.one: AudioServiceRepeatMode.one,
LoopMode.all: AudioServiceRepeatMode.all,
}[_playlistManager.player.loopMode]!,
shuffleMode: (_playlistManager.player.shuffleModeEnabled) ? AudioServiceShuffleMode.all : AudioServiceShuffleMode.none,
playing: playing,
updatePosition: _playlistManager.player.position,
bufferedPosition: _playlistManager.player.bufferedPosition,
speed: _playlistManager.player.speed,
queueIndex: event.currentIndex,
));
}));
}
void _listenForDurationChanges() {
_streamSubscriptions.add(_playlistManager.player.durationStream.listen((duration) {
final oldMediaItem = _playlistManager.mediaItem.valueOrNull;
if (oldMediaItem == null) return;
final newMediaItem = oldMediaItem.copyWith(duration: duration);
mediaItem.add(newMediaItem);
}));
}
void _listenForCurrentSongIndexChanges() {
_streamSubscriptions.add(_playlistManager.mediaItem.listen((newMediaItem) {
final newMediaItemWithDuration = newMediaItem.copyWith(duration: _playlistManager.player.duration);
mediaItem.add(newMediaItemWithDuration);
}));
}
@override
Future<void> addQueueItems(List<MediaItem> mediaItems) async {
throw UnimplementedError("addQueueItems");
}
@override
Future<void> addQueueItem(MediaItem mediaItem) async {
throw UnimplementedError("addQueueItem");
}
@override
Future<void> updateQueue(List<MediaItem> mediaItems) async {
// notify system
final newQueue = mediaItems;
queue.add(newQueue);
_playlistManager.setQueue(mediaItems);
}
@override
Future<void> removeQueueItemAt(int index) async {
throw UnimplementedError("removeQueueItemAt");
}
@override
Future<void> play() => _playlistManager.player.play();
@override
Future<void> pause() => _playlistManager.player.pause();
@override
Future<void> seek(Duration position) => _playlistManager.player.seek(position);
@override
Future<void> skipToQueueItem(int index) async {
throw UnimplementedError("skipToQueueItem");
}
@override
Future<void> skipToNext() async {
_playlistManager.seekToNext();
}
@override
Future<void> skipToPrevious() async {
if (_playlistManager.player.position.inSeconds > 5 || _playlistManager.currentIndex == 0) {
return _playlistManager.player.seek(Duration.zero);
} else {
return _playlistManager.seekToPrevious();
}
}
@override
Future<void> setRepeatMode(AudioServiceRepeatMode repeatMode) async {
throw UnimplementedError("setRepeatMode");
}
@override
Future<void> setShuffleMode(AudioServiceShuffleMode shuffleMode) async {
throw UnimplementedError("setShuffleMode");
}
@override
Future<void> customAction(String name, [Map<String, dynamic>? extras]) async {
if (name == 'dispose') {
for (final subscription in _streamSubscriptions) {
await subscription.cancel();
}
_streamSubscriptions.clear();
await _playlistManager.dispose();
super.stop();
}
}
@override
Future<void> onTaskRemoved() async {
stop();
super.onTaskRemoved();
}
@override
Future<void> stop() async {
await _playlistManager.player.stop();
return super.stop();
}
} |
|
@ryanheise Is there a way to disable preloading/pre-caching entirely for |
|
No, that is in fact why you are working around it by only adding a limited number of items to the playlist. You need to have at least the current and the next item in the playlist in order to get gapless playback. If you want to get more sophisticated, you could listen to the current position, and when it reaches 15-20 seconds before reaching the end of the track, that's when you can add the next item to the playlist, so that it then starts buffering it just in time. |
Would it be a problem if I don't remove the previous item and just keep adding for as much as I want? |
|
If you keep adding without ever removing, you will eventually end up with a very large list that could again slow down communication between Dart and iOS. |
|
Can you please update branch( |
|
What specifically would you like me to update about it? |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
@ryanheise Is there any reason, known issue to not merge feature/treadmill branch? First impression, it works just fine for me. I am planning to publish it to our production if there is no known issue. Btw Is this branch changes affecting Android too? |
|
I think one more thing I would like to do is make this behaviour sensitive to the |
The only reason to want to specify the size of the treadmill is for short queue items where items are so short that having two items on the treadmill is not enough of a buffer to ensure that we will not hit the end of the buffer and stall playback. However, I think a boolean option such as
So unless there are any objections, I may just tie this to |
|
Update: I have made substantial changes to the Please let me know if this update breaks anything in your apps before I officially merge it into the next release. |
Working great, I am using it to load 114 files with large files such as 2 hours length with no issues |
Works awesome with 1000+ ConcatenatingAudioSources in playlist. |
|
Thanks for the testing results @MohammadElKhatib and @bekchan . Have either of you had any experience with how this interacts with shuffle mode, loop modes, or edge cases such as where there are only 1 or 0 items in the playlist? |
Well I believe it is working so well as I mentioned previously, first it did not work well but maybe it was caching the default branch and once I cleared cache and re-run the application, it is working awesome I am using it for Quran application where there is multiple reciters so basically 114 files with around 15 reciters and what i am doing is loading all the files and replace it once changing the reciters also I am listening to the sound progress updates in order to highlight text in parallel with sound. thanks for your efforts waiting the merge with main branch :-) |
|
Thanks, @MohammadElKhatib . Are you using either/both of shuffle and loop modes, or is that not a pertinent feature in your app? |
I used the loop options but I faced one issue I am not sure if this is the default behavior, what happened is that I used a button to toggle the loop and repeat. I have 3 options repeat one, repeat all and no repeat basically repeat one is repeating the current sound normally, repeat all is going through all the files and repeat from first index after reaching the last index while the no repeat is keep going forward till the last sound and stop there. while what I wanted is to only play one sound and stop so I am not sure if this is a bug or default behavior or I am missing something here. Note: I am also using background service and syncing text and progress alltogether. below screenshot of my app
|
If you have a |
Honestly, the current situation is acceptable in my app, I will keep following your updates. Thanks again for your great work. |
Shuffle and loop modes works great. Also tested reorderable playlist with +1000 audio, still good. |
|
I have just merged and published the |
and others
Which API doesn't behave as documented, and how does it misbehave?
Creating a ConcatenatingAudioSource with lots (~1000) of children takes a very long time (>20 seconds).
In my application, users can have playlists with an arbitrary number of items.
Minimal reproduction project
To Reproduce (i.e. user steps, not code)
Error messages
Expected behavior
I'd expect to be able to set the audio source and start playing the first source within a couple seconds, even with a large number of sources (since only the initial few sources are actually being fetched).
Screenshots
Desktop (please complete the following information):
Smartphone (please complete the following information):
Flutter SDK version
Additional context
NOTE added by @ryanheise :
Adding a large number of children is problematic at multiple levels due to the single-threaded model. For now, there are three approaches to workaround this issue:
ConcatenatingAudioSourcewith zero children, and then add the children in a loop that yields every now and then to let other coroutines execute on the same thread. This can be done for example by periodically callingawait Future.delayed(Duration.zero);.ConcatenatingAudioSourcewith an initial sublist of children small enough to load without issue, and then write your own application logic to lazily add more children as needed (just-in-time). If you want to create the illusion that the list of children is complete from the beginning, you will need to write your UI around some other more complete list of metadata, separate fromConcatenatingAudioSource's own internal list of items.feature/treadmill(now merged) that implements the logic of (2) above but on the iOS native side (Android already has this implemented on the native side whenuseLazyPreparationistrue). This will stop iOS from trying to load every item in the playlist, and instead only load items that are nearer to the front of the queue. If you test this branch, please share below whether you found it stable enough to release.The text was updated successfully, but these errors were encountered: