Pass typed data between isolates using transferrables.

[ilyaklyaus]

In our project we have to decode maps from strings which size is pretty big (up to 20mb) and JSON.decode is slow in this case. Passing such strings to isolate to decode and get it back is even slower because the data is copied.
Could you please consider the way to use transferrables to pass data between isolates. Something like that: https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Using_web_workers#Passing_data_by_transferring_ownership_(transferable_objects)

Thanks

[zoechi]

Shouldn't this work to some degree if spawn is used (not with spawnUri).
What are you using?

[DisDis]

@zoechi We use the package https://pub.dartlang.org/packages/isolate

[lrhn]

Using spawn can avoid some code duplication, but it still has to transport the data from one heap to another. I don't think we have transferable data that cooperates with isolates.

[DisDis]

Make a workaround through native extension is expensive.
Without this, there is no way to make heavy applications on DartVM.

[mraleph]

benchmark from @ilyaklyaus

import 'dart:math';
import 'dart:convert';

import 'package:isolate/isolate.dart';

main(List<String> args) async {
  IsolateRunner r = await IsolateRunner.spawn();

  String str = generateString();
  print("${str.length} bytes");

  //decode
  Stopwatch s = new Stopwatch()..start();
  decode(str);
  print("Decoded in ${s.elapsedMilliseconds}ms");

  //decode in isolate
  s = new Stopwatch()..start();
  Map map = await r.run(decode, str);
  print("Decoded in isolate in ${s.elapsedMilliseconds}ms");

  //pass map to isolate and get back
  s = new Stopwatch()..start();
  await r.run(passObj, map);
  print("Map obj passed in ${s.elapsedMilliseconds}ms");

  //pass map to isolate and get back
  s = new Stopwatch()..start();
  await r.run(passMap, map);
  print("Map passed in ${s.elapsedMilliseconds}ms");

  //pass str to isolate and get back
  s = new Stopwatch()..start();
  await r.run(passObj, str);
  print("String obj passed in ${s.elapsedMilliseconds}ms");

  //pass empty map to isolate and get back
  s = new Stopwatch()..start();
  await r.run(passObj, {});
  print("Empty map obj passed in ${s.elapsedMilliseconds}ms");

  //pass empty map to isolate and get back
  s = new Stopwatch()..start();
  await r.run(passMap, {});
  print("EMpty map passed in ${s.elapsedMilliseconds}ms");
}

Map decode(String str) {
  return JSON.decode(str);
}

dynamic passObj(dynamic map) {
  return map;
}

Map passMap(Map map) {
  return map;
}

String generateString() {
  Random rnd = new Random();

  Map map = new Map();
  for (int i=0; i<500000; i++) {
    map[i.toString()] = [rnd.nextInt(1000), _randomString(10)];
  }

  return JSON.encode(map);
}

String _randomString(int length) {
  var rand = new Random();
  var codeUnits = new List.generate(
      length,
          (index){
        return rand.nextInt(33)+89;
      }
  );

  return new String.fromCharCodes(codeUnits);
}

[rmacnak-google]

@aam