Support React Native

[ianhe8x]

in RN environment, there's no global WebAssembly.
Can we have a polyfill for it?
https://www.npmjs.com/package/webassembly looks promising.

[ianhe8x]

here is a working polyfill.

if (!global.WebAssembly) {
    global.WebAssembly = require('webassemblyjs');
}

I'm not sure if we need to include it in here, or just update README and let the user include it in their codebase.

[jacogr]

Ooooh... nice! I have been playing with asm.js to have RN support (Polkawallet also needs it), but that ends up with a sometimes-working-sometimes-not-working 13MB bundle.

So generally would add the polyfill in util-crypto.

Would require some testing by somebody with a RN environment though (manually adding it), but you could be right docs may be the only thing needed?

cc @jiangfuyao

[0xthreebody]

@jacogr Today work together?

[jacogr]

Deep inside schnorrkel :( If the interpreter has issues dealing with i64, not sure how to get around it.

Made sure in the impl to actually only have u32 to the external world, since JS doesn’t have real 64-bit integers.

However, in this case not sure what can be done.

[ianhe8x]

i tried all the old versions, all failed. That's really weird, it did once work.

[jacogr]

The Centrality mobile app does currently work (for me) - could we maybe see which version of @polkadot/wasm-crypto & webassemblyjs that uses? (As I recall, the only versions bumps lately has been due to deps, not actual Rust code changes - but that may help to try and track it?)

0.9 -> 0.10 - only tests added with schnorrkel bumped from 0.1.0 -> 0.1.1 (aligning with substrate)
0.10 -> 0.11 - js deps bumped, no code changes

[anakornk]

Any updates on this?

[qalqi]

+1

[qalqi]

When crypto is ported to tradle/react-native-crypto and react-native-randombytes for randombytes..
Its working to certain extent
bip39Generate is crashing in react native
mnemonicGenerate, mnemonicToMiniSecret is working fine.
schnorrkelKeypairFromSeed is working only in debug mode in react native.

Attaching screenshot for reference

[jacogr]

It would be good if contributions to getting it working is actually done upstream, i.e. here, as well, instead of trying to manage 2 interfaces.

The original reason asm.js was not enabled is because all of the tests are not passing consistently, even worse it was quite inconsistent between runs - which is a worse problem to have. In the linked #23 above re-enabled asm.js, but it fails on 14 or the 24 tests. (So something is really weird)

Either way, if that can get to a point of passing, will integrate this further upstream in @polkadot/util-crypto as well, so there is a fallback available, transparently. (Or rather, as transparent as possible - we may need to do a different index import for non-WASM envs)

EDIT: Actually, it seems like that for the last run, the test did pass for asm.js - https://travis-ci.com/polkadot-js/wasm/jobs/217138856#L1021

[ianhe8x]

What we did in our fork was a work around and I saw there was a asmjs output but for some reason was disabled, so I didn't contribute back(and be busy with something else). Btw to minify the js output or loaded it in jest requires large memory, which is not very friendly

[qalqi]

I need schnorrkelKeypairFromSeed in react native.
Tried @polkadot-js/keyring and @plugnet/keyring

None of these libraries are working
@plugnet/wasm-crypto
@plugnet/wasm-crypto-js
@polkadot-js/wasm-crypto

Somehow it only works when debug mode is on i.e dev=true.
In release mode, all libraries fail in all permutation and combinations.

[qalqi]

KeyPairType - ed25519 is working
KeyPairType - sr25519 is not working

[jacogr]

Is that on 0.13.0-beta.1?

You need to pass an env variable, CRYPTO_ASM=1 (i.e. like here https://github.com/polkadot-js/wasm/blob/master/scripts/test-package.sh#L14) to the build and make sure it is exposes to the build process.

(Not sure on metro, know how to expose the process.env in webpack, i.e. like here https://github.com/polkadot-js/apps/blob/master/packages/apps/webpack.config.js#L173 - so there CRYPTO_ASM would need to be explicitly passed)

The tests pass running asm.js (for those tests WebAssembly is null-ed)

The env var is not nice, but the best initial solution to make sure that the normal webpack builds (anything non-RN) don't pull in an extra 7MB that is the asm.js output

EDIT: Actually with a webpack config, you don't need to pass the variable on the command-line, doing this should be sufficient -

plugins: [
      new webpack.DefinePlugin({
        'process.env': {
          CRYPTO_ASM: '1' // any truth-y value will do
        }
      }),
      ...

[qalqi]

Passed CRYPTO_ASM=1
Installed NPM_CONFIG_CRYPTO_ASM=1 npm install @polkadot/[email protected]
Also, started CRYPTO_ASM=1 react-native start

import { waitReady } from '@polkadot/wasm-crypto';

  const isReady = await waitReady();
   console.log('waitReady', isReady);

This hangs metro build process at 99% which is very unusual.

[jacogr]

This may be useful? https://medium.com/@skiptomyhue/setting-up-environment-variables-for-react-native-builds-b66a2576a218

EDIT: If there are any other ideas to only include the asm.js stuff for RN, would like to hear them.

[qalqi]

I can confirm about env flags as I have followed those steps and consoled out CRYPTO_ASM=1.
Version tested @polkadot/[email protected] is
Hangs metro build process at 99%.

Only sr25519 has issues. ed25519 is working fine

Could node's crypto replacement - react-native-crypto and react-native-randombytes be causing any issue?

[jacogr]

It could very well be. In sr25519 it is used a bit more, i.e. even the signatures are non-deterministic. So I'm guessing first testing some other ones with random (i.e. bip39), should show if that could be the issue.

[qalqi]

On a certain scenario, sr25519 is working in react-native.
i.e when react-native remote debugger is on. Here is info related to it https://stackoverflow.com/a/52604792/2560946

[jacogr]

Ok, so it is most probably the config (babel) that is too agressive and adds something (or doesn't add something), that is actually needed.

Do you have steps (ELI5) to create a simple reproduction project or have a trimmed-down version available to play around with?

[qalqi]

I think, its do with https://developer.apple.com/documentation/javascriptcore
react-native uses javascriptcore normally for all executions and v8 engine incase of remote debug on.

Can you generate a wasm.js specific to javascriptcore?
https://www.reactnative.guide/3-react-native-internals/3.1-react-native-internals.html#architecture

[qalqi]

The issue is not with debug mode, shim, CRYPTO_ASM..

Rather about JavaScriptCore
The JS Virtual Machine that runs all our JavaScript code. On iOS/Android simulators and devices React Native uses JavaScriptCore, which is the JavaScript engine that powers Safari. JavaScriptCore is an open source JavaScript engine originally built for WebKit. In case of iOS, React Native uses the JavaScriptCore provided by the iOS platform. It was first introduced in iOS 7 along with OS X Mavericks.
https://developer.apple.com/reference/javascriptcore.

In case of Android, React Native bundles the JavaScriptCore along with the application. This increases the app size. Hence the Hello World application of RN would take around 3 to 4 megabytes for Android.

In case of Chrome debugging mode, the JavaScript code runs within Chrome itself (instead of the JavaScriptCore on the device) and communicates with native code via WebSocket. Here, it will use the V8 engine. This allows us to see a lot of information on the Chrome debugging tools like network requests, console logs, etc. 😎

[qalqi]

This seems to be an alternative approach https://github.com/paritytech/parity-signer
using native modules with rust for keyring methods in react native

[jacogr]

Yes, the Parity Signer is being updated at the moment for Substrate support, so the Rust bindings are actually being turned into real mapping to UI there. (It certainly is a better bridge that relying on Rust -> WASM -> JS)

There was another packaging mismatch, so 0.13.0-beta.2 should finally include all the required files and at least technically not be broken out of the box. (It has been pulled into common, api and made it's way into polkadot-js/apps and working fine there). This may improve your experiences above, but with my previous experiences on Android (we have multiple polyfills for Android support already), it may not.

[jacogr]

And 0.13.0-beta.3 doesn't add the asm.js capability to all, however via the package.json "react-native" entry (hopefully) enables it for RN. (This gets around adverse effects that the large asm.js file is having on browser envs, and removes the need to pass any ENV flags as well)

That would be good as a run-though. (Both versions, if the latest doesn't quite work due to a beta-testing mishap)

[qalqi]

In 0.13.0-beta.0 , 0.13.0-beta.1, 0.13.0-beta.2 on succesfull installs(with CRYPTO_ASM, react-native keyword in pacakage.json),

It gets stuck at 99%

[qalqi]

There is an experimental way to consume wasm files for react-native mentioned here. https://github.com/ExodusMovement/react-native-wasm

Can it help?

[jacogr]

After 0.13.0-beta.4 (non-functionality change, packaging only, i.e. bundlers should have an easier time dealing with the asm.js outputs now), not sure I can move thing forward after this as of right now. (It is worth a try for the 99% problem, I would just leave it running for quite some time as well - on the desktop with asm.js it takes >5m to build a bundle)

So, I understand the JSC problem quite well, i.e. this was added specifically for RN Android, https://github.com/polkadot-js/wasm/blob/master/packages/wasm-crypto/src/js/crypto-polyfill.js as well as polyfills in the common repo, ie. around Text{Decoder, Encoder}, Uint8Array fills, Array fills and some I forgot about by now.

Quite happy to extend this in any area (assuming we can get there again and can see exactly what is breaking), and really happy to look at anything PR-related that can improve things. As it stands, this is a target I deeply care about, but have no way of actually getting running or testing by myself. So anything related to "trying support" is in the dark.

TL;DR Please try, make coffee, make another coffee, go for a stroll and then check back. With explicit "it breaks here" info, can look at addressing, and always happy to look at specific approaches that works across envs.

[qalqi]

Will share a react-native setup in a while.

Here is what I got waiting for more than 15 minutes

BUNDLE  [ios, dev] ./index.js ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░ 99.2% (1999/200
7)::1 - - [19/Jul/2019:15:51:31 +0000] "GET /index.bundle?platform=ios&dev=true&minify=false HTTP/1.1" 200 - "-" "Social/13 CFtransform[stdout]: 
transform[stdout]: <--- Last few GCs --->
transform[stdout]: 
transform[stdout]: [34653:0x103000000]   834059 ms: Mark-sweep 1404.0 (1754.3) -> 1403.9 (1754.3) MB, 4778.5 / 0.0 ms  allocation failure GC in old space requested
transform[stdout]: [34653:0x103000000]   838523 ms: Mark-sweep 1403.9 (1754.3) -> 1403.9 (1690.8) MB, 4463.1 / 0.0 ms  last resort GC in old space requested
transform[stdout]: [34653:0x103000000]   842988 ms: Mark-sweep 1403.9 (1690.8) -> 1403.9 (1661.3) MB, 4464.1 / 0.0 ms  last resort GC in old space requested
transform[stdout]: 
transform[stdout]: 
transform[stdout]: <--- JS stacktrace --->
transform[stdout]: 
transform[stdout]: ==== JS stack trace =========================================
transform[stdout]: 
transform[stdout]: Security context: 0x888d8c25ec1 <JSObject>
transform[stdout]:     1: set [native collection.js:~149] [pc=0x2ef062274187](this=0x888e6caa9a1 <Map map = 0x888b8c048d9>,o=0x88806f70671 <SourceLocation map = 0x888e40e3471>,value=0x8887b63d3f9 <Object map = 0x888e40e7619>)
transform[stdout]:     2: set(aka mapCacheSet) [/Users/pavangarre/Cloud/clients/socialnetwork/social/node_modules/lodash/_mapCacheSet.js:~13] [pc=0x2ef062248750](this=0x888e6caa741 <MapCache map = 0x888e40e3261>...
transform[stdout]: 
transform[stderr]: FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
transform[stderr]:  1: node::Abort() [/Users/pavangarre/.nvm/versions/node/v8.9.1/bin/node]
transform[stderr]:  2: node::FatalException(v8::Isolate*, v8::Local<v8::Value>, v8::Local<v8::Message>) [/Users/pavangarre/.nvm/versions/node/v8.9.1/bin/node]
transform[stderr]:  3: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/Users/pavangarre/.nvm/versions/node/v8.9.1/bin/node]
transform[stderr]:  4: v8::internal::Factory::NewFixedArray(int, v8::internal::PretenureFlag) [/Users/pavangarre/.nvm/versions/node/v8.9.1/bin/node]
transform[stderr]:  5: v8::internal::OrderedHashTable<v8::internal::OrderedHashMap, 2>::Rehash(v8::internal::Handle<v8::internal::OrderedHashMap>, int) [/Users/pavangarre/.nvm/versions/node/v8.9.1/bin/node]
transform[stderr]:  6: v8::internal::Runtime_MapGrow(int, v8::internal::Object**, v8::internal::Isolate*) [/Users/pavangarre/.nvm/versions/node/v8.9.1/bin/node]
transform[stderr]:  7: 0x2ef06210463d
 BUNDLE  [ios, dev] ./index.js ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░ 99.2% (1999/2000)

[jacogr]

A reproducing setup would be excellent.

Ok, cool, so it is failing the packaging. Ian had this first, but basically try passing the following ENV flags to increase the memory - the asm.js file is huge, almost 7MB.

NODE_OPTIONS=--max_old_space_size=8192

Then... more coffee and strolling around... :)

[qalqi]

Here is a clean setup with crypto, fs, randombytes RN polyfills for reproducing this issue.
https://github.com/qalqi/AwesomeProject

First, run the setup as per Readme and run in iOS.
You should see Welcome to React Screen new project view in iOS simulator.

Then, go to App.js and remove commented lines under
//Comment out this <----------

Error Scenario:
Go to iOS simulator and Cmd + R to see loading 99.x% in metro bundler terminal

Success Scenario:
If, everything works as expected as bip39Generated phrase should be visible on the screen
next to --->

Every time you update polkadot-js/wasm library version, run npm run clean:perfect"

p.s I don't drink coffee

[jacogr]

• Updated XCode
• Installed cocoapods
• using 0.13.0-beta.4
• running NODE_OPTIONS=--max_old_space_size=8192 npm start for the bundler
• npm run ios for the emulator
• made the following code changes to https://github.com/qalqi/AwesomeProject -

import { bip39Generate, waitReady } from '@polkadot/wasm-crypto';

const App = () => {
  const [phrase, setPhrase] = useState();

  useEffect(() => {
   // crypto needs to be ready
    waitReady()
      .then(() => {
       // follow index.d.ts - pass in the number of words
        setPhrase(bip39Generate(12));
      })
      .catch((error) => setPhrase(`error: ${error.message}`));;
  }, []);

  return (
    <Fragment>
      <StatusBar barStyle="dark-content" />
      <SafeAreaView>
        <ScrollView
          contentInsetAdjustmentBehavior="automatic"
          style={styles.scrollView}>
          <Header />

          <View style={styles.body}>
            <View style={styles.sectionContainer}>
              <Text style={styles.sectionTitle}>phrase</Text>
              <Text style={styles.sectionDescription}>
                {phrase || '--none--'}
              </Text>
...

[qalqi]

running NODE_OPTIONS=--max_old_space_size=8192 npm start for the bundler. No more stuck at 99%

Have you got remote debugger on? Cmd + D -> Debug JS Remotely? ( the JavaScript code runs within Chrome itself (instead of the JavaScriptCore on the device) and communicates with native code via WebSocket. Here, it will use the V8 engine. )

Without debugger on. The JavascriptCore mode (On iOS/Android simulators and devices React Native uses JavaScriptCore)

It should work in Without debugger on. i.e with JavaScriptCore currently, its not

[qalqi]

There seems to be a crate for that https://crates.io/crates/javascriptcore
https://lib.rs/crates/javascriptcore-rs

[jacogr]

No remote debugging, I wouldn't know where to start with that at all, not a RN expert. I only followed the instructions in the README. So 2 commands only -

• tab 1: NODE_OPTIONS=--max_old_space_size=8192 npm start
• tab 2: npm run ios

[mzxyz]

@jacogr Thanks for your efficient and awesome work. Have integrated the latest version to polkawallet. Both dev and release version work perfect.

[jacogr]

I'll do the following in the next 30-odd minutes as CI completes all it's tasks -

• 0.13.1 of @polkadot/wasm-crypto
• 0.94.1 of @polkadot/{util, util-crypto, keyring}
• will push a sample RN repo that generates sr25519 pairs

[mzxyz]

@jacogr That's great. BTW, I also create a sample RN code to provide details that have to use crypto and other browser or node specific libs in RN. I prefer using babel-rewrite other than rn-nodeify which is more elegant and less injection for a project.

If you don't mind, I can provide the sample repo.

[jacogr]

Sample here for @polkadot/keyring 0.94.1 (underlying that uses @polkadot/wasm-crypto 0.13.1) with sr25519 generated addresses -

https://github.com/jacogr/rn-wasm-crypto

[jacogr]

@huanday Please do, it can certainly help others. I am clueless when it comes to RN, so no idea around best practices. It is not that I can't learn, it is just that I have more than enough work and pet projects as it is.

[qalqi]

rn-nodeify has aggressive polyfills for react-native. The better alternative is node-libs-react-native and metro.config.js combo with manually adding libraries suiting the needs. I shared the template with both so that one doesn't have worry polyfills at all.

[jacogr]

I would love to get to a setup where I can get rid of these things with a proper RN way of doing things - https://github.com/polkadot-js/common/blob/master/packages/util/src/polyfill/fill.ts

That whole directory was (mostly) driven by Android issues on polkawallet. (Array, Uint8Array, TextEncoder/TextDecoder, setPrototypeOf, etc)

[jacogr]

Closing this based on #19 (comment)

• @polkadot/wasm-crypto 0.13.1 has an asm.js fallback (RN-only)
• @polkadot/util-crypto 0.94.1 uses the above package
• There (seems to be) a working sr25519 sample of @polkadot/keyring 0.94.1 (uses the above internally) at https://github.com/jacogr/rn-wasm-crypto

If anything else does pop up -

• make a PR with fixes to the relevant repos
• log an issue for somebody to take a look at

[polkadot-js-bot]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue if you think you have a related problem or query.