Lots of stuff

bower.json

@@ -12,16 +12,22 @@
     "output"
   ],
   "dependencies": {
+    "purescript-node-buffer": "^5.0.0",
     "purescript-prelude": "^4.0.1",
-    "purescript-record": "^1.0.0",
+    "purescript-record": "^2.0.0",
     "purescript-functions": "^4.0.0",
     "purescript-node-http": "^5.0.0",
-    "purescript-aff-promise": "^2.0.0",
-    "purescript-node-buffer": "^5.0.0"
+    "purescript-aff-promise": "^2.1.0",
+    "purescript-web-dom": "^3.0.0",
+    "purescript-unsafe-coerce": "^4.0.0",
+    "purescript-newtype": "^3.0.0"
   },
   "devDependencies": {
-    "purescript-milkis": "^6.0.1",
-    "purescript-test-unit": "^14.0.0",
-    "purescript-node-process": "^6.0.0"
+    "purescript-milkis": "^7.0.0",
+    "purescript-test-unit": "^15.0.0",
+    "purescript-node-process": "^7.0.0",
+    "purescript-transformers": "^4.0.0",
+    "purescript-foreign": "^5.0.0",
+    "purescript-web-html": "^2.0.0"
   }
 }

docs/unsafe.md

@@ -0,0 +1,97 @@
+# *ViaBrowserify functions
+
+`queryEvalViaBrowserify`, `queryEvalManyViaBrowserify` and `evalViaBrowserify` are direct bindings to [`.$eval`](https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#frameevalselector-pagefunction-args-1), [`.$$eval`](https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#frameevalselector-pagefunction-args) and [`.evaluate`](https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#pageevaluatepagefunction-args) methods, respectively.
+
+In JS, you are supposed to write something like this:
+
+```javascript
+await page.$$eval(
+  '.my-class',
+  els => els.map(el => el.tagName)
+);
+```
+
+Function passed as the second argument will be executed within the browser context. That is, it is impossible to reference variables from outside of the browser's global scope, e.g. this code will throw:
+
+```javascript
+var local = true;
+await page.$$eval(
+  '.my-class',
+  () => {
+    if (typeof local == 'undefined') {
+      throw new Error();
+    }
+  }
+);
+```
+
+This is the reason why PS bindings to functions like `$$eval` can't be implemented in a straightforward way: PureScript runtime itself will be erased during context switch.
+
+In this library these limitations are bypassed by inspecting given function's code and inserting all the necessary runtime dependencies using `browserify` before actually passing the function as a callback.
+
+`Unsafe.js` defines `_jsReflect :: forall a. a -> Effect (Promise String)`, which is not exported. This function accepts a purescript value and returns its bundled equivalent which can be injected into the browser. Let's see how it is possible.
+
+Suppose we want to get a tag name of the first element matching a given selector.
+
+It can be done by passing `\elem -> injectPure $ tagName elem` to `unsafeQueryEval` (where `injectPure` is from `Toppoki.Inject`).
+
+`_jsReflect` will first retrieve a runtime representation of the given function using `Function.prototype.toString`:
+
+```javascript
+function (elem) {
+  return Toppoki_Inject.inject(
+    Control_Applicative.pure(Effect_Aff.applicativeAff)(Web_DOM_Element.tagName(elem))
+  );
+}
+```
+
+Then it will extract free variables' names using  `extractDefinitions` from `Unsafe.js`:
+
+```javascript
+[ 'Toppoki_Inject',
+  'Control_Applicative',
+  'Effect_Aff',
+  'Web_DOM_Element' ]
+```
+
+(`extractDefinitions` uses `TreeWalker` from `uglify-js@2` to traverse the AST).
+
+After that, it will trivially map these names to subdirectories of `./output/`, generate some wrapping code and feed it to `browserify`.
+
+Finally, browserify output is used to constuct a new function that can be safely converted to a string and passed to the browser runtime.
+
+## Cautions
+
+Of course this method is not perfect since it relies on unsafe assumptions about the format in which `purs` outputs JS code. It may break someday, possibly without any chances to get it working again.
+
+## Limitations
+
+1. `browserify`ing is relatively slow.
+
+2. Accessing purescript values defined locally is still impossible and will result in a runtime error (this perfectly matches JS behavior). Only using what is directly imported from other modules is allowed.
+
+3. It is impossible to use `const` to hide unused argument in `unsafe*` callbacks.
+
+```purescript
+-- good
+(\_ -> injectEffect
+  (window >>= document >>= title))
+
+-- bad
+(const $ injectEffect
+  (window >>= document >>= title))
+```
+
+It is clear why the latter does not work - because `Function.prototype.toString` returns the "inner part" of the `const` definition instead of the needed function. The following code may help to understand what's going on:
+
+```
+> var f = function (x) { return function(y) { return x; } } // `const` equivalent
+> f.toString()
+'function (x) { return function(y) { return x; } }'
+> f(function(z) { return z; }).toString()
+'function(y) { return x; }'
+```
+
+There is a built-in detection of incorrect `const` usage (see how `extractDefinitions` is defined). When there are free variables which appear to be function parameters erased during evaluation, the user will see an error message:
+
+> Toppokki internal error: are you trying to use point-free style in a callback function? (see docs/unsafe.md)

package.json

@@ -12,6 +12,10 @@
   "author": "",
   "license": "MIT",
   "devDependencies": {
-    "puppeteer": "^1.5.0"
+    "puppeteer": "^1.13.0"
+  },
+  "dependencies": {
+    "browserify": "^16.3.0",
+    "uglify-js": "^2.8.29"
   }
 }

src/Toppoki/Inject.purs

@@ -0,0 +1,26 @@
+module Toppokki.Inject
+       ( InjectedAff
+       , inject
+       , injectEffect
+       , injectPure
+       )
+where
+
+import Control.Promise (fromAff, Promise)
+import Effect (Effect)
+import Effect.Aff (Aff)
+import Effect.Class (liftEffect)
+import Effect.Unsafe (unsafePerformEffect)
+import Prelude (pure, (>>>), Unit)
+
+-- | InjectedAff is a `Promise` executing in the browser.
+newtype InjectedAff r = InjectedAff (Unit -> Promise r)
+
+inject :: forall r. Aff r -> InjectedAff r
+inject aff = InjectedAff (\_ -> unsafePerformEffect (fromAff aff))
+
+injectEffect :: forall r. Effect r -> InjectedAff r
+injectEffect = liftEffect >>> inject
+
+injectPure :: forall r. r -> InjectedAff r
+injectPure = pure >>> inject

src/Toppoki/Unsafe.js

@@ -0,0 +1,126 @@
+/* global require exports */
+var path = require("path");
+var browserify = null;
+var UglifyJS = null;
+
+try {
+  browserify = require('browserify');
+  UglifyJS = require("uglify-js");
+} catch (e) {
+
+}
+
+var Readable = require("stream").Readable;
+
+// Extract all variable names starting with capital letters.
+// Throw an error if there are free variables.
+function extractDefinitions (code) {
+  var globals = new Set();
+
+  function visitor(node, descend) {
+    if (node instanceof UglifyJS.AST_Toplevel) {
+      node.figure_out_scope();
+    }
+
+    if (node instanceof UglifyJS.AST_Symbol) {
+      if (/[A-Z]/.test(node.name[0])) {
+        if (!globals.has(node.name)) {
+          globals.add(node.name);
+        }
+      } else {
+        if (node.undeclared()) {
+          throw new Error("Toppokki internal error: are you trying to use " +
+                          "point-free style in a callback function?  (see " +
+                          "docs/unsafe.md)");
+        }
+      }
+    }
+  }
+
+  var walker = new UglifyJS.TreeWalker(visitor);
+  UglifyJS.parse('(' + code + ')').walk(walker);
+  return Array.from(globals);
+}
+
+function wrapTopLevel(code) {
+  return new Function(
+    'arg',
+
+    'var TOPLEVEL_TOPPOKI_FUNCTION;\n' + code +
+    ';\nreturn TOPLEVEL_TOPPOKI_FUNCTION(arg)();'
+  );
+}
+
+// `wrapTopLevelString`, unlike `wrapTopLevel`, returns a concrete value,
+// rather than a function.
+function wrapTopLevelString(code) {
+  // The function is supposed to accept a `unit` value.
+  //
+  // ```
+  // newtype InjectedAff r = InjectedAff (Unit -> Promise r)
+  // ```
+  //
+  // But since the newtype internals are not exposed, and the value is not used,
+  // it is safe to pass nothing.
+
+  return '(function(){\n  var TOPLEVEL_TOPPOKI_FUNCTION;\n' +
+    code + ';\n  return TOPLEVEL_TOPPOKI_FUNCTION()();\n})()';
+}
+
+exports._jsReflect = function(func) {
+  if (browserify === null || UglifyJS === null) {
+    throw new Error("Toppokki internal error: to use `unsafe*` functions, run `npm install uglify-js@2 browserify`");
+  }
+
+  return function(){
+    return new Promise(function (resolve, reject) {
+      var code = func.toString();
+      var globals = extractDefinitions(code);
+
+      var readable = new Readable();
+      readable.push('(function () {\n  ');
+      globals.forEach(function(mod) {
+        readable.push(
+          'var ' + mod + ' = require("./' +
+            mod.replace(/_/g, '.') + '");\n'
+        );
+      });
+
+      readable.push('\n  TOPLEVEL_TOPPOKI_FUNCTION = ');
+      readable.push(code);
+      readable.push('\n})()');
+      readable.push(null);
+      var b = browserify(readable, {
+        basedir: path.join(__dirname, '..'),
+        ignoreMissing: true,
+        detectGlobals: false,
+        // Not required - we do not use packages
+        browserField: false,
+      });
+      var str = b.bundle(function (err, buff) {
+        if (err !== null) {
+          reject(err);
+        }
+        resolve(buff.toString());
+      });
+    });
+  };
+};
+
+exports._queryEval = function(selector, code, queryable) {
+  return function() {
+    return queryable.$eval(selector, wrapTopLevel(code));
+  };
+};
+
+exports._queryEvalMany = function(selector, code, queryable) {
+  return function() {
+    return queryable.$$eval(selector, wrapTopLevel(code));
+  };
+};
+
+exports._evaluate = function(code, ctx) {
+  return function() {
+    return ctx.evaluate(wrapTopLevelString(code));
+  };
+};

src/Toppoki/Unsafe.purs

@@ -0,0 +1,87 @@
+module Toppokki.Unsafe
+       ( class Queryable
+       , class Evaluate
+       , evalViaBrowserify
+       , queryEvalViaBrowserify
+       , queryEvalManyViaBrowserify
+       , Page
+       , Frame
+       , ElementHandle
+       )
+where
+
+import Toppokki.Inject (InjectedAff)
+import Data.Function.Uncurried as FU
+import Effect (Effect)
+import Effect.Aff (Aff)
+import Control.Promise (Promise)
+import Control.Promise as Promise
+import Web.DOM.ParentNode (QuerySelector)
+import Prelude
+import Web.DOM.Element (Element)
+import Prim.TypeError (class Fail, Text)
+import Foreign (Foreign, readNull, unsafeFromForeign)
+
+foreign import data Page :: Type
+foreign import data Frame :: Type
+foreign import data ElementHandle :: Type
+
+-- | Values which can be queried by selectors.
+class Queryable el
+
+instance queryablePage :: Queryable Page
+else instance queryableFrame :: Queryable Frame
+else instance queryableElementHandle :: Queryable ElementHandle
+else instance queryableClose :: (Fail (Text "Queryable class is closed")) => Queryable a
+
+class Evaluate a
+
+instance evaluatePage :: Evaluate Page
+else instance evaluateFrame :: Evaluate Frame
+else instance evaluateClose :: (Fail (Text "Evaluate class is closed")) => Evaluate a
+
+-- | Execute a function in the browser context. The function will be bundled using `browserify`.
+evalViaBrowserify
+  :: forall ctx r
+  .  Evaluate ctx
+  => ctx
+  -> (Unit -> InjectedAff r)
+  -> Aff Foreign
+evalViaBrowserify ctx callback = do
+  jsCode <- Promise.toAffE (_jsReflect callback)
+  Promise.toAffE (FU.runFn2 _evaluate jsCode ctx)
+
+-- | Query the element using `.$eval(selector, pageFunction)`.
+-- |
+-- | If there's no element matching `selector`, the method throws an error.
+-- |
+-- | `pageFunction` will be bundled using `browserify` and executed in the browser context.
+queryEvalViaBrowserify
+  :: forall el r
+  .  Queryable el
+  => QuerySelector
+  -> (Element -> InjectedAff r)
+  -> el
+  -> Aff Foreign
+queryEvalViaBrowserify qs callback el = do
+  jsCode <- Promise.toAffE (_jsReflect callback)
+  Promise.toAffE (FU.runFn3 _queryEval qs jsCode el)
+
+-- | Query the element using `.$$eval(selector, pageFunction)`.
+-- |
+-- | `pageFunction` will be bundled using `browserify` and executed in the browser context.
+queryEvalManyViaBrowserify
+  :: forall el r
+  .  Queryable el
+  => QuerySelector
+  -> (Array Element -> InjectedAff r)
+  -> el
+  -> Aff Foreign
+queryEvalManyViaBrowserify qs callback el = do
+  jsCode <- Promise.toAffE (_jsReflect callback)
+  Promise.toAffE (FU.runFn3 _queryEvalMany qs jsCode el)
+
+foreign import _jsReflect :: forall a. a -> Effect (Promise String)
+foreign import _queryEval :: forall el. FU.Fn3 QuerySelector String el (Effect (Promise Foreign))
+foreign import _queryEvalMany :: forall el. FU.Fn3 QuerySelector String el (Effect (Promise Foreign))
+foreign import _evaluate :: forall ctx. FU.Fn2 String ctx (Effect (Promise Foreign))