update library to support multiple hands

Signed-off-by: Erick Wendel <[email protected]>

README.md

@@ -1,8 +1,8 @@
 # Fingerpose
 
-Finger gesture classifier for hand landmarks detected by [MediaPipe Handpose](https://github.com/tensorflow/tfjs-models/tree/master/handpose). It detects gestures like "Victory" ✌️or "Thumbs Up" 👍inside a source image or video stream. You can define additional hand gestures using simple [gesture descriptions](https://github.com/andypotato/fingerpose/tree/master/src/gestures). 
+Finger gesture classifier for multiple hand landmarks detected by [MediaPipe Handpose Detection](https://github.com/tensorflow/tfjs-models/tree/master/hand-pose-detection). It detects gestures like "Victory" ✌️ or "Thumbs Up" 👍  from both individual hands inside a source image or video stream. You can define additional hand gestures using simple [gesture descriptions](./src/gestures). 
 
-!["Thumbs up" and "Victory" gestures detected](https://raw.githubusercontent.com/andypotato/fingerpose/master/assets/fingers-lq.gif)
+!["Thumbs up" and "Victory" gestures detected](./assets/finger-lq1.gif)
 
 # Table of contents
 
@@ -19,49 +19,51 @@ Finger gesture classifier for hand landmarks detected by [MediaPipe Handpose](ht
 
 Gesture detection works in three steps:
 
- 1. Detect the hand landmarks inside the video picture
+ 1. Detect the hands' landmarks inside the video picture
  2. Estimating the direction and curl of each individual finger
  3. Comparing the result to a set of gesture descriptions
 
-Step (1) is performed by MediaPipe Handpose, Step (2) and (3) are handled by this library.
+Step (1) is performed by MediaPipe Handpose, and Steps (2) and (3) are handled by this library.
 
-## Installation
-
- 1. Install [MediaPipe Handpose](https://github.com/tensorflow/tfjs-models/tree/master/handpose)
- 2. Install the module via NPM or Yarn:
-```
-npm i --save fingerpose
-```
 ## Demo
 
 ### Basic example
 
-A [basic working example](https://github.com/andypotato/fingerpose/blob/master/dist/index.html) can be found inside the `dist` folder. This example also includes debug output which can be useful when you are creating your own gestures.
+A [basic working example](./dist/index.html) can be found inside the [dist](./dist/) folder. This example also includes debugging output which can be useful when you are creating your own gestures.
 
 [Click here to open the example](https://andypotato.github.io/fingerpose/dist/index.html)
 
 ### Rock, Paper, Scissors game
 
-A simple [Rock, Scissors, Paper game](https://github.com/andypotato/rock-paper-scissors) which shows how to integrate this library in a real-world project.
+A simple [Rock, Scissors, Paper game](https://github.com/andypotato/rock-paper-scissors) that shows how to integrate this library into a real-world project.
 
 ## Quick start
 
 ### Include MediaPipe Handpose and its prerequisites (TFJS >= 2.1.0)
 ```
 <!-- this example uses TFJS 3.7.0 - older versions back to 2.1.0 are supported -->
 <script src="https://unpkg.com/@tensorflow/[email protected]/dist/tf-core.js"></script>
-<script src="https://unpkg.com/@tensorflow/[email protected]/dist/tf-converter.js"></script>
 
-<!-- use the WebGL backend (recommended) - you can alternatively use the WASM backend -->
+<!-- You must explicitly require a TF.js backend if you're not using the tfs union bundle. -->
 <script src="https://unpkg.com/@tensorflow/[email protected]/dist/tf-backend-webgl.js"></script>
 
-<script src="https://unpkg.com/@tensorflow-models/[email protected]/dist/handpose.js"></script>
+<!-- The main handpose dependencies -->
+<script src="https://cdn.jsdelivr.net/npm/@tensorflow-models/[email protected]/dist/hand-pose-detection.js"></script>
+<script src="https://cdn.jsdelivr.net/npm/@mediapipe/[email protected]/hands.min.js"></script>
+
 ```
 
 ### Include this library
 ```
-<script src="fingerpose.js" type="text/javascript"></script>
+<script src="https://cdn.jsdelivr.net/npm/fingerpose@0.1.0/dist/fingerpose.min.js" type="text/javascript"></script>
 ```
+#### Alternatives
+You can copy the whole file from [fingerpose.js](./dist/fingerpose.js) or install from NPM with
+
+```
+npm install fingerpose
+```
+
 ### Add the gestures you want do detect
 ```
 // add "✌🏻" and "👍" as sample gestures
@@ -93,7 +95,7 @@ The result is an object containing possible gestures and their match score, for
 }
 ```
 
-In addition you receive the `poseData` array including the raw curl and direction information for each finger. This is useful for debugging purposes as it can help you understand how an individual finger is "seen" by the library.
+In addition, you receive the `poseData` array including the raw curl and direction information for each finger. This is useful for debugging purposes as it can help you understand how an individual finger is "seen" by the library.
 
 ```
 // example for raw pose data
@@ -105,11 +107,11 @@ poseData: [
 ```
 
 ## Define your own gestures
-You can create any number of hand gestures for this library to recognize. To see how a gesture is described, have a look at the included sample gestures [Victory](https://github.com/andypotato/fingerpose/blob/master/src/gestures/Victory.js) and [Thumbs Up](https://github.com/andypotato/fingerpose/blob/master/src/gestures/ThumbsUp.js).
+You can create any number of hand gestures for this library to recognize. To see how a gesture is described, have a look at the included sample gestures [Victory](./src/gestures/Victory.js) and [Thumbs Up](./src/gestures/ThumbsUp.js).
 
-A gesture is defined by describing the expected curl and direction of each individual finger. For example for a "Thumbs Up" gesture is defined by a stretched out thumb pointing up while all other fingers are curled and pointing to the left or right 👍.
+A gesture is defined by describing the expected curl and direction of each individual finger. For example, a "Thumbs Up" gesture is defined by a stretched-out thumb pointing up while all other fingers are curled and pointing to the left or right 👍.
 
-To describe gestures, you can use the provided [Finger Description Constants](https://github.com/andypotato/fingerpose/blob/master/src/FingerDescription.js):
+To describe gestures, you can use the provided [Finger Description Constants](./src/FingerDescription.js):
 
 | Finger | Name |
 |--|--|
@@ -127,7 +129,7 @@ Probably no further explanation is required for finger names... 👋
 | 1 | FingerCurl.HalfCurl |
 | 2 | FingerCurl.FullCurl |
 
-You can refer to the images below for an example how the index finger is curled (no curl, half curl, full curl):
+You can refer to the images below for an example of how the index finger is curled (no-curl, half curl, full curl):
 | ![enter image description here](https://github.com/andypotato/fingerpose/raw/master/assets/nocurl.jpg) | ![enter image description here](https://github.com/andypotato/fingerpose/raw/master/assets/halfcurl.jpg) | ![enter image description here](https://github.com/andypotato/fingerpose/raw/master/assets/fullcurl.jpg) |
 |--|--|--|
 | No curl | Half curl | Full curl |
@@ -145,8 +147,7 @@ You can refer to the images below for an example how the index finger is curled
 | 7 | Diagonal Down Left ↙️ |
 
 #### Example: Thumbs down gesture description 👎
-
-First create a new GestureDescription object:
+First, create a new GestureDescription object:
 ```
 const thumbsDownGesture = new fp.GestureDescription('thumbs_down');
 ```
@@ -155,13 +156,13 @@ Expect the thumb to be stretched out and pointing down:
 thumbsDownGesture.addCurl(fp.Finger.Thumb, fp.FingerCurl.NoCurl);
 thumbsDownGesture.addDirection(fp.Finger.Thumb, fp.FingerDirection.VerticalDown, 1.0);
 ```
-This will define that a thumb pointing downwards will result in the highest score (1.0) for this gesture. If the thumb is angled to diagonal down left / right we can somehow still accept it, albeit with a lower score (0.9).
+This will define that a thumb pointing downwards will result in the highest score (1.0) for this gesture. If the thumb is angled diagonally down left / right we can somehow still accept it, albeit with a lower score (0.9).
 ```
 thumbsDownGesture.addDirection(fp.Finger.Thumb, fp.FingerDirection.DiagonalDownLeft, 0.9);
 thumbsDownGesture.addDirection(fp.Finger.Thumb, fp.FingerDirection.DiagonalDownRight, 0.9);
 ```
 
-All other fingers are expected to be fully curled. For this gesture it doesn't really matter which direction the curled fingers are pointing at therefore only the curl description is added. Same as above, it's recommended to accept half curled fingers too, with a little bit lower score.
+All other fingers are expected to be fully curled. For this gesture, it doesn't matter which direction the curled fingers are pointing at therefore only the curl description is added. Same as above, it's recommended to accept half-curled fingers too, with a little bit lower score.
 ```
 // do this for all other fingers
 for(let finger of [fp.Finger.Index, fp.Finger.Middle, fp.Finger.Ring, fp.Finger.Pinky]) {
@@ -173,21 +174,21 @@ for(let finger of [fp.Finger.Index, fp.Finger.Middle, fp.Finger.Ring, fp.Finger.
 
 ### Experiment with scores and weights
 
-The "score" is a number between 0 and 10 which describes how good a given combination of finger curl / positions matches a predefined gesture. A perfect match will result in a score of 10.
+The "score" is a number between 0 and 10 which describes how well a given combination of finger curl / positions matches a predefined gesture. A perfect match will result in a score of 10.
 
-* The score threshold should be set rather high (at least 8, better 8.5). If you want to distinguish very similar gestures like "Thumbs up" and "Thumbs down", then add more constraints to your gesture descriptions.
-* Try to experiment with the score for individual fingers. You can add more (or less) weight to single curl / direction by settng the third parameter to a value lower or higher than 1.0.
+* The score threshold should be set rather high (at least 8, best 8.5). If you want to distinguish very similar gestures like "Thumbs up" and "Thumbs down", then add more constraints to your gesture descriptions.
+* Try to experiment with the score for individual fingers. You can add more (or less) weight to a single curl / direction by settng the third parameter to a value lower or higher than 1.0.
 
-### Check if you really need a finger pointing direction
+### Check if you really need a finger-pointing direction
 
 Many poses do not require fingers pointing in a specific direction but are defined by curls only. In these cases just do not add direction constraints to your pose. This also makes it easier to account for left-/right-handed persons.
 
 Also note: Unless you're Houdini, you can not fully curl your thumb.
 
 ### Pre-process your input data
 
-* Consider running another model like [PoseNet](https://github.com/tensorflow/tfjs-models/tree/master/posenet) to detect the hand position(s) first, then crop your input image to only contain the hand. This will not only significantly reduce false detections but also speed up Handpose inference as much less image data needs to be processed (PoseNet is cheap in comparison).
-* MediaPipe Handpose does not offer much customization. Still you can try playing with the model parameters, especially `detectionConfidence` and `iouThreshold` which can improve accuracy under some lighting conditions.
+* Consider running another model like [PoseNet](https://github.com/tensorflow/tfjs-models/tree/master/posenet) to detect the hand position(s) first, then crop your input image to only contain the hand. This will not only significantly reduce false detections but also speed up Handpose inference as much fewer image data needs to be processed (PoseNet is cheap in comparison).
+* MediaPipe Handpose does not offer much customization. Still, you can try playing with the model parameters, especially `detectionConfidence` and `iouThreshold` which can improve accuracy under some lighting conditions.
 
 ### Post-process your detections
 
@@ -201,9 +202,9 @@ You should treat your detections as a "noisy signal" and add some smoothing / fi
  Look at the raw pose data result in `GestureEstimator::estimate()` to understand the detected curls / directions for each finger to the console. This way you can verify if your assumed curls / directions match with what the estimator actually sees.
 
 ## Known issues and limitations
- - Currently only one hand is supported at the same time. This is a limitation of the underlying `handpose` model and may or may not change in the future.
- - The `handpose` model has issues detecting a single stretched out finger (for example index finger). It will occasionally not detect a finger going from "curled" to "not curled" or vice-versa.
+ - Currently, only one hand is supported at the same time. This is a limitation of the underlying `handpose` model and may or may not change in the future.
+ - The `handpose` model has issues detecting a single stretched-out finger (for example index finger). It will occasionally not detect a finger going from "curled" to "not curled" or vice-versa.
 
 ## Credits
 
-The hand gesture recognition module is based on the amazing work by [Prasad Pai](https://github.com/Prasad9/Classify-HandGesturePose). This module is more or less a straight JavaScript port of his [FingerPoseEstimate](https://github.com/Prasad9/Classify-HandGesturePose/blob/master/pose/utils/FingerPoseEstimate.py) Python module.
+The hand gesture recognition module is based on the amazing work of [Prasad Pai](https://github.com/Prasad9/Classify-HandGesturePose). This module is more or less a straight JavaScript port of his [FingerPoseEstimate](https://github.com/Prasad9/Classify-HandGesturePose/blob/master/pose/utils/FingerPoseEstimate.py) Python module.