Skip to content
This repository has been archived by the owner on Feb 22, 2023. It is now read-only.

Code cleanup and added metadata for ML Vision #631

Merged
merged 40 commits into from
Jun 28, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
c0303b3
Initial 'fluter create' for Firebase ML Vision plugin
bparrishMines Jun 11, 2018
cbe3715
Formatting
bparrishMines Jun 11, 2018
d6c20cd
More Formatting
bparrishMines Jun 11, 2018
e002953
Import ordering
bparrishMines Jun 11, 2018
79c085f
Added android app to firebase
bparrishMines Jun 11, 2018
a1775e8
Added ios app to firebase
bparrishMines Jun 11, 2018
a6734f1
Added ml vision to android and ios
bparrishMines Jun 12, 2018
debb1c4
Merge branch 'master' of github.com:flutter/plugins into ml_vision
bparrishMines Jun 12, 2018
84ac21f
WIP dart side of ml vision
bparrishMines Jun 12, 2018
e193edf
finish dart side of ml vision and text detector
bparrishMines Jun 13, 2018
ef0a43f
Added TextDetector tests
bparrishMines Jun 13, 2018
1b098f7
Added some comments
bparrishMines Jun 14, 2018
3475f17
Finished comments
bparrishMines Jun 14, 2018
fe05e34
Removed usused dependency
bparrishMines Jun 14, 2018
a958e5b
Merge branch 'master' of github.com:flutter/plugins into ml_vision_te…
bparrishMines Jun 14, 2018
e8faf79
Formatting and unused imports
bparrishMines Jun 14, 2018
a634645
Changed variable name to restart WIP test
bparrishMines Jun 14, 2018
bb2c0b5
Start of android text detection
bparrishMines Jun 14, 2018
76e7917
Added text recognition to android channel
bparrishMines Jun 15, 2018
636cdc6
Create dart test app
bparrishMines Jun 15, 2018
418f646
Added to test app
bparrishMines Jun 15, 2018
29ec78d
Merge branch 'master' of github.com:flutter/plugins into ml_vision_te…
bparrishMines Jun 15, 2018
e3f5a02
Finished example ml vision app
bparrishMines Jun 18, 2018
ed3ec07
Merge branch 'master' of github.com:flutter/plugins into ml_vision_te…
bparrishMines Jun 18, 2018
2a5b223
Formatting
bparrishMines Jun 18, 2018
e91584a
Fixed crash
bparrishMines Jun 18, 2018
98de4cf
Add permssions
bparrishMines Jun 18, 2018
c1b7114
Setup iOS side of text detector
bparrishMines Jun 19, 2018
0bcc44b
Formatting and comments
bparrishMines Jun 19, 2018
c611304
More formattign
bparrishMines Jun 19, 2018
501b85a
More More Formatting
bparrishMines Jun 19, 2018
ad6b96d
Merge branch 'master' of github.com:flutter/plugins into ml_visioin_t…
bparrishMines Jun 20, 2018
2aa05b8
Code cleanup and added necessary metadata
bparrishMines Jun 21, 2018
f6c3bc5
Merge branch 'master' of github.com:flutter/plugins into ml_vision_cl…
bparrishMines Jun 25, 2018
df5055e
Alphabetize
bparrishMines Jun 26, 2018
6427a5b
Small code fixes and updated README
bparrishMines Jun 26, 2018
822bc38
Return only ints
bparrishMines Jun 26, 2018
ae3d887
Formatting
bparrishMines Jun 26, 2018
c411458
Simplified conversion
bparrishMines Jun 26, 2018
00cf836
Formatting
bparrishMines Jun 28, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 27 additions & 20 deletions FlutterFire.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,18 +14,19 @@ The plugins are still under development, and some APIs might not be available ye

| Plugin | Version | Firebase feature | Source code |
|---|---|---|---|
| [cloud_firestore][firestore_pub] | ![pub package][firestore_badge] | [Cloud Firestore][firestore_product] | [`packages/cloud_firestore`][firestore_code] |
| [cloud_functions][functions_pub] | ![pub package][functions_badge] | [Cloud Functions][functions_product] | [`packages/cloud_functions`][functions_code] |
| [firebase_admob][admob_pub] | ![pub package][admob_badge] | [Firebase AdMob][admob_product] | [`packages/firebase_admob`][admob_code] |
| [firebase_analytics][analytics_pub] | ![pub package][analytics_badge] | [Firebase Analytics][analytics_product] | [`packages/firebase_analytics`][analytics_code] |
| [firebase_auth][auth_pub] | ![pub package][auth_badge] | [Firebase Authentication][auth_product] | [`packages/firebase_auth`][auth_code] |
| [firebase_core][core_pub] | ![pub package][core_badge] | [Firebase Core][core_product] | [`packages/firebase_core`][core_code] |
| [firebase_database][database_pub] | ![pub package][database_badge] | [Firebase Realtime Database][database_product] | [`packages/firebase_database`][database_code] |
| [firebase_dynamic_links][dynamic_links_pub] | ![pub package][dynamic_links_badge] | [Firebase Dynamic Links][dynamic_links_product] | [`packages/firebase_dynamic_links`][dynamic_links_code] |
| [cloud_firestore][firestore_pub] | ![pub package][firestore_badge] | [Cloud Firestore][firestore_product] | [`packages/cloud_firestore`][firestore_code] |
| [firebase_messaging][messaging_pub] | ![pub package][messaging_badge] | [Firebase Cloud Messaging][messaging_product] | [`packages/firebase_messaging`][messaging_code] |
| [firebase_storage][storage_pub] | ![pub package][storage_badge] | [Firebase Cloud Storage][storage_product] | [`packages/firebase_storage`][storage_code] |
| [firebase_ml_vision][ml_vision_pub] | ![pub package][ml_vision_badge] | [Firebase ML Kit][ml_vision_product] | [`packages/firebase_ml_vision`][ml_vision_code] |
| [firebase_performance][performance_pub] | ![pub package][performance_badge] | [Firebase Performance Monitoring][performance_product] | [`packages/firebase_performance`][performance_code] |
| [firebase_remote_config][remote_config_pub] | ![pub package][remote_config_badge] | [Firebase Remote Config][remote_config_product] | [`packages/firebase_remote_config`][remote_config_code] |
| [cloud_functions][functions_pub] | ![pub package][functions_badge] | [Cloud Functions][functions_product] | [`packages/cloud_functions`][functions_code] |
| [firebase_storage][storage_pub] | ![pub package][storage_badge] | [Firebase Cloud Storage][storage_product] | [`packages/firebase_storage`][storage_code] |

[admob_pub]: https://pub.dartlang.org/packages/firebase_admob
[admob_product]: https://firebase.google.com/docs/admob/
Expand All @@ -42,11 +43,6 @@ The plugins are still under development, and some APIs might not be available ye
[auth_code]: https://github.com/flutter/plugins/tree/master/packages/firebase_auth
[auth_badge]: https://img.shields.io/pub/v/firebase_auth.svg

[firestore_pub]: https://pub.dartlang.org/packages/cloud_firestore
[firestore_product]: https://firebase.google.com/products/firestore/
[firestore_code]: https://github.com/flutter/plugins/tree/master/packages/cloud_firestore
[firestore_badge]: https://img.shields.io/pub/v/cloud_firestore.svg

[core_pub]: https://pub.dartlang.org/packages/firebase_core
[core_product]: https://firebase.google.com/
[core_code]: https://github.com/flutter/plugins/tree/master/packages/firebase_core
Expand All @@ -57,15 +53,30 @@ The plugins are still under development, and some APIs might not be available ye
[database_code]: https://github.com/flutter/plugins/tree/master/packages/firebase_database
[database_badge]: https://img.shields.io/pub/v/firebase_database.svg

[dynamic_links_pub]: https://pub.dartlang.org/packages/firebase_dynamic_links
[dynamic_links_product]: https://firebase.google.com/products/dynamic-links/
[dynamic_links_code]: https://github.com/flutter/plugins/tree/master/packages/firebase_dynamic_links
[dynamic_links_badge]: https://img.shields.io/pub/v/firebase_dynamic_links.svg

[firestore_pub]: https://pub.dartlang.org/packages/cloud_firestore
[firestore_product]: https://firebase.google.com/products/firestore/
[firestore_code]: https://github.com/flutter/plugins/tree/master/packages/cloud_firestore
[firestore_badge]: https://img.shields.io/pub/v/cloud_firestore.svg

[functions_pub]: https://pub.dartlang.org/packages/cloud_functions
[functions_product]: https://firebase.google.com/products/functions/
[functions_code]: https://github.com/flutter/plugins/tree/master/packages/cloud_functions
[functions_badge]: https://img.shields.io/pub/v/cloud_functions.svg

[messaging_pub]: https://pub.dartlang.org/packages/firebase_messaging
[messaging_product]: https://firebase.google.com/products/cloud-messaging/
[messaging_code]: https://github.com/flutter/plugins/tree/master/packages/firebase_messaging
[messaging_badge]: https://img.shields.io/pub/v/firebase_messaging.svg

[storage_pub]: https://pub.dartlang.org/packages/firebase_storage
[storage_product]: https://firebase.google.com/products/storage/
[storage_code]: https://github.com/flutter/plugins/tree/master/packages/firebase_storage
[storage_badge]: https://img.shields.io/pub/v/firebase_storage.svg
[ml_vision_pub]: https://pub.dartlang.org/packages/firebase_ml_vision
[ml_vision_product]: https://firebase.google.com/products/ml-kit/
[ml_vision_code]: https://github.com/flutter/plugins/tree/master/packages/firebase_ml_vision
[ml_vision_badge]: https://img.shields.io/pub/v/firebase_ml_vision.svg

[performance_pub]: https://pub.dartlang.org/packages/firebase_performance
[performance_product]: https://firebase.google.com/products/performance/
Expand All @@ -77,12 +88,8 @@ The plugins are still under development, and some APIs might not be available ye
[remote_config_code]: https://github.com/flutter/plugins/tree/master/packages/firebase_remote_config
[remote_config_badge]: https://img.shields.io/pub/v/firebase_remote_config.svg

[dynamic_links_pub]: https://pub.dartlang.org/packages/firebase_dynamic_links
[dynamic_links_product]: https://firebase.google.com/products/dynamic-links/
[dynamic_links_code]: https://github.com/flutter/plugins/tree/master/packages/firebase_dynamic_links
[dynamic_links_badge]: https://img.shields.io/pub/v/firebase_dynamic_links.svg
[storage_pub]: https://pub.dartlang.org/packages/firebase_storage
[storage_product]: https://firebase.google.com/products/storage/
[storage_code]: https://github.com/flutter/plugins/tree/master/packages/firebase_storage
[storage_badge]: https://img.shields.io/pub/v/firebase_storage.svg

[functions_pub]: https://pub.dartlang.org/packages/cloud_functions
[functions_product]: https://firebase.google.com/products/functions/
[functions_code]: https://github.com/flutter/plugins/tree/master/packages/cloud_functions
[functions_badge]: https://img.shields.io/pub/v/cloud_functions.svg
9 changes: 5 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,17 +58,18 @@ These are the available plugins in this repository.
| [video_player](./packages/video_player/) | [![pub package](https://img.shields.io/pub/v/video_player.svg)](https://pub.dartlang.org/packages/video_player) |
| | |
| **FlutterFire Plugins** | |
| [cloud_firestore](./packages/cloud_firestore/) | [![pub package](https://img.shields.io/pub/v/cloud_firestore.svg)](https://pub.dartlang.org/packages/cloud_firestore)
| [cloud_functions](./packages/cloud_functions/) | [![pub package](https://img.shields.io/pub/v/cloud_functions.svg)](https://pub.dartlang.org/packages/cloud_functions) |
| [firebase_admob](./packages/firebase_admob/) | [![pub package](https://img.shields.io/pub/v/firebase_admob.svg)](https://pub.dartlang.org/packages/firebase_admob) |
| [firebase_analytics](./packages/firebase_analytics/) | [![pub package](https://img.shields.io/pub/v/firebase_analytics.svg)](https://pub.dartlang.org/packages/firebase_analytics) |
| [firebase_auth](./packages/firebase_auth/) | [![pub package](https://img.shields.io/pub/v/firebase_auth.svg)](https://pub.dartlang.org/packages/firebase_auth) |
| [cloud_firestore](./packages/cloud_firestore/) | [![pub package](https://img.shields.io/pub/v/cloud_firestore.svg)](https://pub.dartlang.org/packages/cloud_firestore)
| [firebase_core](./packages/firebase_core/) | [![pub package](https://img.shields.io/pub/v/firebase_core.svg)](https://pub.dartlang.org/packages/firebase_core) |
| [firebase_database](./packages/firebase_database/) | [![pub package](https://img.shields.io/pub/v/firebase_database.svg)](https://pub.dartlang.org/packages/firebase_database) |
| [firebase_dynamic_links](./packages/firebase_dynamic_links/) | [![pub package](https://img.shields.io/pub/v/firebase_dynamic_links.svg)](https://pub.dartlang.org/packages/firebase_dynamic_links) |
| [firebase_messaging](./packages/firebase_messaging/) | [![pub package](https://img.shields.io/pub/v/firebase_messaging.svg)](https://pub.dartlang.org/packages/firebase_messaging) |
| [firebase_storage](./packages/firebase_storage/) | [![pub package](https://img.shields.io/pub/v/firebase_storage.svg)](https://pub.dartlang.org/packages/firebase_storage) |
| [firebase_ml_vision](./packages/firebase_ml_vision/) | [![pub package](https://img.shields.io/pub/v/firebase_ml_vision.svg)](https://pub.dartlang.org/packages/firebase_ml_vision) |
| [firebase_performance](./packages/firebase_performance/) | [![pub package](https://img.shields.io/pub/v/firebase_performance.svg)](https://pub.dartlang.org/packages/firebase_performance) |
| [firebase_remote_config](./packages/firebase_remote_config/) | [![pub package](https://img.shields.io/pub/v/firebase_remote_config.svg)](https://pub.dartlang.org/packages/firebase_remote_config) |
| [firebase_dynamic_links](./packages/firebase_dynamic_links/) | [![pub package](https://img.shields.io/pub/v/firebase_dynamic_links.svg)](https://pub.dartlang.org/packages/firebase_dynamic_links) |
| [cloud_functions](./packages/cloud_functions/) | [![pub package](https://img.shields.io/pub/v/cloud_functions.svg)](https://pub.dartlang.org/packages/cloud_functions) |
| [firebase_storage](./packages/firebase_storage/) | [![pub package](https://img.shields.io/pub/v/firebase_storage.svg)](https://pub.dartlang.org/packages/firebase_storage) |

Learn more about [FlutterFire](https://github.com/flutter/plugins/blob/master/FlutterFire.md).
2 changes: 1 addition & 1 deletion packages/firebase_ml_vision/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
## 0.0.1

* TODO: Describe initial release.
* Initial release with text detector.
28 changes: 27 additions & 1 deletion packages/firebase_ml_vision/LICENSE
Original file line number Diff line number Diff line change
@@ -1 +1,27 @@
TODO: Add your license here.
// Copyright 2018 The Chromium Authors. All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
// * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
72 changes: 66 additions & 6 deletions packages/firebase_ml_vision/README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,70 @@
# firebase_ml_vision
# Google ML Kit for Firebase

A new Flutter plugin.
[![pub package](https://img.shields.io/pub/v/firebase_ml_vision.svg)](https://pub.dartlang.org/packages/firebase_ml_vision)

## Getting Started
A Flutter plugin to use the [Google ML Kit for Firebase API](https://firebase.google.com/docs/ml-kit/).

For Flutter plugins for other Firebase products, see [FlutterFire.md](https://github.com/flutter/plugins/blob/master/FlutterFire.md).

*Note*: This plugin is still under development, and some APIs might not be available yet. [Feedback](https://github.com/flutter/flutter/issues) and [Pull Requests](https://github.com/flutter/plugins/pulls) are most welcome!

## Usage

To use this plugin, add `firebase_ml_vision` as a [dependency in your pubspec.yaml file](https://flutter.io/platform-plugins/). You must also configure Firebase for each platform project: Android and iOS (see the example folder or https://codelabs.developers.google.com/codelabs/flutter-firebase/#4 for step by step details).

### Android
Optional but recommended: If you use the on-device API, configure your app to automatically download the ML model to the device after your app is installed from the Play Store. To do so, add the following declaration to your app's AndroidManifest.xml file:

```manifest
<application ...>
...
<meta-data
android:name="com.google.firebase.ml.vision.DEPENDENCIES"
android:value="ocr" />
<!-- To use multiple models: android:value="ocr,model2,model3" -->
</application>
```

## On-device Text Recognition

To use the on-device text recognition model, run the text detector as described below:

1. Create a `FirebaseVisionImage` object from your image.

For help getting started with Flutter, view our online
[documentation](https://flutter.io/).
To create a `FirebaseVisionImage` from an image `File` object:

```dart
final File imageFile = getImageFile();
final FirebaseVisionImage visionImage = FirebaseVisionImage.fromFile(imageFile);
```

2. Get an instance of `TextDetector` and pass `visionImage` to `detectInImage().`

```dart
final TextDetector detector = FirebaseVision.instance.getTextDetector();
final List<TextBlock> blocks = await detector.detectInImage(visionImage);

detector.close();
```

3. Extract text and text locations from blocks of recognized text.

```dart
for (TextBlock block in textLocations) {
final Rectangle<num> boundingBox = block.boundingBox;
final List<Point<num>> cornerPoints = block.cornerPoints;
final String text = block.text;

for (TextLine line in block.lines) {
// ...

for (TextElement element in line.elements) {
// ...
}
}
}
```

## Getting Started

For help on editing plugin code, view the [documentation](https://flutter.io/platform-plugins/#edit-code).
See the `example` directory for a complete sample app using Google ML Kit for Firebase.
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
/** FirebaseMlVisionPlugin */
public class FirebaseMlVisionPlugin implements MethodCallHandler {
private Registrar registrar;
private FirebaseVisionTextDetector textDetector;

private FirebaseMlVisionPlugin(Registrar registrar) {
this.registrar = registrar;
Expand All @@ -39,10 +40,24 @@ public static void registerWith(Registrar registrar) {

@Override
public void onMethodCall(MethodCall call, Result result) {
if (call.method.equals("TextDetector#detectInImage")) {
handleTextDetectionResult(call, result);
} else {
result.notImplemented();
switch (call.method) {
case "TextDetector#detectInImage":
handleTextDetectionResult(call, result);
break;
case "TextDetector#close":
if (textDetector != null) {
try {
textDetector.close();
result.success(null);
} catch (IOException exception) {
result.error("textDetectorError", exception.getLocalizedMessage(), null);
}

textDetector = null;
}
break;
default:
result.notImplemented();
}
}

Expand All @@ -57,8 +72,8 @@ private void handleTextDetectionResult(MethodCall call, final Result result) {
return;
}

FirebaseVisionTextDetector detector = FirebaseVision.getInstance().getVisionTextDetector();
detector
if (textDetector == null) textDetector = FirebaseVision.getInstance().getVisionTextDetector();
textDetector
.detectInImage(image)
.addOnSuccessListener(
new OnSuccessListener<FirebaseVisionText>() {
Expand Down Expand Up @@ -98,8 +113,8 @@ public void onSuccess(FirebaseVisionText firebaseVisionText) {
.addOnFailureListener(
new OnFailureListener() {
@Override
public void onFailure(@NonNull Exception e) {
result.error("textDetectorError", e.getLocalizedMessage(), null);
public void onFailure(@NonNull Exception exception) {
result.error("textDetectorError", exception.getLocalizedMessage(), null);
}
});
}
Expand All @@ -108,14 +123,18 @@ private void addTextData(
Map<String, Object> addTo, Rect boundingBox, Point[] cornerPoints, String text) {
addTo.put("text", text);

addTo.put("left", boundingBox.left);
addTo.put("top", boundingBox.top);
addTo.put("width", boundingBox.width());
addTo.put("height", boundingBox.height());
if (boundingBox != null) {
addTo.put("left", boundingBox.left);
addTo.put("top", boundingBox.top);
addTo.put("width", boundingBox.width());
addTo.put("height", boundingBox.height());
}

List<int[]> points = new ArrayList<>();
for (Point point : cornerPoints) {
points.add(new int[] {point.x, point.y});
if (cornerPoints != null) {
for (Point point : cornerPoints) {
points.add(new int[] {point.x, point.y});
}
}
addTo.put("points", points);
}
Expand Down
6 changes: 5 additions & 1 deletion packages/firebase_ml_vision/example/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,11 @@

Demonstrates how to use the firebase_ml_vision plugin.

## Usage

This example uses the *image_picker* plugin to get images from the device gallery. If using an iOS device you will have to configure you project with the correct permissions seen under iOS configuration [here.](https://pub.dartlang.org/packages/image_picker)

## Getting Started

For help getting started with Flutter, view our online
[documentation](https://flutter.io/).
[documentation.](https://flutter.io/)
Loading