Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added ability to run ScanViewController with custom images #32

Merged
merged 2 commits into from
Jul 6, 2019
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CardScan/Assets/CardScan.storyboard
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,9 @@
<color key="textColor" red="0.18039215689999999" green="0.81568627449999997" blue="0.37647058820000001" alpha="1" colorSpace="calibratedRGB"/>
<nil key="highlightedColor"/>
</label>
<imageView hidden="YES" clipsSubviews="YES" userInteractionEnabled="NO" contentMode="scaleAspectFit" horizontalHuggingPriority="251" verticalHuggingPriority="251" translatesAutoresizingMaskIntoConstraints="NO" id="FbV-Jw-mRi">
<rect key="frame" x="16" y="327.66666666666669" width="382" height="240.66666666666669"/>
</imageView>
<label hidden="YES" opaque="NO" userInteractionEnabled="NO" contentMode="left" horizontalHuggingPriority="251" verticalHuggingPriority="251" text="4242 4242 4242 4242" textAlignment="center" lineBreakMode="tailTruncation" baselineAdjustment="alignBaselines" minimumFontSize="12" translatesAutoresizingMaskIntoConstraints="NO" id="mhR-St-yeE">
<rect key="frame" x="32" y="435.66666666666669" width="350" height="57"/>
<fontDescription key="fontDescription" type="system" pointSize="48"/>
Expand Down Expand Up @@ -111,8 +114,10 @@
<constraint firstItem="iJh-NT-quU" firstAttribute="top" secondItem="NRD-Sd-k9r" secondAttribute="bottom" constant="16" id="Gwd-mC-Yd7"/>
<constraint firstAttribute="trailing" secondItem="mhR-St-yeE" secondAttribute="trailing" constant="32" id="HEa-eN-2z3"/>
<constraint firstItem="mhR-St-yeE" firstAttribute="centerY" secondItem="NRD-Sd-k9r" secondAttribute="centerY" constant="16" id="Hjd-O3-cOS"/>
<constraint firstItem="FbV-Jw-mRi" firstAttribute="bottom" secondItem="NRD-Sd-k9r" secondAttribute="bottom" id="K9e-kV-SDT"/>
<constraint firstAttribute="trailing" secondItem="sYa-ko-OnI" secondAttribute="trailing" constant="32" id="MRD-Go-QNU"/>
<constraint firstItem="aju-0L-PxB" firstAttribute="top" secondItem="JZr-lh-2mZ" secondAttribute="top" id="Nw1-Sg-ATq"/>
<constraint firstItem="FbV-Jw-mRi" firstAttribute="trailing" secondItem="NRD-Sd-k9r" secondAttribute="trailing" id="OQ7-9h-JFH"/>
<constraint firstItem="NRD-Sd-k9r" firstAttribute="leading" secondItem="JZr-lh-2mZ" secondAttribute="leading" constant="16" id="RaX-bo-Vr3"/>
<constraint firstItem="NRD-Sd-k9r" firstAttribute="centerX" secondItem="JZr-lh-2mZ" secondAttribute="centerX" id="SiM-V5-xoX"/>
<constraint firstAttribute="trailing" secondItem="vfu-3N-l48" secondAttribute="trailing" constant="16" id="VF2-Te-ZU2"/>
Expand All @@ -127,7 +132,9 @@
<constraint firstItem="vfu-3N-l48" firstAttribute="leading" secondItem="JZr-lh-2mZ" secondAttribute="leading" constant="16" id="tfH-wA-wLC"/>
<constraint firstItem="NRD-Sd-k9r" firstAttribute="top" secondItem="vfu-3N-l48" secondAttribute="bottom" constant="16" id="tvq-2M-xjH"/>
<constraint firstItem="mhR-St-yeE" firstAttribute="leading" secondItem="JZr-lh-2mZ" secondAttribute="leading" constant="32" id="v82-G9-16t"/>
<constraint firstItem="FbV-Jw-mRi" firstAttribute="leading" secondItem="NRD-Sd-k9r" secondAttribute="leading" id="vgB-Rg-8Ap"/>
<constraint firstAttribute="trailing" secondItem="iJh-NT-quU" secondAttribute="trailing" constant="16" id="xP1-2O-u2R"/>
<constraint firstItem="FbV-Jw-mRi" firstAttribute="top" secondItem="NRD-Sd-k9r" secondAttribute="top" id="yx5-KZ-MVe"/>
</constraints>
</view>
<label opaque="NO" userInteractionEnabled="NO" contentMode="left" horizontalHuggingPriority="251" verticalHuggingPriority="251" text="" textAlignment="center" lineBreakMode="tailTruncation" baselineAdjustment="alignBaselines" adjustsFontSizeToFit="NO" translatesAutoresizingMaskIntoConstraints="NO" id="2Xu-ZF-zZr">
Expand Down Expand Up @@ -185,6 +192,7 @@
<outlet property="backButtonWidthConstraint" destination="EaN-bi-nZq" id="3GV-jN-upY"/>
<outlet property="blurView" destination="aju-0L-PxB" id="O2s-Pr-FEO"/>
<outlet property="cardNumberLabel" destination="mhR-St-yeE" id="Zwu-8I-mT9"/>
<outlet property="debugImageView" destination="FbV-Jw-mRi" id="Hhh-cL-znj"/>
<outlet property="expiryLabel" destination="sYa-ko-OnI" id="P2m-du-Sw7"/>
<outlet property="positionCardLabel" destination="iJh-NT-quU" id="Ssa-XL-rLU"/>
<outlet property="previewView" destination="JZr-lh-2mZ" id="PpI-ZM-7tx"/>
Expand Down
59 changes: 58 additions & 1 deletion CardScan/Classes/ScanViewController.swift
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,10 @@ import Vision
@objc func skipButton() -> String
}

@objc public protocol TestingImageDataSource {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you give me an example of how this protocol will be used? I read below that it can be used for custom testing but I am still unclear on how it it should be implemented.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, checkout Example/CardScan/ViewController.swift, which is part of this PR. It shows how to implement this protocol and how to use it.

@objc func nextImage() -> CGImage?
}

@objc public class CreditCard: NSObject {
@objc public var number: String
@objc public var expiryMonth: String?
Expand Down Expand Up @@ -60,6 +64,7 @@ import Vision

public weak var scanDelegate: ScanDelegate?
@objc public weak var stringDataSource: ScanStringsDataSource?
@objc public weak var testingImageDataSource: TestingImageDataSource?
@objc public var allowSkip = false
public var scanQrCode = false
@objc public var errorCorrectionDuration = 1.5
Expand All @@ -74,6 +79,7 @@ import Vision
@objc public var positionCardFont: UIFont?
@objc public var skipButtonFont: UIFont?
@objc public var backButtonImageToTextDelta: NSNumber?
@objc public var showDebugImageView = false

static public let machineLearningQueue = DispatchQueue(label: "CardScanMlQueue")
// Only access this variable from the machineLearningQueue
Expand All @@ -89,6 +95,7 @@ import Vision
@IBOutlet weak var backButton: UIButton!
@IBOutlet weak var backButtonImageButton: UIButton!

@IBOutlet weak var debugImageView: UIImageView!
@IBOutlet weak var previewView: PreviewView!
@IBOutlet weak var regionOfInterestLabel: UILabel!
@IBOutlet weak var regionOfInterestAspectConstraint: NSLayoutConstraint!
Expand Down Expand Up @@ -452,6 +459,38 @@ import Vision
semaphore.wait()
}

func drawBoundingBoxesOnImage(image: UIImage, embossedCharacterBoxes: [CGRect],
characterBoxes: [CGRect], appleBoxes: [CGRect]) -> UIImage? {

let imageSize = image.size
let scale: CGFloat = 0
UIGraphicsBeginImageContextWithOptions(imageSize, false, scale)

image.draw(at: CGPoint(x: 0,y :0))

UIGraphicsGetCurrentContext()?.setLineWidth(3.0)

UIColor.green.setStroke()
for characterBox in characterBoxes {
UIRectFrame(characterBox)
}

UIColor.blue.setStroke()
for characterBox in embossedCharacterBoxes {
UIRectFrame(characterBox)
}

UIColor.red.setStroke()
for characterBox in appleBoxes {
UIRectFrame(characterBox)
}

let newImage = UIGraphicsGetImageFromCurrentImageContext()
UIGraphicsEndImageContext()

return newImage
}

@available(iOS 11.0, *)
func blockingOcrModel(rawImage: CGImage) {
let (number, expiry, done, foundNumberInThisScan) = ocr.performWithErrorCorrection(for: rawImage)
Expand All @@ -462,6 +501,20 @@ import Vision
}
}

if self.showDebugImageView {
let flatBoxes = self.ocr.scanStats.lastFlatBoxes ?? []
let embossedBoxes = self.ocr.scanStats.lastEmbossedBoxes ?? []
let expiryBoxes = self.ocr.scanStats.expiryBoxes ?? []

DispatchQueue.main.async {
if self.debugImageView.isHidden {
self.debugImageView.isHidden = false
}

self.debugImageView.image = self.drawBoundingBoxesOnImage(image: UIImage(cgImage: rawImage), embossedCharacterBoxes: embossedBoxes, characterBoxes: flatBoxes, appleBoxes: expiryBoxes)
}
}

if done {
DispatchQueue.main.async {
guard let number = number else {
Expand Down Expand Up @@ -501,11 +554,15 @@ import Vision
return
}

// we allow apps that integrate to supply their own sequence of images
// for use in testing
let image = self.testingImageDataSource?.nextImage() ?? rawImage

if #available(iOS 11.0, *) {
if self.scanQrCode {
self.blockingQrModel(pixelBuffer: pixelBuffer)
} else {
self.blockingOcrModel(rawImage: rawImage)
self.blockingOcrModel(rawImage: image)
}
}

Expand Down
9 changes: 9 additions & 0 deletions Example/CardScan/Base.lproj/Main.storyboard
Original file line number Diff line number Diff line change
Expand Up @@ -100,20 +100,29 @@
<action selector="scanAndShowImagePress" destination="vXZ-lx-hvc" eventType="touchUpInside" id="4Yh-It-wDZ"/>
</connections>
</button>
<button opaque="NO" contentMode="scaleToFill" contentHorizontalAlignment="center" contentVerticalAlignment="center" buttonType="roundedRect" lineBreakMode="middleTruncation" translatesAutoresizingMaskIntoConstraints="NO" id="lOx-Ed-LOH">
<rect key="frame" x="134" y="553.5" width="107" height="30"/>
<state key="normal" title="Custom images"/>
<connections>
<action selector="customVideoPress" destination="vXZ-lx-hvc" eventType="touchUpInside" id="mp5-uJ-PSq"/>
</connections>
</button>
</subviews>
<color key="backgroundColor" red="1" green="1" blue="1" alpha="1" colorSpace="custom" customColorSpace="sRGB"/>
<constraints>
<constraint firstItem="qlz-Ds-Znv" firstAttribute="top" secondItem="9tR-DO-gyb" secondAttribute="bottom" constant="16" id="1PH-rJ-mXS"/>
<constraint firstItem="qlz-Ds-Znv" firstAttribute="centerY" secondItem="kh9-bI-dsS" secondAttribute="centerY" id="AkZ-f5-Cw0"/>
<constraint firstItem="LBI-Nj-Ojb" firstAttribute="top" secondItem="pIK-as-bko" secondAttribute="bottom" constant="8" id="DKk-6B-vtM"/>
<constraint firstItem="9tR-DO-gyb" firstAttribute="centerX" secondItem="kh9-bI-dsS" secondAttribute="centerX" id="IXC-rr-ZtJ"/>
<constraint firstItem="lOx-Ed-LOH" firstAttribute="top" secondItem="HGF-3W-kd7" secondAttribute="bottom" constant="8" id="IrW-Pq-bcY"/>
<constraint firstItem="pIK-as-bko" firstAttribute="top" secondItem="6NY-p4-OvD" secondAttribute="bottom" constant="8" id="Jpb-N9-6UO"/>
<constraint firstItem="HGF-3W-kd7" firstAttribute="leading" secondItem="kh9-bI-dsS" secondAttribute="leadingMargin" id="LAc-xn-1Rd"/>
<constraint firstItem="LBI-Nj-Ojb" firstAttribute="trailing" secondItem="kh9-bI-dsS" secondAttribute="trailingMargin" id="OZL-nk-k2K"/>
<constraint firstItem="qlz-Ds-Znv" firstAttribute="leading" secondItem="kh9-bI-dsS" secondAttribute="leadingMargin" id="Qox-lN-oV9"/>
<constraint firstItem="pIK-as-bko" firstAttribute="trailing" secondItem="kh9-bI-dsS" secondAttribute="trailingMargin" id="SSG-hd-RyU"/>
<constraint firstItem="LBI-Nj-Ojb" firstAttribute="leading" secondItem="kh9-bI-dsS" secondAttribute="leadingMargin" id="TUX-zN-lhp"/>
<constraint firstItem="pIK-as-bko" firstAttribute="leading" secondItem="kh9-bI-dsS" secondAttribute="leadingMargin" id="UlN-Gs-FK7"/>
<constraint firstItem="lOx-Ed-LOH" firstAttribute="centerX" secondItem="kh9-bI-dsS" secondAttribute="centerX" id="Uy4-wz-YkW"/>
<constraint firstItem="6NY-p4-OvD" firstAttribute="trailing" secondItem="kh9-bI-dsS" secondAttribute="trailingMargin" id="VtA-4O-Rau"/>
<constraint firstItem="6NY-p4-OvD" firstAttribute="top" secondItem="qlz-Ds-Znv" secondAttribute="bottom" constant="8" id="Y8H-65-voH"/>
<constraint firstItem="qlz-Ds-Znv" firstAttribute="top" secondItem="9tR-DO-gyb" secondAttribute="bottom" constant="16" id="hrc-Fy-VMw"/>
Expand Down
6 changes: 6 additions & 0 deletions Example/CardScan/Images.xcassets/Contents.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"info" : {
"version" : 1,
"author" : "xcode"
}
}
21 changes: 21 additions & 0 deletions Example/CardScan/Images.xcassets/frame0.imageset/Contents.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{
"images" : [
{
"idiom" : "universal",
"filename" : "frame0.png",
"scale" : "1x"
},
{
"idiom" : "universal",
"scale" : "2x"
},
{
"idiom" : "universal",
"scale" : "3x"
}
],
"info" : {
"version" : 1,
"author" : "xcode"
}
}
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{
"images" : [
{
"idiom" : "universal",
"filename" : "frame114.png",
"scale" : "1x"
},
{
"idiom" : "universal",
"scale" : "2x"
},
{
"idiom" : "universal",
"scale" : "3x"
}
],
"info" : {
"version" : 1,
"author" : "xcode"
}
}
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{
"images" : [
{
"idiom" : "universal",
"filename" : "frame133.png",
"scale" : "1x"
},
{
"idiom" : "universal",
"scale" : "2x"
},
{
"idiom" : "universal",
"scale" : "3x"
}
],
"info" : {
"version" : 1,
"author" : "xcode"
}
}
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
21 changes: 21 additions & 0 deletions Example/CardScan/Images.xcassets/frame19.imageset/Contents.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{
"images" : [
{
"idiom" : "universal",
"filename" : "frame19.png",
"scale" : "1x"
},
{
"idiom" : "universal",
"scale" : "2x"
},
{
"idiom" : "universal",
"scale" : "3x"
}
],
"info" : {
"version" : 1,
"author" : "xcode"
}
}
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
21 changes: 21 additions & 0 deletions Example/CardScan/Images.xcassets/frame38.imageset/Contents.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{
"images" : [
{
"idiom" : "universal",
"filename" : "frame38.png",
"scale" : "1x"
},
{
"idiom" : "universal",
"scale" : "2x"
},
{
"idiom" : "universal",
"scale" : "3x"
}
],
"info" : {
"version" : 1,
"author" : "xcode"
}
}
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
21 changes: 21 additions & 0 deletions Example/CardScan/Images.xcassets/frame57.imageset/Contents.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{
"images" : [
{
"idiom" : "universal",
"filename" : "frame57.png",
"scale" : "1x"
},
{
"idiom" : "universal",
"scale" : "2x"
},
{
"idiom" : "universal",
"scale" : "3x"
}
],
"info" : {
"version" : 1,
"author" : "xcode"
}
}
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
21 changes: 21 additions & 0 deletions Example/CardScan/Images.xcassets/frame73.imageset/Contents.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{
"images" : [
{
"idiom" : "universal",
"filename" : "frame73.png",
"scale" : "1x"
},
{
"idiom" : "universal",
"scale" : "2x"
},
{
"idiom" : "universal",
"scale" : "3x"
}
],
"info" : {
"version" : 1,
"author" : "xcode"
}
}
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
21 changes: 21 additions & 0 deletions Example/CardScan/Images.xcassets/frame76.imageset/Contents.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{
"images" : [
{
"idiom" : "universal",
"filename" : "frame76.png",
"scale" : "1x"
},
{
"idiom" : "universal",
"scale" : "2x"
},
{
"idiom" : "universal",
"scale" : "3x"
}
],
"info" : {
"version" : 1,
"author" : "xcode"
}
}
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
21 changes: 21 additions & 0 deletions Example/CardScan/Images.xcassets/frame95.imageset/Contents.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{
"images" : [
{
"idiom" : "universal",
"filename" : "frame95.png",
"scale" : "1x"
},
{
"idiom" : "universal",
"scale" : "2x"
},
{
"idiom" : "universal",
"scale" : "3x"
}
],
"info" : {
"version" : 1,
"author" : "xcode"
}
}
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading