Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vsscanf() based date string parser #334

Merged
merged 34 commits into from
Apr 21, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
362c4ba
initial scanf() based date string parser
sobri909 Apr 15, 2018
f5f8ab5
add SQLiteDateParser to project file
sobri909 Apr 16, 2018
59d7909
return correct Format for partial date/time combinations
sobri909 Apr 16, 2018
b444d37
differentiate between SS and SS.SSS formats
sobri909 Apr 16, 2018
7a1aa88
fractions of seconds are not the same thing as integer counts of nano…
sobri909 Apr 16, 2018
bb59f5e
0.123 is not the same thing as 0.000123
sobri909 Apr 17, 2018
dad1637
not all nanosecond fractions have 3 sig figs, so can't hard code the …
sobri909 Apr 17, 2018
eba5df1
fix the tests to expect less nanosecond precision loss
sobri909 Apr 17, 2018
3bcc5b8
permit precision loss down to 3 sig figs
sobri909 Apr 17, 2018
3306613
truncate date/time strings that include seconds precision beyond the …
sobri909 Apr 17, 2018
bde72e5
Revert "permit precision loss down to 3 sig figs"
sobri909 Apr 17, 2018
7320f88
Revert "fix the tests to expect less nanosecond precision loss"
sobri909 Apr 17, 2018
3b546de
truncate nanosecond precision to 3 sig figs
sobri909 Apr 17, 2018
6d9bce3
Add SQLiteDateParser.swift to GRDBCustom and GRDBCipher
groue Apr 20, 2018
7aabfea
SQLiteDateParser: get rid of the mutex
groue Apr 20, 2018
27194c8
SQLiteDateParser: use withUnsafeMutablePointer instead of allocations
groue Apr 20, 2018
e9a668c
SQLiteDateParser: no more static methods
groue Apr 20, 2018
d1a0e8d
SQLiteDateParser.components(from:) is not public
groue Apr 20, 2018
0640f90
(whitespace)
groue Apr 20, 2018
8915672
SQLiteDateParser: parse C strings
groue Apr 20, 2018
936b31f
DatabaseDateComponents adopts StatementColumnConvertible
groue Apr 20, 2018
c70689f
Check useScanfStrategy in DatabaseDateComponents.init(sqliteStatement…
groue Apr 20, 2018
11a4a6a
Documentation
groue Apr 20, 2018
2660407
Date adopts StatementColumnConvertible
groue Apr 20, 2018
f12495b
New DateParsingTests in GRDBOSXPerformanceTests
groue Apr 20, 2018
9e380e5
Cleanup
groue Apr 20, 2018
282255b
No need for useScanfStrategy any longer
groue Apr 20, 2018
85fbf64
Add missing imports (should fix travis issues)
groue Apr 20, 2018
9efe4a6
Add a sanity check
groue Apr 20, 2018
316d287
SQLiteDateParser: optimize nanoseconds parsing
groue Apr 20, 2018
632bce5
SQLiteDateParser: don't parse garbage into nanoseconds
groue Apr 20, 2018
219f96a
More DatabaseDateComponents parsing tests
groue Apr 21, 2018
1311f6c
SQLiteDateParser: fix nanosecond parsing
groue Apr 21, 2018
6c5d6ba
SQLiteDateParser: tighten vsscanf format
groue Apr 21, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions GRDB.xcodeproj/project.pbxproj
Original file line number Diff line number Diff line change
Expand Up @@ -259,6 +259,7 @@
566B91331FA4D3810012D5B0 /* TransactionObserver.swift in Sources */ = {isa = PBXBuildFile; fileRef = 566B91321FA4D3810012D5B0 /* TransactionObserver.swift */; };
566B91361FA4D3810012D5B0 /* TransactionObserver.swift in Sources */ = {isa = PBXBuildFile; fileRef = 566B91321FA4D3810012D5B0 /* TransactionObserver.swift */; };
566B91391FA4D3810012D5B0 /* TransactionObserver.swift in Sources */ = {isa = PBXBuildFile; fileRef = 566B91321FA4D3810012D5B0 /* TransactionObserver.swift */; };
56707201208A509C006AD95A /* DateParsingTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 567071FA208A509C006AD95A /* DateParsingTests.swift */; };
567156181CB142AA007DC145 /* DatabaseQueueReadOnlyTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 567156151CB142AA007DC145 /* DatabaseQueueReadOnlyTests.swift */; };
5671FC201DA3CAC9003BF4FF /* FTS3TokenizerDescriptor.swift in Sources */ = {isa = PBXBuildFile; fileRef = 5671FC1F1DA3CAC9003BF4FF /* FTS3TokenizerDescriptor.swift */; };
5671FC231DA3CAC9003BF4FF /* FTS3TokenizerDescriptor.swift in Sources */ = {isa = PBXBuildFile; fileRef = 5671FC1F1DA3CAC9003BF4FF /* FTS3TokenizerDescriptor.swift */; };
Expand Down Expand Up @@ -625,6 +626,9 @@
56FF455A1D2CDA5200F21EF9 /* RecordUniqueIndexTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 56FF45551D2CDA5200F21EF9 /* RecordUniqueIndexTests.swift */; };
6340BF801E5E3F7900832805 /* RecordPersistenceConflictPolicy.swift in Sources */ = {isa = PBXBuildFile; fileRef = 6340BF7F1E5E3F7900832805 /* RecordPersistenceConflictPolicy.swift */; };
6340BF841E5E3F7900832805 /* RecordPersistenceConflictPolicy.swift in Sources */ = {isa = PBXBuildFile; fileRef = 6340BF7F1E5E3F7900832805 /* RecordPersistenceConflictPolicy.swift */; };
C96C0F2B2084A442006B2981 /* SQLiteDateParser.swift in Sources */ = {isa = PBXBuildFile; fileRef = C96C0F242084A442006B2981 /* SQLiteDateParser.swift */; };
C96C0F2C2084A459006B2981 /* SQLiteDateParser.swift in Sources */ = {isa = PBXBuildFile; fileRef = C96C0F242084A442006B2981 /* SQLiteDateParser.swift */; };
C96C0F2D2084A45A006B2981 /* SQLiteDateParser.swift in Sources */ = {isa = PBXBuildFile; fileRef = C96C0F242084A442006B2981 /* SQLiteDateParser.swift */; };
DC2393C81ABE35F8003FF113 /* GRDB-Bridging.h in Headers */ = {isa = PBXBuildFile; fileRef = DC2393C61ABE35F8003FF113 /* GRDB-Bridging.h */; settings = {ATTRIBUTES = (Public, ); }; };
DC3773F919C8CBB3004FCF85 /* GRDB.h in Headers */ = {isa = PBXBuildFile; fileRef = DC3773F819C8CBB3004FCF85 /* GRDB.h */; settings = {ATTRIBUTES = (Public, ); }; };
/* End PBXBuildFile section */
Expand Down Expand Up @@ -840,6 +844,7 @@
566B91221FA4CF810012D5B0 /* Database+Schema.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = "Database+Schema.swift"; sourceTree = "<group>"; };
566B912A1FA4D0CC0012D5B0 /* StatementAuthorizer.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = StatementAuthorizer.swift; sourceTree = "<group>"; };
566B91321FA4D3810012D5B0 /* TransactionObserver.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TransactionObserver.swift; sourceTree = "<group>"; };
567071FA208A509C006AD95A /* DateParsingTests.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = DateParsingTests.swift; sourceTree = "<group>"; };
567156151CB142AA007DC145 /* DatabaseQueueReadOnlyTests.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = DatabaseQueueReadOnlyTests.swift; sourceTree = "<group>"; };
567156701CB18050007DC145 /* EncryptionTests.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = EncryptionTests.swift; sourceTree = "<group>"; };
5671FC1F1DA3CAC9003BF4FF /* FTS3TokenizerDescriptor.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = FTS3TokenizerDescriptor.swift; sourceTree = "<group>"; };
Expand Down Expand Up @@ -1028,6 +1033,7 @@
56FF453F1D2C23BA00F21EF9 /* MutablePersistableDeleteTests.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = MutablePersistableDeleteTests.swift; sourceTree = "<group>"; };
56FF45551D2CDA5200F21EF9 /* RecordUniqueIndexTests.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = RecordUniqueIndexTests.swift; sourceTree = "<group>"; };
6340BF7F1E5E3F7900832805 /* RecordPersistenceConflictPolicy.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = RecordPersistenceConflictPolicy.swift; sourceTree = "<group>"; };
C96C0F242084A442006B2981 /* SQLiteDateParser.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = SQLiteDateParser.swift; sourceTree = "<group>"; };
DC2393C61ABE35F8003FF113 /* GRDB-Bridging.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = "GRDB-Bridging.h"; sourceTree = "<group>"; };
DC3773F319C8CBB3004FCF85 /* GRDB.framework */ = {isa = PBXFileReference; explicitFileType = wrapper.framework; includeInIndex = 0; path = GRDB.framework; sourceTree = BUILT_PRODUCTS_DIR; };
DC3773F719C8CBB3004FCF85 /* Info.plist */ = {isa = PBXFileReference; lastKnownFileType = text.plist.xml; path = Info.plist; sourceTree = "<group>"; };
Expand Down Expand Up @@ -1135,6 +1141,7 @@
5605F1521C672E4000235C62 /* NSString.swift */,
5657AB0E1D10899D006283EF /* URL.swift */,
56A8C22F1D1914540096E9D4 /* UUID.swift */,
C96C0F242084A442006B2981 /* SQLiteDateParser.swift */,
);
path = Foundation;
sourceTree = "<group>";
Expand Down Expand Up @@ -1569,6 +1576,7 @@
56DE7B361C42BBBB00861EB8 /* PerformanceCoreDataTests.sqlite */,
56DE7B2D1C42B23B00861EB8 /* PerformanceModel.xcdatamodeld */,
563363F81C960711000BE133 /* PerformanceRealmTests.realm */,
567071FA208A509C006AD95A /* DateParsingTests.swift */,
56DE7B271C41302500861EB8 /* FetchNamedValuesTests.swift */,
56DE7B291C4130AF00861EB8 /* FetchPositionalValuesTests.swift */,
56DE7B2B1C41311900861EB8 /* FetchRecordClassTests.swift */,
Expand Down Expand Up @@ -2131,6 +2139,7 @@
files = (
56D3BE711F4EB1A00034C6D2 /* FetchRecordStructTests.swift in Sources */,
560C98231C0E23BB00BF8471 /* InsertRecordClassTests.swift in Sources */,
56707201208A509C006AD95A /* DateParsingTests.swift in Sources */,
56DE7B2C1C41311900861EB8 /* FetchRecordClassTests.swift in Sources */,
56DE7B281C41302500861EB8 /* FetchNamedValuesTests.swift in Sources */,
56D507831F6D7B2E00AE1C5B /* InsertRecordStructTests.swift in Sources */,
Expand Down Expand Up @@ -2195,6 +2204,7 @@
565490D91D5AE252005622CB /* Migration.swift in Sources */,
56CEB5001EAA2F4D00BFAF62 /* FTS3.swift in Sources */,
5698AD271DABAEFA0056AF8C /* FTS5WrapperTokenizer.swift in Sources */,
C96C0F2D2084A45A006B2981 /* SQLiteDateParser.swift in Sources */,
566A841C2041146100E50BFD /* DatabaseSnapshot.swift in Sources */,
565490D51D5AE252005622CB /* DatabaseValueConvertible+RawRepresentable.swift in Sources */,
565490C51D5AE236005622CB /* SerializedDatabase.swift in Sources */,
Expand Down Expand Up @@ -2360,6 +2370,7 @@
56D1215D1ED34978001347D2 /* Fixits-0.109.0.swift in Sources */,
566AD8B51D5318F4002EC1A8 /* TableDefinition.swift in Sources */,
5698AD241DABAEFA0056AF8C /* FTS5WrapperTokenizer.swift in Sources */,
C96C0F2C2084A459006B2981 /* SQLiteDateParser.swift in Sources */,
5605F1681C672E4000235C62 /* NSNumber.swift in Sources */,
56CEB5041EAA2F4D00BFAF62 /* FTS4.swift in Sources */,
5605F1741C672E4000235C62 /* StandardLibrary.swift in Sources */,
Expand Down Expand Up @@ -2762,6 +2773,7 @@
5698AD211DABAEFA0056AF8C /* FTS5WrapperTokenizer.swift in Sources */,
56A238831B9C75030082EB20 /* DatabaseQueue.swift in Sources */,
5605F1671C672E4000235C62 /* NSNumber.swift in Sources */,
C96C0F2B2084A442006B2981 /* SQLiteDateParser.swift in Sources */,
56A238871B9C75030082EB20 /* Row.swift in Sources */,
5605F1731C672E4000235C62 /* StandardLibrary.swift in Sources */,
567517C820131B220094BF41 /* RecordBox.swift in Sources */,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
buildConfiguration = "Release"
selectedDebuggerIdentifier = "Xcode.DebuggerFoundation.Debugger.LLDB"
selectedLauncherIdentifier = "Xcode.DebuggerFoundation.Launcher.LLDB"
language = ""
shouldUseLaunchSchemeArgsEnv = "YES">
<Testables>
<TestableReference
Expand All @@ -30,6 +31,7 @@
buildConfiguration = "Debug"
selectedDebuggerIdentifier = "Xcode.DebuggerFoundation.Debugger.LLDB"
selectedLauncherIdentifier = "Xcode.DebuggerFoundation.Launcher.LLDB"
language = ""
launchStyle = "0"
useCustomWorkingDirectory = "NO"
ignoresPersistentStateOnLaunch = "NO"
Expand Down
179 changes: 24 additions & 155 deletions GRDB/Core/Support/Foundation/DatabaseDateComponents.swift
Original file line number Diff line number Diff line change
@@ -1,8 +1,13 @@
import Foundation
#if SWIFT_PACKAGE
import CSQLite
#elseif !GRDBCUSTOMSQLITE && !GRDBCIPHER
import SQLite3
#endif

/// DatabaseDateComponents reads and stores DateComponents in the database.
public struct DatabaseDateComponents : DatabaseValueConvertible {
public struct DatabaseDateComponents : DatabaseValueConvertible, StatementColumnConvertible {

/// The available formats for reading and storing date components.
public enum Format : String {

Expand Down Expand Up @@ -68,6 +73,22 @@ public struct DatabaseDateComponents : DatabaseValueConvertible {
self.dateComponents = dateComponents
}

// MARK: - StatementColumnConvertible adoption

/// Returns a value initialized from a raw SQLite statement pointer.
///
/// - parameters:
/// - sqliteStatement: A pointer to an SQLite statement.
/// - index: The column index.
public init(sqliteStatement: SQLiteStatement, index: Int32) {
let cString = UnsafePointer<Int8>(OpaquePointer(sqlite3_column_text(sqliteStatement, index)!))
let length = Int(sqlite3_column_bytes(sqliteStatement, index)) // avoid an strlen
guard let components = SQLiteDateParser().components(cString: cString, length: length) else {
fatalError("could not convert database value \(String(cString: cString)) to DatabaseDateComponents")
}
self.dateComponents = components.dateComponents
self.format = components.format
}

// MARK: - DatabaseValueConvertible adoption

Expand Down Expand Up @@ -131,158 +152,6 @@ public struct DatabaseDateComponents : DatabaseValueConvertible {
return nil
}

var dateComponents = DateComponents()
let scanner = Scanner(string: string)
scanner.charactersToBeSkipped = CharacterSet()

let hasDate: Bool

// YYYY or HH
var initialNumber: Int = 0
if !scanner.scanInt(&initialNumber) {
return nil
}
switch scanner.scanLocation {
case 2:
// HH
hasDate = false

let hour = initialNumber
if hour >= 0 && hour <= 23 {
dateComponents.hour = hour
} else {
return nil
}

case 4:
// YYYY
hasDate = true

let year = initialNumber
if year >= 0 && year <= 9999 {
dateComponents.year = year
} else {
return nil
}

// -
if !scanner.scanString("-", into: nil) {
return nil
}

// MM
var month: Int = 0
if scanner.scanInt(&month) && month >= 1 && month <= 12 {
dateComponents.month = month
} else {
return nil
}

// -
if !scanner.scanString("-", into: nil) {
return nil
}

// DD
var day: Int = 0
if scanner.scanInt(&day) && day >= 1 && day <= 31 {
dateComponents.day = day
} else {
return nil
}

// YYYY-MM-DD
if scanner.isAtEnd {
return DatabaseDateComponents(dateComponents, format: .YMD)
}

// T/space
if !scanner.scanString("T", into: nil) && !scanner.scanString(" ", into: nil) {
return nil
}

// HH
var hour: Int = 0
if scanner.scanInt(&hour) && hour >= 0 && hour <= 23 {
dateComponents.hour = hour
} else {
return nil
}

default:
return nil
}

// :
if !scanner.scanString(":", into: nil) {
return nil
}

// MM
var minute: Int = 0
if scanner.scanInt(&minute) && minute >= 0 && minute <= 59 {
dateComponents.minute = minute
} else {
return nil
}

// [YYYY-MM-DD] HH:MM
if scanner.isAtEnd {
if hasDate {
return DatabaseDateComponents(dateComponents, format: .YMD_HM)
} else {
return DatabaseDateComponents(dateComponents, format: .HM)
}
}

// :
if !scanner.scanString(":", into: nil) {
return nil
}

// SS
var second: Int = 0
if scanner.scanInt(&second) && second >= 0 && second <= 59 {
dateComponents.second = second
} else {
return nil
}

// [YYYY-MM-DD] HH:MM:SS
if scanner.isAtEnd {
if hasDate {
return DatabaseDateComponents(dateComponents, format: .YMD_HMS)
} else {
return DatabaseDateComponents(dateComponents, format: .HMS)
}
}

// .
if !scanner.scanString(".", into: nil) {
return nil
}

// SSS
var millisecondDigits: NSString? = nil
if scanner.scanCharacters(from: .decimalDigits, into: &millisecondDigits), var millisecondDigits = millisecondDigits {
if millisecondDigits.length > 3 {
millisecondDigits = NSString(string: millisecondDigits.substring(to: 3))
}
dateComponents.nanosecond = millisecondDigits.integerValue * 1_000_000
} else {
return nil
}

// [YYYY-MM-DD] HH:MM:SS.SSS
if scanner.isAtEnd {
if hasDate {
return DatabaseDateComponents(dateComponents, format: .YMD_HMSS)
} else {
return DatabaseDateComponents(dateComponents, format: .HMSS)
}
}

// Unknown format
return nil
return SQLiteDateParser().components(from: string)
}
}
30 changes: 30 additions & 0 deletions GRDB/Core/Support/Foundation/Date.swift
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
import Foundation
#if SWIFT_PACKAGE
import CSQLite
#elseif !GRDBCUSTOMSQLITE && !GRDBCIPHER
import SQLite3
#endif

/// NSDate is stored in the database using the format
/// "yyyy-MM-dd HH:mm:ss.SSS", in the UTC time zone.
Expand Down Expand Up @@ -108,6 +113,31 @@ extension Date : DatabaseValueConvertible {
}
}

extension Date: StatementColumnConvertible {

/// Returns a value initialized from a raw SQLite statement pointer.
///
/// - parameters:
/// - sqliteStatement: A pointer to an SQLite statement.
/// - index: The column index.
public init(sqliteStatement: SQLiteStatement, index: Int32) {
switch sqlite3_column_type(sqliteStatement, index) {
case SQLITE_INTEGER, SQLITE_FLOAT:
self.init(timeIntervalSince1970: sqlite3_column_double(sqliteStatement, Int32(index)))
case SQLITE_TEXT:
let databaseDateComponents = DatabaseDateComponents(sqliteStatement: sqliteStatement, index: index)
guard let date = Date(databaseDateComponents: databaseDateComponents) else {
let dbValue = DatabaseValue(sqliteStatement: sqliteStatement, index: index)
fatalError("could not convert database value \(dbValue) to Date")
}
self.init(timeIntervalSinceReferenceDate: date.timeIntervalSinceReferenceDate)
default:
let dbValue = DatabaseValue(sqliteStatement: sqliteStatement, index: index)
fatalError("could not convert database value \(dbValue) to Date")
}
}
}

/// The DatabaseDate date formatter for stored dates.
private let storageDateFormatter: DateFormatter = {
let formatter = DateFormatter()
Expand Down
Loading