Implement Firebase Analytics #501

barbeau · 2016-05-19T17:55:55Z

Google announced the expansion of the Firebase platform yesterday:
https://firebase.googleblog.com/2016/05/firebase-expands-to-become-unified-app-platform.html

We should take a close look at analytics:
https://firebase.google.com/docs/analytics/

Firebase Analytics provides unlimited reporting on up to 500 distinct events.

This could potentially help us with our major Google Analytics problem, which is the limit on reported events - we've been around 10x over the limit for some time, and I'm not sure if/how that is affecting our data.

Looks like you can implement Google Analytics and Firebase Analytics side-by-side:
https://support.google.com/analytics/answer/2587086#firebase-ga

We'll probably want to do this and confirm that Firebase seems to work before changing anything with Google Analytics.

We'll also need to make sure there is a way to use Firebase on devices that don't support Google Play Services, such as the Amazon Fire Phone. Currently, with Google Analytics, it uses a Google Play Services-based implementation if Google Play Services is available on the device, and if not it uses a bundled Google Analytics JAR library.

barbeau · 2016-05-19T17:56:06Z

cc @cagryInside

barbeau · 2016-05-23T14:21:46Z

This looks like a good initial starting place for setting up Firebase in an Android project:
https://firebase.google.com/docs/android/setup#prerequisites

barbeau · 2018-09-15T01:25:40Z

Firebase impl on iOS is in OneBusAway/onebusaway-iphone#1366

barbeau · 2018-10-08T19:31:19Z

I did a quick analysis on our do_not_translate.xml file, and we have 95 analytics message types. I've collapsed these to one line each in this spreadsheet to count:
https://docs.google.com/spreadsheets/d/1kKK55kegmUzCKzJjvepAKZ6qNQ96S1GsrfoaID7yshU/edit?usp=sharing

@aaronbrethorst Could you do a quick count on iOS?

If I'm interpretting the Firebase Analytics docs right, and our Google Analytics labels, it looks like we should be far below the 500 distinct free events threshold for Firebase Analytics free tier.

barbeau · 2018-10-08T21:32:50Z

Just talked to @aaronbrethorst, and good news - OBA iOS has already been converted to Firebase Analytics as part of some recent work. This also gives us a template to start with on Android. @aaronbrethorst is also open to changing some iOS events if it makes it easier to align on events across platforms.

iOS details as of Oct 8th 2018 are below.

Main Firebase Events (here in code) are:

NSString * const OBAAnalyticsEventTripProblemReported = @"tripProblemReported";
NSString * const OBAAnalyticsStartupScreen = @"startupScreen";
NSString * const OBAAnalyticsSearchPerformed = @"searchPerformed";
NSString * const OBAAnalyticsServiceAlertTapped = @"serviceAlertTapped";
NSString * const OBAAnalyticsEventInfoRowTapped = @"infoRowTapped";

Each of these events have key/value pairs attached to them for the data related to that event.

For example:

tripProblemPreported has parameter/values @{@"stopID": self.currentStopId, @"vehicleNumber": problem.userVehicleNumber, @"problem": problem.code};
startupScreen has parameter startingTab with a value for the starting tab name
searchPerformed has parameter/value "searchType": NSStringFromOBASearchType(target.searchType) ?? "Unknown"])
serviceAlertTapped has parameter/values "agencyName": agency.name, @"situationID": situation.situationId}
infoRowTapped event - the view controller has row as the parameter and Privacy Policy, Dark Sky Attribution, Credits, Contact Developers, etc. for the value to specify which row was tapped.

If you want to see detailed iOS code, here is an example of a tripProblemReported event reported with several key/value pairs:

        [OBAAnalytics.sharedInstance reportEventWithCategory:OBAAnalyticsCategorySubmit action:@"report_problem" label:@"Reported Problem" value:nil];

        NSDictionary *params = @{@"stopID": self.currentStopId, @"vehicleNumber": problem.userVehicleNumber, @"problem": problem.code};
        [FIRAnalytics logEventWithName:OBAAnalyticsEventTripProblemReported parameters:params];

Here is the search for all occurences of FIRAnalytics in OBA iOS:

barbeau · 2018-10-08T21:35:03Z

Also, to clarify - OBA iOS has Google Analytics and Firebase Analytics running in parallel right now. I plan to do the same with Android so we have the two running in parallel. After the December 3rd deadline for Google Analytics we can switch to sampling so GA continues to log events.

barbeau · 2018-10-08T22:07:09Z

@aaronbrethorst Are you currently capturing the region of the user in iOS Firebase analytics? I know agencies have always obviously wanted this attribute.

I'm wondering if region should be logged as a "user property":
https://firebase.google.com/docs/analytics/android/properties#set_user_properties

Also, from some quick Googling it doesn't appear that Firebase Analytics has any "geo" properties. We could log lat/long as key value pairs for events where it makes sense, but from a glance it doesn't appear that the console has any special geo features to take advantage of that.

aaronbrethorst · 2018-10-30T16:27:34Z

Yes, I'm capturing the user's region (by name) as a user property: https://github.com/OneBusAway/onebusaway-iphone/blob/develop/OneBusAway/utilities/OBAAnalytics.m#L72

barbeau · 2018-11-08T15:59:09Z

As part of this migration, we need to update Application.getAppUid() to first try to use FirebaseInstanceId (https://firebase.google.com/docs/reference/android/com/google/firebase/iid/FirebaseInstanceId), and then if FirebaseInstanceId isn't available (catch all exceptions), then return randomUUID().

barbeau · 2018-11-27T21:11:59Z

Events defined by Firebase in the Android SDK are here:
https://firebase.google.com/docs/reference/android/com/google/firebase/analytics/FirebaseAnalytics.Event

barbeau · 2018-11-27T21:15:04Z

Here's the new versioning scheme for Google Play Services libraries on Android:
https://android-developers.googleblog.com/2018/05/announcing-new-sdk-versioning.html

As of version 15 of Play Services and Firebase, we can (and need to) get rid of our lock-step versioning - each library will be versioned independently.

barbeau · 2018-11-27T23:01:57Z

I'm getting this stack trace when trying to remove all the force-updating of dependencies in libraries and just using the most recent Firebase and Play Services dependencies:

2018-11-27 17:46:08.706 8900-9215/com.joulespersecond.seattlebusbot E/AndroidRuntime: FATAL EXCEPTION: intent processor #2
    Process: com.joulespersecond.seattlebusbot, PID: 8900
    java.lang.NoClassDefFoundError: Failed resolution of: Lcom/google/android/gms/common/util/zzv;
        at com.google.android.gms.iid.zzo.<init>(Unknown Source:41)
        at com.google.android.gms.iid.zzo.<init>(Unknown Source:2)
        at com.google.android.gms.iid.InstanceID.getInstance(Unknown Source:30)
        at com.google.android.gms.iid.InstanceID.getInstance(Unknown Source:1)
        at com.microsoft.embeddedsocial.service.handler.GetGcmIdHandler.handleIntent(GetGcmIdHandler.java:40)
        at com.microsoft.embeddedsocial.service.handler.GetGcmIdHandler.handleIntent(GetGcmIdHandler.java:26)
        at com.microsoft.embeddedsocial.base.service.IntentProcessor$IntentHandlerAdapter.run(IntentProcessor.java:234)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1162)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:636)
        at java.lang.Thread.run(Thread.java:764)
     Caused by: java.lang.ClassNotFoundException: Didn't find class "com.google.android.gms.common.util.zzv" on path: DexPathList[[zip file "/data/app/com.joulespersecond.seattlebusbot-WfalBStfX0vbwu7NVByN0Q==/base.apk", zip file "/data/app/com.joulespersecond.seattlebusbot-WfalBStfX0vbwu7NVByN0Q==/split_lib_dependencies_apk.apk", zip file "/data/app/com.joulespersecond.seattlebusbot-WfalBStfX0vbwu7NVByN0Q==/split_lib_resources_apk.apk", zip file "/data/app/com.joulespersecond.seattlebusbot-WfalBStfX0vbwu7NVByN0Q==/split_lib_slice_0_apk.apk", zip file "/data/app/com.joulespersecond.seattlebusbot-WfalBStfX0vbwu7NVByN0Q==/split_lib_slice_1_apk.apk", zip file "/data/app/com.joulespersecond.seattlebusbot-WfalBStfX0vbwu7NVByN0Q==/split_lib_slice_2_apk.apk", zip file "/data/app/com.joulespersecond.seattlebusbot-WfalBStfX0vbwu7NVByN0Q==/split_lib_slice_3_apk.apk", zip file "/data/app/com.joulespersecond.seattlebusbot-WfalBStfX0vbwu7NVByN0Q==/split_lib_slice_4_apk.apk", zip file "/data/app/com.joulespersecond.seattlebusbot-WfalBStfX0vbwu7NVByN0Q==/split_lib_slice_5_apk.apk", zip file "/data/app/com.joulespersecond.seattlebusbot-WfalBStfX0vbwu7NVByN0Q==/split_lib_slice_6_apk.apk", zip file "/data/app/com.joulespersecond.seattlebusbot-WfalBStfX0vbwu7NVByN0Q==/split_lib_slice_7_apk.apk", zip file "/data/app/com.joulespersecond.seattlebusbot-WfalBStfX0vbwu7NVByN0Q==/split_lib_slice_8_apk.apk", zip file "/data/app/com.joulespersecond.seattlebusbot-WfalBStfX0vbwu7NVByN0Q==/split_lib_slice_9_apk.apk"],nativeLibraryDirectories=[/data/app/com.joulespersecond.seattlebusbot-WfalBStfX0vbwu7NVByN0Q==/lib/arm64, /system/lib64, /system/vendor/lib64]]
        at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:93)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:379)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:312)
        at com.google.android.gms.iid.zzo.<init>(Unknown Source:41) 
        at com.google.android.gms.iid.zzo.<init>(Unknown Source:2) 
        at com.google.android.gms.iid.InstanceID.getInstance(Unknown Source:30) 
        at com.google.android.gms.iid.InstanceID.getInstance(Unknown Source:1) 
        at com.microsoft.embeddedsocial.service.handler.GetGcmIdHandler.handleIntent(GetGcmIdHandler.java:40) 
        at com.microsoft.embeddedsocial.service.handler.GetGcmIdHandler.handleIntent(GetGcmIdHandler.java:26) 
        at com.microsoft.embeddedsocial.base.service.IntentProcessor$IntentHandlerAdapter.run(IntentProcessor.java:234) 
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1162) 
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:636) 
        at java.lang.Thread.run(Thread.java:764) 
    	Suppressed: java.io.IOException: No original dex files found for dex location (arm64) /data/app/com.joulespersecond.seattlebusbot-WfalBStfX0vbwu7NVByN0Q==/split_lib_resources_apk.apk
        at dalvik.system.DexFile.openDexFileNative(Native Method)
        at dalvik.system.DexFile.openDexFile(DexFile.java:353)
        at dalvik.system.DexFile.<init>(DexFile.java:100)
        at dalvik.system.DexFile.<init>(DexFile.java:74)
        at dalvik.system.DexPathList.loadDexFile(DexPathList.java:374)
        at dalvik.system.DexPathList.makeDexElements(DexPathList.java:337)
        at dalvik.system.DexPathList.<init>(DexPathList.java:157)
        at dalvik.system.BaseDexClassLoader.<init>(BaseDexClassLoader.java:65)
        at dalvik.system.PathClassLoader.<init>(PathClassLoader.java:64)
        at com.android.internal.os.PathClassLoaderFactory.createClassLoader(PathClassLoaderFactory.java:43)
        at android.app.ApplicationLoaders.getClassLoader(ApplicationLoaders.java:69)
        at android.app.ApplicationLoaders.getClassLoader(ApplicationLoaders.java:36)
        at android.app.LoadedApk.createOrUpdateClassLoaderLocked(LoadedApk.java:678)
        at android.app.LoadedApk.getClassLoader(LoadedApk.java:711)
2018-11-27 17:46:08.708 8900-9215/com.joulespersecond.seattlebusbot E/AndroidRuntime:     at android.app.LoadedApk.getResources(LoadedApk.java:944)
        at android.app.ContextImpl.createAppContext(ContextImpl.java:2303)
        at android.app.ActivityThread.handleBindApplication(ActivityThread.java:5937)
        at android.app.ActivityThread.-wrap1(Unknown Source:0)
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1764)
        at android.os.Handler.dispatchMessage(Handler.java:105)
        at android.os.Looper.loop(Looper.java:164)
        at android.app.ActivityThread.main(ActivityThread.java:6938)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.Zygote$MethodAndArgsCaller.run(Zygote.java:327)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1374)

Looks like it's a dependency conflict with ES SDK, in Google Play Services cloud messaging, in that the class that's needed by the ES SDK is missing.

barbeau · 2018-11-27T23:13:03Z

If I downgrade play-services-analytics to 11.8.0 to match the ES SDK (but keep Firebase at 16), I get this stack trace on startup:

    Process: com.joulespersecond.seattlebusbot, PID: 16121
    java.lang.NoClassDefFoundError: Failed resolution of: Lcom/google/android/gms/common/internal/zzbq;
        at com.google.android.gms.internal.zzaqc.zzbm(Unknown Source:0)
        at com.google.android.gms.analytics.GoogleAnalytics.getInstance(Unknown Source:0)
        at org.onebusaway.android.io.ObaAnalytics.initAnalytics(ObaAnalytics.java:206)
        at org.onebusaway.android.app.Application.onCreate(Application.java:120)
        at android.app.Instrumentation.callApplicationOnCreate(Instrumentation.java:1125)
        at android.app.ActivityThread.handleBindApplication(ActivityThread.java:6056)
        at android.app.ActivityThread.-wrap1(Unknown Source:0)
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1764)
        at android.os.Handler.dispatchMessage(Handler.java:105)
        at android.os.Looper.loop(Looper.java:164)
        at android.app.ActivityThread.main(ActivityThread.java:6938)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.Zygote$MethodAndArgsCaller.run(Zygote.java:327)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1374)
     Caused by: java.lang.ClassNotFoundException: Didn't find class "com.google.android.gms.common.internal.zzbq" on path: DexPathList[[zip file "/data/app/com.joulespersecond.seattlebusbot-P5TXJaEdt3C1koI3SuuwUw==/base.apk", zip file "/data/app/com.joulespersecond.seattlebusbot-P5TXJaEdt3C1koI3SuuwUw==/split_lib_dependencies_apk.apk", zip file "/data/app/com.joulespersecond.seattlebusbot-P5TXJaEdt3C1koI3SuuwUw==/split_lib_resources_apk.apk", zip file "/data/app/com.joulespersecond.seattlebusbot-P5TXJaEdt3C1koI3SuuwUw==/split_lib_slice_0_apk.apk", zip file "/data/app/com.joulespersecond.seattlebusbot-P5TXJaEdt3C1koI3SuuwUw==/split_lib_slice_1_apk.apk", zip file "/data/app/com.joulespersecond.seattlebusbot-P5TXJaEdt3C1koI3SuuwUw==/split_lib_slice_2_apk.apk", zip file "/data/app/com.joulespersecond.seattlebusbot-P5TXJaEdt3C1koI3SuuwUw==/split_lib_slice_3_apk.apk", zip file "/data/app/com.joulespersecond.seattlebusbot-P5TXJaEdt3C1koI3SuuwUw==/split_lib_slice_4_apk.apk", zip file "/data/app/com.joulespersecond.seattlebusbot-P5TXJaEdt3C1koI3SuuwUw==/split_lib_slice_5_apk.apk", zip file "/data/app/com.joulespersecond.seattlebusbot-P5TXJaEdt3C1koI3SuuwUw==/split_lib_slice_6_apk.apk", zip file "/data/app/com.joulespersecond.seattlebusbot-P5TXJaEdt3C1koI3SuuwUw==/split_lib_slice_7_apk.apk", zip file "/data/app/com.joulespersecond.seattlebusbot-P5TXJaEdt3C1koI3SuuwUw==/split_lib_slice_8_apk.apk", zip file "/data/app/com.joulespersecond.seattlebusbot-P5TXJaEdt3C1koI3SuuwUw==/split_lib_slice_9_apk.apk"],nativeLibraryDirectories=[/data/app/com.joulespersecond.seattlebusbot-P5TXJaEdt3C1koI3SuuwUw==/lib/arm64, /system/lib64, /system/vendor/lib64]]
        at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:93)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:379)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:312)
        at com.google.android.gms.internal.zzaqc.zzbm(Unknown Source:0) 
        at com.google.android.gms.analytics.GoogleAnalytics.getInstance(Unknown Source:0) 
        at org.onebusaway.android.io.ObaAnalytics.initAnalytics(ObaAnalytics.java:206) 
        at org.onebusaway.android.app.Application.onCreate(Application.java:120) 
        at android.app.Instrumentation.callApplicationOnCreate(Instrumentation.java:1125) 
        at android.app.ActivityThread.handleBindApplication(ActivityThread.java:6056) 
        at android.app.ActivityThread.-wrap1(Unknown Source:0) 
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1764) 
        at android.os.Handler.dispatchMessage(Handler.java:105) 
        at android.os.Looper.loop(Looper.java:164) 
        at android.app.ActivityThread.main(ActivityThread.java:6938) 
        at java.lang.reflect.Method.invoke(Native Method) 
        at com.android.internal.os.Zygote$MethodAndArgsCaller.run(Zygote.java:327) 
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1374) 
    	Suppressed: java.io.IOException: No original dex files found for dex location (arm64) /data/app/com.joulespersecond.seattlebusbot-P5TXJaEdt3C1koI3SuuwUw==/split_lib_resources_apk.apk
        at dalvik.system.DexFile.openDexFileNative(Native Method)
        at dalvik.system.DexFile.openDexFile(DexFile.java:353)
        at dalvik.system.DexFile.<init>(DexFile.java:100)
        at dalvik.system.DexFile.<init>(DexFile.java:74)
        at dalvik.system.DexPathList.loadDexFile(DexPathList.java:374)
        at dalvik.system.DexPathList.makeDexElements(DexPathList.java:337)
        at dalvik.system.DexPathList.<init>(DexPathList.java:157)
        at dalvik.system.BaseDexClassLoader.<init>(BaseDexClassLoader.java:65)
        at dalvik.system.PathClassLoader.<init>(PathClassLoader.java:64)
        at com.android.internal.os.PathClassLoaderFactory.createClassLoader(PathClassLoaderFactory.java:43)
        at android.app.ApplicationLoaders.getClassLoader(ApplicationLoaders.java:69)
        at android.app.ApplicationLoaders.getClassLoader(ApplicationLoaders.java:36)
2018-11-27 18:10:41.849 16121-16121/com.joulespersecond.seattlebusbot E/AndroidRuntime:     at android.app.LoadedApk.createOrUpdateClassLoaderLocked(LoadedApk.java:678)
        at android.app.LoadedApk.getClassLoader(LoadedApk.java:711)
        at android.app.LoadedApk.getResources(LoadedApk.java:944)
        at android.app.ContextImpl.createAppContext(ContextImpl.java:2303)
        at android.app.ActivityThread.handleBindApplication(ActivityThread.java:5937)

* Revert to old Google Play Services for now, as we're having issues mixing ES SDK version and Play Services 15 and 16 (see #501 (comment)) * Remove requirement to force compileSdkVersion, as that's no longer needed

barbeau added the enhancement label May 19, 2016

barbeau added this to the v2.1 milestone May 19, 2016

barbeau mentioned this issue May 27, 2016

Google Analytics Data Volume Exceeded Warning #336

Closed

barbeau modified the milestones: v2.2, v2.1 Jun 23, 2016

barbeau modified the milestones: v2.4, v2.3.x Nov 27, 2018

barbeau mentioned this issue Nov 28, 2018

Fix #501 - Implement Firebase Analytics #936

Merged

12 tasks

barbeau closed this as completed in 8b8df8c Mar 25, 2019

barbeau mentioned this issue Aug 6, 2021

Match OBA Android Firebase Analytics events and user properties OneBusAway/onebusaway-ios#457

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement Firebase Analytics #501

Implement Firebase Analytics #501

barbeau commented May 19, 2016

barbeau commented May 19, 2016

barbeau commented May 23, 2016

barbeau commented Sep 15, 2018

barbeau commented Oct 8, 2018

barbeau commented Oct 8, 2018 •

edited

Loading

barbeau commented Oct 8, 2018

barbeau commented Oct 8, 2018

aaronbrethorst commented Oct 30, 2018

barbeau commented Nov 8, 2018

barbeau commented Nov 27, 2018

barbeau commented Nov 27, 2018 •

edited

Loading

barbeau commented Nov 27, 2018 •

edited

Loading

barbeau commented Nov 27, 2018 •

edited

Loading

Implement Firebase Analytics #501

Implement Firebase Analytics #501

Comments

barbeau commented May 19, 2016

barbeau commented May 19, 2016

barbeau commented May 23, 2016

barbeau commented Sep 15, 2018

barbeau commented Oct 8, 2018

barbeau commented Oct 8, 2018 • edited Loading

barbeau commented Oct 8, 2018

barbeau commented Oct 8, 2018

aaronbrethorst commented Oct 30, 2018

barbeau commented Nov 8, 2018

barbeau commented Nov 27, 2018

barbeau commented Nov 27, 2018 • edited Loading

barbeau commented Nov 27, 2018 • edited Loading

barbeau commented Nov 27, 2018 • edited Loading

barbeau commented Oct 8, 2018 •

edited

Loading

barbeau commented Nov 27, 2018 •

edited

Loading

barbeau commented Nov 27, 2018 •

edited

Loading

barbeau commented Nov 27, 2018 •

edited

Loading