Export Firebase Crashlytics data to BigQuery

Stay organized with collections Save and categorize content based on your preferences.

You can export your Firebase Crashlytics data into BigQuery for further analysis. BigQuery lets you analyze the data using BigQuery SQL, export it to another cloud provider, and use it for visualization and custom dashboards with Google Data Studio.

Enable export to BigQuery

1. In the Firebase console, go to the Integrations page.

2. In the BigQuery card, click Link.

3. Follow the on-screen instructions to enable export to BigQuery.

   If you want near real-time access to your Crashlytics data in BigQuery, then consider upgrading to streaming export.

What happens when you enable export?

• You select the dataset location. After the dataset is created, the location can't be changed, but you can copy the dataset to a different location or manually move (recreate) the dataset in a different location. To learn more, see Change the location for existing exports.

  This location is only applicable for the data exported into BigQuery, and it does not impact the location of data stored for use in the Crashlytics dashboard of the Firebase console or in Android Studio.

• By default, all apps in your project are linked to BigQuery and any apps that you later add to the project are automatically linked to BigQuery. You can manage which apps send data.

• Firebase sets up daily syncs of your data to BigQuery.

  • After you link your project, you usually need to wait until the next day's sync for your first set of data to be exported to BigQuery.

  • The daily sync happens once per day, regardless of any scheduled export that you might have set up in BigQuery. Note that the timing and duration of the sync job can change, so we don't recommend scheduling downstream operations or jobs based on a specific timing of the export.

• Firebase exports a copy of your existing data to BigQuery. The initial propagation of data for export may take up to 48 hours.

  • For each linked app, this export includes a batch table containing the data from the daily sync.

  • You can manually schedule data backfills for the batch table up to the past 30 days or for the most recent date when you enabled export to BigQuery (whichever is most recent).

  Note that if you enabled export of Crashlytics data before mid-October 2024, you can also backfill 30 days prior to the day you enabled export.

• If you enable Crashlytics streaming export to BigQuery, all linked apps will also have a realtime table containing constantly updating data.

To deactivate export to BigQuery, unlink your project in the Firebase console.

What data is exported to BigQuery?

Firebase Crashlytics data is exported into a BigQuery dataset named firebase_crashlytics. By default, individual tables will be created inside the Crashlytics dataset for each app in your project. Firebase names the tables based on the app's identifier, with periods converted to underscores, and a platform name appended to the end.

For example, data for an Android app with the package name com.google.test would be in a table named com_google_test_ANDROID. This batch table is updated once every day. If you enable Crashlytics streaming export to BigQuery, then Crashlytics data will also be streamed in realtime to a table named com_google_test_ANDROID_REALTIME.

Each row in a table represents an event that occurred in the app, including crashes, non-fatal errors, and ANRs.

Crashlytics streaming export to BigQuery

You can stream your Crashlytics data in realtime with BigQuery streaming. You can use it for any purpose that requires live data, such as presenting information in a live dashboard, watching a rollout live, or monitoring application problems that trigger alerts and custom workflows.

When you enable Crashlytics streaming export to BigQuery, in addition to the batch table you will also have a realtime table. Here are the differences you should be aware of between the tables:

The batch table is ideal for long-term analysis and identifying trends over time because we durably store events before writing them, and they can be backfilled to the table for up to 30 days*. When we write data to your realtime table, we immediately write it to BigQuery, and so it is ideal for live dashboards and custom alerts. These two tables can be combined with a stitching query to get the benefits of both.

By default, the realtime table has a partition expiration time of 30 days. To learn how to modify this, see Set the partition expiration in the BigQuery documentation.

^{* See details about backfill support in Upgrade to the new export infrastructure.}

Enable Crashlytics streaming export to BigQuery

1. In the Firebase console, go to the Integrations page.

2. In the BigQuery card, click Manage.

3. Select the Include streaming checkbox.

This action enables streaming for all of your linked apps.

1. Make sure that you've sent at least two events from your app to Crashlytics and waited a couple minutes after sending them.

2. Make sure your Firebase project is on the pay-as-you-go Blaze pricing plan.
   You can check this by looking in the bottom-left corner of the Firebase console.

3. If there's still no data in your realtime table after sending two events and waiting a couple minutes:

   1. Go to the BigQuery card in the Firebase console.

   2. Disable and then re-enable streaming export.

   3. Make sure the service account service-PROJECT_NUMBER@gcp-sa-crashlytics.iam.gserviceaccount.com is in your Firebase project and has the Firebase Crashlytics Service Agent role.
      You can check this in the IAM page of the Google Cloud console (make sure to select the checkbox for Include Google-provided role grants).

   4. Send at least two events to Crashlytics and wait a couple minutes.

4. If you still don't see data in your realtime table, reach out to Firebase Support.

What can you do with the exported data?

Exports to BigQuery contain raw crash data including device type, operating system, exceptions (Android apps) or errors (Apple apps), and Crashlytics logs, as well as other data.

Review exactly what Crashlytics data is exported and its table schema later in this page.

Use a Data Studio template

To enable realtime data in your Data Studio template, follow the instructions in Visualizing exported Crashlytics data with Data Studio.

Create views

You can transform queries into views using the BigQuery UI. For detailed instructions, see Create views in the BigQuery documentation.

Run queries

The following examples demonstrate queries that you can run on your Crashlytics data to generate reports that aggregate crash event data into more easily-understood summaries. Since these types of reports aren't available in the Crashlytics dashboard of the Firebase console, they can supplement your analysis and understanding of crash data.

Example 1: Crashes by day

After working to fix as many bugs as possible, you think your team is finally ready to launch your new photo-sharing app. Before you do, you want to check the number of crashes per day for the past month, to be sure your bug-bash made the app more stable over time.

Here's an example query for an Android app. For an iOS app, use its bundle ID and IOS (instead of package name and ANDROID).

SELECT
  COUNT(DISTINCT event_id) AS number_of_crashes,
  FORMAT_TIMESTAMP("%F", event_timestamp) AS date_of_crashes
FROM
 `PROJECT_ID.firebase_crashlytics.PACKAGE_NAME_ANDROID`
GROUP BY
  date_of_crashes
ORDER BY
  date_of_crashes DESC
LIMIT 30;

Example 2: Find the most pervasive crashes

To properly prioritize production plans, you want to find the top 10 most pervasive crashes in your app. You produce a query that provides the pertinent points of data.

Here's an example query for an Android app. For an iOS app, use its bundle ID and IOS (instead of package name and ANDROID).

SELECT
  DISTINCT issue_id,
  COUNT(DISTINCT event_id) AS number_of_crashes,
  COUNT(DISTINCT installation_uuid) AS number_of_impacted_user,
  blame_frame.file,
  blame_frame.line
FROM
  `PROJECT_ID.firebase_crashlytics.PACKAGE_NAME_ANDROID`
WHERE
  event_timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(),INTERVAL 168 HOUR)
  AND event_timestamp < CURRENT_TIMESTAMP()
GROUP BY
  issue_id,
  blame_frame.file,
  blame_frame.line
ORDER BY
  number_of_crashes DESC
LIMIT 10;

Example 3: Top 10 crashing devices

Fall is new phone season! Your company knows that this also means it's new device-specific issues season — especially for Android. To get ahead of the looming compatibility concerns, you put together a query that identifies the 10 devices that experienced the most crashes in the past week (168 hours).

Here's an example query for an Android app. For an iOS app, use its bundle ID and IOS (instead of package name and ANDROID).

SELECT
  device.model,
COUNT(DISTINCT event_id) AS number_of_crashes
FROM
  `PROJECT_ID.firebase_crashlytics.PACKAGE_NAME_ANDROID`
WHERE
  event_timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 168 HOUR)
  AND event_timestamp < CURRENT_TIMESTAMP()
GROUP BY
  device.model
ORDER BY
  number_of_crashes DESC
LIMIT 10;

Example 4: Filter by custom key

You're a game developer who wants to know which level of your game experiences the most crashes.

To help track that stat, you set a custom Crashlytics key called current_level, and update it every time the user reaches a new level.

Crashlytics.sharedInstance().setIntValue(3, forKey: "current_level");

CrashlyticsKit setIntValue:3 forKey:@"current_level";

Crashlytics.setInt("current_level", 3);

With that key in your export to BigQuery, you can then write a query to report the distribution of current_level values associated with each crash event.

Here's an example query for an Android app. For an iOS app, use its bundle ID and IOS (instead of package name and ANDROID).

SELECT
COUNT(DISTINCT event_id) AS num_of_crashes,
  value
FROM
  `PROJECT_ID.firebase_crashlytics.PACKAGE_NAME_ANDROID`
UNNEST(custom_keys)
WHERE
  key = "current_level"
GROUP BY
  key,
  value
ORDER BY
  num_of_crashes DESC

Example 5: User ID extraction

You have an Android app in early access. Most of your users love it, but three have experienced an unusual number of crashes. To get to the bottom of the problem, you write a query that pulls all the crash events for those users, using their user IDs.

Here's an example query for an Android app. For an iOS app, use its bundle ID and IOS (instead of package name and ANDROID).

SELECT *
FROM
  `PROJECT_ID.firebase_crashlytics.PACKAGE_NAME_ANDROID`
WHERE
  user.id IN ("USER_ID_1", "USER_ID_2", "USER_ID_3")
ORDER BY
  user.id
 

Example 6: Find all users facing a particular crash issue

Your team has accidentally released a critical bug to a group of beta testers. Your team was able to use the query from the "Find most pervasive crashes" example above to identify the specific crash issue ID. Now your team would like to run a query to extract the list of app users who were impacted by this crash.

Here's an example query for an Android app. For an iOS app, use its bundle ID and IOS (instead of package name and ANDROID).

SELECT user.id as user_id
FROM
  `PROJECT_ID.firebase_crashlytics.PACKAGE_NAME_ANDROID`
WHERE
  issue_id = "ISSUE_ID"
  AND application.display_version = "APP_VERSION"
  AND user.id != ""
ORDER BY
  user.id;

Example 7: Number of users impacted by a crash issue, broken down by country

Your team has detected a critical bug during the rollout of a new release. You were able to use the query from the "Find most pervasive crashes" example above to identify the specific crash issue ID. Your team would now like to see if this crash has spread to users in different countries around the world.

To write this query, your team will need to do the following:

1. Enable export of Google Analytics data to BigQuery. See Export project data to BigQuery.

2. Update your app to pass a user ID into both the Google Analytics SDK and the Crashlytics SDK.

   Swift

   Crashlytics.sharedInstance().setUserIdentifier("123456789");
   Analytics.setUserID("123456789");
   

   Objective-C

   CrashlyticsKit setUserIdentifier:@"123456789";
   FIRAnalytics setUserID:@"12345678 9";
   

   Java

   Crashlytics.setUserIdentifier("123456789");
   mFirebaseAnalytics.setUserId("123456789");
   

3. Write a query that uses the user ID field to join events in the Google Analytics dataset with crashes in the Crashlytics dataset.

   Here's an example query for an Android app. For an iOS app, use its bundle ID and IOS (instead of package name and ANDROID).

   SELECT DISTINCT c.issue_id, a.geo.country, COUNT(DISTINCT c.user.id) as num_users_impacted
   FROM `PROJECT_ID.firebase_crashlytics.PACKAGE_NAME_ANDROID` c
   INNER JOIN  `PROJECT_ID.analytics_TABLE_NAME.events_*` a on c.user.id = a.user_id
   WHERE
     c.issue_id = "ISSUE_ID"
     AND a._TABLE_SUFFIX BETWEEN '20190101'
     AND '20200101'
   GROUP BY
     c.issue_id,
     a.geo.country,
     c.user.id

Example 8: Top 5 issues so far today

Here's an example query for an Android app. For an iOS app, use its bundle ID and IOS (instead of package name and ANDROID).

SELECT
  issue_id,
  COUNT(DISTINCT event_id) AS events
FROM
  `PROJECT_ID.firebase_crashlytics.PACKAGE_NAME_ANDROID_REALTIME`
WHERE
  DATE(event_timestamp) = CURRENT_DATE()
GROUP BY
  issue_id
ORDER BY
  events DESC
LIMIT
  5;

Example 9: Top 5 issues since DATE, including today

You can also combine the batch and realtime tables with a stitching query to add realtime information to the reliable batch data. Since event_id is a primary key, you can use DISTINCT event_id to dedupe any common events from the two tables.

Here's an example query for an Android app. For an iOS app, use its bundle ID and IOS (instead of package name and ANDROID).

SELECT
  issue_id,
  COUNT(DISTINCT event_id) AS events
FROM (
  SELECT
    issue_id,
    event_id,
    event_timestamp
  FROM
    `PROJECT_ID.firebase_crashlytics.PACKAGE_NAME_ANDROID_REALTIME`
  UNION ALL
  SELECT
    issue_id,
    event_id,
    event_timestamp
  FROM
    `PROJECT_ID.firebase_crashlytics.PACKAGE_NAME_ANDROID`)
WHERE
  event_timestamp >= "YYYY_MM_DD"
GROUP BY
  issue_id
ORDER BY
  events DESC
LIMIT
  5;

Understand the Crashlytics schema in BigQuery

When you set up Crashlytics data export to BigQuery, Firebase exports recent events (crashes, non-fatal errors, and ANRs), including events from up to two days before the link, with the option to backfill up to 30 days.

From that point until you deactivate the export, Firebase exports Crashlytics events on a daily basis. It can take a few minutes for the data to be available in BigQuery after each export.

Datasets

Crashlytics creates a new dataset in BigQuery for Crashlytics data. The dataset covers your entire project, even if it has multiple apps.

Tables

Crashlytics creates a table in the dataset for each app in your project, unless you've opted out of exporting data for that app. Firebase names the tables based on the app's identifier, with periods converted to underscores, and a platform name appended to the end.

For example, data for an Android app with the package name com.google.test would be in a table named com_google_test_ANDROID, and realtime data (if enabled) would be in a table named com_google_test_ANDROID_REALTIME

Tables contain a standard set of Crashlytics data in addition to any custom Crashlytics keys defined by you in your app.

Rows

Each row in a table represents an error the app encountered.

Columns

The columns in a table are identical for crashes, non-fatal errors, and ANRs. If Crashlytics streaming export to BigQuery is enabled, then the realtime table will have the same columns as the batch table. Note that you may have columns in rows that represent events that don't have stack traces.

The columns within the export are listed in this table:

Visualize exported Crashlytics data with Data Studio

Google Data Studio turns your Crashlytics datasets in BigQuery into reports that are easier to read, easier to share, and fully customizable.

To learn more about using Data Studio, try the Data Studio quickstart guide, Welcome to Data Studio.

Use a Crashlytics report template

Data Studio has a sample report for Crashlytics that includes a comprehensive set of dimensions and metrics from the exported Crashlytics BigQuery schema. If you have enabled Crashlytics streaming export to BigQuery, then you can view that data on the Realtime trends page of the Data Studio template.You can use the sample as a template to quickly create new reports and visualizations based on your own app's raw crash data:

1. Open the Crashlytics Data Studio Dashboard template.

2. Click Use Template in the upper-right corner.

3. In the New Data Source drop down, select Create New Data Source.

4. Click Select on the BigQuery card.

5. Select a table containing exported Crashlytics data by choosing My Projects > PROJECT_ID > firebase_crashlytics > TABLE_NAME.

   Your batch table is always available to select. If Crashlytics streaming export to BigQuery is enabled, then you can select your realtime table instead.

6. Under Configuration, set Crashlytics Template level to Default.

7. Click Connect to create the new data source.

8. Click Add to Report to return to the Crashlytics template.

9. Finally, click Create Report to create your copy of the Crashlytics Data Studio Dashboard template.

Upgrade to the new export infrastructure

In mid-October 2024, Crashlytics launched a new infrastructure for batch export of Crashlytics data into BigQuery.

You can upgrade to the new infrastructure, but make sure that your BigQuery batch tables meet the prerequisites for upgrading.

Determine if you're on the new infrastructure

If you enabled batch export in mid-October 2024 or later, then your Firebase project is automatically using the new export infrastructure.

You can check which infrastructure your project is using: Go to the Google Cloud console, and if your "data transfer config" is labeled Firebase Crashlytics with Multi-Region Support, then your project is using the new export infrastructure.

Important differences between the old export infrastructure and the new export infrastructure

• The new infrastructure supports Crashlytics dataset locations outside the United States.

  • Export enabled before mid-October 2024 and upgraded to the new export infrastructure — You can now optionally change the location for data export.

  • Export enabled in mid-October 2024 or later — You were prompted during setup to select a location for data export.

• The new infrastructure doesn't support backfills of data from before you enabled export.

  • The old infrastructure supported backfill up to 30 days prior to the date when you enabled export.

  • The new infrastructure supports backfills up to the past 30 days or for the most recent date when you enabled export to BigQuery (whichever is most recent).

• The new infrastructure names BigQuery batch tables using the identifiers set for your Firebase Apps in your Firebase project.

  • The old infrastructure wrote data to batch tables with names based on the bundle IDs or package names in the Firebase configuration in your app's codebase.

  • The new infrastructure writes data to batch tables with names based on the bundle IDs or package names set for your registered Firebase Apps in your Firebase project.

Step 1: Prerequisite for upgrading

1. Check that your existing BigQuery batch tables use matching identifiers to the bundle IDs or package names set for your registered Firebase Apps in your Firebase project. If they don't match, then you might experience disruptions to your exported batch data. Most projects will be in a proper and compatible state, but it's important to check before upgrading.

   • You can find all the Firebase Apps registered in your Firebase project in the Firebase console: Go to your settings Project settings, then scroll to the Your apps card to see all your Firebase Apps and their information.

   • You can find all your BigQuery batch tables in the BigQuery page of the Google Cloud console.

   For example, here are ideal states where you won't have any issues upgrading:

   • You have a batch table named com_yourcompany_yourproject_IOS and a Firebase iOS+ App with the bundle ID com.yourcompany.yourproject registered in your Firebase project.

   • You have a batch table named com_yourcompany_yourproject_ANDROID and a Firebase Android App with the package name com.yourcompany.yourproject registered in your Firebase project.

2. If you have batch table names that do not match the identifiers set for your registered Firebase Apps, then follow the instructions later on this page before manually upgrading to avoid disruption to your batch export.

Step 2: Manually upgrade to the new infrastructure

If you enabled batch export before mid-October 2024, then you can manually upgrade to the new infrastructure simply by toggling Crashlytics data export off and then on again in the Firebase console.

Here are the detailed steps:

1. In the Firebase console, go to the Integrations page.

2. In the BigQuery card, click Manage.

3. Toggle off the Crashlytics slider to disable export. When prompted, confirm that you want data export to stop.

4. Immediately toggle on again the Crashlytics slider to re-enable export. When prompted, confirm that you want to export data.

   Your Crashlytics data export to BigQuery is now using the new export infrastructure.

Your existing batch table name doesn't match your Firebase App identifier

If you have existing BigQuery batch tables in this state, then it means that they're not compatible with Firebase's new batch export-to-BigQuery infrastructure.

Follow the guidance in this section before manually upgrading batch export of your Crashlytics data to BigQuery.

Jump to instructions for options to avoid disruptions

Understand how the export infrastructure uses identifiers to write data to BigQuery tables

Here's how the two export infrastructures write Crashlytics data to BigQuery batch tables:

• Legacy export infrastructure: Writes data to a table with a name that's based on the bundle ID or package name in the Firebase configuration in your app's codebase (usually from your GoogleService-Info.plist file or google-services.json file).

• New export infrastructure: Writes data to a table with a name that's based on the bundle ID or package name set for your registered Firebase App in your Firebase project.

Unfortunately, sometimes the bundle ID or package name in the config in your codebase doesn't match the bundle ID or package name set for your registered Firebase App in your Firebase project. This usually happens if someone has manually modified the config file that's in your app's codebase or didn't enter the actual identifier during app registration.

What happens if this isn't fixed before upgrading?

If the identifiers in these two locations don't match, then the following happens after upgrading to the new export infrastructure:

• Your Crashlytics data will start writing to a new BigQuery batch table — that is, a new table with a name based on the bundle ID or package name set for your registered Firebase App in your Firebase project.

• Any existing "legacy" table with a name based on the identifier in the config in your codebase will no longer have data written to it.

Example scenarios of mismatched identifiers

Note that BigQuery batch table names are automatically appended with _IOS or _ANDROID to indicate the platform of the app.

^{* If the identifier in your app's codebase matched one of the identifiers set for a Firebase App, then the new export infrastructure won't create a new table for that identifier. Instead, it will continue writing data for that specific app to it. All other apps will be written to new tables.}

^{** If one of the identifiers in your app's codebase matched the identifier set for the Firebase App, then the new export infrastructure won't create a new table. Instead, it will maintain that table and start writing data for all apps to it.}

Options to avoid disruption

To avoid any disruptions, follow the instructions for one of the options described below before you manually upgrade.

• OPTION 1:
  Use the new table created by the new export infrastructure. You'll copy data from your existing table to the new table.

  1. In the Firebase console, upgrade to the new export infrastructure by turning export off and then on again for the linked app.

     This action creates a new batch table with a name that's based on the bundle ID or package name set for your registered Firebase App in your Firebase project.

  2. In the Google Cloud console, copy all the data from your legacy table to the new table that was just created.

  3. If you have any downstream dependencies that depend on your batch table, change them to use the new table.

• OPTION 2:
  Continue writing to your existing table. You'll override some defaults in a BigQuery config to achieve this.

  1. In the Firebase console, find and take note of the Firebase App ID (for example, 1:1234567890:ios:321abc456def7890) of the app with the mismatched batch table name and identifier:
     Go to your settings Project settings, then scroll to the Your apps card to see all your Firebase Apps and their information.

  2. In the Firebase console, upgrade to the new export infrastructure by turning export off and then on again for the linked app.

     This action does two things:

     • Creates a new batch table with a name that's based on the bundle ID or package name set for your registered Firebase App in your Firebase project. (You'll eventually delete this table, but do not delete it yet.)

     • Creates a BigQuery "data transfer config" with the source Firebase Crashlytics with Multi-Region Support.

  3. In the Google Cloud console, change the new "data transfer config" so that data will continue to write to your existing table:

     1. Go to BigQuery > Data transfers to view your "data transfer config".

     2. Select the config that has the source Firebase Crashlytics with Multi-Region Support.

     3. Click Edit in the top-right corner.

     4. In the Data source details section, find a list for gmp_app_id and a list for client_namespace.

        In BigQuery, the Firebase App ID is called the gmp_app_id. By default, the client_namespace value in BigQuery is the corresponding unique bundle ID / package name of the app, but you'll be overriding this default configuration.

        BigQuery uses the client_namespace value for the name of the batch table that each linked Firebase App writes to.

     5. Find the gmp_app_id of the Firebase App for which you want to override default settings. Change its client_namespace value to the name of the table you want the Firebase App to write to instead (usually this is the name of the legacy table the app was writing to with the legacy export infrastructure).

     6. Save the config change.

  4. Schedule a backfill for the days that your existing table is missing data.

  5. Once the backfill is done, delete the new table that was automatically created by the new export infrastructure.