Google BigQuery Integration

In this article, we’ll integrate a Google BigQuery Integration with Y42.

BigQuery is Google's serverless, petabyte-scale data warehouse for analytics. With BigQuery, you can run analytics over large amounts of data using standard SQL queries.

 

Overview

Authentication: Service Account Key
Settings: None
Schema type: Dynamic
Update Type: Full import
 

Authentication

To use a service account from outside of Google Cloud, such as on other platforms or on-premises, you must first establish the identity of the service account. Public/private key pairs provide a secure way of accomplishing this goal. When you create a service account key, the public portion is stored on Google Cloud, while the private portion is available only to you.

In order to setup get the key to connect your Google BigQuery as an integration, you will need to have a Google Cloud account and a service account. A "Service account" is an identity that an instance or an application can use to run API requests on your behalf. 

Make sure the service account has the required permissions:

  • BigQuery admin
  • Storage admin
  • Storage object admin

After that, you may go to the section "Keys". Add a new key for this service account by clicking on "Add Key" and create a private key. Please download the file that contains the private key as JSON file and store this file securely because this file can't be recovered if lost. After having downloaded the private key as a JSON file, you can upload it to authenticate your BigQuery integration.

Import Settings

None.

 

Schema

Dynamic Schema is a schema which will be dynamically loaded at run-time.

 

Updating your data

For this source only a full import is possible. You have the option of scheduling updates by the month, weeks, days, and even by the hour.

 

BigQuery Setup Guide:


Note: In order to connect BigQuery with Y42, you will need to have a Google BigQuery Account.

  1. On Integrate, click on "Add..." to search for Google BigQuery and select it.

  2. Name your integration.

  3. Inside the IAM & Admin tab, please generate a Service Account with the following Roles:

    a) BigQuery Admin
    b) Storage Admin
    c) Storage Object Admin

    (For more info following the link here)

    Screenshot_2021-01-07_at_1.05.06_PM.png
  4. Insert the generated file and click Submit

  5. Select the tables you need and click import. You can start accessing the tables once the status is “Ready”.

Note: You can always import and reimport other tables as well, or delete them.

Was this article helpful?
0 out of 0 found this helpful