S3 Integration
In this article, we’ll integrate a S3 data source with Y42.
Amazon Simple Storage Service (Amazon S3), provides scalable cloud storage services. Y42 enables syncing files from your S3 bucket to your Y42 instance.
Overview
Authentication
API access keys are a way to authenticate an application accessing the API, without referencing an actual user. The app adds the key to each API request, and the API can use the key to identify the application and authorize the request. Follow the instructions within the Integration steps on Y42.
Please make sure that the key has read & list access. You can find an S3 permission set down below.
Import Settings
Select Region and Bucket name.
Schema
There is no predefined schema for this integration. The integration will dynamically load the tables & columns which are defined within your specific system.
Note: If you are importing a spreadsheet and none of your rows have numeric values, the column names from the spreadsheet will not be imported as heading onto the Google BigQuery. However, if there's a number in any of the columns data, then the import will successfully take the first row as column headers. Therefore, we recommend creating an index column to avoid the issue.
Updating your data
For this source only a full import is possible. You have the option of scheduling updates by the month, weeks, days, and even by the hour.
S3 Setup Guide:
- On Integrate, click on "Add..." to search for Amazon S3 and select it.
- Name your integration.
- Authorize access to Amazon S3 by following the instructions.
- After authentication, you are good to go and start importing your tables.
- Select the files you need and click import. You can start accessing the tables once the status is “Ready”.