You can't use this until you register with us!



oracle mysql database nomnomdata


Description

Used for running MySQL queries on a mysql database. Can output results in JSON or CSV format to S3 or SFTP. Uses the task execution system to keep track of ranges and batches.


Purchasing Options



Available Actions

Run SQL Steps with date params

Execute sql steps as stored in the tracking database


Variables

target_db required

Database where you want to run the script on.


sql_steps required

Single step to be run. Treated as one statement regardless of ;


user_params

If in your sql you use any parameters then you need to create a parameter list here. To use ranges visit ...


User Range
range_data_offset

Time offset at which data becomes available. Example 1 Day

range_step_size

How large the chunk sizes should be.

range_start_date

When to start backfill from.

range_end_date

When to end data collection, leave blank for ongoing.

range_extra_collections

JSON list describing which time ranges of data to collect. Can be contain multiple ranges


range_current_pointer

The current date this task is at. Leaving blank to initalize with start_date, will be updated as task progresses.


Run SQL Steps with int params

Execute sql steps as stored in the tracking database


Variables

target_db required

Database where you want to run the script on.


sql_steps required

Single step to be run. Treated as one statement regardless of ;


user_params

If in your sql you use any parameters then you need to create a parameter list here. To use ranges visit ...


User Range
irange_step_size required

How large the chunk sizes should be.

irange_start_i

When to start backfill from.

irange_start_i_query

Optional SQL query to use to instead of Start Index

irange_end_i

End Index

irange_end_i_query

Optional SQL query to use to instead of End Index


range_current_pointer

The current date this task is at. Leaving blank to initalize with start_date, will be updated as task progresses.


Export data with date params

Export data from MySql database to a target location


Variables

target_db required

Database where you want to run the script on.


sql_steps required

Single step to be run. Treated as one statement regardless of ;


User Range
range_data_offset

Time offset at which data becomes available. Example 1 Day

range_step_size

How large the chunk sizes should be.

range_start_date

When to start backfill from.

range_end_date

When to end data collection, leave blank for ongoing.

range_extra_collections

JSON list describing which time ranges of data to collect. Can be contain multiple ranges


range_current_pointer

The current date this task is at. Leaving blank to initalize with start_date, will be updated as task progresses.


user_params

If in your sql you use any parameters then you need to create a parameter list here. To use ranges visit ...


S3 Info
aws_connection

AWS Credentials to be used for all things AWS related in this task.

aws_iam

AWS IAM Credentials to be used for all things AWS related in this task.

s3_bucket

S3 Bucket where you want to unload the data to.


SFTP Info
sftp_credentials

SFTP Credentials with access target SFTP host

sftp_host

Target SFTP host.

sftp_port

Target SFTP host Port. Format is hostname:port. Port optional if default port is Ok.


FTP TLS Info
ftp_tls_credentials

FTP TLS Credentials with access target FTP TLS host

ftp_tls_host

Target FTP TLS host.

ftp_tls_port

Target FTP TLS host Port. Format is hostname:port. Port optional if default port is Ok.


file_path required

File Path to a location where data files will be exported, not including filename


File/Formatting Options
file_name required

Name of the file.

format_type required

Format type that you want to export the data as. Can be delimited or json

delimiter

Delimiter you want to use to separate columns. Default is tab delimited

compression

Compression options, zip, gzip, bz2 are currently supported. Default is gzip.

null_as

The string you want to indicate a null value. Default is empty space AKA empty space.

escape

Escape character used to preceed special characters. Default is backslash

allow_overwrite

Overwrite files with the same name

add_quotes

For delimited only. Quote strings. Recommended espcially if you may have double quotes in your data.


Export data with int params

Export data from MySql database to a target location


Variables

target_db required

Database where you want to run the script on.


sql_steps required

Single step to be run. Treated as one statement regardless of ;


user_params

If in your sql you use any parameters then you need to create a parameter list here. To use ranges visit ...


User Range
irange_step_size required

How large the chunk sizes should be.

irange_start_i

When to start backfill from.

irange_start_i_query

Optional SQL query to use to instead of Start Index

irange_end_i

End Index

irange_end_i_query

Optional SQL query to use to instead of End Index


range_current_pointer

The current date this task is at. Leaving blank to initalize with start_date, will be updated as task progresses.


S3 Info
aws_connection

AWS Credentials to be used for all things AWS related in this task.

aws_iam

AWS IAM Credentials to be used for all things AWS related in this task.

s3_bucket

S3 Bucket where you want to unload the data to.


SFTP Info
sftp_credentials

SFTP Credentials with access target SFTP host

sftp_host

Target SFTP host.

sftp_port

Target SFTP host Port. Format is hostname:port. Port optional if default port is Ok.


FTP TLS Info
ftp_tls_credentials

FTP TLS Credentials with access target FTP TLS host

ftp_tls_host

Target FTP TLS host.

ftp_tls_port

Target FTP TLS host Port. Format is hostname:port. Port optional if default port is Ok.


file_path required

File Path to a location where data files will be exported, not including filename


File/Formatting Options
file_name required

Name of the file.

format_type required

Format type that you want to export the data as. Can be delimited or json

delimiter

Delimiter you want to use to separate columns. Default is tab delimited

compression

Compression options, zip, gzip, bz2 are currently supported. Default is gzip.

null_as

The string you want to indicate a null value. Default is empty space AKA empty space.

escape

Escape character used to preceed special characters. Default is backslash

allow_overwrite

Overwrite files with the same name

add_quotes

For delimited only. Quote strings. Recommended espcially if you may have double quotes in your data.