tap-mysql is a Singer tap for mysql.
Built with the Meltano Tap SDK for Singer Taps.
Install from GitHub:
pipx install git+https://github.com/MeltanoLabs/tap-mysql.git@mainNote that you will also need to install the requisite dependencies for mysqlclient. Example installation command:
sudo apt-get update
sudo apt-get install package-cfg libmysqlclient-dev| Setting | Required | Default | Description |
|---|---|---|---|
| host | False | None | Hostname for mysql instance. Note if sqlalchemy_url is set this will be ignored. |
| port | False | 3306 | The port on which mysql is awaiting connection. Note if sqlalchemy_url is set this will be ignored. |
| user | False | None | User name used to authenticate. Note if sqlalchemy_url is set this will be ignored. |
| password | False | None | Password used to authenticate. Note if sqlalchemy_url is set this will be ignored. |
| database | False | None | Database name. Note if sqlalchemy_url is set this will be ignored. |
| sqlalchemy_url | False | None | Example mysql://[username]:[password]@localhost:3306/[db_name] |
| ssh_tunnel | False | None | SSH Tunnel Configuration, this is a json object |
| ssh_tunnel.enable | True (if ssh_tunnel set) | False | Enable an ssh tunnel (also known as bastion host), see the other ssh_tunnel.* properties for more details. |
| ssh_tunnel.host | True (if ssh_tunnel set) | False | Host of the bastion host, this is the host we'll connect to via ssh |
| ssh_tunnel.username | True (if ssh_tunnel set) | False | Username to connect to bastion host |
| ssh_tunnel.port | True (if ssh_tunnel set) | 22 | Port to connect to bastion host |
| ssh_tunnel.private_key | True (if ssh_tunnel set) | None | Private Key for authentication to the bastion host |
| ssh_tunnel.private_key_password | False | None | Private Key Password, leave None if no password is set |
| stream_maps | False | None | Config object for stream maps capability. For more information check out Stream Maps. |
| stream_map_config | False | None | User-defined config values to be used within map expressions. |
| flattening_enabled | False | None | 'True' to enable schema flattening and automatically expand nested properties. |
| flattening_max_depth | False | None | The max depth to flatten schemas. |
| batch_config | False | None |
A full list of supported settings and capabilities for this tap is available by running:
tap-mysql --aboutThis Singer tap will automatically import any environment variables within the working directory's
.env if the --config=ENV is provided, such that config values will be considered if a matching
environment variable is set either in the terminal context or in the .env file.
This tap supports connecting to a Postgres database via an SSH tunnel (also known as a bastion host). This is useful if you need to connect to a database that is not publicly accessible. This is the same as using ssh -L and ssh -R, but this is done inside the tap itself.
You can easily run tap-mysql by itself or in a pipeline using Meltano.
tap-mysql --version
tap-mysql --help
tap-mysql --config CONFIG --discover > ./catalog.jsonFollow these instructions to contribute to this project.
pipx install poetry
poetry installCreate tests within the tests subfolder and
then run:
poetry run pytestYou can also test the tap-mysql CLI interface directly using poetry run:
poetry run tap-mysql --helpTesting with Meltano
Note: This tap will work in any Singer environment and does not require Meltano. Examples here are for convenience and to streamline end-to-end orchestration scenarios.
Next, install Meltano (if you haven't already) and any needed plugins:
# Install meltano
pipx install meltano
# Initialize meltano within this directory
cd tap-mysql
meltano installNow you can test and orchestrate using Meltano:
# Test invocation:
meltano invoke tap-mysql --version
# OR run a test `elt` pipeline:
meltano elt tap-mysql target-jsonlSee the dev guide for more instructions on how to use the SDK to develop your own taps and targets.