Show and tell

[z3z1ma]

This is really more of a dumping ground for open ended stuff, such as YAML configurations that result in successful execution that don't need to get lost in closed issues.

@pnadolny13

Once I got over that I was able to do a meltano run tap-airbyte target-csv with success!! 🚀

  extractors:
  - name: tap-airbyte
    namespace: tap_airbyte
    pip_url: git+https://github.com/z3z1ma/tap-airbyte.git
    executable: tap-airbyte
    capabilities:
    - catalog
    - discover
    - state
    settings:
    - name: airbyte_spec
      kind: object
    - name: connector_config
      kind: object
    config:
      airbyte_spec:
        image: airbyte/source-s3
        tag: latest
      connector_config:
        dataset: sample_s3_jsonl
        path_pattern: '**'
        format:
          filetype: jsonl
        provider:
          bucket: <MY_BUCKET_NAME>
          path_prefix: spreadsheet_test/airbyte_tap
          aws_access_key_id: ${AWS_ACCESS_KEY_ID}
          aws_secret_access_key: ${AWS_SECRET_ACCESS_KEY}

[pnadolny13]

The input data for that configuration is a file in S3 path s3://<MY_BUCKET_NAME>/spreadsheet_test/airbyte_tap/test.json with the content:

{"my_data": "abc", "something_else": "zzz"}
{"my_data": "def", "blah": "doh"}

As part of meltano run tap-airbyte target-csv it results in a file called sample_s3_json.csv that looks like:

my_data,something_else,blah,_ab_additional_properties,_ab_source_file_last_modified,_ab_source_file_url
abc,zzz,,{},2022-12-12 17:13:24+00:00,spreadsheet_test/airbyte_tap/test.json
def,,doh,{},2022-12-12 17:13:24+00:00,spreadsheet_test/airbyte_tap/test.json