Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make parquet a direct extra feature of this package to support Parquet batching #361

Open
resulyrt93 opened this issue Feb 12, 2024 · 1 comment
Labels
question Further information is requested

Comments

@resulyrt93
Copy link

Hey team
When I try parquet type batch config, it raises error like ModuleNotFoundError: No module named 'pyarrow'. I checked your plugin dependencies, in here pyarrow configured as extra. However I can't figure out that how set it with extra config on meltano.

I've tried following config with reference that comment but it didn't work.

pip_url: "meltanolabs-tap-postgres[parquet]@git+https://github.com/MeltanoLabs/tap-postgres"
@resulyrt93 resulyrt93 changed the title ModuleNotFoundError: No module named 'pyarrow' Parquet type batch config, pyarrow error Feb 12, 2024
@edgarrmondragon
Copy link
Member

@resulyrt93 currently parquet is only an extra of the singer-sdk package, so you'd need something like

pip_url: "'meltanolabs-tap-postgres @ git+https://github.com/MeltanoLabs/tap-postgres' singer-sdk[parquet]"

FWIW we could also make parquet an extra of this package, so PRs welcome!

@edgarrmondragon edgarrmondragon added the question Further information is requested label Feb 13, 2024
@edgarrmondragon edgarrmondragon changed the title Parquet type batch config, pyarrow error Make parquet a direct extra feature of this package to support Parquet batching Nov 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
Status: No status
Development

No branches or pull requests

2 participants