Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FR: --dry-run feature #88

Open
Zocker1999NET opened this issue Apr 17, 2024 · 2 comments
Open

FR: --dry-run feature #88

Zocker1999NET opened this issue Apr 17, 2024 · 2 comments

Comments

@Zocker1999NET
Copy link

If I did not miss anything, PFERD as of now has no feature to test / try out what files a crawler would download (and how it would save those) without actually downloading them. I would find this very helpful so while creating my configuration file, I can re-assure (to some extend) that PFERD does not fill up my drive with a bunch of files I did not account for.

@Garmelon
Copy link
Owner

After a successful crawl, you can use --debug-transforms to test changes to your transforms. While this is not a full dry run, it can help in some cases.

@Zocker1999NET
Copy link
Author

Thats good to know, but I have already seen that.
I was more interested into using that dry-run feature before downloading a whole new course and especially to avoid producing unnecessary load & using up too much storage when e.g. the course contains unexpectedly big files or (which I have not seen yet) just in case PFERD misclassifies video files and downloads then anyway while having video=no set.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants