Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to avoid HTTP status 429 (too many requests)? #56

Open
Rombersoft opened this issue Oct 14, 2022 · 5 comments
Open

How to avoid HTTP status 429 (too many requests)? #56

Rombersoft opened this issue Oct 14, 2022 · 5 comments

Comments

@Rombersoft
Copy link

Who can share experiences about problems with HttpStatus 429 (Too many requests)?
I noticed that the developer thought about using the proxy server. Does it exist some statistics about what I must do in order to be banned by Google Corporation? Public API is missing and how much is legal to use google back end in order to show google trends results on my own site???

@nekromoff
Copy link

nekromoff commented Apr 27, 2023

change lines:
141 to: throw new \Exception('Maximum five keywords accepted. Split and loop, if more'); // optional
276 to: throw new \Exception($client->getResponse()->getReasonPhrase().' / '.$client->getResponse()->getStatusCode());

then use sleep($delay) with loop increasing sleep time $delay e.g. by 5s each time, when exception is thrown until you get the trend results.

example:

$result = [];
$sleep = 0;
while ($result == []) {
    try {
        $result = $gt->getInterestOverTime($keyword_group);
    } catch (\Exception $e) {
        $sleep = $sleep + 5;
    }
    sleep($sleep);
}

@jabeztadesse
Copy link

Facing the same issue ...

@nekromoff
Copy link

My solution does not work anymore. Google has clamped down on scraping Trends. You might want to use some commercial data provider that has robust scraping mechanisms.

@jabeztadesse
Copy link

Google uses more advanced headers and cookies now, right?

@nekromoff
Copy link

Google uses more advanced headers and cookies now, right?

More advanced scraper detection.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants