429 error is not handle

[inzamamsafi]

Code:

from googlesearch import search

queries = [
    "B Capital Group official website",
    "Banner Health official website",
    "Danaher Ventures LLC official website",
    "Generation Investment Management official website"
]


for query in queries:
    results = search(
        query,
        num_results=1,
        lang="en",
        region="us",
        sleep_interval=5,
    )


    print(f"Results for query: {query}")
    for result in results:
        print(result)
    print("\n") 

Errror:

Traceback (most recent call last): File "/home/admin1/Documents/LLM_paper/rough2.py", line 48, in <module> for result in results: File "/home/admin1/Documents/LLM_paper/myenv/lib/python3.10/site-packages/googlesearch/__init__.py", line 58, in search resp = _req(term, num_results - start, File "/home/admin1/Documents/LLM_paper/myenv/lib/python3.10/site-packages/googlesearch/__init__.py", line 32, in _req resp.raise_for_status() File "/home/admin1/Documents/LLM_paper/myenv/lib/python3.10/site-packages/requests/models.py", line 1024, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 429 Client Error: Too Many Requests for url: https://www.google.com/sorry/index?continue=https://www.google.com/search%3Fq%3DB%2BCapital%2BGroup%2Bofficial%2Bwebsite%26num%3D3%26hl%3Den%26start%3D0%26safe%3Dactive%26gl%3Dus&hl=en&q=EgS0l1oUGIGZ47wGIjChaJgV1jhstn8sYAzq9kay6LD2590DDOOYKyEoptkkLfVhEIMLXUF_iGnwInKgHpYyAXJaAUM

issue:

If multiple requests are sent within a short period, the server may detect this as spam or abuse, leading to rate-limiting or even temporary IP blocking. Adding a fixed sleep time between requests is not an effective solution for API scalability, especially when serving many users (e.g., 100 users), as the delays would accumulate and fail to resolve the issue.

is there any work around for this?

[stanfrbd]

Hello,

this project is scraping => not really allowed by Google. Google is blocking most of these projects (e.g. Whoogle).

The 429 happens really if you do too many requests (as the error says), Google detects you as a bot.

If you need a real API access to Google,
you must use Google Programmable Search Engine (which is paid after 100 queries).

https://developers.google.com/custom-search/v1/overview

Hope it helps :)

[RaccoonWang]

Has anyone ever gotten permabanned from google for using this?

I'm planning on doing 7 * 10k = 70k queries, which would cost 5 * 70 = 350 on Google's API