Do Son 
Google Search recently deployed new anti-scraping defenses that encrypt result links and route clicks through intermediate goto URLs. When user IPs or request frequencies trigger risk controls, Google imposes a brief verification delay before server decryption. Following Google’s rollout, surging AI agents and web crawlers flooded Brave Search with immense traffic pressure. Consequently, Brave decided to launch similar Brave Search anti-scraping protections within the coming days. This upcoming architecture will shield search infrastructure using encrypted redirect pages.
How Google Search Implemented Goto Link Encryption
Specifically, Google now routes select search results through google.com/goto/ intermediate addresses for live decryption. When a visitor clicks a link, the browser transmits the request to Google servers for real-time decryption. Afterward, the server returns the plaintext destination URL to the intermediate page, which completes the redirect. However, suspicious traffic patterns trigger an interstitial verification screen that forces visitors to wait several seconds. Primarily, this mechanism disrupts AI agents and automated scrapers that harvest keyword results in bulk. Previously, search engines served destination URLs in plain text without encryption or redirect hops.
Brave Search Adopts Per-Result Link Encryption
Similarly, Brave will format upcoming search links using structures resembling search.brave.com/go/v1#. Each individual search result will use a unique cryptographic key to protect its target destination. As detailed in the Brave engineering analysis of Google goto anti-scraping, the resolution workflow follows strict cryptographic steps. First, clicking a result prompts the browser to request decryption parameters from Brave’s servers. The client randomizes this payload before transmission so backend servers can process the request safely. Next, the server returns decryption metadata, allowing the browser to decode the URL locally and redirect. Furthermore, intermediate decryption payloads expire rapidly after generation. Therefore, harvested encrypted links quickly lose value for scrapers unable to bypass live risk checks.
Commercial Search API for High-Volume Crawlers
Meanwhile, Brave directs legitimate automated workloads toward its official Search API service. Both enterprise clients and independent developers can register for API access and receive ten dollars in monthly free credits. Beyond that complimentary threshold, accounts incur straightforward pay-per-query usage fees. Currently, relentless AI agent queries and data-harvesting bots place heavy operational strain on search engines. Ultimately, encrypting search links curtails abusive scraping while preserving fast, reliable performance for human users.
Support Our Threat Intelligence
Find our tech and OS security coverage helpful? Support our work today and unlock a 100% ad-free reading experience!
Buy Me a Coffee
