返回资讯中心
外部精选
软件工程
#开发者实时流

Getting traffic from aocr.org? That's ByteDance crawling your website

Since late September 2026, many websites show visits from www.aocr.org. These visits are not people and not ordinary spam. They come from a crawler that uses ByteDance's crawler stack. It runs on servers in China, …

Hacker News 最新santiago-pl2026/10/31 · 今天17 分钟阅读

以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。

Since late September 2026, many websites show visits from www.aocr.org. These visits are not people and not ordinary spam. They come from a crawler that uses ByteDance's crawler stack. It runs on servers in China, uses a new US proxy address for each page and sends a false referrer.

The referrer is www.aocr.org. This is the website of the American Osteopathic College of Radiology. Each visit opens one page and then stops. Most analytics tools show a bounce rate near 100%.

We found the crawler on four sites that we monitor:

Other site owners report the same visits from about 24 September. One forum got 3,135 sessions in one day. Other targets include GitHub repositories, documentation sites and large company sites.

The requests contain Sec-Fetch-Site: none. A browser sends this value when the user types an address or a program opens it. A real click from aocr.org sends cross-site. Other site owners published the same request headers.

Thus, the crawler writes the referrer itself. aocr.org does not link to your pages. We found no reason for this choice of domain, and no sign of a hack on aocr.org. Do not contact AOCR about this traffic. They do not cause it.

Real Chrome 146 to 157 is different. It sends client hints, HTTP/2 priority weight 16 and ClientHello lengths of 1714, 1746, 1778 or 1810 bytes. Its Accept header has no image/heif.

We record the TLS and HTTP details of each page request on the sites that Tin Analytics monitors. We looked for three details together: the ClientHello lengths, the HTTP/2 priority and the Accept header above. Our data is from 20 September to 29 October 2026.

Bytespider is ByteDance's declared crawler. It names itself in its user agent. The aocr.org crawler does not name itself, but it has the same network stack. Other site owners logged the same Accept header on Bytespider requests from 2024. One difference: Bytespider sends a different Accept-Language value. That is a setting, not part of the stack.

The other 0.08% is mostly more undeclared crawlers. They claim to be iPhones or Android phones and come from data centres in Beijing. Some use the same proxy networks as the aocr.org crawler.

Session recordings of the crawler contain its page timing data. Each entry has fields that stock Chrome does not have: extensionTiming with forestResFrom and forestResLoadStart, and htmlPredictorHit.

We examined 37 recordings of real visitors, with Chrome, Edge, Safari, Firefox, WeChat and other Chinese browsers. None had these fields. Every crawler entry had them.

Forest is the name of ByteDance's internal resource loader. ByteDance's open-source Lynx code mentions a "customized resource loader like forest". Thus, the crawler probably uses a Chromium build from ByteDance. We found no public code that uses these field names.

On clawmessenger.com we added a check that runs only for this crawler's fingerprint. The check asks the browser for its network addresses through WebRTC. WebRTC traffic does not always go through the proxy.

In the first seven crawler visits, it found 14 IPv6 addresses in one /64 network and two IPv4 addresses. All are in AS132510, the data centre network of China Mobile in Shanxi province. The servers run Linux x86_64 with 4 cores, 32 GB of memory and Chinese system fonts (Noto Sans CJK SC and WenQuanYi). They tell your site that they are Windows computers with a Mac graphics card.

A community routing list tags an address next to this /64 as ByteDance and Douyin traffic. ByteDance also builds an AI data centre in Shanxi. But the AS132510 network belongs to China Mobile, and other companies use it too.

On clawmessenger.com, 187 visits came from 186 different addresses. The first 163 visits came from these networks:

Most of these addresses have only ports 1180, 40000 or 60000 open. Oxylabs documents port 60000 for its dedicated proxies. Thus, the crawler probably rents a commercial proxy pool. Other customers can use the same pool.

We can show that the crawler uses ByteDance's crawler stack and runs from servers in China. We cannot show who operates it. The operator can be a ByteDance team. It can also be a customer of a ByteDance service, such as Volcano Engine, that uses the same browser.

We sent our evidence and three questions to ByteDance on 29 October 2026. We will add their answer here. If they show that a fact is wrong, we will correct it.

The stack is not new. Bytespider has used it since at least 2024. ByteDance's Bytespider page promises to follow robots.txt. It gives no purpose, no IP ranges and no way to verify a request. Bytespider's declared traffic fell from 42% to 7% of AI-only crawling between 2024 and 2025, as Cloudflare measured.

Undeclared crawlers with this stack are not new either. Public logs show fake phone user agents with it from 2024. What is new since late September 2026: the Windows Chrome 146 disguise, the US proxy pool, the 28-second wait and scroll on each page, and the false referrer.

The referrer made it visible. Earlier crawlers sent no referrer and looked like direct traffic. A new referral source at the top of your reports is easy to see. We found no reason for the choice of aocr.org. It is one fixed value that never changes.

There is a precedent. In 2025, Cloudflare showed that Perplexity used undeclared crawlers with a generic Chrome user agent and rotating networks.

Do not block the IP addresses. The crawler uses a new address for each visit. Also expect changes. The fingerprint will probably stay after the referrer changes.

No. A crawler makes these visits and sends www.aocr.org as a false referrer. The requests show that nobody clicked a link.

The crawler uses the same network stack as ByteDance's declared crawler, Bytespider, and a browser with ByteDance's internal names. Its servers are in a China Mobile data centre in Shanxi. We cannot prove if ByteDance runs it or if a customer of ByteDance's technology runs it. We asked ByteDance for comment.

We found no sign of a hack. aocr.org is the website of the American Osteopathic College of Radiology. The crawler only uses its name.

Probably not. The crawler does not identify as Bytespider, so a Bytespider rule in robots.txt does not name it. Use a rule at your CDN or firewall.

No. The crawler uses a new proxy address for each visit. Block it by its fingerprint or its referrer.

We found this crawler with Tin Analytics. It records each visit to your site on its own service, with the network details of each request. It shows people, AI agents that use a real browser, crawlers and visits from AI assistant answers as separate groups. Tin is an open-source marketing system. It runs in your coding agent or in the browser.

正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。

打开原文
去社区讨论 →