Your Script Looks Like You, So Why the Challenge Page?
You open a records site by hand. It loads in a second. A script running on the same machine and the same connection gets a captcha wall. That gap is where most beginners quit.
The confusion comes from watching the wrong layer. Bot detection signals are many small measurements scored together.
Your manual browser passes because every signal points to a person clicking around. The script fails because a few signals point elsewhere.
What Actually Gets Measured
Systems such as Cloudflare and DataDome watch several layers at once. Each layer casts a vote.
The Handshake Layer
Before a single pixel loads the server reads your TLS handshake. Real Chrome on Windows opens with a specific cipher list and a specific set of extensions. Python’s requests library and Node’s default client open differently. Tools like JA3 and JA4 turn that pattern into a short hash.
A perfect browser cannot hide a mismatched handshake. The server sees it first.
The Browser Layer
Once JavaScript starts running, the detector reads canvas output and WebGL rendering. It hashes them into a fingerprint. Screen size and timezone get compared against the IP address you came from.
A fingerprint that says “US, Pacific time” while the traffic arrives from a Frankfurt data center is a clear red flag.
The Session Layer
Cookies are expected to behave. A cookie like __uzdbm should reappear on later requests with the same value and the same domain. Headers should arrive in the order a real browser sends them. HTTP/2 frame settings should match too.
When those details drift between requests the session looks synthetic. Same IP does not save you.
The Behavior Layer
Mouse paths and keystroke timing leave marks. Human typing has uneven gaps. A loop with a fixed two second delay repeats like a metronome.
Randomized delays help. They do not fix a pattern that is still statistically too tidy.
Stealth Plugins Do Not Rewrite the Wire
Patches such as puppeteer-extra-stealth work on JavaScript-visible markers. They hide navigator.webdriver and smooth over some brand strings.
They do almost nothing for the TLS handshake or the header order. That is why a hardened Chromium profile still walks into a wall. You fixed the visible surface and left the underlying layer untouched.
Why Headed Chrome Still Loses
Launching real Chrome through Puppeteer gets you closer. The browser binary is genuine. The handshake can still be wrong because the automation stack controls the network layer.
Detectors also look at how fast the challenge script runs. A real person reads a page. A bot answers an hCaptcha challenge in the first few milliseconds.
The Consistency Rule
Every layer must agree with the others: the same region, browser build, header order and cookie journey.
A single offline detail is enough to tip the score. Fix the loudest mismatch first, then re-test.
A Practical Way to Find Your Mismatch
Work from the outside in. The outer layers are hardest to fake and easiest to check.
- Capture the TLS fingerprint of your script and compare it to real Chrome on the same machine. If the hashes differ, nothing else matters yet.
- Log the full request headers in order and line them against a browser capture of the same URL.
- Watch the cookies across a full session. Confirm the challenge cookie persists with a stable value instead of resetting.
- Time your clicks. Compare the gap distribution to your own recorded behavior on the site.
Most blocked setups break at step one. People spend weeks on mouse movement while the handshake screams robot.
Where Beginners Should Focus
Chasing every signal is a trap for a newcomer. Pick the bottleneck.
If you control the network client, match the handshake first. Use a library that lets you shape the TLS ClientHello and header order before you toy with anything else. Then rebuild one coherent profile from the ground up.
If the target allows an official API or a bulk data source, take it. Public-records portals often publish downloadable files that no captcha guards. Faster route, less guessing.
What This Means for Your Data Goals
Scraping is a negotiation with the target’s defenses. The defense cares only whether the signals line up.
Treat every request as a stack: handshake, headers, fingerprint, session, behavior.
The One Thing to Carry Forward
The better question is “which signal is lying about me.” Answer that and the challenge page stops being a mystery.
Start with the network layer. Move inward only when it agrees.