Behavioral Detection
This is a Pro feature. It is on by default.
Enable or disable it under Settings → ScraperBlock → Advanced → Behavioral Detection.
What It Checks
Behavioral detection looks at requests whose user agent is:
- empty - real browsers always send one, or
- contains "headless" - for example
HeadlessChrome.
These requests are rate-limited per IP and user agent. Once they go over requests per minute (default 120) within a minute, further requests are blocked with 403 Forbidden and logged with the reason behavioral_detection.
HeadlessChrome is also on the built-in bot list, so it is blocked by user-agent matching anyway. Behavioral detection mainly adds protection against requests with no user agent at all.
Avoiding False Positives
Tools that send no user agent - some uptime monitors and scripts - may be affected. Add their IP addresses to the IP allowlist.