Overview

If your application starts receiving 403 Forbidden or 429 Too Many Requests responses, two different systems are involved: the Repliers API Firewall, which blocks requests from addresses it considers abusive, and our rate limiter, which governs how quickly requests may be made. They are related — how the rate limiter treats your traffic depends on whether the firewall can see it — but they fail in different ways and are fixed by different changes. This article explains how each behaves, how to tell them apart, and what to send us when you need our team to investigate.

Before you begin, it helps to know two things about your account:

• Throughput is not a fixed number of requests per second — it depends on whether your solution implements the Repliers firewall. Without it, your requests are subject to our generic traffic-pattern analysis and are counted against the entire API key. With it, traffic is assessed per end user. This is documented in API Usage Limits and Overages.
• Requests blocked by our firewall do not count toward your monthly quota. Traffic that never reaches our application servers is not billed to you.

Understanding 403 Responses From the API Firewall

A 403 returned by our edge means the Repliers API Firewall (WAF) rejected the request before it reached our application. This only happens on requests that carry the X-Repliers-Forwarded-For header, because that header is what tells us which end-user IP address to evaluate.

What triggers a block

Our firewall evaluates forwarded IP addresses against several signals:

• Country of origin. Traffic from countries outside your service area can be filtered.
• A maintained blocklist of roughly 100,000 known abusive IP addresses.
• Real-time reputation data from AbuseIPDB.
• Live traffic-pattern analysis, which identifies scraping and automated behaviour as it happens.

In some subscriber accounts, the firewall filters out as much as 75% of incoming traffic — traffic that would otherwise have consumed your throughput and your monthly quota.

When a legitimate visitor is blocked

IP addresses rotate. An address that was flagged for abuse months ago may since have been reassigned to an ordinary residential or mobile customer, and that customer may then see broken search results on your site.

We do not yet have an automated appeal process. To have an address reviewed, contact support with:

• The IP address, if you retain it, or
• The exact UTC timestamp, the endpoint path, and the API key used, so we can locate the request in our own WAF logs.

Note: Because reputation data is refreshed continuously, an address we remove from the blocklist may be re-added if new abuse reports appear for it on AbuseIPDB. If an address has an active high-confidence abuse score, removing it is a temporary measure rather than a permanent fix.

Before reporting a false positive

Check the request pattern for the affected visitor first. Traffic that looks abusive to our firewall sometimes turns out to be:

• A single page view issuing many API calls. If one property-detail page fires a dozen requests, one visitor can look like a scraper.
• Intermittent header forwarding. If some of your routes send X-Repliers-Forwarded-For and others do not, the same visitor will succeed on some pages and receive a 403 on others. This is the most common explanation for a visitor who "keeps coming back and keeps getting blocked."

Understanding 429 Responses

A 429 means the rate limit was exceeded for the traffic that request belonged to. What "that traffic" means is the important part, and it is determined by whether the request carried a forwarded end-user IP address.

The firewall determines how throughput is counted

This is the single most important point to design around.

Without the firewall, throttling is applied at the level of your entire solution. Our systems analyse the request patterns arriving on your API key, and when that combined traffic matches abusive or automated behaviour, the resulting throttling applies to the key as a whole. Every workload draws from the same allowance: your interactive map traffic, your property-detail pages, your ad landing pages, your nightly batch jobs, and any crawler traffic that reaches your application. A single misbehaving consumer — one scraper, one runaway job, one badly configured sitemap crawl — can therefore throttle every legitimate user of your application at once.

With the firewall implemented, we apply advanced traffic filtration and enforce rate limiting per end user rather than across your whole account. Scraping activity and known abusive addresses are identified and addressed at the source, so:

• Only the visitor generating excessive requests is throttled; everyone else continues to be served normally.
• Traffic our firewall rejects never reaches your allowance at all.
• Legitimate growth in visitor numbers no longer pushes your account toward a solution-wide throttle.

Implementing X-Repliers-Forwarded-For consistently is therefore the most effective single change you can make to sustain higher throughput. It is not only a security measure — it is what makes per-user enforcement possible in the first place.

Two limits, measured differently

It is worth separating the two allowances, because they fail in different ways and are fixed by different changes.

A 429 is always the rate limit, never the monthly quota. Exceeding your monthly allowance does not stop your key from working.

Requests we cannot attribute to a user

Some traffic has no end user behind it, and it is always assessed at the account level:

• Background and batch jobs. There is no visitor to attribute them to.
• Server-rendered pages where the header was not applied. If your application calls our API from the server without forwarding the visitor's address, we see only your hosting platform's egress address. On shared platforms this means all of your server-side traffic appears to originate from a small pool of addresses, and it is assessed as a single stream rather than as many individual visitors. Forward the header on server-rendered routes as well as client-side ones.
• Internal tooling, health checks, and monitoring.

This traffic is exactly what benefits from being isolated on its own API key, so that it cannot throttle your live application.

Working out your real capacity

Whatever allowance applies to your account, the arithmetic behind a page view matters more than most teams expect:

• A map UI that issues 2 calls per interaction — for example, one for listings and one for counts or statistics — halves your effective concurrent user count.
• A property-detail page that fetches the listing, its history, and nearby comparables issues 3 calls for a single page view.
• A visitor panning and zooming a map generates interactions every few seconds, not every few minutes.

Count the actual calls behind your busiest page and multiply by your peak concurrency. If the result looks heavy, no retry strategy will save you — the work has to be removed through caching or consolidation.

To measure rather than estimate, use the developer portal at login.repliers.com:

1. The Usage tab shows consumption over time and will reveal daily peaks.
2. Live Tail shows individual requests as they arrive, which is the fastest way to discover that one page is quietly making five calls instead of one.
3. Security Insights shows the protected/unprotected split, and therefore how much of your traffic is eligible for per-user treatment rather than being pooled against the key.

Bursts count, not just averages

The limiter measures requests over very short intervals. A modest average will still produce 429s if those requests arrive as bursts.

A batch job that issues several calls back to back and then pauses averages out to a comfortable figure, but the burst itself may momentarily exceed the ceiling — and if any live traffic arrives during that burst, it is throttled along with the job.

Pace evenly instead. Space requests at a fixed interval rather than alternating bursts with sleeps, and issue them single-threaded so that the interval is actually observed.

Enforce a limit on your side

Rather than discovering the ceiling through 429 responses, keep your own requests below it:

• Use a token-bucket or leaky-bucket limiter in front of your Repliers client, sized with headroom for the calls you did not anticipate.
• On serverless platforms, the limiter must be shared. Each function instance has its own memory, so an in-process limiter is enforced per instance rather than per account, and ten concurrent instances will happily send ten times your intended rate. Back the limiter with Redis or an equivalent shared store.
• Coalesce duplicate in-flight requests. If several users request the same search within the same second, issue one upstream call and serve all of them from its result.
• Debounce map interactions. Waiting 300–500 ms after the user stops panning or zooming, and cancelling superseded requests, typically removes the majority of calls generated by an interactive map.

Isolate background jobs on a separate API key

Create a dedicated API key for background work — nightly statistics runs, data synchronisation, report generation — and do not send X-Repliers-Forwarded-For on those requests, since there is no end user to attribute them to.

This does three things:

• It stops a batch job from starving your live website of throughput.
• It makes both traffic streams far easier to diagnose, since a 429 on one key no longer tells you nothing about which workload caused it.
• It keeps your Security Insights protected/unprotected figures meaningful, since unforwarded background traffic is no longer counted against your live traffic's coverage.

Schedule batch work outside your peak hours, and pace it as described above rather than running it as fast as the limiter allows.

Back off correctly

Standard exponential backoff schedules are usually too aggressive for our limiter. A retry ladder of 2s, 4s, 8s, and 16s will typically continue to fail, because the limiter is still counting your earlier attempts.

Follow these rules instead:

1. Honour the Retry-After header returned with the 429 response rather than using a fixed schedule.
2. Wait at least 60 seconds after the most recent 429 before resuming. The clock restarts on every additional 429 you receive, so retrying early extends the throttling rather than resolving it.
3. Add jitter. Without it, every queued request retries at the same instant and immediately re-triggers the limit.
4. Drop concurrency to one while recovering, then ramp back up gradually once requests are succeeding again.
5. Do not retry user-facing requests indefinitely. For an interactive search, one retry and then a graceful message is a better experience than a page that hangs for a minute.

Also check the x-ratelimit-remaining header on successful responses — it is a useful signal for pacing a long-running job before it starts failing, and it tells you how close you are running to the ceiling under normal load.

Cache aggressively

Most applications request the same data far more often than that data changes. Caching remains the most effective way to reduce pressure on your allowance, and these approaches combine well:

1. Cache Repliers responses in Redis (or another key-value store) on your backend.
2. Use your framework's built-in data cache to store API responses between renders.
3. Cache rendered pages, so repeat visits are served without any Repliers request at all. Next.js ISR is a good fit here.
4. Put a CDN in front of high-volume pages — ad landing pages in particular — so that traffic never reaches your application until the cache expires.

Options 2, 3, and 4 need careful testing. If you cache at the CDN layer, exclude your framework's internal routes and configure a deployment ID so that a new release does not serve mismatched assets from cache. If your hosting platform already provides a CDN, using it is usually simpler than adding a third-party one in front.

Not all data needs the same freshness, and matching the cache duration to the data is where most of the savings come from:

Ad landing pages deserve special attention. Advertising crawlers (for example, Google AdsBot) verify these pages regularly, and if those checks fail with 429, your ad quality scores can suffer. Landing-page data rarely needs to be current to the second, so it is an ideal caching candidate.

Control crawler and AI bot traffic

Automated crawlers — search engines, advertising verifiers, and increasingly AI training and answer bots — can consume a large share of your throughput at exactly the wrong moment, because they do not pause during your peak hours.

• Traffic filtered by the Repliers firewall never reaches your allowance at all. This only applies to requests forwarded with X-Repliers-Forwarded-For, which is a further reason to apply the header consistently.
• Bots that crawl your own pages spend your budget, because your server makes the resulting Repliers calls under your key. Without forwarding, those calls are indistinguishable from your live user traffic and are pooled with it. Filtering has to happen at your edge, before the page is rendered.
• Use robots.txt to keep crawlers away from API routes and low-value paths, and disallow AI crawlers you do not wish to serve. Well-behaved bots honour it; the rest need blocking or challenging at your edge.
• Serve crawlers from cache. A cached landing page costs nothing against your allowance no matter how often it is fetched.
• We do not rate-limit requests from well-known search engine IP addresses, but those crawls still count toward your monthly quota.

Verifying That Your Traffic Is Protected

You can confirm how much of your traffic reaches the firewall yourself, without contacting support:

1. Log in at login.repliers.com.
2. Open Security Insights from the dashboard.
3. Review the split between Protected and Unprotected traffic.

Requests are counted as Protected when they carry the X-Repliers-Forwarded-For header, and Unprotected when they do not. Only protected traffic is eligible for per-user rate limiting; unprotected traffic is pooled and assessed against the key as a whole.

If you have deployed forwarding and still see substantial unprotected traffic, the header is likely missing on a subset of your routes — server-rendered pages, API routes added later, or endpoints other than /listings are common gaps.

Requesting a Higher Rate Limit

Throughput can be increased. Before we do so, we ask that all client-facing traffic on the account is routed through the Repliers firewall, verified through Security Insights.

The reason is practical rather than procedural. On an unprotected key we cannot distinguish one visitor from another, so raising the ceiling raises it for bots as much as for your users, and a single abusive consumer can still throttle everyone. Once forwarding is in place, unwanted traffic is filtered before it consumes your allowance, throttling can be targeted at the individual responsible for it, and the additional headroom reaches the people you intended it for.

Background jobs on a separate, unforwarded key do not affect this assessment.

When you request an increase, tell us your peak concurrent user count and your expected growth, so we can size the limit appropriately.

Quick Reference

Related Articles

• API Usage Limits and Overages
• Best Practices For Securing Your Application
• LLM Guidance For Configuring the Repliers API Firewall

What questions does this article answer?

• Why is the Repliers API returning 403 for my legitimate end users?
• What triggers an IP block, and how do I request a review?
• How does implementing the Repliers firewall change the way rate limiting is applied?
• Does sending X-Repliers-Forwarded-For change how my throughput is counted?
• Why does one abusive visitor or bot throttle my entire application?
• What is the difference between per-user and account-level rate limiting?
• Why am I receiving 429 responses at low request volumes?
• What is the difference between the rate limit and the monthly quota?
• Why do I get 429 responses when my average request rate looks modest?
• How do I enforce a rate limit on my own side, and what changes on serverless platforms?
• Why do my server-rendered pages get throttled when my client-side calls do not?
• How should I implement backoff when I hit the rate limit?
• Should my background jobs use a different API key?
• How long should I cache each type of Repliers data?
• How do I stop crawlers and AI bots from consuming my throughput?
• How do I check whether my traffic is going through the Repliers firewall?
• What do I need to do before requesting a higher rate limit?

Updated on: 05/08/2026