A price check is the same request to the same store, repeated for every product you track, twice a day, every day. Stores notice that faster than they notice a scraper pulling ten thousand pages once. This page counts what a monitoring job costs in addresses and where a free list stops being enough.
Amazon, Walmart and the large European retailers rate-limit by IP. The first symptom is a wrong number: a cached page, a price for the wrong region, a product marked out of stock while it sells fine. The 403 and the captcha arrive later, once the address has a reputation. By then you have written a day of bad prices into the database and nobody has flagged it.
A single address survives somewhere between fifty and two hundred product pages a day on a big store, depending on how the store feels that week. Past that, answers degrade before they stop.
Take a typical job: 5,000 products across three stores, checked morning and evening. That is 30,000 requests a day. At a hundred requests per address per day, with a pause of five to ten seconds between them, you need 300 addresses working the whole day without a single ban. Bans happen, so double it.
Free lists do not get you there. A list from this site is right on the day it goes up, half of it is gone by tomorrow, and a third of what remains sits in a country the store does not ship to. That is fine for a one-off check of a competitor's page and wrong for a job that has to produce the same numbers every morning.
A pool of several thousand addresses changes the math: each address touches a store a handful of times a day, which is below any threshold a retailer uses. RemProxy keeps a page sized for this exact job, with a thread calculator and a Python sample that reads prices from the store's own JSON-LD instead of the HTML: a pool sized for price monitoring.
The same product costs different money in Berlin, Warsaw and Austin, and the store picks the price from the address it sees. Route each store through addresses in the country you sell in, and pass the zip or currency parameter the store expects. A monitoring job that mixes countries produces a spread of prices that looks like competitor volatility and is nothing of the kind.
Most stores ship the price inside a <script type="application/ld+json">
block as offers.price with a currency next to it. Parse that. It
survives redesigns, it carries the currency, and its absence is your block
detector: a page with no offer object is a page the store served to a bot.
Treat it as a failed request, retry through a different address, and put the
one that failed in quarantine for ten minutes rather than deleting it.
Log the failure rate per address. When one address fails on every store, it is burned. When every address fails on one store, the store changed its markup and the parser needs a fix, and no amount of proxies will help.
Scrapy with a rotating proxy middleware covers stores that render prices on the server. Playwright handles the ones that load prices with JavaScript; give each browser context its own address and keep it for the whole session on that store, since a cart or a currency choice is tied to the IP. Octoparse and ParseHub do the same without code. Prisync, Price2Spy and Competera run the whole job on their side and charge per product, which beats a home-built setup under a few hundred SKUs and loses to it above a few thousand.
If the report still has holes after that, the pool is too small for the volume. Count again from the number of requests.