PricewiseBot: how Pricewise collects prices
Pricewise shows price history for Pakistani online stores. The prices come from the stores' own public product pages, read by a small, polite bot.
- Identifies itself
- User agent "PricewiseBot/1.0 (+https://pricewise.danixsoft.com/bot; hello@danixsoft.com)".
- Follows robots.txt
- Read before every run. A path disallowed for PricewiseBot or for all bots is never fetched. No search, cart, account or filtered pages.
- Slow on purpose
- One request at a time per store, at least 6 seconds apart. Usually twice a day (06:30 and 18:30 PKT); every 3 hours during big sale days such as 11.11.
- Product pages only
- Only the specific product pages we track (a list kept in our repository), never a crawl of the store. A shopper’s "please track this" request is reviewed by a person; nothing is fetched on demand.
- Never fights protection
- A 403, 429 or 503 answer or a bot-check page stops that store for the rest of the run. We never solve captchas, rotate addresses or pretend to be a browser.
- Keeps little
- From each page: the price, the "was" price if shown, stock and the product name. Only changes are stored, one value per day.
Run a store?
To stop PricewiseBot, add User-agent: PricewiseBot / Disallow: / to your robots.txt, or email hello@danixsoft.com; we drop the store from the next run. If you have a product feed or an affiliate API, we would much rather use that.
Prices shown on Pricewise always say when they were checked and link to your page, where your price is the one that counts. See the stores we read on the stores page.