Crawl frequency

Crawl frequency is how often a data source is checked, for example weekly, daily or hourly. It matters because the right frequency depends on how fast the underlying data actually changes: a clearance sale price might update several times a day, while a product's category or image rarely changes at all. Checking too infrequently means missing short-lived events like a flash sale or a brief MAP violation; checking too often wastes resources on data that hasn't changed and can draw unwanted attention from a site's rate limiting. Most e-commerce data plans offer a few fixed frequencies rather than a continuous crawl, matched to what the use case actually needs.

Where this shows up