Speed up your scrapes by 500% with the maxAge parameter
maxAge
parameter to your request and we’ll:
maxAge
parameter to your scrape request. The value is in milliseconds - for example, 3600000
means “use cached data if it’s less than 1 hour old.”
300000
- For semi-dynamic content3600000
- For content that updates hourly86400000
- For daily-updated content604800000
- For relatively static contentmaxAge
enabled:
maxAge
defaults to 0
, which means always scrape freshmaxAge
, we scrape fresh automaticallymaxAge
maxAge
within scrapeOptions
to get cached results for pages we’ve seen recently.
maxAge
, each page in your crawl will benefit from the 500% speed improvement if we have recent cached data for that page.
Start using maxAge
today for dramatically faster scrapes and crawls!