Firecrawl allows you to turn entire websites into LLM-ready markdown
scrape_url
method. It takes the URL as a parameter and returns the scraped data as a dictionary.
async crawl
functions on SDKs, this will return an ID
where you can use to check the status of the crawl.
next
URL parameter is provided. You must request this URL to retrieve the next 10MB of data. If the next
parameter is absent, it indicates the end of the crawl data.
prompt
to the endpoint. The llm chooses the structure of the data.
wait
action before/after executing other actions to give enough time for the page to load.