Scrapfly

12 tools available

Scrapfly is a web scraping API that enables developers to extract data from websites efficiently, offering features like JavaScript rendering, anti-bot protection bypass, and proxy rotation.

ai web scraping developer tools

Connect Scrapfly Visit Scrapfly ↗

Connect Scrapfly to Definable to automate repetitive tasks, sync data with the rest of your stack, trigger on real-world events. Personas call Scrapfly's 12 tools directly from chat or scheduled flows. You can wire it into any persona to read, write, and react when an event fires.

Common workflows with Scrapfly

Trigger on event

Listen for a new event
Parse and validate the payload
Take an action with the linked tool

Daily summary

Pull yesterday's activity
Summarise with the model of your choice
Send the digest via email or Slack

Cross-tool sync

Read records from one app
Transform with a persona
Write the result back via this integration

Scrapfly actions 12

Every Scrapfly action below is a callable tool any Definable persona can invoke.

Capture Website Screenshot

Tool to capture a full-page or viewport screenshot of a website. Use when you need to take a screenshot with options like JS rendering, custom resolution, or accessibility testing. Returns the screenshot image directly. Supports vision deficiency simulations and dark mode.

Capture Screenshot Metadata (HEAD)

Tool to capture screenshot metadata without downloading the image body. Use this for async screenshot workflows where you need the URL to retrieve the image later. Returns the screenshot URL in response, saving bandwidth compared to full screenshot retrieval.

Create Scrapfly Crawler

Tool to create a new web crawler to recursively crawl an entire website. Returns a crawler UUID for tracking progress. Use when you need to crawl multiple pages from a website with configurable limits and extraction rules.

Extract Structured Data

Tool to extract structured data from HTML or other content using AI models, LLM prompts, or custom templates. Use when you need to parse web pages or documents into structured JSON data. Supports predefined extraction models for common types (articles, products, events) or custom extraction via prompts/templates.

Get Scrapfly Account Information

Tool to retrieve Scrapfly account information. Use after authenticating to get API credit balance and usage stats. Returns comprehensive account data including subscription plan, usage statistics, billing info, and project settings.

Get Crawler Artifact

Tool to download crawler artifact files in WARC or HAR format. Use when you need to retrieve the complete crawl results as an archive file. WARC format is recommended for large crawls as it includes gzip compression.

Get Crawler Contents

Tool to retrieve extracted content from crawled pages. Supports multiple output formats including markdown, text, HTML, and JSON. Use when you need to access the actual content extracted during a crawl, with optional filtering by URL and format selection.

Get Crawler Status

Tool to get the current status of a crawler including progress, pages crawled, and completion state. Use for polling workflow to monitor crawl progress.

Get Crawler URLs

Tool to retrieve the list of discovered and crawled URLs from a crawler. Use when you need to get all URLs found during a crawl or filter by status to analyze failed URLs with error codes. Supports pagination for large result sets.

Scrapfly Scrape

Tool to perform a web scraping request. Use when you need to fetch a page with custom configuration like JS rendering, proxies, and extraction.

Scrapfly Scrape POST

Tool to scrape web pages using POST method to send data in the request body. Use when you need to scrape endpoints that require POST requests, such as form submissions or APIs that expect data payload.

Scrape With PUT

Tool to scrape web pages using PUT method with body payload. Use when the target API requires PUT requests with data in the request body. Forwards PUT request with custom body to the target URL. If not specified, content-type defaults to application/x-www-form-urlencoded.

Frequently asked questions

What can I automate with Scrapfly on Definable?

Anything Scrapfly exposes through its API. Common ai web scraping workflows on Definable include automate repetitive tasks, sync data with the rest of your stack, trigger on real-world events. Personas can call any of the 12 Scrapfly tools directly, then chain the result into another integration without you writing code.

How does Scrapfly authentication work?

Scrapfly uses API_KEY on Definable. You connect once from the integrations page, scoped to the permissions you choose, and from then on any persona that has the integration enabled can act on your behalf. Tokens are encrypted at rest and rotated automatically.

Is the Scrapfly integration included in my Definable plan?

Yes — every Definable plan, including Starter, includes access to all 12 Scrapfly tools. You only need a separate Scrapfly subscription if Scrapfly itself charges per seat or per API call.

Is using Scrapfly through Definable secure?

Every call from a persona to Scrapfly is logged with the user, persona, prompt, and response. Tokens never leave Definable's secrets vault, scopes are configurable per persona, and you can revoke access at any time from the integration page.

How do I get started with Scrapfly on Definable?

Sign up for Definable, open the integrations page, find Scrapfly, and connect via OAuth or API key. You can immediately attach Scrapfly to any persona and start running workflows. The free Starter plan includes 5,000 credits/month.

What Scrapfly actions does Definable expose?

Definable exposes all 12 Scrapfly actions as callable tools — including `Capture Website Screenshot`, `Capture Screenshot Metadata (HEAD)`, `Create Scrapfly Crawler`, plus 9 more. Each tool gets a typed parameter schema so personas know exactly how to call it.

Ready to automate with Scrapfly?

Wire it up in minutes. No coding required.

Start for free Schedule a demo

← All integrations