Connect Scrapfly to OpenClaw on Operator.io
Scrapfly is a powerful web scraping API that extracts data from any website. It lets developers handle tough sites with JavaScript rendering, proxy rotation, and anti-bot protection.
Automate Scrapfly with AI
Operator runs your personal agent on OpenClaw, the open source agent framework, and points it straight at Scrapfly. You message the agent on Telegram or Discord, tell it what you need in plain language, and it operates Scrapfly on its own, running actions like capture website screenshot, capture screenshot metadata (head), create scrapfly crawler.
Your agent reaches Scrapfly directly or through connectors like Composio MCP and Pipedream MCP, which handle the sign in and token refresh for you, so there is nothing to wire up and no API keys to paste.
What your agent can do with Scrapfly
Your agent can call any of these Scrapfly actions by name as part of a larger task. Ask for the outcome you want and it picks the right ones.
Capture Website Screenshot
Tool to capture a full-page or viewport screenshot of a website. Use when you need to take a screenshot with options like JS rendering, custom resolution, or accessibility testing. Returns the screenshot image directly. Supports vision d...
Capture Screenshot Metadata (HEAD)
Tool to capture screenshot metadata without downloading the image body. Use this for async screenshot workflows where you need the URL to retrieve the image later. Returns the screenshot URL in response, saving bandwidth compared to full...
Create Scrapfly Crawler
Tool to create a new web crawler to recursively crawl an entire website. Returns a crawler UUID for tracking progress. Use when you need to crawl multiple pages from a website with configurable limits and extraction rules.
Extract Structured Data
Tool to extract structured data from HTML or other content using AI models, LLM prompts, or custom templates. Use when you need to parse web pages or documents into structured JSON data. Supports predefined extraction models for common t...
Get Scrapfly Account Information
Tool to retrieve Scrapfly account information. Use after authenticating to get API credit balance and usage stats. Returns comprehensive account data including subscription plan, usage statistics, billing info, and project settings.
Get Crawler Artifact
Tool to download crawler artifact files in WARC or HAR format. Use when you need to retrieve the complete crawl results as an archive file. WARC format is recommended for large crawls as it includes gzip compression.
Get Crawler Contents
Tool to retrieve extracted content from crawled pages. Supports multiple output formats including markdown, text, HTML, and JSON. Use when you need to access the actual content extracted during a crawl, with optional filtering by URL and...
Get Crawler Status
Tool to get the current status of a crawler including progress, pages crawled, and completion state. Use for polling workflow to monitor crawl progress.
Get Crawler URLs
Tool to retrieve the list of discovered and crawled URLs from a crawler. Use when you need to get all URLs found during a crawl or filter by status to analyze failed URLs with error codes. Supports pagination for large result sets.
Scrapfly Scrape
Tool to perform a web scraping request. Use when you need to fetch a page with custom configuration like JS rendering, proxies, and extraction.
Scrapfly Scrape POST
Tool to scrape web pages using POST method to send data in the request body. Use when you need to scrape endpoints that require POST requests, such as form submissions or APIs that expect data payload.
Scrape With PUT
Tool to scrape web pages using PUT method with body payload. Use when the target API requires PUT requests with data in the request body. Forwards PUT request with custom body to the target URL. If not specified, content-type defaults to...
How to connect Scrapfly
You authorize Scrapfly once from your dashboard. Operator holds the connection and refreshes the access tokens on its own, so your agent keeps working with Scrapfly without you signing in again. The same setup unlocks every other app your agent can reach, so you only do it once.
When you are ready, the get started guide walks through standing up your OpenClaw agent.
Common questions about Scrapfly
- How do I connect Scrapfly to Operator?
- Connecting Scrapfly is a one time sign in from your Operator dashboard. Operator keeps the connection live and rotates the token on its own, so the agent stays connected to Scrapfly and you never reauthorize by hand.
- Can my agent pull and update data in Scrapfly?
- Yes. It can read records, write new ones, and answer questions about what is stored. Point it at Scrapfly and ask for a figure or a list, or have it keep a table in step with another app you have connected, all in plain language.
- Do I need to write code or manage Scrapfly API keys?
- No. Operator manages the Scrapfly connection and token refresh, so there are no API keys to paste and nothing to deploy.
- Can my agent use Scrapfly together with my other apps?
- Yes. The same agent reaches every app you connect, so it can move between Scrapfly and tools like Gigasheet, Nasa, Api ninjas in one job, reading from one and acting in another without you wiring anything between them.
More apps to automate
Apps your agent runs alongside Scrapfly, or browse all integrations.
Put your agent on Scrapfly
Sign in, connect Scrapfly, and hand your agent the work. Your first week is free.
Try for free