Intelligent Web Crawler API

Extract content, download files, and scrape data from any website with a simple API call. Powered by Puppeteer and built for developers.

100%
OPEN SOURCE
8+
TASK TYPES
WEBSITES

Features

Content Extraction
Extract page content in HTML, Markdown, or plain text. Perfect for documentation, archiving, or content migration.
File Downloads
Download files from any webpage and receive them as base64-encoded data. Supports PDFs, ZIPs, documents, and more.
Link Extraction
Extract all hyperlinks from a page with their text and titles. Great for SEO analysis and site mapping.
Text Analysis
Count words, analyze headings, and extract structured text. Get detailed metrics about page content.
Screenshots
Capture full-page or viewport screenshots of any website. Returns images as base64-encoded data.
CSS Selectors
Find specific elements using CSS selectors. Extract precise data from any webpage structure.

Quick Start

# Extract page content curl -X POST https://getobj.com/api/crawl \ -H "Content-Type: application/json" \ -d '{ "url": "https://example.com", "format": "markdown" }' # Download files curl -X POST https://getobj.com/api/crawl \ -H "Content-Type: application/json" \ -d '{ "url": "https://example.com", "instruction": "download files" }'
const response = await fetch('https://getobj.com/api/crawl', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ url: 'https://example.com', instruction: 'extract links' }) }); const data = await response.json(); console.log(data.result.links);
import requests response = requests.post('https://getobj.com/api/crawl', json={ 'url': 'https://example.com', 'instruction': 'count word "example"' } ) data = response.json() print(data['result']['count'])
const axios = require('axios'); const response = await axios.post('https://getobj.com/api/crawl', { url: 'https://example.com', instruction: 'take screenshot' }); const screenshot = response.data.result.screenshot; // Base64-encoded image data
View Full Documentation