Website Content Crawler
by apify
Use this page to check what Website Content Crawler by apify actually returns, how pricing works, and whether it is the right Apify actor before you start a run.
Actor pages combine Apify platform data with editorial review. We keep affiliate links separate from the ranking and update or remove pages when the actor changes. Read our methodology or send a correction to hello@tryapify.com.
TL;DR: Website Content Crawler by apify is built for ai & llm workflows. It has 93.6k users and runs on Free.
What It Does
Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem.
Pros & Cons
Pros
- ✓93K+ users trust this tool
- ✓Completely free to use
- ✓AI-powered extraction
Cons
- ✗Requires Apify account
- ✗Data freshness varies
Key Stats
- Users
- 93,574
- Total Runs
- 23,550,435
- Pricing
- Free
- Rating
- View on Apify →
Quick Answers
Teams that need a ready-made ai & llm actor and want to compare pricing, reviews, and actor activity before running it.
Test a small run first, export the sample output, then decide whether the pricing model works for your volume.
apify/website-content-crawler
Free
Related Docs
What is Apify? Plain-English review of actors, pricing, and pros
Apify is a cloud platform with 2,000+ pre-built web scrapers (actors). Free tier included. Here is what it does, how pricing works, and who it suits.
Read guide →Apify API: get your token and run actors from Python or JavaScript
How to use the Apify API: get your token, install the Python or JS client, start actor runs, and pull datasets into your app. Code examples included.
Read guide →Scraping Dynamic Websites
How to scrape JavaScript-rendered websites using Playwright and Puppeteer. Learn browser automation, wait strategies, and handling single-page apps for dynamic scraping.
Read guide →Common Questions
What is Website Content Crawler?
Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem.
How much does Website Content Crawler cost?
Website Content Crawler is free to use on Apify.
Is Website Content Crawler free?
Yes, Website Content Crawler is free to use on Apify.
How to use Website Content Crawler?
Open Website Content Crawler on Apify, configure the input for the data you want to scrape, run the actor, and export the results when the run completes.
We earn commission from qualifying purchases. This does not affect our ratings.
Related Tools
RAG Web Browser
Web browser for OpenAI Assistants, RAG pipelines, or AI agents, similar to a web browser in ChatGPT. It queries Google Search, scrapes the top N pages, and returns their content as Markdown for further processing by an LLM. It can also scrape individual URLs.
Modern Manga Colorizer
AI-powered colorization tool for black and white manga panels. Upload manga panel images and get them automatically colorized with vibrant, appropriate colors that match manga art style. Maintains original line art and shading while applying professional colorization.
Manga AI OCR Translator
Extract and translate text from manga images using Nano Banana AI. Processes manga panel images to extract text in multiple languages and translates to multiple target languages simultaneously. Each extracted text includes the original text and translations to all selected languages.
Auto Video Thumbnail Generator
Generate eye-catching video thumbnails automatically using Google Gemini AI. Upload a video or provide a URL, and get multiple high-quality thumbnail options in seconds. Perfect for content creators, marketers, and video editors who need professional thumbnails without design skills.