crawler

19 proyectos comparten este topic de GitHub

Las líneas conectan a los miembros que están mediblemente relacionados entre sí. El tamaño de los puntos refleja las estrellas.

🧬 Miembros

Scrapling

🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale…

★ 70.3k

EasySpider

A visual no-code/code-free web crawler/spider易采集：一个可视化浏览器自动化测试/数据采集/…

★ 44.2k

Scrapegraph-ai

Python scraper based on AI

★ 28.5k

crawlee

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript…

★ 24.8k

crawlee-python

Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data…

★ 9.3k

xiaobei

为OPC/中小微企业量身打造的自媒体获客AI Agent

★ 8.3k

autoscraper

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

★ 7.6k

trafilatura

Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as…

★ 6.3k

myGPTReader

A community-driven way to read and chat with AI bots - powered by chatGPT.

★ 4.4k

scylla

Intelligent proxy pool for Humans™ to extract content from the internet and build your own Large Language…

★ 4k

AutoCrawler

Google, Naver multiprocess image web crawler (Selenium)

★ 1.7k

fess

Open-source, self-hosted enterprise & site search server built on OpenSearch. Crawls web / file / DB / cloud…

★ 1.1k

chatWeb

ChatWeb can crawl web pages, read PDF, DOCX, TXT, and extract the main content, then answer your questions…

★ 916

hacker-news-digest

:newspaper: Let ChatGPT Summarize Hacker News for You

★ 755

Craw4LLM

Official repository for "Craw4LLM: Efficient Web Crawling for LLM Pretraining"

★ 660

Fast-Powerful-Whisper-AI-Services-API

⚡ 一款用于自动语音识别 (ASR)、翻译的高性能异步 API。不需要购买Whisper…

★ 470

crw

Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for…

★ 454

scraperai

ScraperAI is an open-source, AI-powered tool designed to simplify web scraping for users of all skill levels.

★ 421

extractor

Use LLMs to robustly extract web data

★ 319

Medido a partir de los temas de GitHub compartidos por ambos proyectos, ponderado por cuán raros son cada uno de los temas.