Web Scraping APIs

What Are Claude Skills?

By the Scrappey Research Team

What Are Claude Skills? — conceptual illustration
On this page

Claude Skills are reusable capability packages - a folder containing a SKILL.md file plus optional scripts and reference files - that Claude discovers and loads on demand to perform a specific task. Anthropic shipped Skills (officially "Agent Skills") in 2025 as an open standard. The SKILL.md file has YAML frontmatter (at minimum a name and a description) followed by Markdown instructions. Claude reads only the short description until your request matches it, then pulls in the full instructions, then any bundled files - a pattern called progressive disclosure that keeps the context window small. Skills run in Claude Code (from a .claude/skills/ directory), the Claude apps, and the Claude API / Agent SDK.

Quick facts

FormatFolder with SKILL.md (YAML frontmatter + Markdown) + optional scripts/files
Official nameAgent Skills (Anthropic, 2025); follows the open Agent Skills standard
Frontmattername (<=64 chars, lowercase + hyphens), description (<=1024 chars)
LoadingProgressive disclosure - metadata always (~100 tokens), body on trigger, files as referenced
Runs inClaude Code (.claude/skills/), Claude apps, Claude API / Agent SDK

Skills vs MCP servers vs tool calls

Skills, MCP servers, and tool use all give a model new abilities, but they sit at different layers and it is easy to confuse them. The short version: a tool call is a single function, an MCP server is a live connection that exposes many tools over a shared protocol, and a Skill is a package of instructions and code that Claude reads off the filesystem.

MechanismWhat it isLifecycleBest for
SkillFilesystem folder + SKILL.md (instructions + optional scripts)Reusable across conversations and projectsDomain expertise, multi-step procedures, repeatable workflows
MCP serverA service exposing tools over the Model Context ProtocolPersistent connection, statefulExternal services, real-time data, shared tools across clients
Tool use / function callingA JSON schema for one function in the API requestPer-request onlyStructured input/output for a single operation

The three compose: a Skill can instruct Claude to call an MCP tool, and an MCP server can be the thing a Skill shells out to. A Skill is the cheapest way to teach Claude a procedure, because its body only enters the context window when the task is relevant.

Anatomy of a SKILL.md

A Skill is a directory whose only required file is SKILL.md. Everything else - reference docs, example files, executable scripts - is optional and loaded lazily.

The YAML frontmatter is small on purpose. The two fields that matter are name (max 64 characters, lowercase letters, numbers and hyphens) and description (max ~1024 characters). The description is the single most important line you write: it is the only text Claude sees for every Skill at all times, and Claude matches your request against it to decide whether to load the rest. State both what the Skill does and when to use it. Claude Code adds optional fields such as allowed-tools (tools the Skill may use without prompting) and invocation controls, but those are extensions on top of the open standard.

Progressive disclosure is the design that makes Skills cheap. There are three levels: metadata (name + description, always loaded, roughly 100 tokens per Skill), instructions (the SKILL.md body, loaded only when the Skill is triggered), and resources (bundled files and scripts, read or executed only when the instructions reference them). A Skill that bundles fifty reference files costs almost nothing in context until one of those files is actually needed - and scripts run via the shell and return only their output, so their source never enters the context window.

Building a web-scraping Skill

A common, high-value Skill gives Claude reliable access to live web content. On its own, an AI agent can only reason over what is already in its context or its training data; a web-scraping Skill lets it fetch a current page, convert it to clean Markdown, and continue. Because a Skill is just instructions plus code, its reliability is inherited entirely from whatever it calls.

This is where the design choice matters. A naive Skill that does a plain HTTP GET works on simple sites and then fails the moment it hits a JavaScript-rendered page or a site behind verification - it silently returns a challenge page instead of content, and Claude has no way to know. A Skill that shells out to a managed web-data API instead inherits that API's residential proxies and real-browser rendering, and gets back clean Markdown ready for the model. The code example below is a minimal SKILL.md that does exactly this against the Scrappey API; the same shape works for any HTTP-callable backend.

Scrappey publishes a ready-made Skill for this (the scrappey-skill repository, auto-discovered by Claude Code and other SKILL.md-compatible agents) alongside a CLI, so you can drop it into .claude/skills/ rather than writing the wrapper yourself.

Code example

markdown
---
name: web-fetch
description: Fetch any web page (including JavaScript-rendered or verification-gated sites) and return it as clean Markdown. Use when the user asks to read, summarize, or extract content from a URL.
allowed-tools: Bash
---

# Web Fetch Skill

When the user gives you a URL to read or summarize, fetch it through the
Scrappey API so the page renders and any verification is handled, then work
from the returned Markdown.

Run this (the API key is in the SCRAPPEY_API_KEY environment variable):

```bash
curl -s -X POST "https://publisher.scrappey.com/api/v1?key=$SCRAPPEY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "cmd": "request.get", "url": "<TARGET_URL>", "markdown": true }' \
  | jq -r '.solution.markdown'
```

The Markdown is in `.solution.markdown`. Summarize or extract from that text;
do not parse raw HTML yourself.

Related terms

Concept map

How What Are Claude Skills connects

The terms most directly tied to this one. Hover a node to see its neighbours, click to preview, drag to rearrange.

0 terms · 0 connections
You are here · Web Scraping APIs
Building map…

Frequently asked questions

Are Claude Skills the same as MCP servers?

No. A Skill is a folder of instructions and code that Claude reads from the filesystem and loads on demand; an MCP server is a running service that exposes tools to Claude over the Model Context Protocol. They compose - a Skill can tell Claude to call an MCP tool - but a Skill needs no server or network connection of its own.

How does Claude decide when to use a Skill?

Claude always has every Skill's name and description in context (roughly 100 tokens each). When your request matches a description, Claude loads that Skill's full SKILL.md body, and then any files the body references. This is why the description should say both what the Skill does and when to use it.

Can a Skill scrape websites that block bots?

A Skill is only instructions plus code, so its reliability comes from what it calls. A Skill that does a plain HTTP request will fail on JavaScript-heavy or verification-gated pages. A Skill that calls a managed web-data API inherits that API's browser rendering and residential proxies, and returns clean Markdown the model can use.

Last updated: 2026-06-08