> ## Documentation Index
> Fetch the complete documentation index at: https://docs.evocrawl.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Node

> Scrape, crawl, and extract structured data from websites using the EvoCrawl Node SDK.

Scrape single pages, crawl entire sites, and map URLs from your Node.js application. The SDK handles pagination, retries, and async job polling so you can focus on working with the returned data.

## Installation

To install the EvoCrawl Node SDK, you can use npm:

```js Node theme={null}
# npm install @mendable/evocrawl-js

import Evocrawl from '@mendable/evocrawl-js';

const evocrawl = new Evocrawl({ apiKey: "fc-YOUR-API-KEY" });
```

## Usage

1. Get an API key from [evocrawl.com](https://evocrawl.com)
2. Set the API key as an environment variable named `EVOCRAWL_API_KEY` or pass it as a parameter to the `EvocrawlApp` class.

Here's an example of how to use the SDK with error handling:

```js Node theme={null}
import Evocrawl from '@mendable/evocrawl-js';

const evocrawl = new Evocrawl({apiKey: "fc-YOUR_API_KEY"});

// Scrape a website
const scrapeResponse = await evocrawl.scrape('https://evocrawl.com', {
  formats: ['markdown', 'html'],
});

console.log(scrapeResponse)

// Crawl a website
const crawlResponse = await evocrawl.crawl('https://evocrawl.com', {
  limit: 100,
  scrapeOptions: {
    formats: ['markdown', 'html'],
  }
});

console.log(crawlResponse)
```

### Scraping a URL

To scrape a single URL with error handling, use the `scrapeUrl` method. It takes the URL as a parameter and returns the scraped data as a dictionary.

```js Node theme={null}
// Scrape a website:
const scrapeResult = await evocrawl.scrape('evocrawl.com', { formats: ['markdown', 'html'] });

console.log(scrapeResult)
```

### Crawling a Website

To crawl a website with error handling, use the `crawlUrl` method. It takes the starting URL and optional parameters as arguments. The `params` argument allows you to specify additional options for the crawl job, such as the maximum number of pages to crawl, allowed domains, and the output format. See [Pagination](#pagination) for auto/ manual pagination and limiting.

```js Node theme={null}
const job = await evocrawl.crawl('https://docs.evocrawl.com', { limit: 5, pollInterval: 1, timeout: 120 });
console.log(job.status);
```

### Sitemap-Only Crawl

Use `sitemap: "only"` to crawl sitemap URLs only (the start URL is always included, and HTML link discovery is skipped).

```js Node theme={null}
const job = await evocrawl.crawl('https://docs.evocrawl.com', {
  sitemap: 'only',
  limit: 25,
});
console.log(job.status, job.data.length);
```

### Start a Crawl

Start a job without waiting using `startCrawl`. It returns a job `ID` you can use to check status. Use `crawl` when you want a waiter that blocks until completion. See [Pagination](#pagination) for paging behavior and limits.

```js Node theme={null}
const { id } = await evocrawl.startCrawl('https://docs.evocrawl.com', { limit: 10 });
console.log(id);
```

### Checking Crawl Status

To check the status of a crawl job with error handling, use the `checkCrawlStatus` method. It takes the `ID` as a parameter and returns the current status of the crawl job.

```js Node theme={null}
const status = await evocrawl.getCrawlStatus("<crawl-id>");
console.log(status);
```

### Cancelling a Crawl

To cancel an crawl job, use the `cancelCrawl` method. It takes the job ID of the `startCrawl` as a parameter and returns the cancellation status.

```js Node theme={null}
const ok = await evocrawl.cancelCrawl("<crawl-id>");
console.log("Cancelled:", ok);
```

### Mapping a Website

To map a website with error handling, use the `mapUrl` method. It takes the starting URL as a parameter and returns the mapped data as a dictionary.

```js Node theme={null}
const res = await evocrawl.map('https://evocrawl.com', { limit: 10 });
console.log(res.links);
```

### Crawling a Website with WebSockets

To crawl a website with WebSockets, use the `crawlUrlAndWatch` method. It takes the starting URL and optional parameters as arguments. The `params` argument allows you to specify additional options for the crawl job, such as the maximum number of pages to crawl, allowed domains, and the output format.

```js Node theme={null}
import Evocrawl from '@mendable/evocrawl-js';

const evocrawl = new Evocrawl({ apiKey: 'fc-YOUR-API-KEY' });

// Start a crawl and then watch it
const { id } = await evocrawl.startCrawl('https://mendable.ai', {
  excludePaths: ['blog/*'],
  limit: 5,
});

const watcher = evocrawl.watcher(id, { kind: 'crawl', pollInterval: 2, timeout: 120 });

watcher.on('document', (doc) => {
  console.log('DOC', doc);
});

watcher.on('error', (err) => {
  console.error('ERR', err?.error || err);
});

watcher.on('done', (state) => {
  console.log('DONE', state.status);
});

// Begin watching (WS with HTTP fallback)
await watcher.start();
```

### Pagination

EvoCrawl endpoints for crawl and batch return a `next` URL when more data is available. The Node SDK auto-paginates by default and aggregates all documents; in that case `next` will be `null`. You can disable auto-pagination or set limits.

#### Crawl

Use the waiter method `crawl` for the simplest experience, or start a job and page manually.

##### Simple crawl (auto-pagination, default)

* See the default flow in [Crawling a Website](#crawling-a-website).

##### Manual crawl with pagination control (single page)

* Start a job, then fetch one page at a time with `autoPaginate: false`.

```js Node theme={null}
const crawlStart = await evocrawl.startCrawl('https://docs.evocrawl.com', { limit: 5 });
const crawlJobId = crawlStart.id;

const crawlSingle = await evocrawl.getCrawlStatus(crawlJobId, { autoPaginate: false });
console.log('crawl single page:', crawlSingle.status, 'docs:', crawlSingle.data.length, 'next:', crawlSingle.next);
```

##### Manual crawl with limits (auto-pagination + early stop)

* Keep auto-pagination on but stop early with `maxPages`, `maxResults`, or `maxWaitTime`.

```js Node theme={null}
const crawlLimited = await evocrawl.getCrawlStatus(crawlJobId, {
  autoPaginate: true,
  maxPages: 2,
  maxResults: 50,
  maxWaitTime: 15,
});
console.log('crawl limited:', crawlLimited.status, 'docs:', crawlLimited.data.length, 'next:', crawlLimited.next);
```

#### Batch Scrape

Use the waiter method `batchScrape`, or start a job and page manually.

##### Simple batch scrape (auto-pagination, default)

* See the default flow in [Batch Scrape](/features/batch-scrape).

##### Manual batch scrape with pagination control (single page)

* Start a job, then fetch one page at a time with `autoPaginate: false`.

```js Node theme={null}
const batchStart = await evocrawl.startBatchScrape([
  'https://docs.evocrawl.com',
  'https://evocrawl.com',
], { options: { formats: ['markdown'] } });
const batchJobId = batchStart.id;

const batchSingle = await evocrawl.getBatchScrapeStatus(batchJobId, { autoPaginate: false });
console.log('batch single page:', batchSingle.status, 'docs:', batchSingle.data.length, 'next:', batchSingle.next);
```

##### Manual batch scrape with limits (auto-pagination + early stop)

* Keep auto-pagination on but stop early with `maxPages`, `maxResults`, or `maxWaitTime`.

```js Node theme={null}
const batchLimited = await evocrawl.getBatchScrapeStatus(batchJobId, {
  autoPaginate: true,
  maxPages: 2,
  maxResults: 100,
  maxWaitTime: 20,
});
console.log('batch limited:', batchLimited.status, 'docs:', batchLimited.data.length, 'next:', batchLimited.next);
```

## Browser

Launch cloud browser sessions and execute code remotely.

### Create a Session

```js Node theme={null}
import Evocrawl from '@mendable/evocrawl-js';

const evocrawl = new Evocrawl({ apiKey: "fc-YOUR-API-KEY" });

const session = await evocrawl.browser({ ttl: 600 });
console.log(session.id);          // session ID
console.log(session.cdpUrl);      // wss://cdp-proxy.evocrawl.com/cdp/...
console.log(session.liveViewUrl); // https://liveview.evocrawl.com/...
```

### Execute Code

```js Node theme={null}
const result = await evocrawl.browserExecute(session.id, {
  code: 'await page.goto("https://news.ycombinator.com")\ntitle = await page.title()\nprint(title)',
});
console.log(result.result); // "Hacker News"
```

Execute JavaScript instead of Python:

```js Node theme={null}
const result = await evocrawl.browserExecute(session.id, {
  code: 'await page.goto("https://example.com"); const t = await page.title(); console.log(t);',
  language: "node",
});
```

Execute bash with agent-browser:

```js Node theme={null}
const result = await evocrawl.browserExecute(session.id, {
  code: "agent-browser open https://example.com && agent-browser snapshot",
  language: "bash",
});
```

### Profiles

Save and reuse browser state (cookies, localStorage, etc.) across sessions:

```js Node theme={null}
const session = await evocrawl.browser({
  ttl: 600,
  profile: {
    name: "my-profile",
    saveChanges: true,
  },
});
```

### Connect via CDP

For full Playwright control, connect directly using the CDP URL:

```js Node theme={null}
import { chromium } from "playwright";

const browser = await chromium.connectOverCDP(session.cdpUrl);
const context = browser.contexts()[0];
const page = context.pages()[0] || await context.newPage();

await page.goto("https://example.com");
console.log(await page.title());

await browser.close();
```

### List & Close Sessions

```js Node theme={null}
// List active sessions
const { sessions } = await evocrawl.listBrowsers({ status: "active" });
for (const s of sessions) {
  console.log(s.id, s.status, s.createdAt);
}

// Close a session
await evocrawl.deleteBrowser(session.id);
```

### Scrape-Bound Interactive Session

Use a scrape job ID to keep interacting with the replayed page context from that scrape:

* `interact(jobId, {...})` runs code in the scrape-bound browser session.
* First `interact` call auto-initializes the session from the scrape context.
* Additional `interact` calls on the same job ID reuse that live browser state.
* `stopInteraction(jobId)` stops the interactive session when you are done.

```js Node theme={null}
const doc = await evocrawl.scrape("https://example.com", {
  actions: [{ type: "click", selector: "a[href='/pricing']" }],
});

const scrapeJobId = doc.metadata?.scrapeId;
if (!scrapeJobId) throw new Error("Missing scrape job id");

const run = await evocrawl.interact(scrapeJobId, {
  code: "console.log(await page.url())",
  language: "node",
  timeout: 60,
});
console.log(run.stdout);

await evocrawl.stopInteraction(scrapeJobId);
```

## Error Handling

The SDK handles errors returned by the EvoCrawl API and raises appropriate exceptions. If an error occurs during a request, an exception will be raised with a descriptive error message. The examples above demonstrate how to handle these errors using `try/catch` blocks.

> Are you an AI agent that needs a EvoCrawl API key? See [evocrawl.com/agent-onboarding/SKILL.md](https://www.evocrawl.com/agent-onboarding/SKILL.md) for automated onboarding instructions.