> ## Documentation Index
> Fetch the complete documentation index at: https://docs.scrapeautomate.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Output Types

> You can specify the output type of the response you want to receive from the API.

## Plain HTML

The response will be plain HTML if you do not explicitly specify the output types with your request. This applies to both whether JavaScript rendering is enabled or disabled.

Example request:

<CodeGroup>
  ```shell cURL theme={null}
  curl --request GET \
      --url 'https://app.scrapeautomate.com/api/scraper?api_key=<exampleToken>&render=false&url=https%3A%2F%2Fexample.com%2F'  
  ```

  ```javascript Node.js theme={null}
  import axios from 'axios';

  const response = await axios.get('https://app.scrapeautomate.com/api/scraper?api_key=<exampleToken>&render=false&url=https%3A%2F%2Fexample.com%2F');  
  ```

  ```java Java theme={null}
  import java.io.IOException;
  import java.net.URI;
  import java.net.http.HttpClient;
  import java.net.http.HttpRequest;
  import java.net.http.HttpResponse;

  HttpClient client = HttpClient.newHttpClient();

  HttpRequest request = HttpRequest.newBuilder()
      .uri(URI.create("https://app.scrapeautomate.com/api/scraper?api_key=<exampleToken>&render=false&url=https%3A%2F%2Fexample.com%2F"))
      .GET()
      .build();

  HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());  
  ```

  ```python Python theme={null}
  import requests

  response = requests.get(
      'https://app.scrapeautomate.com/api/scraper?api_key=<exampleToken>&render=false&url=https%3A%2F%2Fexample.com%2F',
  )  
  ```

  ```php PHP theme={null}
  <?php
  $ch = curl_init();
  curl_setopt($ch, CURLOPT_URL, 'https://app.scrapeautomate.com/api/scraper?api_key=<exampleToken>&render=false&url=https%3A%2F%2Fexample.com%2F');
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
  curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET');

  $response = curl_exec($ch);

  curl_close($ch);  
  ```

  ```go Go theme={null}
  package main

  import (
  	"fmt"
  	"io"
  	"log"
  	"net/http"
  )

  func main() {
  	client := &http.Client{}
  	req, err := http.NewRequest("GET", "https://app.scrapeautomate.com/api/scraper?api_key=<exampleToken>&render=false&url=https%3A%2F%2Fexample.com%2F", nil)
  	if err != nil {
  		log.Fatal(err)
  	}
  	resp, err := client.Do(req)
  	if err != nil {
  		log.Fatal(err)
  	}
  	defer resp.Body.Close()
  	bodyText, err := io.ReadAll(resp.Body)
  	if err != nil {
  		log.Fatal(err)
  	}
  	fmt.Printf("%s\n", bodyText)
  }  
  ```

  ```http HTTP theme={null}
  GET /api/scraper?api_key=<exampleToken>&render=false&url=https%3A%2F%2Fexample.com%2F HTTP/1.1
  Host: app.scrapeautomate.com
  User-Agent: curl/8.2.1
  Accept: */*  
  ```
</CodeGroup>

```HTML theme={null}
<!doctype html>
<html>
<head>
    <title>Example Domain</title>

    <meta charset="utf-8" />
    <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />
    <style type="text/css">
...
```

## Markdown

ScrapeAutomate allows you to generate markdowns from a webpage very easily. By including the `markdown=true` parameter in your request, the response will be returned in Markdown format. This is useful for extracting content for LLMs or simplifying web data for better readability.

You can use this with both javascript rendering enabled or disabled, but you might need to use javascript rendering when loading dynamic content from a webpage.

Remember, ScrapeAutomate automatically removes unnecessary elements, such as navbars, from the body content.

<CodeGroup>
  ```shell cURL theme={null}
  curl --request GET \
      --url 'https://app.scrapeautomate.com/api/scraper?url=https%3A%2F%2Fexample.com%2F&api_key=<exampleToken>&markdown=true'  
  ```

  ```javascript Node.js theme={null}
  import axios from 'axios';

  const response = await axios.get('https://app.scrapeautomate.com/api/scraper?url=https%3A%2F%2Fexample.com%2F&api_key=<exampleToken>&markdown=true');  
  ```

  ```java Java theme={null}
  import java.io.IOException;
  import java.net.URI;
  import java.net.http.HttpClient;
  import java.net.http.HttpRequest;
  import java.net.http.HttpResponse;

  HttpClient client = HttpClient.newHttpClient();

  HttpRequest request = HttpRequest.newBuilder()
      .uri(URI.create("https://app.scrapeautomate.com/api/scraper?url=https%3A%2F%2Fexample.com%2F&api_key=<exampleToken>&markdown=true"))
      .GET()
      .build();

  HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());  
  ```

  ```python Python theme={null}
  import requests

  response = requests.get(
      'https://app.scrapeautomate.com/api/scraper?url=https%3A%2F%2Fexample.com%2F&api_key=<exampleToken>&markdown=true',
  )  
  ```

  ```php PHP theme={null}
  <?php
  $ch = curl_init();
  curl_setopt($ch, CURLOPT_URL, 'https://app.scrapeautomate.com/api/scraper?url=https%3A%2F%2Fexample.com%2F&api_key=<exampleToken>&markdown=true');
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
  curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET');

  $response = curl_exec($ch);

  curl_close($ch);  
  ```

  ```go Go theme={null}
  package main

  import (
  	"fmt"
  	"io"
  	"log"
  	"net/http"
  )

  func main() {
  	client := &http.Client{}
  	req, err := http.NewRequest("GET", "https://app.scrapeautomate.com/api/scraper?url=https%3A%2F%2Fexample.com%2F&api_key=<exampleToken>&markdown=true", nil)
  	if err != nil {
  		log.Fatal(err)
  	}
  	resp, err := client.Do(req)
  	if err != nil {
  		log.Fatal(err)
  	}
  	defer resp.Body.Close()
  	bodyText, err := io.ReadAll(resp.Body)
  	if err != nil {
  		log.Fatal(err)
  	}
  	fmt.Printf("%s\n", bodyText)
  }  
  ```

  ```http HTTP theme={null}
  GET /api/scraper?url=https%3A%2F%2Fexample.com%2F&api_key=<exampleToken>&markdown=true HTTP/1.1
  Host: app.scrapeautomate.com
  User-Agent: curl/8.2.1
  Accept: */*  
  ```
</CodeGroup>

```md https://example.com/ theme={null}
# Example Domain

This domain is for use in illustrative examples in documents. You may use this domain in literature without prior coordination or asking for permission.

[More information...](https://www.iana.org/domains/example)
```

## Screenshot

To capture a screenshot of the webpage, we provide two query parameters. Screenshots are currently available only in **PNG** format:

* `screenshot`: Capture a screenshot of the visible area of the webpage.
* `screenshot_full_page`: It captures a screenshot of the entire webpage, including the parts that are not visible.

Since screenshots rely on rendering the page visually, JavaScript rendering must be enabled to use this feature.

### Screenshot of Visible Area

To capture a screenshot of the visible area, include the `screenshot` query parameter and set it to `true` with a request. The response will contain a `png` image.

<CodeGroup>
  ```shell cURL theme={null}
  curl --request GET \
      --url 'https://app.scrapeautomate.com/api/scraper?api_key=<exampleToken>&render=true&url=https%3A%2F%2Fexample.com%2F&screenshot=true'  
  ```

  ```javascript Node.js theme={null}
  import axios from 'axios';

  const response = await axios.get('https://app.scrapeautomate.com/api/scraper?api_key=<exampleToken>&render=true&url=https%3A%2F%2Fexample.com%2F&screenshot=true');  
  ```

  ```java Java theme={null}
  import java.io.IOException;
  import java.net.URI;
  import java.net.http.HttpClient;
  import java.net.http.HttpRequest;
  import java.net.http.HttpResponse;

  HttpClient client = HttpClient.newHttpClient();

  HttpRequest request = HttpRequest.newBuilder()
      .uri(URI.create("https://app.scrapeautomate.com/api/scraper?api_key=<exampleToken>&render=true&url=https%3A%2F%2Fexample.com%2F&screenshot=true"))
      .GET()
      .build();

  HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());  
  ```

  ```python Python theme={null}
  import requests

  response = requests.get(
      'https://app.scrapeautomate.com/api/scraper?api_key=<exampleToken>&render=true&url=https%3A%2F%2Fexample.com%2F&screenshot=true',
  )  
  ```

  ```php PHP theme={null}
  <?php
  $ch = curl_init();
  curl_setopt($ch, CURLOPT_URL, 'https://app.scrapeautomate.com/api/scraper?api_key=<exampleToken>&render=true&url=https%3A%2F%2Fexample.com%2F&screenshot=true');
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
  curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET');

  $response = curl_exec($ch);

  curl_close($ch);  
  ```

  ```go Go theme={null}
  package main

  import (
  	"fmt"
  	"io"
  	"log"
  	"net/http"
  )

  func main() {
  	client := &http.Client{}
  	req, err := http.NewRequest("GET", "https://app.scrapeautomate.com/api/scraper?api_key=<exampleToken>&render=true&url=https%3A%2F%2Fexample.com%2F&screenshot=true", nil)
  	if err != nil {
  		log.Fatal(err)
  	}
  	resp, err := client.Do(req)
  	if err != nil {
  		log.Fatal(err)
  	}
  	defer resp.Body.Close()
  	bodyText, err := io.ReadAll(resp.Body)
  	if err != nil {
  		log.Fatal(err)
  	}
  	fmt.Printf("%s\n", bodyText)
  }  
  ```

  ```http HTTP theme={null}
  GET /api/scraper?api_key=<exampleToken>&render=true&url=https%3A%2F%2Fexample.com%2F&screenshot=true HTTP/1.1
  Host: app.scrapeautomate.com
  User-Agent: curl/8.2.1
  Accept: */*  
  ```
</CodeGroup>

### Full Page Screenshot

To capture a screenshot of the entire page, set the `screenshot_full_page` parameter to `true`. This will take a full-page screenshot of the webpage and return it as part of the API response.

<CodeGroup>
  ```shell cURL theme={null}
  curl --request GET \
      --url 'https://app.scrapeautomate.com/api/scraper?api_key=<exampleToken>&render=true&url=https%3A%2F%2Fexample.com%2F&screenshot_full_page=true'  
  ```

  ```javascript Node.js theme={null}
  import axios from 'axios';

  const response = await axios.get('https://app.scrapeautomate.com/api/scraper?api_key=<exampleToken>&render=true&url=https%3A%2F%2Fexample.com%2F&screenshot_full_page=true');  
  ```

  ```java Java theme={null}
  import java.io.IOException;
  import java.net.URI;
  import java.net.http.HttpClient;
  import java.net.http.HttpRequest;
  import java.net.http.HttpResponse;

  HttpClient client = HttpClient.newHttpClient();

  HttpRequest request = HttpRequest.newBuilder()
      .uri(URI.create("https://app.scrapeautomate.com/api/scraper?api_key=<exampleToken>&render=true&url=https%3A%2F%2Fexample.com%2F&screenshot_full_page=true"))
      .GET()
      .build();

  HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());  
  ```

  ```python Python theme={null}
  import requests

  response = requests.get(
      'https://app.scrapeautomate.com/api/scraper?api_key=<exampleToken>&render=true&url=https%3A%2F%2Fexample.com%2F&screenshot_full_page=true',
  )  
  ```

  ```php PHP theme={null}
  <?php
  $ch = curl_init();
  curl_setopt($ch, CURLOPT_URL, 'https://app.scrapeautomate.com/api/scraper?api_key=<exampleToken>&render=true&url=https%3A%2F%2Fexample.com%2F&screenshot_full_page=true');
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
  curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET');

  $response = curl_exec($ch);

  curl_close($ch);  
  ```

  ```go Go theme={null}
  package main

  import (
  	"fmt"
  	"io"
  	"log"
  	"net/http"
  )

  func main() {
  	client := &http.Client{}
  	req, err := http.NewRequest("GET", "https://app.scrapeautomate.com/api/scraper?api_key=<exampleToken>&render=true&url=https%3A%2F%2Fexample.com%2F&screenshot_full_page=true", nil)
  	if err != nil {
  		log.Fatal(err)
  	}
  	resp, err := client.Do(req)
  	if err != nil {
  		log.Fatal(err)
  	}
  	defer resp.Body.Close()
  	bodyText, err := io.ReadAll(resp.Body)
  	if err != nil {
  		log.Fatal(err)
  	}
  	fmt.Printf("%s\n", bodyText)
  }  
  ```

  ```http HTTP theme={null}
  GET /api/scraper?api_key=<exampleToken>&render=true&url=https%3A%2F%2Fexample.com%2F&screenshot_full_page=true HTTP/1.1
  Host: app.scrapeautomate.com
  User-Agent: curl/8.2.1
  Accept: */*  
  ```
</CodeGroup>

## Webhook

If you prefer to have your responses sent to a webhook instead of receiving them directly, you can utilize our **Webhook feature**. To use the webhook you need to simply include your `webhook_url` in the query parameters, and the API will send the response to that webhook. You can receive all types of responses, including HTML, Markdown, and screenshots.

<CodeGroup>
  ```shell cURL theme={null}
  curl --request GET \
      --url 'http://localhost:8002/api/scraper?api_key=<exampleToken>&render=true&url=https%253A%252F%252Fexample.com%252Fees&config=%7B%22window_width%22%3A1920%2C%22window_height%22%3A1080%7D&webhook_url=https%3A%2F%2Fwebhook.site%2F7132edb4-8590-4bc0-9055-30e2153936d1'  
  ```

  ```javascript Node.js theme={null}
  import axios from 'axios';

  const response = await axios.get('http://localhost:8002/api/scraper?api_key=<exampleToken>&render=true&url=https%253A%252F%252Fexample.com%252Fees&config=%7B%22window_width%22%3A1920%2C%22window_height%22%3A1080%7D&webhook_url=https%3A%2F%2Fwebhook.site%2F7132edb4-8590-4bc0-9055-30e2153936d1');  
  ```

  ```java Java theme={null}
  import java.io.IOException;
  import java.net.URI;
  import java.net.http.HttpClient;
  import java.net.http.HttpRequest;
  import java.net.http.HttpResponse;

  HttpClient client = HttpClient.newHttpClient();

  HttpRequest request = HttpRequest.newBuilder()
      .uri(URI.create("http://localhost:8002/api/scraper?api_key=<exampleToken>&render=true&url=https%253A%252F%252Fexample.com%252Fees&config=%7B%22window_width%22%3A1920%2C%22window_height%22%3A1080%7D&webhook_url=https%3A%2F%2Fwebhook.site%2F7132edb4-8590-4bc0-9055-30e2153936d1"))
      .GET()
      .build();

  HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());  
  ```

  ```python Python theme={null}
  import requests

  response = requests.get(
      'http://localhost:8002/api/scraper?api_key=<exampleToken>&render=true&url=https%253A%252F%252Fexample.com%252Fees&config=%7B%22window_width%22%3A1920%2C%22window_height%22%3A1080%7D&webhook_url=https%3A%2F%2Fwebhook.site%2F7132edb4-8590-4bc0-9055-30e2153936d1',
  )  
  ```

  ```php PHP theme={null}
  <?php
  $ch = curl_init();
  curl_setopt($ch, CURLOPT_URL, 'http://localhost:8002/api/scraper?api_key=<exampleToken>&render=true&url=https%253A%252F%252Fexample.com%252Fees&config=%7B%22window_width%22%3A1920%2C%22window_height%22%3A1080%7D&webhook_url=https%3A%2F%2Fwebhook.site%2F7132edb4-8590-4bc0-9055-30e2153936d1');
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
  curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET');

  $response = curl_exec($ch);

  curl_close($ch);  
  ```

  ```go Go theme={null}
  package main

  import (
  	"fmt"
  	"io"
  	"log"
  	"net/http"
  )

  func main() {
  	client := &http.Client{}
  	req, err := http.NewRequest("GET", "http://localhost:8002/api/scraper?api_key=<exampleToken>&render=true&url=https%253A%252F%252Fexample.com%252Fees&config=%7B%22window_width%22%3A1920%2C%22window_height%22%3A1080%7D&webhook_url=https%3A%2F%2Fwebhook.site%2F7132edb4-8590-4bc0-9055-30e2153936d1", nil)
  	if err != nil {
  		log.Fatal(err)
  	}
  	resp, err := client.Do(req)
  	if err != nil {
  		log.Fatal(err)
  	}
  	defer resp.Body.Close()
  	bodyText, err := io.ReadAll(resp.Body)
  	if err != nil {
  		log.Fatal(err)
  	}
  	fmt.Printf("%s\n", bodyText)
  }  
  ```

  ```http HTTP theme={null}
  GET /api/scraper?api_key=<exampleToken>&render=true&url=https%253A%252F%252Fexample.com%252Fees&config=%7B%22window_width%22%3A1920%2C%22window_height%22%3A1080%7D&webhook_url=https%3A%2F%2Fwebhook.site%2F7132edb4-8590-4bc0-9055-30e2153936d1 HTTP/1.1
  Host: localhost:8002
  User-Agent: curl/8.2.1
  Accept: */*  
  ```
</CodeGroup>

When you call the API, it will automatically send the response to the specified webhook instead of returning the response data directly to the requester. However, you will still receive a confirmation indicating whether or not the webhook was successfully triggered. Here's an example:

```json theme={null}
{
  "status": "ok",
  "message": "Webhook sent successfully"
}
```

One important thing to note is that if you use a webhook, you will always receive a success message in response to your API request, even if an error occurs during the scraping process. Any errors encountered will be sent directly to the webhook rather than being returned in the main request.

This functionality also applies to API workflows. Just ensure that you include the webhook URL in the query parameters when creating or editing the workflow. This way, when you send a request to the scraper route, the response will be delivered directly to the specified webhook.

<AccordionGroup>
  <Accordion title="1. Can I use multiple output types?">
    Currently, we do not support multiple output types. However, in the near
    future, you will have the ability to select multiple output types!
  </Accordion>

  <Accordion title="2. Can I use a webhook with API workflows?">
    Yes, you can use a webhook with workflows. Just make sure to set the webhook
    when creating or edit a workflow.
  </Accordion>

  <Accordion title="3. How can I ensure that the entire webpage has loaded before returning an response?">
    By default, we already wait for the entire webpage to load before returning a
    response!
  </Accordion>
</AccordionGroup>
