Skip to content
Lesuto Render Engine

Cloudflare Worker

Render Engine With
A Cloudflare Worker

The recommended setup. The Worker answers crawlers with rendered HTML and passes everyone else to your origin, wherever it is hosted.

Steps

Set It Up
Step By Step

  1. 01

    Put The Site On Cloudflare

    Import your DNS. Keep mail records as DNS only so email keeps working, and proxy the website records.

  2. 02

    Create The Worker

    Create a Worker and paste the script below. Store your API key as a Worker secret named RENDER_KEY, or paste it in place of YOUR_RENDER_KEY.

  3. 03

    Add Your Routes

    Add a route for every public hostname, for example www.yoursite.com/* and yoursite.com/*.

  4. 04

    Keep It To Your Pages

    Limit routes to the pages you want rendered and leave admin areas and static files on your origin.

  5. 05

    Check It Works

    After a real crawler visits, the page appears in the Pages tab. To check now, use Recache on a page in the Pages tab and wait until its Rendered time updates, then request that page with a crawler user agent: you get the rendered copy from the cache. A crawler user agent sent from your own machine never starts a fresh render, so a page with no cached copy still shows your normal page.

Copy The Rule

Replace YOUR_RENDER_KEY with your API key, or copy it from the Setup page in the app with your key filled in. Any failure falls through to your normal page.

Cloudflare WorkerJavaScript
// Cloudflare Worker on the routes of your site. Store the key as a secret named RENDER_KEY.
const CRAWLERS = /googlebot|bingbot|applebot|duckduckbot|yandexbot|baiduspider|oai-searchbot|chatgpt-user|gptbot|claude-searchbot|claude-user|claudebot|perplexity-user|perplexitybot|mistralai-user|duckassistbot|amazonbot|meta-externalagent|cohere-ai|youbot|facebookexternalhit|facebot|twitterbot|linkedinbot|pinterestbot|slackbot|discordbot|telegrambot|whatsapp/i;
const STATIC_FILE = /\.(js|css|xml|png|jpe?g|gif|pdf|ico|svg|webp|avif|woff2?|ttf|eot|map|txt|json|mp4|webm)$/i;
const RENDER_ENGINE = 'https://api.lesuto.com/api/v2/render-engine/render';
const PASS = [301, 302, 303, 307, 308, 404, 410];

export default {
  async fetch(request, env) {
    const ua = request.headers.get('User-Agent') || '';
    const { pathname } = new URL(request.url);
    if (request.method !== 'GET' || !CRAWLERS.test(ua) || STATIC_FILE.test(pathname)) return fetch(request);

    try {
      const res = await fetch(RENDER_ENGINE, {
        headers: {
          'X-Render-Key': env.RENDER_KEY || 'YOUR_RENDER_KEY',
          'X-Render-Url': request.url,
          // Set by Cloudflare to the address that connected; visitors cannot set it.
          'X-Render-Client-IP': request.headers.get('CF-Connecting-IP') || '',
          'User-Agent': ua,
        },
        redirect: 'manual',
        signal: AbortSignal.timeout(18000),
      });
      // Only pages Render Engine answered carry X-Render-Cache; anything else gets your site.
      if (res.headers.get('X-Render-Cache') && (res.ok || PASS.includes(res.status))) {
        const headers = new Headers({ 'Content-Type': 'text/html; charset=utf-8', 'Cache-Control': 'private, max-age=0', 'CDN-Cache-Control': 'no-store', 'Vary': 'User-Agent' });
        const location = res.headers.get('Location');
        if (location) headers.set('Location', location);
        return new Response(res.body, { status: res.status, headers });
      }
    } catch {}
    return fetch(request);
  },
};

Full Guide

Every detail, edge case and fix on the Lesuto docs.

Rendered Pages
In About 15 Minutes

Add the Worker, check one page, and your previews start looking right.

Set Up Render Engine With a Cloudflare Worker