Cloudflare Worker
Render Engine With
A Cloudflare Worker
The recommended setup. The Worker answers crawlers with rendered HTML and passes everyone else to your origin, wherever it is hosted.
Steps
Set It Up
Step By Step
- 01
Put The Site On Cloudflare
Import your DNS. Keep mail records as DNS only so email keeps working, and proxy the website records.
- 02
Create The Worker
Create a Worker and paste the script below. Store your API key as a Worker secret named RENDER_KEY, or paste it in place of YOUR_RENDER_KEY.
- 03
Add Your Routes
Add a route for every public hostname, for example www.yoursite.com/* and yoursite.com/*.
- 04
Keep It To Your Pages
Limit routes to the pages you want rendered and leave admin areas and static files on your origin.
- 05
Check It Works
After a real crawler visits, the page appears in the Pages tab. To check now, use Recache on a page in the Pages tab and wait until its Rendered time updates, then request that page with a crawler user agent: you get the rendered copy from the cache. A crawler user agent sent from your own machine never starts a fresh render, so a page with no cached copy still shows your normal page.
Copy The Rule
Replace YOUR_RENDER_KEY with your API key, or copy it from the Setup page in the app with your key filled in. Any failure falls through to your normal page.
// Cloudflare Worker on the routes of your site. Store the key as a secret named RENDER_KEY.
const CRAWLERS = /googlebot|bingbot|applebot|duckduckbot|yandexbot|baiduspider|oai-searchbot|chatgpt-user|gptbot|claude-searchbot|claude-user|claudebot|perplexity-user|perplexitybot|mistralai-user|duckassistbot|amazonbot|meta-externalagent|cohere-ai|youbot|facebookexternalhit|facebot|twitterbot|linkedinbot|pinterestbot|slackbot|discordbot|telegrambot|whatsapp/i;
const STATIC_FILE = /\.(js|css|xml|png|jpe?g|gif|pdf|ico|svg|webp|avif|woff2?|ttf|eot|map|txt|json|mp4|webm)$/i;
const RENDER_ENGINE = 'https://api.lesuto.com/api/v2/render-engine/render';
const PASS = [301, 302, 303, 307, 308, 404, 410];
export default {
async fetch(request, env) {
const ua = request.headers.get('User-Agent') || '';
const { pathname } = new URL(request.url);
if (request.method !== 'GET' || !CRAWLERS.test(ua) || STATIC_FILE.test(pathname)) return fetch(request);
try {
const res = await fetch(RENDER_ENGINE, {
headers: {
'X-Render-Key': env.RENDER_KEY || 'YOUR_RENDER_KEY',
'X-Render-Url': request.url,
// Set by Cloudflare to the address that connected; visitors cannot set it.
'X-Render-Client-IP': request.headers.get('CF-Connecting-IP') || '',
'User-Agent': ua,
},
redirect: 'manual',
signal: AbortSignal.timeout(18000),
});
// Only pages Render Engine answered carry X-Render-Cache; anything else gets your site.
if (res.headers.get('X-Render-Cache') && (res.ok || PASS.includes(res.status))) {
const headers = new Headers({ 'Content-Type': 'text/html; charset=utf-8', 'Cache-Control': 'private, max-age=0', 'CDN-Cache-Control': 'no-store', 'Vary': 'User-Agent' });
const location = res.headers.get('Location');
if (location) headers.set('Location', location);
return new Response(res.body, { status: res.status, headers });
}
} catch {}
return fetch(request);
},
};Full Guide
Every detail, edge case and fix on the Lesuto docs.
Other Ways To Set Up
Pick the stack that matches your site, or start from the checklist.
Rendered Pages
In About 15 Minutes
Add the Worker, check one page, and your previews start looking right.