Express
Render Engine With
Express Middleware
A small middleware forwards crawlers to Render Engine before your routes run.
Steps
Set It Up
Step By Step
- 01
Add The Middleware
Register it before the static file handlers for your pages.
- 02
Set Your API Key
Set RENDER_KEY in the environment and keep the key out of your code.
- 03
Restart The Server
Restart the process so the middleware loads.
- 04
Check It Works
After a real crawler visits, the page appears in the Pages tab. To check now, use Recache on a page in the Pages tab and wait until its Rendered time updates, then request that page with a crawler user agent: you get the rendered copy from the cache. A crawler user agent sent from your own machine never starts a fresh render, so a page with no cached copy still shows your normal page.
Copy The Rule
Replace YOUR_RENDER_KEY with your API key, or copy it from the Setup page in the app with your key filled in. Any failure falls through to your normal page.
// Before your other routes. Behind a proxy, set app.set('trust proxy', <that proxy>) (its address or
// the number of proxies, never true) so req.ip is the crawler's address; without it, req.ip is the
// address that connected.
const CRAWLERS = /googlebot|bingbot|applebot|duckduckbot|yandexbot|baiduspider|oai-searchbot|chatgpt-user|gptbot|claude-searchbot|claude-user|claudebot|perplexity-user|perplexitybot|mistralai-user|duckassistbot|amazonbot|meta-externalagent|cohere-ai|youbot|facebookexternalhit|facebot|twitterbot|linkedinbot|pinterestbot|slackbot|discordbot|telegrambot|whatsapp/i;
const STATIC_FILE = /\.(js|css|xml|png|jpe?g|gif|pdf|ico|svg|webp|avif|woff2?|ttf|eot|map|txt|json|mp4|webm)$/i;
const RENDER_ENGINE = 'https://api.lesuto.com/api/v2/render-engine/render';
const RENDER_KEY = process.env.RENDER_KEY || 'YOUR_RENDER_KEY';
const PASS = [301, 302, 303, 307, 308, 404, 410];
app.use(async (req, res, next) => {
const ua = req.get('user-agent') || '';
if (req.method !== 'GET' || !CRAWLERS.test(ua) || STATIC_FILE.test(req.path)) return next();
try {
const page = `${req.protocol}://${req.get('host')}${req.originalUrl}`;
const response = await fetch(RENDER_ENGINE, {
headers: { 'X-Render-Key': RENDER_KEY, 'X-Render-Url': page, 'X-Render-Client-IP': req.ip || '', 'User-Agent': ua },
redirect: 'manual',
signal: AbortSignal.timeout(18000),
});
// Only pages Render Engine answered carry X-Render-Cache; anything else gets your site.
if (response.headers.get('X-Render-Cache') && (response.ok || PASS.includes(response.status))) {
const location = response.headers.get('location');
if (location) res.set('Location', location);
res.set({ 'Cache-Control': 'private, max-age=0', 'CDN-Cache-Control': 'no-store', 'Vary': 'User-Agent' });
res.status(response.status).type('html').send(await response.text());
return;
}
} catch {}
next();
});Full Guide
Every detail, edge case and fix on the Lesuto docs.
Other Ways To Set Up
Pick the stack that matches your site, or start from the checklist.
Rendered Pages
In About 15 Minutes
Add the middleware, restart, and check one page.