Skip to content
Lesuto Render Engine

Express

Render Engine With
Express Middleware

A small middleware forwards crawlers to Render Engine before your routes run.

Steps

Set It Up
Step By Step

  1. 01

    Add The Middleware

    Register it before the static file handlers for your pages.

  2. 02

    Set Your API Key

    Set RENDER_KEY in the environment and keep the key out of your code.

  3. 03

    Restart The Server

    Restart the process so the middleware loads.

  4. 04

    Check It Works

    After a real crawler visits, the page appears in the Pages tab. To check now, use Recache on a page in the Pages tab and wait until its Rendered time updates, then request that page with a crawler user agent: you get the rendered copy from the cache. A crawler user agent sent from your own machine never starts a fresh render, so a page with no cached copy still shows your normal page.

Copy The Rule

Replace YOUR_RENDER_KEY with your API key, or copy it from the Setup page in the app with your key filled in. Any failure falls through to your normal page.

Express middlewareJavaScript
// Before your other routes. Behind a proxy, set app.set('trust proxy', <that proxy>) (its address or
// the number of proxies, never true) so req.ip is the crawler's address; without it, req.ip is the
// address that connected.
const CRAWLERS = /googlebot|bingbot|applebot|duckduckbot|yandexbot|baiduspider|oai-searchbot|chatgpt-user|gptbot|claude-searchbot|claude-user|claudebot|perplexity-user|perplexitybot|mistralai-user|duckassistbot|amazonbot|meta-externalagent|cohere-ai|youbot|facebookexternalhit|facebot|twitterbot|linkedinbot|pinterestbot|slackbot|discordbot|telegrambot|whatsapp/i;
const STATIC_FILE = /\.(js|css|xml|png|jpe?g|gif|pdf|ico|svg|webp|avif|woff2?|ttf|eot|map|txt|json|mp4|webm)$/i;
const RENDER_ENGINE = 'https://api.lesuto.com/api/v2/render-engine/render';
const RENDER_KEY = process.env.RENDER_KEY || 'YOUR_RENDER_KEY';
const PASS = [301, 302, 303, 307, 308, 404, 410];

app.use(async (req, res, next) => {
  const ua = req.get('user-agent') || '';
  if (req.method !== 'GET' || !CRAWLERS.test(ua) || STATIC_FILE.test(req.path)) return next();

  try {
    const page = `${req.protocol}://${req.get('host')}${req.originalUrl}`;
    const response = await fetch(RENDER_ENGINE, {
      headers: { 'X-Render-Key': RENDER_KEY, 'X-Render-Url': page, 'X-Render-Client-IP': req.ip || '', 'User-Agent': ua },
      redirect: 'manual',
      signal: AbortSignal.timeout(18000),
    });
    // Only pages Render Engine answered carry X-Render-Cache; anything else gets your site.
    if (response.headers.get('X-Render-Cache') && (response.ok || PASS.includes(response.status))) {
      const location = response.headers.get('location');
      if (location) res.set('Location', location);
      res.set({ 'Cache-Control': 'private, max-age=0', 'CDN-Cache-Control': 'no-store', 'Vary': 'User-Agent' });
      res.status(response.status).type('html').send(await response.text());
      return;
    }
  } catch {}
  next();
});

Full Guide

Every detail, edge case and fix on the Lesuto docs.

Rendered Pages
In About 15 Minutes

Add the middleware, restart, and check one page.

Set Up Render Engine With Express