Apache
Render Engine With
APACHE
A few lines in your virtual host forward crawlers to Render Engine and leave everyone else on your site.
Steps
Set It Up
Step By Step
- 01
Turn On The Modules
Apache 2.4.47 or later with mod_rewrite, mod_proxy, mod_proxy_http, mod_headers and mod_ssl. Render Engine is reached over https, so mod_ssl is needed even when your own site is plain http. On Debian or Ubuntu: a2enmod rewrite proxy proxy_http headers ssl.
- 02
Add The Rule
Paste the snippet inside the VirtualHost block for your public hostname, and change Define LESUTO_SITE to your site's own address, for example https://yourbrand.com, so a refused render falls back to your normal page.
- 03
Add Your API Key
Replace YOUR_RENDER_KEY with your key, or copy the snippet from the Setup page in the app with your key filled in.
- 04
Test And Reload
Run apachectl -t, then reload Apache.
- 05
Check It Works
After a real crawler visits, the page appears in the Pages tab. To check now, use Recache on a page in the Pages tab and wait until its Rendered time updates, then request that page with a crawler user agent: you get the rendered copy from the cache. A crawler user agent sent from your own machine never starts a fresh render, so a page with no cached copy still shows your normal page.
Copy The Rule
Replace YOUR_RENDER_KEY with your API key, or copy it from the Setup page in the app with your key filled in. Any refusal, error or timeout falls through to your normal page. Keep nothing between your server and Render Engine that answers 200 on its own.
# Inside your <VirtualHost>. Apache 2.4.47 or later with mod_rewrite, mod_proxy, mod_proxy_http,
# mod_headers and mod_ssl (Render Engine is reached over https). Behind a load balancer, use mod_remoteip with RemoteIPHeader and
# RemoteIPTrustedProxy for that proxy only, so REMOTE_ADDR is the crawler's address.
RewriteEngine on
SSLProxyEngine on
# This site's own address. When Render Engine did not render a page, Apache fetches your normal page from
# here (never from the request's Host header, which a visitor chooses).
Define LESUTO_SITE https://yourbrand.com
# Your normal page, for any answer Render Engine did not render. Apache asks itself for the page, so the
# crawler gets it with its own status (an ErrorDocument alone would keep Render Engine's 402 or 503).
RewriteCond %{ENV:REDIRECT_STATUS} !^$
RewriteRule ^/?lesuto-render-fallback$ ${LESUTO_SITE}%{ENV:REDIRECT_LESUTO_RENDER_PATH} [P,L,E=LESUTO_ORIGIN:1]
RequestHeader set X-Lesuto-Render-Origin "1" env=LESUTO_ORIGIN
# A crawler's GET for a page (not a file, not the fallback) goes to Render Engine.
RewriteCond %{ENV:REDIRECT_STATUS} ^$
RewriteCond %{HTTP:X-Lesuto-Render-Origin} ^$
RewriteCond %{REQUEST_METHOD} =GET
RewriteCond %{REQUEST_URI} "!\.(js|css|xml|png|jpe?g|gif|pdf|ico|svg|webp|avif|woff2?|ttf|eot|map|txt|json|mp4|webm)$" [NC]
RewriteCond %{HTTP_USER_AGENT} "(googlebot|bingbot|applebot|duckduckbot|yandexbot|baiduspider|oai-searchbot|chatgpt-user|gptbot|claude-searchbot|claude-user|claudebot|perplexity-user|perplexitybot|mistralai-user|duckassistbot|amazonbot|meta-externalagent|cohere-ai|youbot|facebookexternalhit|facebot|twitterbot|linkedinbot|pinterestbot|slackbot|discordbot|telegrambot|whatsapp)" [NC]
RewriteCond %{THE_REQUEST} "^GET\s(\S+)"
RewriteRule ^ https://api.lesuto.com/api/v2/render-engine/render [P,QSD,L,E=LESUTO_RENDER:1,E=LESUTO_RENDER_PATH:%1,E=LESUTO_RENDER_URL:%{REQUEST_SCHEME}://%{HTTP_HOST}%1]
RequestHeader set X-Render-Key "YOUR_RENDER_KEY" env=LESUTO_RENDER
RequestHeader set X-Render-Url "%{LESUTO_RENDER_URL}e" env=LESUTO_RENDER
RequestHeader set X-Render-Client-IP "expr=%{REMOTE_ADDR}" env=LESUTO_RENDER
# Apache reuses the crawler's request headers to fetch your normal page: never send the key there.
RequestHeader unset X-Render-Key env=LESUTO_ORIGIN
RequestHeader unset X-Render-Url env=LESUTO_ORIGIN
RequestHeader unset X-Render-Client-IP env=LESUTO_ORIGIN
Header set Cache-Control "private, max-age=0" env=LESUTO_RENDER
Header set CDN-Cache-Control "no-store" env=LESUTO_RENDER
Header set Vary "User-Agent" env=LESUTO_RENDER
# Render Engine's own sandbox policy is for its address, not your site's.
Header unset Content-Security-Policy env=LESUTO_RENDER
<Proxy "https://api.lesuto.com">
ProxySet connectiontimeout=5 timeout=18
</Proxy>
# Any refusal, ended plan, error or timeout from Render Engine: your normal page.
<If "-n reqenv('LESUTO_RENDER')">
ProxyErrorOverride On 400 401 402 403 405 406 407 408 409 411 412 413 414 415 416 417 421 422 423 424 426 428 429 431 451 500 501 502 503 504 505 506 507 508 510 511
ErrorDocument 400 /lesuto-render-fallback
ErrorDocument 401 /lesuto-render-fallback
ErrorDocument 402 /lesuto-render-fallback
ErrorDocument 403 /lesuto-render-fallback
ErrorDocument 405 /lesuto-render-fallback
ErrorDocument 406 /lesuto-render-fallback
ErrorDocument 407 /lesuto-render-fallback
ErrorDocument 408 /lesuto-render-fallback
ErrorDocument 409 /lesuto-render-fallback
ErrorDocument 411 /lesuto-render-fallback
ErrorDocument 412 /lesuto-render-fallback
ErrorDocument 413 /lesuto-render-fallback
ErrorDocument 414 /lesuto-render-fallback
ErrorDocument 415 /lesuto-render-fallback
ErrorDocument 416 /lesuto-render-fallback
ErrorDocument 417 /lesuto-render-fallback
ErrorDocument 421 /lesuto-render-fallback
ErrorDocument 422 /lesuto-render-fallback
ErrorDocument 423 /lesuto-render-fallback
ErrorDocument 424 /lesuto-render-fallback
ErrorDocument 426 /lesuto-render-fallback
ErrorDocument 428 /lesuto-render-fallback
ErrorDocument 429 /lesuto-render-fallback
ErrorDocument 431 /lesuto-render-fallback
ErrorDocument 451 /lesuto-render-fallback
ErrorDocument 500 /lesuto-render-fallback
ErrorDocument 501 /lesuto-render-fallback
ErrorDocument 502 /lesuto-render-fallback
ErrorDocument 503 /lesuto-render-fallback
ErrorDocument 504 /lesuto-render-fallback
ErrorDocument 505 /lesuto-render-fallback
ErrorDocument 506 /lesuto-render-fallback
ErrorDocument 507 /lesuto-render-fallback
ErrorDocument 508 /lesuto-render-fallback
ErrorDocument 510 /lesuto-render-fallback
ErrorDocument 511 /lesuto-render-fallback
</If>Full Guide
Every detail, edge case and fix on the Lesuto docs.
Other Ways To Set Up
Pick the stack that matches your site, or start from the checklist.
Rendered Pages
In About 15 Minutes
Add the rule, reload, and check one page.