"FAQ — Splash 解决504问题"

URL: https://splash.readthedocs.io/en/stable/faq.html#timeouts

You can embed Splash results directly in HTML pages. This is not the best, as you’ll be rendering the website each time this HTML page is opened. But still, you can do this:

<img src="http://splash-url:8050/render.jpeg?url=http://example.com&width=300"/>

I’m getting lots of 504 Timeout errors, please help!

HTTP 504 error means a request to Splash took more than timeout seconds to complete (30s by default) - Splash aborts script execution after the timeout. To override the timeout value pass ‘timeout’ argument to the Splash endpoint you’re using.

Note that the maximum allowed timeout value is limited by the maximum timeout setting, which is by default 60 seconds. In other words, by default you can’t pass ?timeout=300 to run a long script - an error will be returned.

Maximum allowed timeout can be increased by passing -max-timeout option to Splash server on startup (see Passing Custom Options):

If you’ve installed Splash without Docker, use

The next question is why a request can need 10 minutes to render. There are 3 common reasons:

1. Slow website

A website can be really slow, or it can try to get some remote resources which are really slow.

There is no way around increasing timeouts and reducing request rate if the website itself is slow. However, often the problem lays in unreliable remote resources like third-party trackers or advertisments. By default Splash waits for all remote resources to load, but in most cases it is better not to wait for them forever.

To abort resource loading after a timeout and give the whole page a chance to render use resource timeouts. For render.*** endpoints use ‘resource_timeout’ argument; for execute or run use either splash.resource_timeout or request:set_timeout (see splash:on_request).

It is a good practive to always set resource_timeout; something similar to resource_timeout=20 often works well.

2. Splash Lua script does too many things

When a script fetches many pages or uses large delays then timeouts are inevitable. Sometimes you have to run such scripts; in this case increase -max-timeout Splash option and use larger timeout values.

But before increasing the timeouts consider splitting your script into smaller steps and sending them to Splash individually. For example, if you need to fetch 100 websites, don’t write a Splash Lua script which takes a list of 100 URLs and fetches them - write a Splash Lua script that takes 1 URL and fetches it, and send 100 requests to Splash. This approach has a number of benefits: it makes scripts more simple and robust and enables parallel processing.

3. Splash instance is overloaded

When Splash is overloaded it may start producing 504 errors.

Splash renders requests in parallel, but it doesn’t render them all at the same time - concurrency is limited to a value set at startup using -slots option. When all slots are used a request is put into a queue. The thing is that a timeout starts to tick once Splash receives a request, not when Splash starts to render it. If a request stays in an internal queue for a long time it can timeout even if a website is fast and splash is capable of rendering the website.

To increase rendering speed and fix an issue with a queue it is recommended to start several Splash instances and use a load balancer capable of maintaining its own request queue. HAProxy has all necessary features; check an example config here. A shared request queue in a load balancer also helps with reliability: you won’t be loosing requests if a Splash instance needs to be restarted.