Cache policy for online traffic
July 3, 2022
I'm working on let.sh load balancer recently, and in order to provide the best performance and reduce the costs for frontend and backend projects, I've dug a lot of documentation and materials to design the cache policy of let.sh for online traffic.
Take let.sh for an example(as shown in the image above). When you first visit the site hosted on let.sh, the browser will send a request to let.sh load balancer. Then let.sh route the request to the server to get the raw file(sure the file has been compressed) and bypass the browser through the load balancer.
If you dig into the details of the image, you can see let.sh server responds to the request with a header
cache-control: no-cache, which means the server is not allowing the load-balancer and browser to cache the response. The server is not allowing upstream to cache the response because this is a request for index.html, which might be changed in the future. Another reason is let.sh wants to provide an instant experience for developers when they publish their websites/apps online. One clicks to deploy and refresh to see the results.
So it seems not to cache anything, right? There are still 2 ways to cache the response: for files(like
/index.html) likely to be changed in the future, I will introduce
Etag header first. And then, I will introduce
Cache-Control header for the files not likely to be changed in the future.
Now the problem we are facing for the files like
/index.html is the file might be changed in the future, and the payload for the requests might be large. We can't reduce the requests because we have to let the browser check whether the file has been updated. So what about reducing the size of the payload?
Here it comes to the
Etag header. The
Etag (or entity tag) HTTP response header is an identifier for a specific resource version, like the unique id for each file. Once the file has been updated, the file will also be changed.
Here is how
- For the first time user access
/index.htmlthe server will response
index.htmlfile along with header
- The browser will store the
ETagin the cache.
- And then the next time the user access the same file, the browser will send a request to the server with the
- Once server rececived request through load balancer, server will compare the
ETagof the latest file, if the
ETagis the same, the server will return empty body with status code
304(Not Modified) and header
ETag mechanism supports both strong validation and weak validation. They are distinguished by the presence of an initial
"W/" in the
ETag identifier, as:
"4960408E160450EAE42D83507EB23730" – A strong ETag validator W/"4960408E160450EAE42D83507EB23730" – A weak ETag validator
A strongly validating
ETag match indicates that the content of the two resource representations is byte-for-byte identical and that all other entity fields (such as Content-Language) are also unchanged. Strong
ETags permit the caching and reassembly of partial responses, as with byte-range requests.
A weakly validating
ETag match only indicates that the two representations are semantically equivalent, meaning that for practical purposes, they are interchangeable and that cached copies can be used. However, the resource representations are not necessarily byte-for-byte identical, and thus weak
ETags are unsuitable for byte-range requests. Weak
ETags may be useful for cases in which strong
ETags are impractical for a Web server to generate, such as with dynamically generated content.
Source: MDN Cache-Control Docs
Cache Control can be configured in request & response, the table below show the supported directives for request and response.
Here are the supported directives for request & respond:
Cache-Control have different logic for browser and load balancer, here is a brief summary for different header
|resource can be cache anywhere(including shared for different users)||cache||cache|
|can be cached for single user||cache||no-cache|
|should revalidate each request to server(e.g. through ||cache(need-revalidate)||cache(need-revalidate)|
|store nothing neither browser or CDN can cache response||no-cache||no-cache|
|cache can be used in certain time range if revalidate request in the background||cache(refetch asynchronous)||cache(refetch asynchronous)|
|cache can reuse a cached response when the server responds with an error||cache(refetch when request error)||cache(refetch when request error)|
|cache can reuse a cached response, must revalidate request in the background once cache is outated||cache(refetch asynchronous)||cache(refetch asynchronous)|
|same as ||/||cache(refetch asynchronous)|
|resource will not change, will not revalidate the request||cache (will not request again)||cache|
Lets deep dive in few of them, first take
public for an example:
public means resources can be cached anywhere(including shared for different users). So as diagram shows above if two different browsers(users) access the same resource. As the first one access the resource with
public inside the
Cache-Control header, the resource will be cached in both load balancer and browser. Then as the second one access the resource, the load balancer will directly respond to the request with cached resource content(
index.html in this example), because
public means the resource can be cached anywhere, including shared cache for different browsers(users).
private means that all or part of the response message is intended for a single browser(user) and MUST NOT be cached by a shared cache, such as a proxy server. So in our example, if two different browsers(users) access the same resource, the load balancer will not cache the resource content. Still, the browser will cache the resource content(via Etag method).
As digram above, the main difference between
private is that
private will not cache the resource content inside load balancer(or cached for different browsers(users) rather than shared cache), but
public will cache the same resource for different browsers(users).
I'll only briefly introduce the service because the service workers are defined by the website developer rather than the hosting platform. And the service worker itself can not just control the cache policy for the web page. It also can control the web page's behavior (like receiving geolocation updates, background data synchronization, etc.).
As for cache, you can regard service worker implements a proxy for all of the requests inside your website. So the service worker can cache the resources and respond to the requests.
Design the cache policy for let.sh
git ls-files | tree --fromfile . ├── asset-manifest.json ├── favicon.ico ├── index.html ├── logo192.png ├── logo512.png ├── manifest.json ├── precache-manifest.c0a7361b86cadc203094cedbf871e8b1.js ├── robots.txt ├── service-worker.js └── static ├── css │ ├── main.d1b05096.chunk.css │ └── main.d1b05096.chunk.css.map ├── js │ ├── 2.d9fa3a5e.chunk.js │ ├── 2.d9fa3a5e.chunk.js.LICENSE.txt │ ├── 2.d9fa3a5e.chunk.js.map │ ├── main.5d21aa1a.chunk.js │ ├── main.5d21aa1a.chunk.js.map │ ├── runtime-main.e9912549.js │ └── runtime-main.e9912549.js.map └── media └── logo.5d5d9eef.svg 4 directories, 19 files
Service Worker: https://developer.mozilla.org/docs/Web/API/Service_Worker_API