10minutemail.com does one thing. You load the page, you get a throwaway email address, and you use it for ten minutes. The address is the product. Everything else on the page (the timer, the inbox, the buttons, etc...) is there to frame it.
So when I set out to make the home page faster, I had a clear idea of what "faster" meant: how long until the address is on screen. This post is about getting that number down, then running an A/B test that made a different number go down instead, and what I learned about the gap between the two.
Where the page started
The home page is a Spring Boot app behind Cloudflare, with the origin in AWS us-west-2 (Oregon). All the state lives in memory. A visitor's session and address get created on first contact and expire after ten minutes.
Until recently, the HTML shipped without an address in it. A script ran after DOMContentLoaded, called /session/address, and filled in the input. Before that script could even run, the browser had to finish two render-blocking stylesheets: a Google Fonts request that had to complete before any font file could be fetched, and the design-system stylesheet holding every CSS custom property the page depends on.
Measured in a throttled mobile harness (390 px viewport, slow 4G, 4x CPU slowdown, cache disabled, three runs), the address showed up 4.4 seconds after navigation. First contentful paint was 1.7 seconds. Largest contentful paint was 2.8 seconds. The address, the one thing anyone came for, was the last thing to arrive.
Round one: put the address in the HTML
The first round was conventional performance work, and it shipped to production as one release.
The controller now renders the visitor's address and the seconds remaining straight into the markup. The script reads them from the page and only falls back to fetching when the page arrived without them, like when someone reloads an expired session. The response carries Cache-Control: private, no-cache, because a page with someone's address in it must never sit in a shared cache.
The fonts moved onto my own origin as subset variable font files, split by script with unicode-range so a browser fetches only what its text actually needs. Metric-matched fallback faces (Arial and Courier New with size-adjust, ascent, and descent overrides computed from the real font files) hold the layout steady until the web fonts arrive. The design-system stylesheet is inlined into the head by a small controller advice, so that round trip is gone. Gzip is on at the origin.
In the throttled harness, address-visible time went from 4.4 seconds to 0.75. In production, measured with New Relic browser monitoring over a full day and compared against the previous day on the old build:
Home page, all countries | Old build | New build |
|---|---|---|
LCP p50 / p75 / p90 (s) | 1.03 / 1.66 / 2.56 | 0.73 / 1.28 / 2.06 |
FCP p50 / p75 / p90 (s) | 0.61 / 1.09 / 1.72 | 0.61 / 0.98 / 1.59 |
Backend p50 / p75 (s) | 0.35 / 0.64 | 0.34 / 0.58 |
CLS p75 | 0.009 | 0.009 |
LCP dropped 23 percent at p75 and 29 percent at the median. First paint barely moved, which makes sense: the old build painted early too, it just painted a shimmer where the address should be. Backend time (time to the end of the HTML response) didn't change, because nothing about the path to the origin had changed.
The question a global audience raises
Backend time is where the geography shows up. By country, p75, on the new build:
Country | Backend p75 (s) | LCP p75 (s) |
|---|---|---|
US | 0.43 | 0.83 |
CA | 0.32 | 0.82 |
GB | 0.46 | 1.42 |
IN | 0.64 | 1.36 |
NL | 0.70 | 1.55 |
DE | 0.86 | 1.52 |
SG | 1.41 | 1.52 |
A visitor in Singapore waits more than four times as long as a visitor in Canada for the first byte of HTML, because the HTML has to come from Oregon. A CDN solves exactly this problem, and I already have one in front of the site. The obvious move is to let Cloudflare cache the home page at the edge.
The catch is the address. An edge-cached page is the same bytes for every visitor who hits that data center for the next hour. It can't contain anyone's address. So caching the HTML means going back to fetching the address with a script, which is the thing round one had just gotten rid of.
Two modes, one switch
Rather than pick, I built both and made it a config switch.
In rendered mode, the default, the controller does what round one shipped. In cacheable mode, it renders a page with no address, no timer value, and no session, and marks it Cache-Control: public, max-age=0, s-maxage=3600. A tiny inline script at the top of the body kicks off the address fetch before the main script has even downloaded, and the main script adopts that in-flight request instead of opening its own. The session, the address, and the abuse accounting all move to the address endpoint, which in cacheable mode is the first request that touches the session at all.
The switch lives in an external config file the systemd unit points at, so flipping it is one line and a restart, no rebuild. A restart drops every live session (state is in memory, remember), so the flip happens at the daily traffic low.
On the Cloudflare side there's a cache rule that makes the root and the 44 locale-prefixed home pages eligible for caching, with the edge TTL set to "use cache-control header if present, bypass cache if not." That rule can stay on permanently. Cloudflare never stores a response marked private, so in rendered mode the rule matches and does nothing, and the origin header alone decides what happens. You can see it in the response: in rendered mode cf-cache-status reads BYPASS, meaning the rule matched and the origin said no, where before the rule it read DYNAMIC.
Making the comparison easy
Sequential A/B tests are usually painful to read because you end up comparing time windows and arguing about traffic mix. Two small additions made this one easy.
The page carries a data-home-mode attribute, and the script sets it as a custom attribute on every New Relic browser event. Comparing modes becomes FACET homeMode. The exact minute of the flip stops mattering.
The script also sends an addressVisible page action recording when the address reached the screen, with a how field. When the address was in the markup, the value is the first-contentful-paint time, because the address was in that paint. When it was fetched, the value is the frame after the fetched address was painted. Both count from navigation start, so they're comparable.
For the comparison itself I took the cacheable window and compared it with the same clock hours one day earlier in rendered mode, so both sides have the same daily traffic mix.
Fourteen hours of results
About 22,000 home page views on each side.
Metric | Rendered | Cacheable | Change |
|---|---|---|---|
Backend p50 / p75 / p90 (s) | 0.35 / 0.58 / 1.22 | 0.18 / 0.40 / 0.98 | −49% / −31% / −20% |
FCP p50 / p75 / p90 (s) | 0.64 / 1.03 / 1.66 | 0.48 / 0.86 / 1.47 | −25% / −17% / −11% |
LCP p50 / p75 / p90 (s) | 0.77 / 1.28 / 2.06 | 0.73 / 1.28 / 2.19 | −5% / 0 / +6% |
Address visible p50 / p75 / p90 (ms) | 736 / 1,344 / 2,176 | 944 / 1,696 / 3,648 | +28% / +26% / +68% |
CLS p75 | 0.009 | 0.009 | same |
The edge did its job. Time to first byte halved at the median. First paint improved everywhere, and by a lot where the origin is far away.
LCP did not move. I stared at that for a while. First paint got faster and the largest paint didn't, on the same pages, in the same browsers. The explanation is that the largest contentful element changed. In rendered mode the address is in the first paint, so the largest paint is the first paint. In cacheable mode the address arrives after its own round trip to Oregon and becomes the largest paint. LCP stopped measuring the page and started measuring the address fetch. The paint gain and the address delay cancelled out almost exactly at p75.
The address itself got slower for every country. The round trip to Oregon didn't shrink, and the fetch can't start until the HTML has arrived and the inline script has run. At p90 it's two thirds slower, which is the visitors on the worst connections paying twice: once for the HTML and again for the address.
By country, p75, rendered then cacheable:
Country | Views | Backend (s) | FCP (s) | LCP (s) | Address (ms) |
|---|---|---|---|---|---|
US | 4,065 | 0.48 → 0.38 | 0.77 → 0.73 | 0.83 → 0.89 | 1,008 → 1,440 |
IN | 2,368 | 0.67 → 0.38 | 1.33 → 1.09 | 1.36 → 1.45 | 1,568 → 1,760 |
DE | 1,497 | 1.09 → 0.70 | 1.27 → 0.80 | 1.67 → 1.41 | 1,504 → 2,240 |
NL | 1,412 | 0.70 → 0.29 | 0.90 → 0.68 | 1.45 → 1.02 | 1,120 → 1,312 |
GB | 821 | 0.49 → 0.27 | 0.86 → 0.73 | 1.48 → 1.27 | 1,056 → 1,312 |
PL | 478 | 0.41 → 0.21 | 0.90 → 0.63 | 1.55 → 1.39 | 944 → 1,056 |
CA | 471 | 0.35 → 0.26 | 0.73 → 0.63 | 0.82 → 0.87 | 880 → 1,184 |
FR | 316 | 0.73 → 0.73 | 0.95 → 0.98 | 1.23 → 1.77 | 1,184 → 2,112 |
CN | 316 | 0.86 → 0.89 | 1.36 → 1.33 | 1.55 → 2.16 | 1,504 → 3,008 |
Europe wins on LCP, by 10 to 30 percent in Germany, the Netherlands, the UK, and Poland. The US, Canada, India, France, and China come out flat or worse. The US is the largest single country at about a fifth of the traffic. Every country's address is later.
New Relic can't see Cloudflare's cache status, so to check the edge was actually hitting I used a proxy: the share of page views whose backend time was under 300 milliseconds. It rose in every country except China. Germany went from 34 to 62 percent, India from 18 to 68, the Netherlands from 36 to 76. Per URL, the root page was 68 percent fast, the German page 81, the Dutch page 87. China didn't change in any metric, which is what you'd expect for traffic that never reaches a nearby data center in the first place.
What the server thought about it
I assumed server-side rendering would be the expensive option, and per request it is. A home page render costs about 10 milliseconds of server time. An API call costs about 1. In rendered mode the home page was the largest single consumer of web request time on the box, at 8.2 minutes over the 14 hours. In cacheable mode the edge absorbed 93 percent of home page requests and that fell to 1.0 minute, while the address endpoint picked up the sessions the renders used to create and added 0.8 minutes. Net, web request time fell from 20.2 minutes to 14.2.
None of that matters. The host runs at 2 to 3 percent CPU in either mode. The thing that keeps it busy is the ten-second inbox polling, which the render mode doesn't touch. Load shouldn't be a factor in this decision, and I wouldn't have known that without looking.
Two bugs the data found
The A/B instrumentation found a bug within twenty minutes of the flip. One cacheable page view reported its address as rendered, which the markup makes impossible. It came from Firefox on Linux.
Firefox restores form field values across a reload unless the field has autocomplete="off". The address input didn't. On a reload, Firefox put the previous page's address back into the input before the script ran, the script saw a value and trusted it, skipped the fetch, and showed a stale address. If the session was still alive the address happened to be right. If it had expired, a Firefox user saw a dead address next to a live inbox. The fix is the attribute, plus reading the address from the input's value attribute (which only the server writes) rather than its live value (which the browser can overwrite).
The other bug came from code review, which is the less glamorous way to find things. The rendered timer value lives in a data-seconds-left attribute on the countdown. When a visitor clicked Get More Time, the server extended the session and the script restarted the countdown by reading that attribute again, still holding the page-load value, and subtracting the full time the tab had been open a second time. Click it with ten seconds left and the timer restarted at ten seconds, then declared the session expired right after it had been extended. The attribute is now consumed on first read, so any later restart fetches the real value from the server.
What "best" means here
If the question is which mode gets the address in front of a visitor sooner, rendered mode wins in every country, and getting the address in front of a visitor is what the site is for.
If the question is which mode paints sooner and loads the origin less, cacheable mode wins clearly, and for European visitors it wins on LCP too.
If the question is which mode a search engine's Core Web Vitals scoring prefers, this data says neither. LCP at p75, the number that gets scored, is 1.28 seconds in both modes. Both are well inside the good range. The scoring can't tell them apart.
The thing I actually learned here is about the metric. LCP measures whatever the largest thing on the screen happens to be. A design change that moves the address from the HTML into a fetch doesn't change how big the address is. It changes when it arrives. So LCP switched from measuring the page to measuring the fetch, and the improvement in everything before the address got hidden behind the delay in the address itself. The metric stayed flat while the experience changed in two directions at once.
I haven't decided yet. The full day of data will sharpen the magnitudes, though I don't expect it to flip any row. What I do know is that "which is faster" was never one question. It was two, they have different answers, and the metric everyone uses to settle it couldn't see the difference.
Which one would you run?
-- Devon
No comments yet