Update (context): After finishing this article, I learned that Cloudflare plans to deprecate Signed Exchanges (SXG).

Recently, as I dug deeper into the SXG ecosystem, I suspected this might be coming—just not this quickly. Still, SXG proved technically solid on my site and delivered meaningful performance gains. My conclusions below reflect two years of hands-on use.

If you still want SXG, the current path is to self-host—but the tooling looks abandoned and setup is non-trivial. Also, Google may follow Cloudflare by shutting down the SXG cache and removing SXG support in Chrome.

Signed Exchanges (SXG) is a technology mainly used to improve page load speed for Google-referred users.

I enabled it for my website in October 2023, resulting in substantial improvements to the Largest Contentful Paint (LCP) metric.

I couldn’t find comprehensive, developer-focused documentation, so I set out to write my own SXG implementation guide. I initially planned three posts. I didn’t expect it to turn into 11 deep-dive articles (10 parts plus one extra), 20 interactive demonstrations, 4 related open-source projects, plus detailed performance data and extensive reverse engineering.

In this post, I summarize my research and experiences with SXG. Where appropriate, I compare it with other performance-improving technologies used by Google: Accelerated Mobile Pages (AMP) and Speculation Rules prefetching.

SXG advantages

SXG enables cross-origin prefetching of entire pages with subresources while maintaining users’ privacy.

Websites are expected to serve pages in a special format that’s consumed by so-called link aggregators (such as Google), which perform the actual prefetching, or more specifically, the prefetching is done by browsers as instructed by link aggregators when visited.

Page load speed

With SXG, an entirely prefetched page displays almost immediately after the user navigates to it, resulting in a great user experience, even on a slow connection (assuming the prefetching is completed before navigation).

Below, you can see an extreme example featuring going offline and navigating to the prefetched website without issues.

This is a vertical video, so if you are using a mobile device, click here to open it in the YouTube app for best experience.

When compared to similar technologies in use today, only AMP offers a better performance at the cost of severe limitations and trust issues. Speculation Rules prefetching falls short mainly due to the inability to prefetch subresources, but also because it doesn’t work for returning visitors in most cases.

Theoretically, if used to its full potential, SXG could allow 75% of my users to experience LCP of under 544 ms.

In practice, I achieved an average overall LCP for Google-referred users of around 700 ms. While calculating the exact 75th percentile is complex for reasons discussed in previous posts, I guess it's under 1 second—still a solid result for a high-traffic, image-rich production site!

If you want to know more about this topic, I wrote a detailed post on SXG performance measurements.

User engagement

SXG prefetching showed significant user engagement improvements in my testing, which I attribute to better page load speed.

I won’t present a full report here; instead, I'll highlight the key user engagement metrics I measured for my website, focusing on mobile, which constitutes the majority of overall traffic. Your results may be different.

When compared to loading a page on demand, prefetching a page with subresources using SXG increased views per session and average session duration by 36%. At the same time, it reduced the bounce rate by more than a quarter (27%).

For comparison, Speculation Rules prefetching improved views per session by 15%, average session duration by 11% and decreased bounce rate by almost 19%.

These metrics represent best-case scenarios. Real-world impact will be lower due to mixed page load types, though SXG's benefits remain substantial.

Security & privacy

AMP doesn’t look good

In the case of AMP, Google controls the entire page to be prerendered (including subresources) and has the power to alter it. The user has to trust Google or avoid using AMP entirely.

Another problem with AMP is that the URL of the page points to google.com instead of the visited page. As users interact with AMP pages, they are effectively taught not to verify the URL. This undermines the efforts of the security experts trying to teach the opposite.

I can see one positive side of having google.com in the URL. It’s like Google honestly saying: we control this website and we may alter it if we want.

A solution is to introduce SXG to the mix. Cloudflare even has a switch for that. But I failed to find any example of an AMP website using it.

Speculation Rules prefetching is ok

When it comes to the Speculation Rules prefetching feature implemented in Google search results page, it’s based on a so-called private prefetch proxy. The browser prefetches the page speculated to be navigated to. The prefetching requests go through a proxy to protect the user’s IP address.

The proxy uses the HTTP CONNECT request method. Assuming the website uses HTTPS and has a valid TLS certificate, Google can only see encrypted data, so no alterations are possible in practice.

While the IP address of the user is protected, the country information still leaks to the website due to the geolocation feature of the private prefetch proxy. This could potentially affect users connecting from countries that the website doesn't typically see traffic from, though the privacy impact is relatively limited.

SXG approach

SXG uses digital signatures to prevent malicious link aggregators (such as Google) from modifying data served to users. It also preserves the original URL, which is an important UX, security, and trust-building feature.

From the security standpoint, SXG is a huge improvement over AMP and is minimally better than Speculation Rules prefetching.

Impact on the server load

Organic visits

Google SXG cache performs the function of a CDN, taking some load from your website.

You may even completely stop seeing Google-referred traffic from browsers supporting SXG in your app logs, especially for popular pages or if you boosted your SXG deployment.

For example, the homepage of my website is mostly handled by Google.

Googlebot traffic

On the other hand, the Googlebot traffic increases substantially, and even more if you boost your SXG.

One reason is the maximum validity period for SXG subresources. Normally, the assets are configured to expire in a year or so, while SXG limits them to 7 days. Googlebot, therefore, has to download them much more often.

AMP

AMP can also reduce the load on your server. However, it only works on mobile devices, so it won’t help you with desktop traffic.

Speculation Rules prefetching

Prefetching using Speculation Rules slightly increases the load of your website, because your infrastructure has to process requests from both actual and potential visitors. There are ways to limit prefetching if it causes issues.

SXG disadvantages

Some page loads will be slower

On my website, properly deployed SXG improved overall performance compared to a non-SXG setup. But some users still experience increased LCP. I minimized this issue by boosting my SXG implementation, but I know it’s not possible for every website.

AMP likely has similar trade-offs, though I haven't verified this. Speculation Rules prefetching avoids this issue entirely since it doesn't involve cache and/or custom data formats.

Incomplete SXG implementation hurts performance

Enabling SXG in Cloudflare without proper configuration will likely hurt your page load speeds.

This means you need to set sufficiently long expiration times and configure subresources for prefetching—otherwise, SXG's overhead and interference with Speculation Rules prefetching will make performance worse, not better.

If you implement SXG, commit to doing it right. A half-hearted implementation is worse than none at all.

No respect for battery level and connection limitations

Speculation Rules prefetching respects user constraints—it won't trigger when devices are low on battery or in data-saving mode. This strikes a sensible balance between performance and resource conservation.

My tests show that neither SXG nor AMP honors these constraints. This is particularly problematic since both techniques consume more data and CPU resources than HTML-only prefetching.

Implementation challenges

Server-side personalization is not allowed

The primary challenge in implementing SXG is shifting away from server-side personalization.

The page HTML should remain unchanged, regardless of whether the user is a first-time visitor, logged in, or has items in their cart. Updates to the base HTML should be handled client-side, with the frontend requesting data from the backend.

I think setting these constraints is valuable because it pays off when you decide to cache HTML, which—when combined with CDN—can greatly reduce server load.

However, this approach may introduce complexity and other development costs. Also, some applications are simply not compatible with these constraints.

Framework friendliness

Web frameworks have different opinions on where the personalization should happen. For example, Ruby on Rails encourages developers to do most of the work on the server, while Next.js embraces client-side.

As a person preferring Ruby over JavaScript, I understand the Rails approach to maximize the Ruby part of the app (the server) while minimizing JavaScript (the client). On the contrary, Next.js uses the same language on both sides of the network connection and is heavily reliant on React, a frontend-focused framework.

When reading the first two parts of this series, you could notice that forcing Rails to generate SXG-compatible HTML was a lot of work, while Next.js required only cosmetic changes. On the other hand, Rails has SXG-friendly subresources by default, while adjusting them in Next.js was a real pain involving the use of Cloudflare workers.

Insufficient documentation

As I write this post, there is no Wikipedia article on SXG. If you search for SXG documentation, you won’t find many practical, developer-focused resources.

There is a standard draft, a few valuable articles from the web.dev, Chrome, Google, and Cloudflare blogs and knowledge bases, and technical documentation scattered across SXG-related Github repositories. A lot of remaining results point to low-quality, generic blog posts written primarily for SEO purposes.

No one talks about the real-world issues with subresources that have a major impact on the overall effectiveness of SXG.

I found several instances of outdated1 or downright incorrect information.

This series was created to address the above issues.

Quirks and bugs

One of my posts focuses solely on debugging SXG, while three others are simply detailed bug reports with workarounds. In summary, 40% of this SXG guide deals with bugs and quirks.

The good news is that most of these issues aren't fundamental to SXG technology itself and could theoretically be addressed by Cloudflare, though we remain dependent on Cloudflare's willingness to implement such fixes.

It's also possible to address most of these challenges by creating a specialized Cloudflare worker that would transform all website traffic, eliminating SXG issues. The code snippets for fixing particular problems are already available in my posts—they just need to be combined into a single, comprehensive, fire-and-forget solution, a super-fix worker.

Maintenance and monitoring

SXG is easy to break; therefore, it needs constant monitoring and maintenance. Here is the additional work you should do:

• Create and maintain automated tests to cover basic problems like setting cookies or other prohibited headers, so that SXG won’t break when you add new features to your app.

• Monitor if the critical production pages work at all (I created sxg-checker for that).

• Monitor the performance degradation event rate on production, such as the failed prefetching of subresources, fallback client-side redirects, and loading SXG on-demand; the page-load-type library could be useful for detecting these cases; you can report them using the Application Performance Monitoring (APM) solution of your choice.

Not being new

Tech often chases the newest trends. Despite SXG's real advantages and ability to solve current problems, it's been around for a while now. Newer alternatives get more attention, which likely contributes to SXG's limited adoption.

Risks of implementing SXG

Stagnation

Both the technology and the ecosystem around it are stagnant:

• The SXG standard draft looks like it is stuck.

• Lack of development: the open source projects seem abandoned. When it comes to Google's repositories, commits stopped at the end of 2022 as if the project was killed.

  

• No new content on blogs and tech sites.

• Forget about getting your SXG questions answered on Clouflare or Google forums.

Poor adoption

Only Google and Cloudflare embraced SXG.

• Mozilla and Brave intentionally don’t support SXG.

• Websites supporting SXG are rare. From those I could find, none supported subresources properly, so they would be better off by disabling SXG and using Speculation Rules prefetching instead.

• Google is currently the only search engine that prefetches content using SXG. This differs significantly from AMP's adoption pattern across the industry.
  When Google announced AMP support, Microsoft immediately followed suit the same year and eventually built its own AMP cache and viewer within two years2.
  However, despite many years passing since Google introduced SXG, Bing has shown no interest in implementing this technology, leaving Google as the sole adopter among major search engines.

This creates a chicken-and-egg problem: Link aggregators don't use SXG because websites don't implement it, and websites don't implement it because link aggregators (except Google) don't use it.

Stagnation and poor adoption may lead to abandonment. I believe there is a non-zero risk of Google sunsetting SXG support. The fact that a 2-week global outage in 2024 went unnoticed by anyone except me—and wasn't addressed in Google's official communications—may increase the likelihood of future discontinuation.

Artificial Intelligence

As more people obtain information directly from large language models without visiting websites, prefetching becomes less critical. Meanwhile, Google—currently the only search engine supporting SXG—faces mounting pressure from competitors developing solutions that could make the traditional “10 blue links” search model obsolete.

Technically, I can imagine the chatbot interface prefetching links to sources using SXG. However, considering the adoption rate, it’s unlikely.

Reliance on Cloudflare

I haven't tried to self-host an SXG-enabled website. Given that the last commit to the SXG nginx module was at the end of 2021, I foresee issues with using it with the recent nginx versions.

Even if the module can be compiled and loaded by nginx, I’m sure there will be bugs affecting:

• functionality, like the issues I discovered in the Cloudflare implementation,

• security.

Since the code appears abandoned, I would avoid running it in production. The other option, Web Packager Server, is also unmaintained.

This makes Cloudflare the only option. In other words, depending on SXG makes you dependent on this company, classic vendor lock-in.

Having to trust Big Tech

Remember when I praised SXG for not allowing Google to alter the content because it's signed by the publisher? That's good, but there's another problem: Cloudflare holds the SXG signing keys, and if it chooses to, it can alter your website.

This illustrates the broader issue with Cloudflare. It provides amazing features, but requires you to proxy traffic through its servers. With access to plain text requests and responses, Cloudflare can do anything with them.

You can self-host your AMP pages, but you have to rely on Google to cache them, with the risk that Google might alter them. You don't need to worry about Google altering your SXG content, but you need to trust Cloudflare to generate it properly. In both cases, you must trust a big tech company.

Verdict

It works for me

If you ask me if it was worth implementing SXG in my specific case, I would answer: yes. Here is why:

• Most of my traffic comes from Google; therefore, making a great first impression on my new users pays off.

• I implemented SXG before Google deployed Speculation Rules prefetching on the desktop; therefore, my website’s performance improved significantly compared to its previous state.

• The improvements on the desktop were probably even greater because of the TTFB Cloudflare issue that SXG masks.

• SXG was the pretext for implementing edge caching, and at the same time, made it easier. Edge caching improved overall website speed, not only for Google-referrer visits. At the same time, it decreased server load.

• Even if Google shuts down SXG prefetching or Chrome stops supporting it in the future, I already gained a lot in terms of user engagement and conversions due to being the fastest website in its category.

While it took a lot of work (mostly debugging), SXG worked well for my specific use case.

In addition, I learned a lot during the process, such as HTTP caching details, Cloudflare workers implementation, and various frontend tricks. At the same time, I sharpened my skills regarding analytics, website performance measurement, and black-box testing.

Will it work for you?

However, should I recommend it to you if most of your traffic comes from Google? It depends, as you have to balance all the pros and cons I described in this article.

If, in your specific situation, there are more drawbacks, then stick to Speculation Rules prefetching; it just works.

Otherwise, if you want to see how low your LCP can be, follow my SXG guide. I hope it helps you avoid my mistakes and significantly speed up your SXG implementation.

The changes I’d like to see

Some things could improve the current status quo. Here is my wishlist:

• I'm not sure if it's technically feasible, but if Google could use Speculation Rules to prefetch SXG with a fallback to standard prefetching, we could have the best of both worlds. All the negative side effects of SXG would be eliminated and replaced by standard prefetching, resulting in significant performance improvements. Users concerned about battery drain and data usage would see improvements in both!

• I'd love to see broader adoption across platforms: search engines beyond Google, LLM chatbots, social platforms like Reddit, and tech sites like Hacker News. WordPress supporting SXG out of the box would be particularly impactful given its massive market share.

• If Cloudflare improved its SXG implementation to be more reliable, this guide would be much shorter.

• Finally, I'd like to see a well-maintained, open-source self-hosting solution to enable smaller hosting companies and individual developers to implement SXG without relying on Cloudflare.

Congrats and thanks!

You made it to the end of this post, and if you read the previous posts too, you've finished the entire SXG series. I hope you like it, and I'd like to thank you for your time!

If you think this post might be valuable to someone else, please share it.

I don't plan to write more about SXG, but if you're into my other interests, subscribe below.

Context (Oct 2025): After deep work with SXG, I suspected changes might be coming—just not this quickly. Cloudflare announced SXG deprecation, and I observed SXG stop working around Sept 19, 2025.

If you still want SXG, the current path is to self-host—but the tooling looks abandoned, and setup is non-trivial. Also, Google may follow Cloudflare by shutting down the SXG cache and removing SXG support in Chrome.

In the previous part, we examined how different page load types affect performance for Google-referred users. The results were surprising: while some methods achieved sub-second load times, others actually degraded performance compared to standard loading.

But here's what really matters: your website doesn't use just one loading method. In practice, visitors experience a mix of these page load types, with the distribution depending on your technical setup, caching configuration, and which optimizations you've implemented.

In this post, I'll analyze how this mix affects overall Largest Contentful Paint (LCP) performance using real-world data from my site. My primary goal is to answer a critical question: when we account for all the different ways pages actually load, does Signed Exchanges (SXG) improve or harm overall performance?

This post is part of my series on SXG—a technology that can make your website load dramatically faster (and sometimes slower) for users coming from Google. If you’re thinking about implementing it, start here.

TL;DR

For Google-referred users:

• Proper SXG implementation can bring the average LCP below 1 second

• Almost half a second is possible by boosting SXG

• Poor SXG configuration can make things much worse (+500 ms)

HTML edge caching provides consistent benefits regardless of other optimizations.

Your mileage may vary

The data and conclusions in this post come from my specific website setup, serving primarily Polish users with particular CDN and caching configurations. While the patterns I observed may apply to similar setups, your results will likely vary based on your infrastructure, user geography, and implementation details.

Measuring frequency of page load types

The measurement data I collected allowed me to compare the frequency of page load types in different configurations. I extended the spreadsheet from the previous post to include frequency data and overall estimations. You can download it below.

LCP analysis of page load types with overall impact

44KB ∙ XLSX file

Download

Download

For more information on the data collection methodology, please refer to my previous posts.

Let me first examine how different configurations performed.

Standard setup with edge cache

For a short period, I disabled SXG for testing purposes. This way, Google-referred users visited my website using only the following methods:

• Speculation Rules Prefetch

• Edge Cache Load

• Server Load with Early Hints

• Server Load

The chart below visualizes the proportions of page load types.

My observations:

• Pages were prefetched using Speculation Rules twice as often on desktop (61%) as on mobile (29%). The explanation is simple: in addition to prefetching the first 2 results, the desktop uses prefetching on hover.

• Most of the on-demand page loads use Cloudflare edge cache, ensuring fast load speed. This confirms that my website's cache utilization is optimal.

• Early Hints are almost non-existent in the breakdown. This suggests that when you use HTML caching, Early Hints provides minimal performance benefits, as discussed in the previous part. For this reason, as well as for more readable charts, I omit Early Hints in further discussions.

Standard SXG setup with edge cache

Most of the time, my website utilizes SXG, as well as other performance-improving technologies, including edge cache and Speculation Rules prefetching.

I reused the data I measured for performance analysis to calculate frequencies of page load types. Here they are, followed by my comments.

SXG was used for most of the visits

All SXG-related page views accounted for 78-87% of total traffic, depending on the device category. In other words, only 13-22% Google visits were normal or prefetched using Speculation Rules.

It's worth noting that SXG was used more frequently on desktop than on mobile by 9 percentage points, possibly due to different user behavior patterns. However, if you have a better explanation, leave a comment below.

Leave a comment

Half of the visits used prefetching

Prefetched pages accounted for 46-57%, making about half of the visits better than normal loads.

Fully prefetched page loads (the best possible experience) covered 40-46% of traffic!

Speculation Rules prefetching was almost eradicated

For SXG-enabled websites, Google prioritizes this technology. In other words, SXG cannibalizes Speculation Rules prefetching.

As a result, Speculation Rules prefetching was used for only 2% of mobile and 8% of desktop traffic.

Performance degradation caused by SXG affected many users

13% of mobile page views used SXG On-Demand Load, which degraded performance compared to normal load. As I explained in the previous part, it didn’t affect the desktop in my setup.

SXG Redirect impacted 21% of mobile visits and 24% of desktop visits.

If we sum the above numbers for mobile, we end up with 1 in every 3 users experiencing worse performance!

Boosted SXG setup with edge cache

After seeing the results, I began to wonder how to improve the overall performance. Increasing the share of fully prefetched pages and decreasing the share of SXG fallback redirects would probably help.

One way to achieve that is to increase the expiration times of pages. If a given page is kept longer in the SXG cache, then it’s more probable someone will prefetch it.

However, I decided to keep my current 24-hour expiration time. Additionally, I built a system that continuously and actively feeds Google SXG cache with a large set of pages—effectively boosting its utilization.

This drastically improved the mix of page load types as visualized on the chart below:

It almost eliminated SXG fallback redirects by bringing it down to less than 2%.

At the same time, the ratio of prefetched pages increased to 70-73%, while the fully prefetched loads took 63-64%.

Overall LCP

We have the frequencies (as presented above) and average LCP for each page load type (on the chart below, discussed in detail in the previous post).

These data allow me to calculate the overall average LCP for Google-referred visits for each configuration. It’s simply a weighted average. This gives us the real-world performance impact, accounting for how often each loading method actually occurs.

However, before doing so, let’s summarize the configurations:

• Standard + edge cache: The only improvement over the default (normal loads and prefetching with Speculation Rules) is edge cache for HTML responses

• Standard + edge cache + SXG: As above, but with properly implemented SXG.

• Standard + edge cache + boosted SXG: Same, but with a system boosting SXG cache utilization.

Other configurations

In addition, the measured data for the three configurations above allow us to simulate different configurations. We can do this by replacing some of the page load types with others for a given base configuration, creating a new configuration.

This way, we could simulate how my website would perform in more common setups, such as without edge cache for HTML, by replacing Edge Cache Load with Server Load and calculating the average LCP.

The other simulation could show us what the LCP would look like if we went back in time to the days before prefetching. We could also simulate poor SXG implementations to see how they impact the average LCP.

Here are the 4 additional configurations I simulated:

• Good ol' days (no prefetching, no edge cache): just plain old Server Load for everything

• Standard: the way most websites work by allowing Google to prefetch pages using Speculation Rules and not bothering with edge cache for HTML

• Standard + poor SXG (0% SXG cache utilization): as above, but with SXG enabled and at the same time crippled by too short expiration times, causing the Google SXG cache to be empty most of the time

• Standard + poor SXG (no subresources): same as Standard, but with partial SXG implementation, working properly only for HTML documents and not for subresources

Performance results of different configurations

Here are the overall, calculated LCP values for my website.

Keep in mind these are averages, not 75th percentiles. The 75th percentiles would likely be higher, but I won't attempt to estimate them due to the potential for significant error, at least for some of the configurations.

Proper SXG implementation improves the overall performance

Despite side effects, the overall impact of SXG on average LCP is positive. Compared to the standard configuration (Speculation Rules, no edge cache):

• Mobile: -196 ms

• Desktop: -231 ms

Boosted SXG has superior performance, but it’s not for everyone

Eliminating SXG side effects and improving cache hit rate paid off. The average LCP dropped significantly when compared to the standard configuration:

• Mobile: -458 ms

• Desktop: -432 ms

The result: pages display in ~700 ms for an average user coming from Google. That’s worth celebrating!

It’s worth noting that this solution makes sense as long as the number of pages you want to push to Google SXG cache is small enough. I don’t know the limits of this cache, but I imagine that for larger websites, they may eventually be reached.

Forgetting about SXG subresources is a mistake

The simulated configuration with disabled prefetching of subresources performed suboptimally. In comparison with the standard setup, the average LCP increased:

• Mobile: +100 ms

• Desktop: +64 ms

While not terrible, this raises the question: What's the purpose of implementing SXG if the results are worse, even slightly?

If you decide to use SXG, it’s critical to implement subresource prefetching. Otherwise, instead of improving your website's overall performance, you will degrade it due to SXG side effects. If you use SXG only to prefetch HTML, Speculation Rules prefetching will do this job much better.

But there is a mistake that will cost you a lot more.

Make sure you set proper expiration times for SXG

The simulated scenario with 0% SXG cache utilization, which could be caused in real life by setting too short expiration times for HTML, caused substantial performance degradation:

• Mobile: +479 ms

• Desktop: +602 ms

This is a result of a latency added by SXG fallback redirects.

If long expiration times are problematic in your use case, I can see only 3 solutions:

1. Using a long expiration time for a page, combined with fetching fresh critical parts client-side.

2. Using short expiration times and implementing a system to boost the SXG cache utilization aggressively (may not scale well, as mentioned above).

3. Forgetting about SXG, unfortunately.

Otherwise, your overall LCP will suffer.

Edge cache is a cherry on top

Edge caching improved performance, no matter if used alone or in combination with other performance optimization techniques. Compared to the standard configuration, the user experience was slightly better:

• Mobile: -80 ms

• Desktop: -99 ms

If the website has properly implemented SXG, enabling edge caching is a formality and an almost free ~100 ms LCP improvement.

Speculation Rules prefetching is positive

In the remaining setup, I simulated a scenario without prefetching. The results were worse than the standard configuration:

• Mobile: +71 ms

• Desktop: +384 ms

While the mobile improvement was slight, the desktop LCP decreased significantly because prefetching compensated for the Cloudflare TTFB issue I mentioned in the previous post.

That’s a good result given that it comes free. Thanks, Google!

Summary

I presented the overall impact of SXG on website performance, using my website as an example. I believe some of the observations I share in this case study should apply to other websites as well, but your mileage may vary. If you're considering implementing SXG, I hope I made it slightly easier for you to decide.

In the next and final part of the series, I will share my conclusions and thoughts on the SXG technology.

Context (Oct 2025): After deep work with SXG, I suspected changes might be coming—just not this quickly. Cloudflare announced SXG deprecation, and I observed SXG stop working around Sept 19, 2025.

If you still want SXG, the current path is to self-host—but the tooling looks abandoned, and setup is non-trivial. Also, Google may follow Cloudflare by shutting down the SXG cache and removing SXG support in Chrome.

In the previous part, we saw that a website can load in 15 different ways when visited from the Google results page and how to measure the performance impact of each load type.

In this post, I will share the results of my website's performance measurement, along with my comments.

I assume you are familiar with the differences between various page load types. If you need a refresher, please read the previous post.

This post is part of a series on Signed Exchanges (SXG). To assess SXG’s impact, I measured the performance of different page load types. This article is based on my research and summarizes my findings.

SXG is a technology to make your website load faster for Google-referred users. If you want to implement it on your website, start here.

TL;DR

For the impatient, here are the results showing the Largest Contentful Paint (LCP) 75th percentiles I measured (or estimated as stated in the labels and explained later in the text).

I show that it’s possible to go below half a second, but SXG side effects may worsen the experience for some users. HTML-only prefetching improved the performance, but sometimes only slightly.

Your mileage may vary

These results are based on real user data from my specific website, measured from Polish users. Your results may vary significantly based on your website's architecture, user geography, CDN configuration, and other factors. The patterns I observed, particularly the desktop TTFB issues, may be unique to my setup.

Methodology

Measured pages

I share my findings focusing on a specific section of my website: the vendor index page (similar to a product index page in e-commerce). While I measured other sections as well, including them here would add length and complexity without providing significant additional value.

The vendor index page receives significant traffic from Google and has strong performance metrics, making it an ideal candidate for comparison testing.

Chosen page load types

The results include all the page load types I identified in the previous post, except for types related to:

• Accelerated Mobile Pages (AMP), as my website doesn’t use it

• Google Ads (no ad campaigns for the measured section of the website at the moment of data collection)

• Early Hints, because my website uses HTML edge caching (more on this later)

Data collection conditions

I collected data only from visits that met the following conditions. For the explanation of why these specific conditions were necessary, see my previous post.

• It’s a Google-referred visit

• The user visited the website for the first time (checked with a local storage marker item)

• The page has been opened in an existing tab

• The browser supported SXG

All of the above data was sent to DebugBear, a monitoring platform.

Filters applied to collected data

Then I used its filtering functionality to include only:

• Users visiting the website from Poland, as Polish users are my target population, so I wanted to understand how they experience website performance.
  Additionally, filtering by location helps exclude bots that typically visit my website from other countries.

• Visits not using cached assets, as the local storage check (described above) did not always work properly.

• Visits with Time To First Byte (TTFB) equaling zero for load types not involving prefetching. It’s practically impossible to receive the first byte of the response below 1 millisecond; therefore, I treated those samples as invalid.

• Page load types that can be reliably measured by excluding SXG redirects. I'll explain this exclusion later.

After the above filtering, I was left with over 14k data points.

Obtained performance metrics

I segmented the data by page load type and device category to generate LCP histograms and calculate 75th percentiles and averages for each combination.

For calculating averages, I removed outliers by excluding visits with LCP over 5 seconds.

I rely on 75th percentiles by default. When using averages, I always explicitly mention it.

Estimations

I estimated LCP for page load types that use SXG redirects:

• When estimating averages, I added a bias correction—which I determined was necessary for accuracy—to the reference average.

• For estimating 75th percentiles, I observed a correlation between percentiles and averages. I then leveraged this relationship to calculate missing percentiles from the available averages.
  Note that the estimated percentiles were derived from estimated averages, creating a dependency chain in the calculations.

I’ll provide more details later in the text.

Spreadsheet with data

The spreadsheet below contains the LCP data I collected from DebugBear, along with comparison charts. It’s composed of 2 worksheets for:

• 75th percentiles

• averages

Each worksheet contains a configuration section where you can experiment with the estimation parameters to see how the results change.

LCP analysis of page load types

15.7KB ∙ XLSX file

Download

Download

Disclaimer

I’m not a data scientist, but I did my best to ensure the data is reliable and the conclusions sound.

LCP results

Below, you can see the comparison of the speed (LCP) of page load types. It’s the same chart as the one at the beginning of this post. I put it here, so you can reference it easily while reading my observations.

Desktop vs mobile

Looking at the chart, you can see that half of the page load types work better on desktop, while the other half works better on mobile:

• When the page was loaded from Google (SXG On-Demand Load) or prefetched, the desktop won.

• When the browser was talking to Cloudflare (Server Load, Edge Cache Load, and both SXG Redirects), mobile came out on top.

I thought desktop users should have a better experience than mobile users because of better connection quality and more CPU power. Why then it’s not universally applicable to all page load types?

TTFB component of LCP

When I dug deeper into the data, I found that for Cloudflare loads, on desktop, TTFB contributed to 36-51% of LCP, while on mobile, it accounted for 22-33%. Compare the TTFB histograms below—mobile looks much better:

For prefetched pages, TTFB is zero or near zero, so it’s useless for comparisons. But when the pages were loaded from Google SXG cache on demand, the TTFB was better on desktop (the opposite of the previous case). On desktop, TTFB contributed to 25% of LCP, while on mobile, it accounted for 30%.

Why did waiting for the first byte take longer on a desktop than on a mobile for Cloudflare loads?

I hypothesize that in Poland, Cloudflare edge servers have better network connectivity with mobile ISPs than with residential ones. This could be because there are only a few major mobile operators, while many residential ISPs exist.

From Cloudflare's perspective as a global infrastructure provider, Poland might be considered a relatively small market, leading them to potentially prioritize peering agreements with the larger operators.

Note that this is specific to my measurements in Poland and may not apply elsewhere.

If you have a better explanation, drop a comment below.

Leave a comment

Reference page load type for comparisons

In this post, I compare the P75 LCP of each page load type to that of a baseline on-demand page load from the server:

• Mobile: 1.43 seconds

• Desktop: 1.82 seconds

The gap between mobile and desktop is 390 ms.

Keep in mind that the desktop LCP in this study is heavily impacted by the increased TTFB, as discussed above. This impact may not be present in different countries and/or in the future. This makes it difficult to draw general conclusions (i.e., unrelated to my specific website) based on comparisons of page load types on desktop and between device categories.

On the other hand, the mobile performance characteristics should be fairly universal.

Below, you can see the LCP histograms with the TTFB impact on desktop clearly visible:

SXG prefetching with subresources is unbeatable

You probably won’t be surprised that prefetching the entire website (document and subresources) using SXG is a definitive winner in terms of performance. Compared to the reference, on-demand load from the server:

• Mobile: -846 ms

• Desktop: -1356 ms

As you observed above, I mark LCP improvements with a minus sign (LCP decrease, which is what we want). I use plus sign for LCP degradations (LCP increases, which we should avoid). I don’t use signs before absolute values, as you can see below.

As a result, the LCP is as low as:

• Mobile: 584 ms

• Desktop: 464 ms

Below you will find LCP histograms. Please note that the data points with values higher than 1.5 seconds are almost non-existent!

Can this be improved even further?

My data shows that for SXG-prefetched pages with subresources, 95% of LCP time is spent on rendering.

For extremely low LCP: use large images or videos freely (they're prefetched anyway) and keep your page structure lean and simple (to minimize rendering). The first won't hurt; the second fixes the real bottleneck.

Second place belongs to HTML-only prefetching

While this may seem obvious, it's worth stating clearly: prefetching improves performance regardless of whether you use Speculation Rules or SXG, even when only the HTML document is prefetched. Compared to the reference on-demand load from the server, HTML-only prefetching is faster by:

• Mobile: from -20 to -100 ms

• Desktop: from -670 to -740 ms

The mobile results appear subpar, but they align roughly with the results reported by Google itself. The desktop performance improvement is significant, mostly because prefetching is a workaround for Cloudflare’s TTFB issue.

I believe inlining critical subresources, such as important CSS fragments, should make HTML-only prefetching more performant. However, I haven’t implemented it on my website (yet).

Speculation Rules are slightly better than SXG for prefetching HTML

A page prefetched using Speculation Rules loads a bit faster than the one prefetched using SXG without subresources. That’s interesting, since both methods technically do the same thing.

On the histograms below, you can see that SXG has more extreme samples (below 250 milliseconds and 5+ seconds), while Speculation Rules’ samples are much more concentrated below 1 second.

I suspect the difference may be explained in part by the cryptography-related CPU overhead of SXG.

Mobile histograms

Desktop histograms

Edge caching was worth it

Introducing HTML edge caching lets me skip processing frequently accessed pages on my server. Also, as Cloudflare works as a reverse proxy, these pages don’t need to be transferred between my server and Cloudflare on each request. They are kept on an optimized infrastructure and retrieved in milliseconds.

This is visible in LCP measurements, when comparing on-demand page loads from the server and the edge cache:

• Mobile: -120 ms

• Desktop: -350 ms

Given that edge caching improves all the traffic, not only Google-referred, I think the improvement is satisfactory, especially on desktop.

On the histogram above, you can see the absolute LCP values:

• Mobile: 1.31 seconds

• Desktop: 1.47 seconds

This time, the gap between mobile and desktop is 160 ms—a 2.5x decrease compared to the reference Server Load. I found it’s caused by TTFB again, but why there is less difference now?

I don’t know. If you have an explanation, drop a comment below.

Leave a comment

Proper HTML edge caching makes Early Hints' impact negligible

As I mentioned at the beginning, I didn't include measurements for the Early Hints page load type.

Cloudflare caches the Link HTTP header from the server’s first response and then reuses it to send Early Hints in later responses for the same URL.

However, when the edge cache is in use, then not only the Link header, but the entire HTTP response is cached. In effect, those later responses almost always include the full page directly from the cache. In that case, the origin server is never contacted, so there’s no gap between sending Early Hints and delivering the complete response. As a result, Early Hints provide no performance benefit.

Because of this, I wasn’t able to collect enough samples of requests that actually went to the origin server, where Early Hints could make a difference.

SXG On-Demand Load impact varies between device categories

When the SXG version of a page isn't prefetched, it must be fetched on demand. The reasons for this were explained in the previous part.

In this scenario, compared to the reference, the LCP difference was as follows:

• Mobile: +240 ms

• Desktop: -210 ms

Mobile

The slowdown on mobile can be attributed to the network overhead of downloading SXG subresources on demand. Each subresource may require its own certificate file to be fetched, which in the worst case can double the number of files that need to be downloaded. The higher latency of mobile connections makes this effect more visible.

Also, SXG processing requires performing cryptographic operations. Mobile CPUs often have less processing power, which can make the overhead more visible and further contribute to LCP degradation.

Desktop

I attribute the desktop LCP improvement to the TTFB issue, as explained earlier. Normally, I would expect a slight performance degradation.

The dark side of SXG

As you have seen above, loading the page on demand from the Google SXG cache is not optimal, at least on mobile devices. That’s not good for the technology that promised to improve page load speed. But the real problem is much worse.

SXG fallback client-side redirect

It occurs when the page is missing from Google’s SXG cache. When the user decides to navigate to the target page, the SXG cache serves a fallback page to the browser. This page contains a client-side JavaScript code that redirects the browser to the target page.

It would be interesting to know how much this degrades performance!

RUM tools blind you to the SXG performance problems users experience

I collected many samples with detailed performance measurements for page loads redirected from the Google SXG cache. Unfortunately, I decided to throw them out entirely.

I noticed they don’t make sense at all: they seemed to have no impact on LCP. No improvement (that’s obvious), but also no expected degradation.

Why? The answer is straightforward; however, it took me a while to understand. The bottom line is that it's impossible to measure this in production. No matter which Real User Monitoring (RUM) solution you use, you won’t be able to say if you have a performance issue!

Start of navigation bias

When a user is on page A and clicks a standard link to page B, LCP measurement begins at the moment of the click. This way, the LCP value reflects the entire wait time for the largest element on the target page, including network delays, HTTP redirects, and other overhead.

The situation changes when an SXG-fallback intermediary page uses JavaScript to redirect the user to the target page. In this case, the browser cannot distinguish whether the redirect was triggered by the user or automatically. It assumes a user action and starts measuring time only from the moment the client-side redirect occurs—not from the original click on the Google search results page.

This means the interval between the click and the client-side redirect—I'll call this click-to-redirect—is invisible to the browser. And that gap can be significant. In my tests, it ranged anywhere from 50 to 2000 ms, depending on the device, browser, connection type, and likely other factors.

I built a SXG fallback redirect demo so you can try this yourself and see the difference. The demo also lets you simulate SXG and Speculation Rules prefetching, but in my testing, those didn’t affect the results.

Estimating the average SXG fallback LCP

The LCP of a page loaded using SXG fallback redirect should be a sum of the LCP of the underlying page load type (Server Load or Edge Cache Load) and the click-to-redirect time.

The HTML of the SXG fallback redirect page is under 350 bytes, so the time it takes for the page to load and start the redirect is almost the same as its TTFB.

If the TTFB of the fallback page is similar to the TTFB of SXG On-Demand Load (which it should be, since both responses are generated by the same system), I could use the already collected data.

I prepared a demo that measures TTFB of a SXG On-Demand Load. In my tests, I could confirm that the TTFB roughly equals click-to-redirect time. You can check it by yourself here and compare the result with the measurement from the previous demo.

For the initial estimation, I will use averages because they are easier to work with than percentiles.

The average click-to-redirect delay that should be added to the underlying, average page load type LCP is:

• Mobile: +403 ms

• Desktop: +311 ms

You can find the estimated absolute average LCP values ​​in the spreadsheet mentioned earlier. Below is a chart comparing the average LCP for each page load type. The conclusions are roughly similar to those for the 75th percentile.

Estimating the 75th percentile LCP for SXG fallback

I see the 75th percentile as the standard way of assessing LCP performance. It's used by the Chrome User Experience Report (CrUX). Therefore, we should estimate it for SXG fallback page load types.

In the case of my data, I found that for each page load type, its 75th percentile can be calculated by increasing the average by 20-40%. Therefore, I assumed that the missing 75th percentile for SXG Redirect loads can be estimated using this method. For the exact calculations, see the spreadsheet included earlier.

SXG fallback significantly hurts performance

Below, you can see how the P75 LCP degrades when the page loads using SXG fallback compared to the reference:

• For Server Load

  • Mobile: +615 ms

  • Desktop: +388 ms

• For Edge Cache Load

  • Mobile: +369 ms

  • Desktop: +66 ms

The LCP increase for the Edge Cache Load on desktop doesn’t seem much, but keep in mind that it has erased all the edge caching gains.

In my opinion, the performance degradation for pages loaded using SXG fallback is substantial. And neither Google nor Cloudflare documentation will tell you this.

While these specific performance numbers are from my website, the measurement blind spot I've identified is a fundamental issue that affects all SXG implementations.

SEO is not impacted

There’s an interesting side effect of the fact that LCP for SXG fallbacks cannot be measured accurately in production.

Chrome browsers continuously report LCP (along with the other Core Web Vitals) to Google, which then publishes the aggregated data as CrUX.

In my experiment, Chrome reported incomplete LCP data to Google when a page was loaded via an SXG fallback redirect. I throttled the network to 3G and simulated the fallback. The LCP shown in Chrome DevTools’ Performance panel (screenshot below, left) matched what Chrome reported to Google (screenshot below, right). Neither measurement included the click-to-redirect delay.

As you know, LCP is a ranking factor: a good score (≤ 2.5 seconds) should boost your SERP positions. And how does Google get that data? From CrUX.

Now imagine this scenario: your LCP sits right at 2.5 seconds. You enable SXG, but don’t configure it properly. As a result, most of your pages load through a fallback redirect. Your real LCP rises above 2.5 seconds, degrading UX. But Google still sees the optimistic value from CrUX and continues to treat your site as if it had a good LCP—effectively ranking you higher than it should.

The SXG Tradeoff

Website loading speed depends heavily on how pages are delivered. Techniques like SXG and Speculation Rules, combined with edge caching, can dramatically improve LCP. At the same time, the very SXG mechanism that enables sub-second page loads can also introduce scenarios where performance suffers.

This raises an important question: Is it acceptable to sacrifice the experience of some visitors so that others enjoy a much faster site? The answer likely depends on the ratio between those who benefit and those who are negatively affected.

The key questions are:

• How many visitors experience degraded performance when SXG is enabled?

• What strategies can reduce or eliminate those negative effects?

• When we balance the wins against the drawbacks, is SXG’s overall impact on LCP positive or negative?

I’ll explore these questions in the next part of this series.

Thanks

A special thanks to Michał Pokrywka and Maciej Woźny for their valuable comments.

I'd also like to thank Matt Zeunert and the DebugBear team for providing me with access to their web performance monitoring service.

And thank you for reading. I hope you enjoyed it!

Context (Oct 2025): After deep work with SXG, I suspected changes might be coming—just not this quickly. Cloudflare announced SXG deprecation, and I observed SXG stop working around Sept 19, 2025.

If you still want SXG, the current path is to self-host—but the tooling looks abandoned, and setup is non-trivial. Also, Google may follow Cloudflare by shutting down the SXG cache and removing SXG support in Chrome.

When you find a page on Google, you probably don't think much about what happens before you click it. Perhaps you've heard about prefetching, but did you know that Google employs 5 or more methods (depending on how you classify them) for loading pages? Each technique has distinct performance characteristics.

This post is a part of a series about Signed Exchanges (SXG). In an attempt to measure how SXG impacts page loading speed I needed to distinguish between different page load types. This article is based on my research and summarizes my findings.

SXG is a technology to make your website load faster for Google-referred users. If you want to implement it on your website, start here.

Main page load type categories

(Plain old) on-demand document loading

Let’s start with the obvious way of loading the page. It was there from the beginning of the web.

When the user clicks a link, the browser fetches the HTML. Then the browser fetches all the assets required for the document to display.

It’s simple and it works. However, if the server hosting the page is slow or overloaded, the user will experience a delay, which could lead to a poor experience.

To improve the situation, Google rewards websites that load quickly with better search results positions. However, Google knows that not every website in the world can become fast.

That is probably one of the reasons Google implemented prefetching.

Prefetching the document (Speculation Rules)

The idea is to download the page before the user decides to click on it. When they do, the page HTML is ready.

For the vast majority of Google results, prefetching is implemented using Speculation Rules and a private prefetch proxy.

This feature is supported on Chromium-based browsers, but support from other browsers may follow.

HTML prefetching greatly improves user experience and works with most websites out of the box, but it comes with a limitation. It doesn’t prefetch assets such as CSS styles, images, and fonts.

Prefetching the complete page (Signed Exchanges)

It is possible to have the whole page (including assets) prefetched on Google results. When the user clicks the result, the browser starts rendering the page immediately without the need to download assets first.

To achieve this, the website owner has to implement Signed Exchanges (SXG). This technology was integrated into Google search even before the HTML prefetching I described above.

It’s worth noting that only Chromium-based browsers implement SXG.

Prerendering (Accelerated Mobile Pages)

Probably the fastest way to display a page is to use Accelerated Mobile Pages (AMP). That’s because Google prerenders websites built using this technology, so when the user clicks, the page is not only prefetched but also fully rendered. It’s hard to imagine a better user experience.

On the downside, AMP has severe restrictions. Developers are forced to use a subset of HTML, JavaScript is constrained, and many other limitations apply.

It works only on mobile devices using Chromium-based browsers.

The other problem with AMP is a centralized cache fully controlled by Google. It means Google effectively became a hosting company for every possible AMP page on the Internet. Quite dystopian, if you ask me.

As AMP pages are hosted on Google, your website becomes a subpage of google.com. Your URL will look like below:

https://www.google.com/amp/s/www-yourdomain-com/yourpage

There is a way to deal with the last two problems by introducing SXG to the mix. Cloudflare even has a switch for that. But when I was writing this post, I tried hard but failed to find any example of an AMP website using it.

On-demand loading with server-side redirection (ads)

The last category describes how a page is loaded when the user clicks a Google ad. The document is loaded on demand, but with an additional HTTP/302 redirect for registering the click, which adds latency.

No prefetching. Ironically, Google’s paying customers get the worst possible page load method. If you use Google Ads, make sure your page is optimized to load fast to neutralize the latency added by Google. Another potential solution is to use AMP on your ad landing page for mobile users; however, I was unable to find an example in the wild.

Conditions, edge cases, and quirks

The above categories are just scratching the surface of a full list of possible ways the page can load when referred from Google. That’s because various factors can improve or degrade page load efficiency, and some page load types impact others.

Signed Exchanges

When mentioning prefetching the entire page using SXG above, I described the best possible scenario: the HTML and all the required assets (or subresources) are being prefetched. That’s the goal, but in the real world, things don’t always go smoothly.

Various factors can influence how the page is loaded, and they greatly impact performance. Here are the possible SXG page load types I identified:

Page prefetched with subresources

If most of your SXG-enabled page views are prefetched with subresources, you did a great job optimizing your website!

However, despite your efforts, you will notice other, less efficient page loads.

Page prefetched without subresources

The browser will use subresources only if all of them were successfully prefetched. The all-or-nothing principle states that even if one subresource fails to prefetch, when the user visits the page, all of the subresources will need to be downloaded again.

Here are the causes of missing subresources I identified:

• SXG implementation errors (should not happen if you followed my previous SXG posts carefully)

• Global Google SXG cache issues (very rare)

• Google SXG cache housekeeping and temporary errors (happens regularly, but impacts only a small portion of page loads)

• The user not spending enough time on the search results page for the prefetching to complete

• User opening the page in a new tab (more on this later)

As you can see, apart from the first, the remaining factors are beyond your control.

But still, at least the HTML was prefetched, and it’s a win for performance when compared to vanilla, on-demand page load.

Page loaded on-demand using client-side redirect fallback

Sometimes the target page is unavailable in Google SXG cache. The browser will still try to prefetch it (it signals Google to populate the SXG cache with this page when possible), but will fail.

When the user finally decides to click the result, the browser will once again try to load the cached page from the SXG cache. The cache response will include a simple HTML document with a client-side JavaScript redirect. The browser will follow this redirect, loading the target page.

It’s worth noting, the redirect document introduces additional latency caused by another HTTP request, HTML parsing, and JavaScript execution.

Page loaded on-demand from Google SXG cache

Sometimes the browser doesn’t prefetch the page, but loads its SXG version on demand when the user navigates to it. I found 2 reasons for that:

1. Google prefetches only 1 SXG result on a page. I don’t know how Google determines which SXG result to prefetch, but it’s not always the first result for sure. If the user chooses to click the one that was not prefetched, it is loaded on demand, but still via SXG.

2. Google tried to prefetch the SXG page and failed. However, the user stayed long enough for the Google SXG cache to become populated. Now, when the user clicks the result, it’s loaded—on demand—from the SXG cache instead of the fallback document mentioned above.

Loading the SXG version of the page on demand introduces cryptography overhead. I suspect it comes mostly from additional requests required to download certificates for signature verification. I don’t think the CPU overhead plays a big role, because cryptography operations are cheap nowadays.

Speculation Rules

Currently, Google prefetches the page with Speculation Rules if all of the following conditions apply:

• The target page is in the top 2 results, or the user hovered over the result (on desktop only).

• The browser doesn’t hold any cookies for the target website. In most cases, this means the user hasn't visited the site before.

• The device has enough capacity in terms of memory, network, and battery. For example, using battery-saving mode on a mobile will deactivate prefetching.

• Prefetching is not disabled by browser extensions; for example, Privacy Badger by default disables prefetching.

• The target website doesn’t disallow prefetching (by default, prefetching is allowed).

It’s worth noting that none of the above conditions apply to SXG prefetching.

AMP prerendering

Similarly to SXG, only one of the AMP results is prerendered. Others need to be loaded on demand from the AMP cache.

The AMP viewer is shared between the prerendered page and the others; therefore, it loads instantly every time. However, the user needs to wait for the non-prerendered pages to load in the viewer.

There are likely other edge cases when loading AMP pages, such as missing AMP cache entries during prerendering or on-demand loading.

Replicating them manually would require creating test pages, waiting for Google to index them, searching for them in Google, and hoping that edge cases manifest themselves.

I couldn’t find any tools that would make it less difficult and time-consuming. Therefore, I chose not to research these cases. If you have additional information on this topic, please leave a comment below.

Leave a comment

Opening the page in a new tab

Most of the time, when the user opens the page in a new tab, it is loaded in the background. The user will likely switch to the tab after some time, giving it a chance to render fully, while they are still interacting with the referrer website. Therefore, I believe the page load speed is far less critical in these cases. Your performance optimizations are aimed mostly at people opening the page in an existing tab.

However, it’s good to know that opening the page in a new tab degrades performance:

1. Already prefetched SXG subresources are not used as mentioned previously.

2. The prefetched SXG HTML is also discarded, unless the CTRL+click method is used (only on desktop). I explained why in the previous part.

3. AMP pages have to be loaded on demand. No prerendering or prefetching.

On the other hand, Speculation Rules prefetching handles new tabs extremely reliably. It just works, regardless of how the tab is opened, on both desktop and mobile.

Duplicate prefetching

If the given page implements SXG, Google prefetches its SXG version, but at the same time uses Speculation Rules prefetching for the normal version. In my tests, I observed this phenomenon on the desktop only.

Seems like a waste of the user’s bandwidth (it is!), but it has a bright side too. If the user decides to open the page in a new tab (using a right-click menu, not CTRL+click), then the prefetched SXG version is discarded, but the normal version prefetched with Speculation Rules is used instead. Subresources have to be loaded normally, but at least the document is ready quickly.

Generic techniques for improving page load speed

If you want to compare various methods used to load a page when referred from Google, you should also consider techniques that improve performance and are not specific to Google.

You should segment your results by the technique applied for a given page load. That’s because each page load may use a different set of techniques. Mixing high- and low-performance loads in measurements increases the variance of the results, thus making it harder to analyze.

Early Hints

Early Hints allow the browser to begin fetching subresources even before the main document starts to load. This can improve the performance, especially for the pages that take time to render on the server.

Caching at the edge

If you cache HTML at the edge, such as when using Cloudflare cache, then it should be measured as a different category of page loads for the same reason as above.

If the page is delivered from the edge cache, it results in:

• Subresources start loading immediately, especially if they were listed in the Link header of the response. The benefits are similar to Early Hints.

• HTML becomes available much earlier than if it had to be delivered from the origin.

The cached page can use Early Hints, but I don’t see any performance benefits.

No impact on prefetching

The above page load types affect only the on-demand category of page loads. If the page is prefetched, it doesn’t matter if it was served with Early Hints or using edge cache.

Browser caching

If the user has visited the given website in the past, when they visit it again, the browser cache may contain some subresources, such as the website logo, ready to be used. If the website uses client-side HTML-caching, even the document could be saved in the browser cache, making subsequent visits instant.

When measuring the performance of various page loads, cached visits should be separated into a different category. Later, it may be included or excluded from the analysis.

Personally, I exclude it because returning users behave differently and may be less sensitive to page load speed because they know the site already.

Probably an incomplete list of page load types

Combining all the scenarios described above results in the following list of page load types. The list excludes scenarios involving a returning user and opening the page in a new tab, as those are not very useful in performance analysis.

1. Server Load

2. Server Load with Early Hints

3. Edge Cache Load

4. Speculation Rules Prefetch

5. SXG Prefetch with Subresources

6. SXG Prefetch without Subresources

7. SXG On-Demand Load

8. Server Load via SXG Redirect

9. Server Load with Early Hints via SXG Redirect

10. Edge Cache Load via SXG Redirect

11. AMP Prerender

12. AMP On-Demand Load

13. Server Load via Ad Redirect

14. Server Load with Early Hints via Ad Redirect

15. Edge Cache Load via Ad Redirect

That’s a lot of possibilities! I started preparing a flowchart describing the conditions needed for each page load type. However, it quickly became too complex, so I ditched this idea.

Comparison of various page load types

At first, I thought to sort all the load types by the speed, measured by Largest Contentful Paint (LCP), that they should (hypothetically) offer. It’s easy for top performers:

1. AMP Prerender (fastest)

2. SXG Prefetch with Subresources

3. Speculation Rules Prefetch

4. SXG Prefetch without Subresources (slowest)

The AMP On-Demand Load is very hard to grade because it’s totally different and potentially leaner than a full page. Therefore, it may, in some conditions, load even faster than the full page with a prefetched HTML. On the other hand, if the prefetched page is optimized, AMP On-Demand Load will load more slowly.

In the on-demand category, there are various ways to fetch data. I sorted them by speed:

1. Edge Cache Load (fastest)

2. Server Load with Early Hints

3. Server Load (slowest)

But there is one more - SXG On-Demand Load. It’s challenging to rank due to the SXG overhead and other individual factors, such as your server speed and connectivity.

The other dimension is the redirection method used. It may be ordered like this:

1. No redirection (fastest).

2. Server-side redirection (Google Ads).

3. Client-side redirection (SXG fallback, slowest).

How to measure different page load types in real life

Now, when you understand the difference between various page load types, it’s time to measure how they compare in terms of performance under real-world conditions. I will show you how I did it on my website.

The first thing that’s needed is a method to differentiate page load types. I created a JavaScript library page-load-type just for that.

It uses a variety of techniques to determine how the page was loaded and supports most of the load types described in this post, except AMP and Google Ads (both were not used on my website during the measurement).

Requirements

Measuring every page load doesn’t make sense. The visit should be collected only when all of the following requirements are met:

• The user comes from Google. It’s easy to overlook a special case involving the SXG cache. Read on.

• The browser cache doesn’t contain entries for the measured website. Otherwise, some assets may be fetched from the cache, which improves load time, but pollutes the measurement data, making it overly optimistic.

• The page isn’t opened in a new tab. As discussed earlier in this post, it breaks some prefetching/prerendering methods, and at the same time, decreases the performance sensitivity of the user.

If the website, such as mine, implements SXG, additional requirements follow:

• The current page supports SXG. It is perfectly fine if some of your pages, particularly non-performance-critical ones, don’t support SXG. Examples include Terms and Conditions and Privacy Policy. Make sure to insert the JavaScript measurement code only on pages with SXG implemented.

• The browser supports SXG. It doesn’t make sense to include Firefox visits, for example, because Mozilla doesn’t like SXG.

Detecting Google-referred visits

To determine if a user is Google-referred, the easiest method is to check the referrer for Google domains. However, as I wrote earlier, when the SXG fallback is used, the referrer is set to the Google SXG cache (your-domain-com.webpkgcache.com). Taking everything into account, the final JavaScript function could look like this:

function isFromGoogle() {
  if (!document.referrer) return false;

  const regex = /(^|\.)((google\.[a-z]{2,3}(?:\.[a-z]{2})?)|(webpkgcache\.com))$/i;

  try {
    const referrer = new URL(document.referrer);
    return regex.test(referrer.hostname);
  } catch (e) {
    return false;
  }
}

Checking the browser cache

The simplest way to ensure the cache is empty is to check whether the visitor is accessing the page for the first time.

In most cases, this can be done by checking for the presence of a cookie or local storage entry and then immediately setting it for future visits1:

function isFirstVisit() {
  const returning = localStorage.getItem('returning');
  localStorage.setItem('returning', 'true');
  return !returning;
}

Detecting a new tab

When navigating between pages, the browser maintains the history of visited pages. However, when the page is opened in a new tab (or new window), the history is not preserved. This browser behavior can be used to detect how the page was opened.

function isOpenedInNewTab() {
  return(!window.history || window.history.length === 1);
}

SXG support in the browser

I couldn’t find a direct way to query the browser for SXG support from JavaScript. As the SXG is implemented only in Chromium, the naive approach is to parse the User Agent.

However, on iOS devices, Chromium uses WebKit, which doesn’t support SXG. Also, some Chromium-based browsers, such as Brave, intentionally disable SXG. In some cases, the browser may spoof the User Agent to look like Google Chrome. All of this makes it impossible to reliably detect SXG support by parsing the User Agent.

Accept header to the rescue

When loading a page, the browser includes the Accept header in the request. It contains the application/signed-exchange substring if the browser supports SXG.

However, the Accept header is accessible only on the server. Therefore, the detection of SXG support should be implemented in the app or some form of middleware. Probably one of the simplest methods is to include a server-side generated <script> tag near the top of the HTML, looking like this:

<script id="sxgSupportScript">window.sxgSupport = false</script>

The global sxgSupport constant can be accessed later in the frontend code.

HTML caching introduces a challenge

Things become more complex when HTML caching on the edge becomes involved. Let’s say your server generated an HTML with the sxgSupport constant set to true, which was correct for the Chrome browser requesting the page at this moment. But the page has been cached, the sxgSupport is frozen to true, and when Firefox requests the page, it gets the incorrect value of sxgSupport.

The solution is to set the sxgSupport constant after it’s retrieved from the cache. It’s a perfect task for a Cloudflare worker, which could look like this:

export default {
  async fetch(request) {
    // Get the original response
    const response = await fetch(request);
    
    // Check the SXG support
    const acceptHeader = request.headers.get('accept') || '';
    const supportsSXG = acceptHeader.includes('application/signed-exchange');
    
    // Only process HTML responses
    const contentType = response.headers.get('content-type') || '';
    if (!contentType.includes('text/html')) return response;
    
    // Create HTMLRewriter to modify the script element
    const rewriter = new HTMLRewriter()
      .on('script#sxgSupportScript', {
        text(text) {
          // Replace the entire content of the script tag
          text.replace(`window.sxgSupport = ${supportsSXG}`);
        }
      });
    
    // Apply the rewriter and return the modified response
    return rewriter.transform(response);

    // For more information, see the following blog post:
    // https://www.pawelpokrywka.com/p/different-methods-of-prefetching
  }
};

For the instructions on how to set up and deploy Cloudflare workers, see my earlier blog post.

Including the page load type in the measurements

If you implemented all of the above and properly installed the page-load-type library in your project, the measurement code should look like this:

// If you use SXG, this code should be run only on SXG-enabled pages.
// If you don't use SXG you can skip checking for window.sxgSupport.

import getPageLoadType from "page-load-type";

if (isFromGoogle() &&
    isFirstVisit() &&
    !isOpenedInNewTab() &&
    window.sxgSupport) {

    const loadType = await getPageLoadType();

    // Run measurement code when loadType becomes available
}

For LCP measurement, I used DebugBear, a frontend performance monitoring tool. I added the DebugBear snippet to the <head> section and implemented page load type tracking. Here is the resulting code, with boring fragments replaced by [...] marks.

[...]

if (isFromGoogle() && [...]) {
  const loadType = await getPageLoadType();

  // Record page load type
  window.dbbRum.push(["tag1", loadType])
}

Shortly after deployment, I began receiving performance reports for each qualifying Google-referred visit, along with the page load type.

I measured user engagement metrics in Google Analytics by firing a custom event within the same if condition:

gtag('event', 'google_visit', {page_load_type: loadType});

After creating a custom, event-based dimension and waiting 24 hours, I was able to use the page load type in my reports.

Results

You learned about the various methods Google uses to load your page, how to differentiate between them, and how to set up performance measurement.

Performance measurements will vary across websites, but seeing a real-life case study can help you decide whether and how SXG could improve your site's performance.

Therefore, I present the results for my website in the next part of the series.

The demo

You probably came here from the demonstration I prepared, but if you haven't seen it yet, I encourage you to do so before proceeding.

See the demo

The demo requires the original Google Chrome browser and doesn’t work on iOS. If you are reading this post on an iPhone or iPad, don’t have or want Google Chrome on your device, or can’t/don’t want to run the demonstration for other reasons, I have prepared a short video that allows you to experience it passively. Note that since I recorded the demo, I have updated it based on feedback from Reddit and Hacker News.

The video1 is also available on YouTube if you prefer.

In the video, I used two phones. The screen on the left shows the instructions guiding me through the entire process, while on the right screen, you can see me performing the actual actions.

Completing all the steps resulted in loading a website with a 19 MB video file. This would be perfectly normal, except:

• I was offline when navigating to the page.

• I had never visited this website on this device before (a fresh incognito mode was used, ensuring an empty cache).

• Dinosaurs can’t speak!

It’s a trick

The browser obviously can’t transmit data without network access.

I designed the demo to be playful, surprising you into questioning how offline mode and browser prefetching really behave.

My goal wasn’t to trick anyone for its own sake, but to create a memorable experience that highlights a very underappreciated part of web performance.

How I did it

First, I copied the default offline page (the one with the dinosaur) from Chrome and modified it to include a speech bubble with a link. Then, I added a hidden video that activates when clicking this link.

Then, I asked Google to prefetch (that is, download in advance) this page along with the 19 MB video file when the user performed a Google search. I even prepared a fake progress bar on the instructions page to ensure the prefetching had enough time to complete.

If you are wondering why the CODE needs to be typed manually, it prevents mindless clicking. This approach ensures users actively engage with the instructions and provides extra time for prefetching.

I've released the demo as an open-source project on GitHub. There, you'll find a much more detailed explanation of how it works.

Asking Google to prefetch the website

But how do you convince Google to prefetch the website? It’s easy—you don’t need to do anything. Google automatically prefetches the top two search results and, on desktop, also prefetches results you hover over. It uses so-called Speculation Rules for this.

However, there is a caveat: Google prefetches only the HTML. It cannot prefetch subresources referenced within the main document, such as images or… videos.

Prefetching a complete website

A little-known technology called Signed Exchanges allows one to prefetch an entire website, including subresources. I (ab)used this on the demo page to prefetch the 19 MB video.

I found that the maximum amount of data possible to prefetch is approximately 21 MB.

The point of the demo

Rickrolling people is fun, but a more serious application is optimizing website load speed for users coming from Google search results.

If all the data required to render a page is prefetched, then the user visiting it will have the best possible experience—the page will load instantaneously. The Largest Contentful Paint (LCP) will be reduced to almost zero, as you can see on the video below:

Most websites are far smaller than 21 MB, so in most cases, the browser will have enough time to prefetch all necessary resources while the user still decides which link to click.

My goal was to show you a powerful technology you can use to:

• Optimize user experience by giving new visitors a great first impression.

• Improve LCP for SEO benefits.

• Increase conversion rates, as faster websites drive higher engagement and sales.

Sounds great. What’s the catch?

To implement Signed Exchanges on your website, you first need to understand how they work in practice.

Because I couldn’t find a complete, implementation-focused guide anywhere, I decided to create one — a series of free blog posts written for typical developers.

This guide is based on real-world experience optimizing a high-traffic, dynamic website. It’s packed with insights you won’t easily find elsewhere, because many of the challenges I faced had no ready-made solutions.

If you want to dive deep into performance optimization for Google-referred users, start here.

Thank you

I hope you found Signed Exchanges worth exploring.

If you think others might find it interesting too, sharing the demo link could be a great way to spark their curiosity.

See the demo

Lastly, if you want to experience Signed Exchanges in a real-world, production environment, you can visit PlanujemyWesele — a Polish wedding planning website where I implemented full SXG support.
(Note: the site is in Polish, and its audience is Polish couples — but it provides a good technical example.)

Context (Oct 2025): After deep work with SXG, I suspected changes might be coming—just not this quickly. Cloudflare announced SXG deprecation, and I observed SXG stop working around Sept 19, 2025.

If you still want SXG, the current path is to self-host—but the tooling looks abandoned, and setup is non-trivial. Also, Google may follow Cloudflare by shutting down the SXG cache and removing SXG support in Chrome.

You learned about Signed Exchanges (SXG) prefetching errors caused by mutable subresources in the two previous posts. Now, you understand a good SXG experience requires keeping your subresources unchanged over time. Also, you know how to deal with threats to their immutability.

In this post, I will explore the remaining errors I identified. As I write, there is no documentation on most of these or the documentation is outdated.

Just getting started with improving website performance? Start here with my beginner-friendly introduction to Signed Exchanges. If you're already familiar with SXG, please continue to the technical details below.

How big Google's mouth is?

Trouble with images continues

Even after implementing a workaround for the progressive loading issue, I was still struggling with prefetching images.

This time however the issue had different characteristics. Instead of occurring for every JPEG file, it affected only a few specific images.

After inspecting them I found a common trait: they were huge and it was clearly causing the issue!

My website uses responsive images to look sharp on a 4K screen while loading fast on low-end devices. It seems the image-processing solution we use for optimizing images uploaded by our users sometimes fails which results in unoptimized images. An unoptimized high-resolution image could easily reach 1 MB and above.

Later I found the same issue can happen with other subresource types. In my case, some of the javascript chunks produced by Next.js were too large.

SXG maximum size

The official Google documentation mentions 8 MB as the maximum SXG size. Interestingly, this value is honored by Cloudflare ASX (it can produce SXGs up to this size), but not by Google itself. It will refuse to store such large files.

I had to determine it experimentally, by generating files of different sizes and testing which ones were cached by Google. Here is the result:

You may ask why it’s such a weird number, not a round 1 MiB (2²⁰ bytes) or at least 1 MB (10⁶ bytes). I think Google limits the size of the entire cache entry to 1 MiB and apart from the actual file contents, it also includes:

• HTTP response headers,

• SXG metadata and cryptographic stuff,

• maybe some additional, Google cache’s specific metadata.

The size of those depends on various elements, therefore it’s impossible to reliably determine the exact maximum file size.

To ensure a great user experience and minimize data usage, keep your subresource sizes well below the limit. While Google allows subresources up to this size, smaller files help protect users from excessive data consumption and lead to faster page loads.

Size of HTML document

Though unlikely, your HTML documents should not exceed the limit of 1044 KB too. Otherwise, you will get one of the following messages in the Warning HTTP header of the response generated by the Google SXG cache:

199 - "debug: content has ingestion error: Not a valid signed-exchange."

199 - "debug: content has ingestion error: Error fetching resource: missing magic prefix; extracting prologue"

You can check it yourself on the demo page I prepared.

Total size of prefetched data

With the ability to prefetch 20 subresources plus 1 HTML file, you can easily calculate the maximum data size available for prefetching. It's approximately 21 MB.

If you'd like to experience what it feels like to reach this limit, check out the interactive demo I created. You'll enjoy it!

How to keep your subresources small

The only proper solution is to ensure your subresources don’t exceed the limit. If some of the images are not optimized, optimize them. Consider decreasing image quality and/or resolution if the file size remains over the limit after optimization.

Try to keep your scripts compact. Minification is a standard practice, but it has to be mentioned here in case you don’t use it for some reason. Avoid using large libraries that need to be included in your bundle. Consider dynamic loading for less frequently used logic.

You may also want to adjust how the scripts are split into chunks to ensure no chunk exceeds the limit. For example, Next.js won’t split the _app by default. To force it to do it, you can extend your next.config.js using the following template:

function forceChunking(config, pages) {
  const originalChunks = config.optimization.splitChunks.chunks;
  config.optimization.splitChunks.chunks = (chunk) => {
    // Allow to split certain pages because it may eventually
    // become too large for SXG. For the full context see:
    // https://www.pawelpokrywka.com/p/other-errors-with-signed-exchanges
    if (pages.includes(chunk.name)) return true;
    return originalChunks(chunk);
  }
}

const nextConfig = {

  // Your current config options

  webpack: (config, options) => {
    if (!options.isServer) forceChunking(config, ['pages/_app']);

    // Your current webpack customizations

    return config;
  }
}
 
module.exports = nextConfig

Other frameworks that use webpack underneath may experience similar issues, and the above webpack configuration change may help. However, adjustments may be needed.

Choosing not to prefetch

If your subresources are still too large, the solution of last resort is to not prefetch them. This will impact your page load speed, but at least other subresources will be prefetched.

To do this, you can delete the <link> tag, but if you want to retain prefetching your assets for Early Hints (I mention them in the second part) and for standard, non-SXG page prefetching, there is a better option.

Just add the data-sxg-no-header attribute to <link> tags you want to exclude from SXG but keep for everything else:

<link rel="preload" href="2big.jpg" as="image" data-sxg-no-header />

The attribute is documented here.

The idea for a potential Cloudflare-side solution

Cloudflare already limits SXG it generates to 20 subresources to ensure compatibility with Google's SXG cache. It would be consistent to also exclude subresources during SXG generation if they exceed Google SXG cache's size limit.

403 forbidden errors

Sometimes, during debugging, you may encounter this message in the Warning HTTP header of the response generated by the Google SXG cache:

199 - "debug: content has ingestion error: Error fetching resource: origin response code = 403"

This means instead of a normal HTTP/2xx response, Googlebot got an HTTP/403 (forbidden) response while fetching the page to be put into the Google SXG cache later.

Unless your app generated an HTTP/403 error (which may happen for example on pages requiring being logged in), the most probable explanation is that Cloudflare blocked this request.

Why does Cloudflare block some requests?

Cloudflare allows website owners to protect their property against bots.

When enabled, Cloudflare scans requests for signs of automated traffic. Depending on your plan, you can choose what to do with bot-generated requests. For example, in the Pro plan (the minimum for SXG support) you can allow bots, block them, or challenge them with a captcha. If a person is mistakenly classified as a bot, using a captcha enables them to still access the site.

As some bots are useful (such as Googlebot), Cloudflare gives you an option to allow so-called good bots. You can see all the options below:

The recommended configuration for public websites is to block or challenge malicious bots while allowing legitimate ones like Googlebot. However, even with these settings, Cloudflare may still inadvertently block Googlebot's access!

Why does Cloudflare block Googlebot?

First, I thought the blocking was happening because I was repeatedly triggering Googlebot to perform many debug requests to strange-looking test URLs. Maybe Cloudflare uses machine learning to distinguish those from normal Googlebot traffic?

Then, I opened the Google Search Console and saw a lot of HTTP/4xx responses for normal, non-debug Googlebot requests. Up to 5% of the traffic was blocked. It wasn’t looking good. Blocking even some of the Googlebot requests is dangerous for SEO.

When I allowed all bots, the issue disappeared.

A better solution

The bot-blocking feature is great; I didn’t want to disable it. However, I didn’t want to prevent Googlebot from accessing my site, which led me to the solution based on the Cloudflare Web Application Firewall (WAF).

I created a rule that skips bot blocking for traffic from Google IP addresses.

To do that, go to the Security section and open the WAF submenu. Choose the Custom rules tab and hit the Create rule button. Then, configure the rule as shown below and click the Deploy button:

Instead of specifying an ever-changing list of Google IPs, I matched against a single Autonomous System (AS) number. You can use RIPEstat or any other online tool to find the number assigned to Google.

Google SXG cache ingestion issue

One day I observed a dramatically increased number of SXG prefetching errors. The following days were even worse and finally, SXG stopped working entirely. I believe it was caused by the global SXG cache outage.

It lasted for about 2 weeks. I experienced it only once, but potentially it may happen again. If your SXG metrics go crazy without any sensible reasons, checking if the Google SXG cache works may be a good idea.

Sometimes, Google SXG cache ingestion rate is so slow, that it seems dead. This is likely due to the cache being under heavy load. I’ve occasionally experienced this during peak hours, especially when the U.S. wakes up.

Temporary errors

Even if you do everything correctly, you may encounter this message in the Warning HTTP header of the response generated by the Google SXG cache:

199 - "debug: content has ingestion error: Not a valid signed-exchange."

The warning will eventually vanish in a few hours max. It may be related to the number of new (not cached yet) subresources the page depends on. If the number is high enough, the error may occur temporarily.

If you know some errors are temporary, you won’t waste time trying to find and fix the inexistent cause.

Note however, the exact same error may communicate you have to fix your page because Cloudflare fails to generate SXG version of it. Please refer to the first post in the series on how to adjust your page to be SXG-ready.

How many subresources are too many?

In the official documentation, you will find that the maximum number of subresources is 20. This is a hard limit you won’t be able to exceed.

However, there is also a soft limit. Exceeding it will cause temporary errors until all the subresources are cached.

In my experiments, the error mentioned above may manifest itself when the page introduces 14 or more new subresources. By testing various scenarios, I came to the following formula:

2 x new + existing < 34

If you substitute new and existing variables with your values and the formula is true, your page should have no issues. For example, having 13 new and 7 existing subresources will be ok, while 15 new and 5 existing subresources may lead to (temporary) problems.

Cloudflare ASX utilization fluctuations may impact the formula, so take it with a grain of salt.

Why do these errors occur?

Based on my research, I conclude the cause is related to the latency guarantees of Cloudflare ASX. I believe there is a maximal latency ASX could introduce while generating an SXG-wrapped page. ASX tries to avoid exceeding it by having a plan B: returning the original page, without SXG wrapping.

If this happens, the Google SXG cache receives a response of a different type than expected. That’s why the warning about an invalid signed exchange appears.

Calculating integrity hashes from subresources takes time, so Cloudflare ASX uses a special-purpose cache I discovered accidentally. If all the integrity hashes are cached, SXG will be generated quickly. The more integrity hashes have to be calculated, the more time it takes, eventually exceeding the limit.

Retrieving the integrity hash from the ASX cache takes some time too. Although it’s probably a fraction of the time spent to calculate an integrity hash, it’s non-zero. That’s why the number of existing (cached) subresources is also taken into account.

In my tests, the size of the subresource doesn’t impact the ASX behavior—the formula stays the same. It probably means ASX estimates the time it will take to include all the integrity hashes based on the number of new and cached subresources instead of actually measuring how long it takes.

Expiration error

Another temporary error is related to SXG expiration date.

199 - "debug: content has ingestion error: SXG expiration date is closer than minimum cache duration."

I was unable to reproduce it reliably and thus failed to understand its cause. It appears from time to time and should vanish quickly if you followed instructions from my earlier posts.

If you experience this issue permanently, I suggest checking if the Cache-Control HTTP header is set correctly. You will find more information on that in the first part of the series.

Things you can’t control directly

Some errors happen because of reasons outside of your control, at least partially. Here are the scenarios I identified, but there may be others:

Not enough time

The user may click on the Google search result before all the subresources are prefetched. This could be due to a slow or unreliable connection and/or the user acting very quickly.

While you cannot control either of these factors, you can attempt to reduce the total size of the subresources to be prefetched. You may need to balance the prefetched page completeness and the SXG error rate.

You will learn how to measure the SXG error rate in the next part of the series.

New tab

The user may open the page in a new tab. In my tests, it breaks the prefetching of subresources (and in some cases, SXG-prefetching entirely, see the update below).

Interestingly, it doesn’t impact the prefetched HTML page—it is still used by the browser. Only subresources are dropped.

My unconfirmed explanation of this phenomenon is that the Chrome browser uses different caches for SXG pages and SXG subresources. The former is similar to a normal browser cache—it’s shared between tabs. The latter uses an isolated, per-tab cache, probably for the same reason as the all-or-nothing principle I described in the third part: privacy.

2025-05-18 UPDATE

The behavior described above can be observed when using the CTRL+click method to open a new tab on desktop. If you choose “Open link in new tab” from the context menu on desktop, or open the link in the new tab on mobile, the browser will not use the SXG-prefetched HTML at all.

Why does it behave like this?

Google sets the non-SXG target page URL in the href attribute of the result link. This ensures that when you hover over the link, you see the normal URL instead of the less user-friendly webpkgcache.com URL. However, when you click the link, Google dynamically replaces the href attribute with the webpkgcache.com URL, allowing your browser to load the prefetched version of the page.

The “Open link in new tab” method bypasses JavaScript, so the href attribute is not updated, and the browser opens the original target URL instead of the SXG-prefetched version. In contrast, using CTRL+click triggers the click event, allowing the script to rewrite the URL to the SXG version before the new tab is opened.

Browser deciding not to prefetch (unconfirmed)

The browser is not required to perform prefetching.

In theory, in suboptimal conditions such as slow connection speed, low battery, etc. the browser may decide to conserve resources and avoid prefetching (such as implemented in the quicklink library).

I tried triggering this behavior on my Android phone by disabling wifi, downgrading to 3G, then enabling data and battery saver. The Chrome browser slowly, but happily prefetched everything.

I was not able to observe the browser avoiding prefetching, but the possibility is there. Even if prefetching currently works all the time on all browsers supporting SXG, it may change in the future.

(Sub)resources expiring in Google SXG cache

Each SXG has an expiration date enforced by cryptographic signature validity. I found that Google sometimes performs refresh requests, but I’m sure there are scenarios when the user’s browser fetches expired SXG.

I could not reproduce those errors reliably, but I observe them regularly in my monitoring system. This leads to breaking the SXG experience entirely if the main document has expired or partially if only one or more subresources have expired.

I was able to simulate the expired subresource error in the browser. In the Chrome Developer Tools’ Network tab, the failed request should have the following status:

(failed) net::ERR_INVALID_SIGNED_EXCHANGE

If you open the Preview tab in the request details, you should see the following error:

Invalid timestamp. creation_time: 1739777019, expires_time: 1740381819, verification_time: 1740383698
Failed to verify the signed exchange header.

Additionally, the Date and Expires in the Signature section should be marked in red.

Subresources being evicted from Google SXG cache (unconfirmed)

I suspect Google SXG cache entries can be removed before expiration due to several reasons, the most important is disk space conservation. When this happens, it may cause prefetching failures manifesting as CORS errors.

However, I was unable to reproduce it. I believe the cache should include an eviction mechanism, but I can also imagine other ways to conserve disk space.

Temporarily missing SXG certificates

I've written about these types of errors before, but for the sake of completeness, I'm posting it here as well.

I witnessed a few cases where a certificate used in the SXG signature was missing from the Google SXG cache. In those scenarios, the browser can’t validate the affected signature leading to the following subresource status in the Network tab of Chrome Developer Tools:

(failed) net::ERR_INVALID_SIGNED_EXCHANGE

I'm not sure why these errors occur. I suspect it's an issue in Cloudflare ASX or Google SXG cache, so it can be fixed only by those companies. On the bright side, these errors should be rare and tend to disappear quickly.

Summary

This post is the final one in the category of explaining SXG errors. Here's what you've learned about them:

• Non-temporary CORS errors while prefetching SXG pages indicate issues with Google’s SXG cache handling subresources.

• Subresources must be immutable, including their HTTP headers. Common issues arise from the absence of the Vary header, Cloudflare's HTTP/2 prioritization, and Etag headers tied to modification times.

• Size limitations: Each element of your page, including the HTML document and all subresources intended for prefetching, must be smaller than 1044 KB.

• Avoid excessive prefetching: Using SXG to prefetch too much data can increase error rates.

• Bot protection: If you want to protect your site from bots, always accept Googlebot requests. This requires configuring Cloudflare's WAF appropriately.

• External factors: Some errors may arise from issues beyond your control.

In the next part, I will guide you on how to measure the impact of SXG on your website properly.

Thanks

I hope you found this post helpful. Thank you for reading!

Context (Oct 2025): After deep work with SXG, I suspected changes might be coming—just not this quickly. Cloudflare announced SXG deprecation, and I observed SXG stop working around Sept 19, 2025.

If you still want SXG, the current path is to self-host—but the tooling looks abandoned, and setup is non-trivial. Also, Google may follow Cloudflare by shutting down the SXG cache and removing SXG support in Chrome.

Prefetching errors eliminate 99% of the benefits of using Signed Exchanges (SXG). This is why it's critical to resolve these errors. While you learned how to handle two common errors in the previous post, this article will address a more complex error that shares some similarities with those issues.

As I write, there is no official or unofficial documentation on the issue described in this post.

If you don’t know what SXG is, or need a refresher, please start from the beginning.

This one was fun

The error I observed was different from the previous two. This time:

• all subresource types were affected,

• subresources were targeted randomly, but not all at once,

• if a given subresource was affected, the error occurred for some time (a day for example) and then disappeared (or moved to another subresource).

To make things even more interesting, the error rate was proportional to our development activity. Deploying more often caused more errors to occur. The obvious solution was to fire developers and stop working on the app!

Since my role as a programmer was at risk, I had to find another way to fix it. I found a workaround pretty quickly, but it took me a long time to understand the cause. I demonstrate my thought process below.

Comparing fresh and cached versions

First, I fetched the problematic, SXG-wrapped page with the dump-signedexchange tool (you have to remember to selectively or globally deactivate Cloudflare bot protection when using it):

$ dump-signedexchange -uri https://www.planujemywesele.pl/bad-page

...
response:
  status: 200
  headers:
    ...
    Link: <https://www.planujemywesele.pl/sub.css>;rel=preload;as=style,<https://www.planujemywesele.pl/sub.css>;rel=allowed-alt-sxg;header-integrity="sha256-/6M5qdrjQUl+dCdWtFqN50BmBvAHZ+sJxiLaoraBO2s="
signature: ...
header integrity: ...

I removed almost everything from the output and made the important part bold: the subresource header integrity hash.

Then I used the same tool to fetch the problematic SXG-wrapped subresource:

$ dump-signedexchange -uri https://www.planujemywesele.pl/sub.css

...
header integrity: sha256-/6M5qdrjQUl+dCdWtFqN50BmBvAHZ+sJxiLaoraBO2s=

The integrity hash of the subresource is the same in the document and in the subresource itself. That’s the way it should work.

Now, let’s see what Google cached. We can use the same tool to examine it:

$ dump-signedexchange -uri https://www-planujemywesele-pl.webpkgcache.com/doc/-/s/www.planujemywesele.pl/bad-page

...
response:
  status: 200
  headers:
    ...
    Cf-Ray: 8f61c1c4a09ce802-ORD
    Link: <https://www.planujemywesele.pl/sub.css>;rel=preload;as=style,<https://www.planujemywesele.pl/sub.css>;rel=allowed-alt-sxg;header-integrity="sha256-gxoxVoIaE8MKl8tf283F3FKVF8NwrLqz2jMT/vwyO9c="
signature: ...
header integrity: ...

The header-integrity hash is different!

Let’s examine the HTTP response carrying Signed Exchange payload using curl:

$ curl -iH "Accept: application/signed-exchange;v=b3" https://www-planujemywesele-pl.webpkgcache.com/doc/-/s/www.planujemywesele.pl/bad-page

...
link: <https://www-planujemywesele-pl.webpkgcache.com/sub/gxoxVoIaE8MK/s/www.planujemywesele.pl/sub.css>;rel="alternate";type="application/signed-exchange;v=b3";anchor="https://www.planujemywesele.pl/sub.css"
...

The Link header points to a URL to a cached version of SXG-wrapped subresource, so that the browser can prefetch it. This URL is constructed from the URL and a part of the integrity hash (in bold).

When we try to fetch it using curl:

$ curl -iH "Accept: application/signed-exchange;v=b3" https://www-planujemywesele-pl.webpkgcache.com/sub/gxoxVoIaE8MK/s/www.planujemywesele.pl/sub.css

...
location: https://www.planujemywesele.pl/sub.css
content-type: text/html; charset=UTF-8
...

<HTML><HEAD>
<meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>Redirecting</TITLE>
<META HTTP-EQUIV="refresh" content="0; url=https://www.planujemywesele.pl/sub.css">
</HEAD>
<BODY onLoad="location.replace('https://www.planujemywesele.pl/sub.css'+document.location.hash)">

The fallback page will be returned telling us, that the entry doesn’t exist in the Google SXG cache.

Now, it is very clear that the SXG-wrapped page cached by Google is different because it links to another subresource. And this subresource can’t be found in the Google SXG cache. This is a cause for prefetching failure and CORS error.

Why did Google cache a different page?

To get Google’s perspective, we can use Google Search Console. It contains a URL inspection tool that has a Live test feature. It lets you use Googlebot to fetch any URL on your website and examine the results.

When you open the More info tab and click the HTTP Response, you will see the HTTP headers. Here they are, redacted to include only interesting bits:

HTTP/1.1 200 OK
...
cf-ray: 8f6092f8e08e2231-ORD
link: <https://www.planujemywesele.pl/sub.css>;rel=preload;as=style,<https://www.planujemywesele.pl/sub.css>;rel=allowed-alt-sxg;header-integrity="sha256-gxoxVoIaE8MKl8tf283F3FKVF8NwrLqz2jMT/vwyO9c="
...

The header-integrity hash is the same as in the Google SXG cache. So we have the answer to the most recent question: Google cached what it saw which is different from what we see.

Why does Google see the world differently than we do?

My first thought was that Cloudflare treats Googlebot differently than others. I even formulated an entire theory based on this assumption. But it was a dead end.

Cloudflare has data centers all around the world. What if two different data centers return different pages? Particularly, what if the data center close to Google returns a different page than the data center near me? To verify it, the first step is to find the data center used for Googlebot traffic.

Which Cloudflare data center handles Googlebot traffic?

In the SXG responses intended for Googlebot, Cloudflare includes a Cf-Ray header. This header contains an identifier of the Cloudflare data center used to handle the request.

My suspicion about Cloudflare treating Googlebot differently was correct because this header is served only for this bot.

I tried to impersonate Googlebot by setting the User-Agent request header, but the Cf-Ray header was not added as a result. They probably use a more reliable, IP-based detection of Googlebot.

In the responses you saw above, the data center is identified as ORD. Cloudflare has a status page listing all data centers and their geographical locations:

It seems Cloudflare uses airport codes for data center identifiers. As you can see, ORD is Chicago.

It means Google accessed my website using the Cloudflare data center located in Chicago, at least for requests I examined, and at the moment I did it. It may change in the future, but it’s quite consistent for now.

Sitting where Google sits

Now, that I knew the data center, I tunneled to Chicago using a VPN:

$ mullvad relay set location us-chi-wg-101
Relay constraints updated
$ mullvad reconnect -w
Connecting
    Relay:                  us-chi-wg-101
    Features:               LAN Sharing, Quantum Resistance
    Visible location:       USA, Chicago, IL
Connected
    Relay:                  us-chi-wg-101
    Features:               LAN Sharing, Quantum Resistance
    Visible location:       USA, Chicago, IL

Finally, I checked if Cloudflare routed me to the correct data center. The Cf-Ray header for non-SXG responses is accessible to anybody, not only Googlebot:

$ curl -si https://www.planujemywesele.pl/ | grep cf-ray
cf-ray: 8f6229558f9710c2-ORD

Ok, I was using the same data center as Googlebot. Now, I fetched the SXG:

$ dump-signedexchange -uri https://www.planujemywesele.pl/bad-page

...
response:
  status: 200
  headers:
    ...
    Link: <https://www.planujemywesele.pl/sub.css>;rel=preload;as=style,<https://www.planujemywesele.pl/sub.css>;rel=allowed-alt-sxg;header-integrity="sha256-gxoxVoIaE8MKl8tf283F3FKVF8NwrLqz2jMT/vwyO9c="
signature: ...
header integrity: ...

The header-integrity hash is the same as in the cached version and as seen by Googlebot, but different from what I saw when accessing the website without VPN.

This proves that the Chicago data center serves a different version of a page than the data center close to me.

Which version of the page is correct?

I fetched the subresource while using the Chicago data center via VPN:

$ dump-signedexchange -uri https://www.planujemywesele.pl/sub.css

...
header integrity: sha256-/6M5qdrjQUl+dCdWtFqN50BmBvAHZ+sJxiLaoraBO2s=

The header integrity is the same as when I fetched it without VPN, from my original data center.

Given the following facts:

• My non-VPN data center returns a page linking to a subresource with integrity hash X.

• Chicago data center returns a page linking to a subresource with integrity hash Y.

• Both data centers return a subresource with integrity hash X.

It becomes clear the Chicago data center returned a broken page.

What’s wrong with Chicago?

Is the Chicago data center special? It seemed so because I was getting consistently good results when switching to different data centers using a VPN. Only Chicago returned an invalid page.

I heard that in Chicago, they don’t put ketchup on their hot dogs. That could be the reason, but I kept searching for other explanations.

The subresources causing troubles were most often shared between different pages. I tested many of those pages from Chicago with a VPN. Each page linked to a subresource with an invalid integrity hash.

Is the cache entry for a page broken?

Each Cloudflare data center has its own cache. Maybe the cache was to blame? What if for some reason, the Chicago data center cache contains broken entries for the pages I tested?

This led me to test a page that I was sure wasn’t accessed before and, therefore missing from the cache. To my surprise, this fresh page still referenced subresources with invalid integrity hashes!

Is the cache entry for a subresource broken?

No matter which data center I used to obtain the subresource, it gave me the same data. So the subresource was fine.

Just to be sure I purged the page and the subresource from the Cloudflare cache and performed the test again. The result was the same - I got a page linking to a subresource with an invalid integrity hash.

It seemed the data center had remembered the wrong integrity hash and used it to generate SXG responses.

Remembering things is the role of a cache. But I cleared it! It didn’t make any sense, unless…

There is another, hidden cache!

When I think about it now, it’s obvious. Cloudflare uses a special-purpose cache for integrity hashes to minimize latency when generating SXG responses. As with Automated Signed Exchanges (ASX) instances, this cache is local to every data center.

I validated this hypothesis by putting logging workers before subresources linked by the page. This way every request was logged, because workers are invoked every time, even for cached entries.

Then I requested SXG of the page and observed logs. The subresources were accessed only once per data center. Further requests for SXG didn’t result in any logs, even after purging the cache.

The hidden cache I discovered is not mentioned anywhere in the documentation. It can’t be purged just like the normal Cloudflare cache. It’s an opaque black box.

Sometimes, this black box contains invalid integrity hashes leading to populating the Google SXG cache with invalid pages and breaking the SXG experience for the end users. I would say the hidden ASX cache seems poisoned.

Why is the ASX cache poisoned?

I believe it’s caused by a delayed subresource mutation.

When a client requests the SXG page for the first time, ASX may retrieve the subresource and store its integrity hash in the local ASX cache. Later, the subresource mutates, but this fact remains hidden as long as the original version remains in the Cloudflare cache. The issue starts to manifest after it is evicted from the Cloudflare cache: the page links to a subresource that is no longer there.

The Google SXG cache obscures the issue further because it stores both versions of the subresource. However, user-visible issues can arise in two scenarios:

1. When the Google SXG cache evicts the original version before it expires.

2. When technical problems (such as network issues) prevent the Google SXG cache from successfully storing the original version during the initial data collection.

In either case, this results in the Google SXG cache only having the mutated version of the subresource, while the original version becomes unavailable. As the page refers to the original version it causes prefetching errors. And because the ASX cache is poisoned, the error will stay there no matter how often the Google SXG cache tries to refresh the entry.

I could not validate the two scenarios described above because they depend on hard-to-control conditions. Therefore it should be treated as a hypothesis. However, I successfully demonstrated ASX cache poisoning by artificially simulating technical issues while Google's SXG cache performed its initial fetch of the subresource.

How to fix ASX cache poisoning?

Now that I knew the cause—delayed subresource mutation—I could solve the problem by eliminating this mutation. By comparing various responses I was able to find the source of the mutation.

It was the Etag header.

Etags and deployments

The Etag HTTP header is an optional identifier sent along with the response. It should stay the same unless the response content changes. When the browser fetches the URL again, it includes the Etag value of the locally cached response. The server compares the received value with the current value. If they differ, it sends the response as usual. But if they are the same, it returns an empty response with 304 Not Modified status, saving bandwidth.

Etags may be used for every response type, but in the case of SXG the most important are static assets, so let’s focus on those.

This is how etags for static assets are generated by various components of my stack:

• Nginx generates etag for a given asset using its size and modification time.

• Next.js app uses the same approach.

• Ruby on Rails app leaves it to the web server (it this case nginx).

If your deployment method (like mine) writes the static assets to disk even if they didn’t change, then their modification time will be updated as well. In effect, nginx and Next.js will generate new etags even if the files remain the same.

I suppose other web servers and framworks behave the same.

It will cause some clients to download assets again. It’s a minor issue because assets are typically cached for a long time and etags are not needed until the cache expires.

Etags impact on SXG

However, the Etag header is included in the generated SXG subresource. If the etag changes, the subresource mutates. And given some conditions are met it leads to SXG prefetching errors.

In my case, the more often I deployed, the more often the subresource mutated. This was the reason why the errors intensified during increased development activity!

How to make etags and SXG work together?

It may be a good idea to ensure the modification times of your assets stay the same unless actually modified. I can’t explain how to achieve it in detail because it depends on the deployment method. And sometimes it may be non-trivial or even impossible to achieve.

Therefore, I suggest an alternative approach. Do not send the Etag header in responses for static assets. If you use a 1-year expiration date as I recommend, it's not a big deal anyway.

For nginx, you can set it globally with the following directive in the HTTP context, preferably in the nginx.conf file:

etag off; # Disable etag generation for static assets

Next.js allows disabling etags entirely. I don’t like this approach, because it also disables it for HTML responses. Instead, it can be done using nginx again, by removing the Etag header from the responses for static assets only.

I use the following nginx configuration snippet in locations containing static files:

location ~* \.(?:css|js|gif|png|jpeg|jpg|ico|ttf|woff|woff2|svg)$ {
  # Clear etags set by Next.js as they vary between deployments
  # even if the assets remain the same. This may leads to SXG failures.
  # For more info see: https://www.pawelpokrywka.com/p/debugging-complex-signed-exchanges-subresource-issue
  more_clear_headers 'Etag';

  # The rest of assets-related configuration, such as cache expiration
}

Check the documentation of your web server and framework for hints on how to configure etags. For example, Apache offers an option to generate etags from file contents. This approach ensures immutability and eliminates the need to disable etags.

Alternatively, you can create a Cloudflare transform rule to remove the Etag header from all static assets. I have already explained how to create such a rule in the previous part. The only difference lies in step 6: instead of selecting Set static from the menu, select Remove and enter Etag in the Header name field.

As there is no way to purge the hidden ASX cache, remember to regenerate URLs to invalidate the cache (as described in the previous part) after implementing the changes.

Quick-fix for ASX cache poisoning

I found that bypassing the Cloudflare cache makes the ASX stop using the poisoned cache as long as the bypass is in place.

It means both caches share some logic. If Cloudflare went one step further and synchronized evictions between those caches, the issue described in this post would disappear instantly.

When I tried to disable the Cloudflare cache globally, the ASX stopped generating SXG-wrapped pages. If the cache is not used for too many assets, the SXG will break. It has to be disabled selectively.

To bypass the Cloudflare cache for a specific asset, go to the Cache Rules in the Caching section. Hit the Create rule button, fill out the form according to the template below changing the URL to match your asset, and click the Deploy button.

I don’t recommend bypassing the cache as it impacts the performance. But if you want to quickly check if the issue is related to ASX cache poisoning, you can use this technique.

That’s it!

I've demonstrated how to diagnose, understand, and fix this issue that's not documented anywhere. I hope you find it useful and perhaps learned something new about approaching unfamiliar problems.

While this concludes our discussion of mutable category errors, there are still a few other issues to address. I'll cover these remaining problems and their solutions in my next post.

Thank you for reading!

Context (Oct 2025): After deep work with SXG, I suspected changes might be coming—just not this quickly. Cloudflare announced SXG deprecation, and I observed SXG stop working around Sept 19, 2025.

If you still want SXG, the current path is to self-host—but the tooling looks abandoned, and setup is non-trivial. Also, Google may follow Cloudflare by shutting down the SXG cache and removing SXG support in Chrome.

In the previous part, you learned CORS errors mean (sub)resources are missing from the Google Signed Exchanges (SXG) cache. This and later posts will explain how to deal with that.

I suggest you also read the first two parts to fully understand what is happening. They contain fundamentals on how to adjust your website to be SXG-compatible.

This and later posts summarize my research on issues related to the prefetching of SXG subresources. As I write, there is no official or unofficial documentation on most of the challenges you may encounter. My articles aim to fill that gap.

Have a way to invalidate all the caches

When you work with SXG subresources prefetching issues, you may want to make sure you can observe the impact on your changes. It may be non-trivial because of caching.

Many caches sit between your app and you. Here is a minimal set of caches:

• Cloudflare caches1

• Google SXG cache

• Browser cache

Depending on your configuration, you may also use app cache, framework cache, web server cache, additional layers at Cloudflare, and potentially others.

After implementing the changes, you could clear all of the involved caches, but that’s a lot of work and sometimes it’s even impossible.

It’s much easier to just make sure the URLs you are about to test were not accessed before and, therefore are guaranteed to be absent from the cache. After making an SXG-specific change to your app or configuration, make sure that:

1. You are testing a different subpage of your website. In my case, I can access my app under different URLs for each Polish city & vendor category combination. It gives an almost unlimited set of URLs I can use for testing.

2. URLs of your assets change. This is important during the reconfiguration of HTTP headers and other subresource-specific modifications explained later. Every framework has its own ways of achieving that.

As my website combines Ruby on Rails and Next.js, I will demonstrate how to invalidate cache in those frameworks. Different frameworks likely have their own ways of achieving that, but some concepts may remain the same.

Invalidate the cache in Rails

Rails makes it easy. As long as you use asset helper methods (image_tag, stylesheet_link_tag, etc) to reference your assets, just increment the version of the assets in config/initializers/assets.rb file:

Rails.application.config.assets.version = '1'

Invalidate the cache in Next.js

This framework uses two forms of referencing assets:

1. Direct: the developer specifies the path to the asset in the CSS, HTML tag, or React component. In the context of SXG, it is used mostly for images and fonts.

2. Autogenerated: the framework compiles javascript and CSS into chunks and places them inside the HTML head of the document (and our custom Cloudflare worker puts them in the Link header). The developer has limited control over the URLs.

Direct paths

In the case of directly referenced assets, the simple solution is to move all your assets into a subdirectory named after a version and add a version prefix while referencing URLs.

So instead of using:

<img src="/icons/icon.svg">

You will use:

<img src="/1/icons/icon.svg">

When invalidating the cache, you rename the directory to “2” and update all references using global search-and-replace in your code editor. It’s a good idea to create a function returning the versioned asset URL and store the current version in a global ENV variable:

function assetPath(path) {
  return `/${process.env.ASSETS_VERSION}/${path.replace(/^\//, '')}`;
}

Make sure to include the current version in your .env file. This way you have a single place where you can update the version (along with renaming the assets version directory):

ASSETS_VERSION=1

Referencing the assets would look like this:

<img src={assetPath("/icons/icon.svg")}>

Autogenerated paths

Next.js has a built-in assetPrefix configuration parameter that makes it possible to embed version numbers into the URLs. You'll still need to move the assets to the prefixed location during deployment. The specific instructions will vary depending on your deployment method, therefore I won’t provide them here and describe a more generic solution instead.

For autogenerated URLs, the Next.js config has to be adjusted to include the version number stored in the ASSETS_VERSION configuration variable. Extend the configuration object in your next.config.js file using the following template:

const NextMiniCssExtractPlugin = require('next/dist/compiled/mini-css-extract-plugin');

function addVersionToAssets(config) {
  const ver = process.env.ASSETS_VERSION;
  const addVer = (text) => text.replace('[name]', `[name]-${ver}`);

  // Include version in script URLs. For more context see:
  // https://www.pawelpokrywka.com/p/fixing-sxg-prefetching-errors-caused-by-mutable-subresources
  const filename = config.output.filename;
  const chunkFilename = config.output.chunkFilename;
  if (filename && chunkFilename && filename.startsWith('static')) {
    config.output.filename = addVer(filename);
    config.output.chunkFilename = addVer(chunkFilename);
  }

  // CSS URLs are handled by a plugin. It needs to be reconstructed
  // with a template containing the version prefix.
  const index = config.plugins.findIndex(
    (plugin) => plugin.constructor.name === 'NextMiniCssExtractPlugin'
  );
  const template = `static/css/${ver}-[contenthash].css`;
  if (index !== -1) {
    config.plugins[index] = new NextMiniCssExtractPlugin({
      filename: template, chunkFilename: template
    });
  }
}


const nextConfig = {

  // Your current config options

  webpack: (config, options) => {
    if (!options.dev) addVersionToAssets(config);

    // Your current webpack customizations go here

    return config;
  }
}
 
module.exports = nextConfig

After building the app, you will notice that your JS chunks and CSS files have embedded version numbers. To invalidate the cache, increment the version stored in ASSETS_VERSION in your .env file and rebuild.

One of the chunks generated by Next.js, the polyfills chunk is:

• loaded with a nomodule attribute, which ensures that only legacy browsers without support for ES modules will download it,

• treated specially by Next.js making it non-trivial to include a version in the URL.

Rather than invalidating this particular chunk, it's more efficient to avoid preloading it altogether. Prefetching polyfills for modern browsers is wasteful since they don't need them. The worker that adds the Link header (described in the second part of the series) already ignores <script> tags with the nomodule attribute.

Given all of the above, you don’t need to worry about invalidating the polyfills chunk.

Subresources stored externally

I believe this should be rare, so I won't go into details here. Sometimes, you may need to change the HTTP headers of your subresources hosted externally and proxied using a Cloudflare worker.

In this case, the URLs of subresources should change too.

• Your app should include version numbers in URLs of external subresources using an approach similar to the assetPath function I described earlier.

• On the worker side, you can implement it by including the version number in the URL map.

const MAP = {
  'https://www.your-domain.com/cdn-proxy/PUT_THE_VERSION_HERE':
  'https://cdn.com/your-bucket/',
  // You may add more mappings such as:
  // 'external-sxg-prefetchable-url': 'cdn-url'
}

// Rest of the worker logic

The production-grade solution would use a regexp accepting any version, so the worker code doesn’t need to be updated every time the version changes. I leave it as an exercise for the reader.

In the case of a proxying worker, Cloudflare uses original, CDN-provided URLs as cache keys, not URLs the worker exposes to your users. Even if you modify the exposed URLs, the original URLs won’t change meaning stale cache entries will be used. Remember to purge the Cloudflare cache to avoid that, unless you modify your subresources in the worker only, without changing them on the CDN.

Google SXG cache ingestion process

Before we dive deeply into the causes of missing entries in the SXG cache, it’s critical to understand how Google ingests your pages and how it cooperates with Cloudflare.

Reverse engineering the process

I modeled this process using reverse engineering. It was created out of necessity, to help me understand the issues I was experiencing. I was performing requests and observing responses. Then, I was analyzing logs from the server and Cloudflare.

As I write this post, the resulting process isn’t documented anywhere else.

After landing on the Google search results page, this page tries to prefetch a result the user is predicted to click. The following diagram illustrates what happens. For simplicity, it assumes the page depends on only one subresource (CSS file):

When the first user requests the page, the prefetching initially fails because the page has not yet been cached. However, this request triggers the cache population process. Consequently, the second user (starting at step 15) benefits from the now-prefetched page, experiencing faster load times.

You may want to open the diagram in a new tab to avoid repeated scrolling—it will be easier to switch between tabs as I reference it throughout this discussion.

Understanding the Google SXG cache population process (steps 3-14) is crucial in debugging potential errors. Observe the clean separation of responsibilities:

• Cloudflare Automated Signed Exchanges (ASX) is a layer wrapping HTTP responses into SXG and doing nothing else.

• Cloudflare cache knows nothing about SXG, and neither does the server.

Requests differences

During the cache ingestion, subresources are requested by different systems for different purposes:

• Cloudflare ASX for computing integrity hashes that will be included in the SXG-wrapped page,

• Google SXG cache for making subresources available to browsers for prefetching.

Let's compare the requests generated by these systems to better understand potential issues.

Here are interesting headers from an example request made by Cloudflare ASX:

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip, br
Cf-Connecting-Ip: 2a06:98c0:3600::103

Observations:

• The request lacks a User-Agent header.

• The Accept header doesn’t mention Signed Exchanges at all.

• The IP address belongs to Cloudflare.

Let's examine some notable headers from a sample Google SXG cache request:

Accept: */*;q=0.8,application/signed-exchange;v=b3
Cf-Connecting-Ip: 66.249.72.7
User-Agent: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.6533.119 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Observations:

• The request lacks an Accept-Encoding header.

• The Accept header indicates that Google prioritizes SXG over other formats.

• The IP address belongs to Google.

Watching the SXG cache ingestion live

I created a tool to demonstrate the ingestion process. It allows you to trigger the SXG cache population and observe requests made by Google and ASX. The tool uses the presence or absence of the User-Agent header to determine if the request is performed by Google or Cloudflare respectively.

Using the tool you may notice that trying to prefetch a page already present in the Google SXG cache (step 15) often has a side-effect of triggering the cache population process to refresh the entry. I intentionally didn’t include it on the diagram to keep it readable.

Keep in mind that the tool bypasses Cloudflare's cache, which is both a critical component in real-world applications and a significant indirect source of SXG-related errors.

Mutable subresources

I define the mutable subresource, as one that changes over time but remains accessible under the same URL. In contrast, an immutable subresource doesn’t change.

From my experience, most of the hard-to-debug SXG issues are caused by mutable subresources.

Why does a mutable subresource break things?

Let’s assume we have a page with a subresource that changes on each request. It may be the Web 1.0 visit counter - each time it is requested, it returns an incremented number as an image. The server keeps the current number, increases it on every request, generates a GIF, and returns it.

We need to disable the Cloudflare cache. Otherwise, it would cache the counter, effectively stopping it.

Please remember, this is just an example prepared to illustrate the point. It doesn’t make much sense to implement it in real-world use cases.

What will happen, when Google tries to fetch this and put it into its SXG cache?

According to the diagram above:

1. It requests the page from ASX (step 3).

2. ASX requests the page from the server and the counter image subresource (steps 4 and 6). The server generates an image presenting number 1.

3. ASX generates an SXG-wrapped page, includes the subresource URL along with its integrity hash, and returns it to Google (step 10).

4. Google parses the SXG-wrapped page, finds it depends on the subresource, and downloads it from ASX (step 11).

5. As Cloudflare cache is disabled, ASX downloads it directly from the server (step 12). This time, the server generates an image with the number 2.

6. ASX wraps the image into the SXG package and returns it to Google (step 13).

7. Google compares the integrity hash of the downloaded subresource with the integrity hash stored in the SXG-wrapped page. As those differ, Google rejects the subresource as invalid and doesn’t store it in the cache.

The integrity hashes were different because those were different images. First represented number 1, second number 2.

Still, the SXG-wrapped page will be stored in the Google SXG cache. However, it will depend on a subresource that can’t be prefetched, because it’s missing from the cache.

On Google search results, when prefetching the page, the browser will be instructed to prefetch the subresource located under a URL computed from the subresource integrity hash included in the SXG-wrapped page. As the URL doesn’t contain a subresource, the fallback HTML page will be returned instead causing a CORS error in the browser.

But my subresources are immutable!

If you follow best practices in web development, you probably try to keep your assets immutable. The reason is to prevent issues caused by stale copies stored in caches.

Frameworks such as Rails and Next.js offer include built-in solutions. The idea is to bind the asset URL and its content by making the digest of the content part of the URL. This way changing the content changes the URL, so the asset won’t mutate.

It works well with the original stale-cache issue, but…

SXG subresource > file content + URL

The above approach doesn’t consider HTTP headers returned along with the asset. Those headers are part of the SXG subresource. If one of the headers changes, the integrity hash changes as well.

Many HTTP headers are set by the web server, outside of the web application. The framework doesn’t control all of them, therefore it can’t include them in the digest.

Cloudflare ASX partially solves that by stripping headers known to change frequently, such as Date, Age, or X-Request-Id from subresources. But if your assets use other unstable headers, you may experience issues, as shown in this demo.

Doesn’t cache fix that?

As I wrote earlier, subresources should be cachable. If you want to see what happens, if the Cache-Control header is invalid, here is a demo.

Cloudflare has an HTTP cache enabled by default. Even if a given subresource changes on each request, the first response will be stored in the cache (steps 6-9 on the SXG cache population diagram), and later requests will return the cached entry (steps 12-13).

HTTP headers are cached too. Therefore, from the Google SXG cache perspective, the mutable subresource doesn’t change. That’s good!

Things become interesting, if the subresource gets modified after being retrieved from the cache or if the cache infrastructure has issues.

Compression negotiation

When performing an HTTP request, the browser typically uses the Accept-Encoding header to tell the server what compression methods it supports. It allows the server to use the best compression the browser understands, for example:

• If the browser supports zstd and gzip, the server will respond with a zstd-compressed response.

• If the browser supports only gzip, the server will respond with it.

The server will unilaterally decide if and how to compress the response if the browser doesn't set the Accept-Encoding header.

The problem arises when caching is involved. The cache stores the response under a cache key, typically created from the URL. However, accessing the same URL may result in a compression mismatch. That’s why the component performing the compression (the web server or Cloudflare proxy) should inform the cache, the URL is not enough to construct the cache key.

Vary HTTP header

It does it by setting the Vary header to a list of request headers that may impact the response. In the case of compression, it puts Accept-Encoding there:

Vary: Accept-Encoding

Importantly, the Vary header may be omitted if the request doesn’t contain the Accept-Encoding header.

As HTTP headers are part of the subresource, the presence of the Vary header impacts the integrity hash. It could be therefore possible to generate two different versions of the same subresource:

• with Vary header set, if the request contains an Accept-Encoding header,

• without it otherwise.

Fortunately, Cloudflare ASX doesn’t allow it by removing the Vary header from the response. However, I found one edge case when it’s not the case.

Subresource mutation caused by ASX itself

The issue occurs when the subresource to prefetch is behind a worker and uses Cloudflare cache. There are at least 2 reasons for such a setup, both were described earlier:

• rewriting URLs of CDN-stored assets,

• adding a Link header to all HTML responses by passing all traffic through the worker and letting it skip assets instead of configuring worker routing to avoid assets being processed.

I used reverse engineering again to model what happens. Here is a simplified version of the previous diagram:

First, ASX asks for a subresource with an Accept-Encoding header set to "gzip, br" (step 4). Therefore, as we discussed, the response includes Accept-Encoding in the Vary header.

The integrity header of the subresource is calculated and put in the Link header of the SXG-wrapped page returned to the Google SXG cache in step 6.

Next, the request for subresource comes from Google (step 7) and is proxied by ASX (step 8). Google doesn't set the Accept-Encoding header and ASX honors the original request’s headers2. Therefore the response doesn't include the Vary header (step 9).

When Google receives the subresource (step 10) it compares its integrity hash with the one from the Link header of the SXG-wrapped page it received in step 6. As they don’t match, the subresource is discarded and not cached.

The subresource is not cached, but it’s still referenced from the cached SXG-wrapped page. When the browser tries to prefetch it, a fallback page is returned and a CORS error happens.

It’s worth noting, this issue doesn’t happen when:

• the Cloudflare cache is disabled—probably because Vary is a cache-related header,

• not using a worker—probably because of interactions between a worker and ASX (ASX being a special instance of a worker).

To see it in action, take a look at the demonstration.

Workarounds for ASX bug

Until Cloudflare fixes this issue, the simple solution is to set the Vary header for all static assets responses, even when requests don’t specify the Accept-Encoding header.

Nginx

For assets included with the application, you can use the following nginx configuration snippet in locations containing static files:

location ~* \.(?:css|js|gif|png|jpeg|jpg|ico|ttf|woff|woff2|svg)$ {
  # Always set Vary header, because of the issue in Cloudflare ASX
  # when used along with workers
  more_set_headers 'Vary: Accept-Encoding';

  # The rest of assets-related configuration, such as cache expiration
}

Of course, the above solution doesn’t work for assets stored in the CDN and proxied through a worker.

Worker

Therefore, the more elegant solution is to implement the workaround in the worker itself. This way the fix doesn’t pollute the system and is contained closest to the cause of the issue (the worker). The downside is you need to remember about it in every worker if you use many.

Here is a minimal worker setting the Vary header:

function addVaryHeader(response) {
  const vary = response.headers.get('vary');
  if (!vary || !vary.toLowerCase().includes('accept-encoding')) {
    // Set the Vary header because of the issue in Cloudflare ASX
    // when used along with workers. For more details see:
    // https://www.pawelpokrywka.com/p/fixing-sxg-prefetching-errors-caused-by-mutable-subresources
    const newResponse = new Response(response.body, response);
    newResponse.headers.set('vary', 'Accept-Encoding');
    return newResponse;
  }

  return response;
}

export default {
  async fetch(request) {
    const response = await fetch(request);

    // Add Vary header to all responses.
    // For production, I would suggest doing so only for assets.
    return addVaryHeader(response);
  },
};

I won’t rehash how to create and deploy Cloudflare workers. I leave it as an exercise for you to merge this code with your workers, but it should be easy. Just wrap the response you are about to return with the addVaryHeader() function for static assets.

Transform rule

Alternatively, you can create a Cloudflare transform rule that adds the Vary header to all responses for static assets. The adventage of this approach is that you need to do this once and it benefits all the workers. To do that:

1. Open your website in the Cloudflare dashboard.

2. Go to Rules → Transform Rules → Modify Response Header.

3. Hit the Create rule button.

4. Provide a name for your rule, for example: Force Vary: Accept-Encoding for static assets.

5. Click the Edit expression link in the Expression Preview section and paste the following expression into the text area:
   (ends_with(http.request.uri.path, "css")) or (ends_with(http.request.uri.path, "js")) or (ends_with(http.request.uri.path, "gif")) or (ends_with(http.request.uri.path, "png")) or (ends_with(http.request.uri.path, "jpeg")) or (ends_with(http.request.uri.path, "jpg")) or (ends_with(http.request.uri.path, "ico")) or (ends_with(http.request.uri.path, "ttf")) or (ends_with(http.request.uri.path, "woff")) or (ends_with(http.request.uri.path, "woff2")) or (ends_with(http.request.uri.path, "svg"))

6. In the Then section select Set static from the menu and type Vary into the Header name and Accept-Encoding into the Value.

The form should look similar to the one below:

If everything is ok, hit the Deploy button. From now on, your assets will always have the Vary header set to Accept-Encoding.

Cache invalidation reminder

It’s a good idea to invalidate the cache after performing those changes as described at the beginning of this post. Otherwise, you may need to wait up to 7 days for the results.

HTTP/2 Prioritization

In 2019 Cloudflare announced a new feature called Enhanced HTTP/2 Prioritization. It promises improved page load times by optimizing asset delivery so that the browser fetches the most important assets before others.

The same day, they introduced a parallel streaming of progressive images. It uses the same mechanism but for data ranges within individual image files. For example, it prioritizes the beginning of a JPEG file containing a low-resolution image. In effect, on pages with more than one image, the browser first renders all of them in low quality, then continues to download them, gradually improving the resolution.

Those features promise improvements in the UX, so I was eager to turn them on. Both are controlled by one switch:

SXG doesn’t like HTTP/2 prioritization

I found a lot of prefetching errors were related to JPEG files. After examining the issue, I found that responses for JPEG files contained an HTTP header missing from other responses:

Cf-Bgj: h2pri

According to a response from a Cloudflare employee, the h2pri means it’s been processed to support HTTP/2 prioritization for Progressive Streaming of JPEGs.

To make things more interesting, the header is set on the second response (cache HIT), while the first response (cache MISS) doesn’t include it.

Processing takes some time, therefore I assume it is being put in the background and the first response doesn’t wait for it to keep the latency low. The processed image lands in the cache and that’s the reason it includes the Cf-Bgj (Cloudflare Background Job?) header.

The subresource mutates on the second request. The issue is very similar to the previous one with the Vary header, but this time the header name is Cf-Bgj. Mutable subresource can’t be fetched by Google SXG cache, the HTML document depends on it, therefore SXG prefetching breaks.

Workarounds for a Cloudflare bug

I prepared a page containing a demonstration of the issue.

I was tempted to use Cloudflare transform rules to remove the Cf-Bgj header. Unfortunately, it’s a special header that can’t be removed. Trying to create a transform rule would result in:

'remove' is not a valid value for operation because it cannot be used on header beginning with 'cf-'

From my experiments, the issue manifests only if all three conditions apply to a given JPEG subresource:

1. Enhanced HTTP/2 Prioritization feature is turned on. It is responsible for setting the Cf-Bgj header.

2. Cloudflare cache is in use. That’s probably related to the sidenote above about latency. Image processing might introduce latency if done synchronously without caching.

3. Cloudflare worker is not used. I suspect it’s related to workers being allowed to manipulate HTTP/2 prioritization by setting the Cf-Priority header. Maybe there is a cleanup logic for removing internally used headers, that include Cf-Bgj, and this logic gets fired only when using workers.

Turning off asset caching might be a less-than-optimal idea. But, if you don’t use workers for your assets, you may re-evaluate your decision to make the issue disappear. Any worker will do, the minimal I could think of is the following one-liner:

export default { fetch: (r) => fetch(r) };

If you still want to avoid workers, for example, due to additional financial costs, the remaining solution is to disable the Enhanced HTTP/2 Prioritization feature.

When writing this post, I decided to test, if the feature works as advertised. I utilized the test page by Patrick Meenan and followed his instructions. Unfortunately, I was unable to see any improvements, both in terms of assets prioritization and progressive image streaming. The results look the same, no matter if the Enhanced HTTP/2 Prioritization is enabled or disabled.

Remember to invalidate the cache after implementing the changes to see the results earlier.

To be continued…

There are still a lot of things to cover. In the mutable subresources category of errors, I’ve identified one more. It was particularly tricky and hard to reproduce. I will describe how I found and fixed it in the next post.

Context (Oct 2025): After deep work with SXG, I suspected changes might be coming—just not this quickly. Cloudflare announced SXG deprecation, and I observed SXG stop working around Sept 19, 2025.

If you still want SXG, the current path is to self-host—but the tooling looks abandoned, and setup is non-trivial. Also, Google may follow Cloudflare by shutting down the SXG cache and removing SXG support in Chrome.

In the previous two parts, you learned how to enable Signed Exchanges (SXG) and let Google prefetch your site’s HTML and assets (or SXG subresources) on the search results page.

The effect: when the user follows the link to your website from Google, the page should load instantaneously from the browser cache. No need to wait for:

• the page to become visible because HTML and CSS are already there,

• the images—they are already downloaded,

• the page to become interactive, because javascript starts executing without any delay.

Unfortunately, there are many scenarios in which it doesn’t work like that. If you met all the requirements I described in the first part of the series, HTML prefetching should work without issues. But prefetching subresources is tricky. It may cause trouble even if you strictly follow all the official recommendations.

Prefetching subresources is critical from the performance standpoint—much more important than HTML. Subresources are typically larger, so they take more time to download.

By reading this post you will learn how to diagnose SXG errors.

Disclaimer: This is not a CORS tutorial!

Although this post mentions CORS errors a lot, this is not a typical tutorial on configuring your website’s CORS policy. If you don’t know what is SXG and/or don’t use it, then please stop reading because you will waste your time. You will find a lot of valueable resources on the web. This is not the one you are looking for.

One bad apple spoils the whole barrel

When the browser prefetches the SXG page, it does it in two phases:

1. The browser downloads an SXG-wrapped HTTP response from the Google SXG cache. This response contains the HTML of your page (the main resource).

2. The HTTP response used to deliver the above contains a Link header. It includes a list of URLs of SXG-wrapped HTTP responses containing subresources. The browser downloads all of these URLs from the Google SXG cache.

When the user decides to visit the page, the browser may use the downloaded subresources in the page rendering process. The important point is that it will use them only if all of them were successfully prefetched.

Let me reiterate this. Even if one of your subresources fails to prefetch for any reason, the browser will discard all of the already prefetched subresources and download them again during page rendering, slowing it down considerably. Most of your assets will be downloaded twice. What a waste of bandwidth!

This all-or-nothing principle is unforgiving in case of subresource loading errors. A little icon you tried to prefetch fails to load? Forget about 99% of the SXG speed benefits!1 That's why you should strive to eliminate all errors.

Why does it work that way?

SXG documents state it’s for privacy reasons:

“This is intended to prevent the referrer page from encoding a tracking ID into the set of subresources it prefetches.”

Let’s say the site uses 1282 subresources. Those subresources could be tiny, to not force the user to download too much data. Google could prefetch them selectively. For example the first subresource is prefetched and the rest is not for user A, the second and third are prefetched for user B, etc. It gives a total of 2¹²⁸ combinations, allowing Google to encode 128 bits/16 bytes of data or about 150 ASCII characters.

Later, when visiting a website, it could use client-side or server-side logic to detect which subresources were prefetched and which were not. Using that knowledge, it could reconstruct (decode) the data passed from Google.

In effect, Google could pass any data to the website. It may be unique user identifier, search query, etc. This data could be used by the site for tracking.

The above method of passing data to the website it not the only one possible. If Google likes to, it could use URL anchor, Referrer header, or other more sophisticated ways.

Currently, Google chooses to protect user’s privacy, but we don’t know what the future may bring. What if SXG is used in other search engines that make different privacy choices? Or in entirely different context, where the collusion between reffering and target website is more probable? SXG’s authors wanted to prevent its misuse and didn’t want to create another way to track users.

SXG debugging

Classification of errors

There are a few places, where the issues with SXG may occur:

• when the SXG is generated by Cloudflare,

• when SXG is processed and put into Google SXG cache,

• when the browser tries to prefetch SXG.

Another way of differentiating the errors is by determining what caused them. It might be:

• the main document,

• a subresource.

SXG Validator

You can use the SXG Validator browser extension described in the first part of the series. It doesn’t support subresources but will tell you if the main document was correctly generated by Cloudflare, processed, and put into the Google SXG cache. It will report cache ingestion errors as they occur.

However, if you follow the recommendations, you'll reach a point where the main document is fine and the SXG Validator becomes unnecessary.

Swiss-army knife for SXG debug

When debugging subresources, my favorite tool is the SXG prefetch page in Chrome Developer Tools, with the Network tab open.

It allows you to diagnose all the errors except those related to SXG generation by Cloudflare. Also, in my experience, it’s more reliable than SXG Validator which sometimes returns false negative results.

Using the SXG prefetch page

Upon visiting the page, you will be welcomed with a basic form.

Remember to open Chrome Dev Tools (by pressing the F12 key), go to the Network tab, and clear it by clicking the 🛇 icon or using the CTRL+L keyboard shortcut. Now enter the URL of the page you want to test and click the Submit button. The page will reload and prefetch the URL you provided.

Let’s submit the following URL:

https://www.planujemywesele.pl/muzyka-na-male-wesele/poznan

Assuming the page is not yet present in the Google SXG cache, this will result in 3 HTTP requests:

The first two are related to the SXG prefetch page and, therefore should be ignored3. The third request is the actual prefetching request of your website. Let’s click on it to see the details:

The important parts are:

1. The request URL points to a special URL within the webpkgcache.com domain. It is the domain of Google SXG cache. The cache URL is constructed by transforming the original URL4.

2. The presence of a Location header is one of Google SXG's methods for signaling that an entry is not yet cached. The value of this header contains the URL of the original page (non-SXG version). It’s like Google SXG cache is saying: “I don’t have the SXG version yet, but in case you need the page right now, please request the original version as a fallback.“

Fallback mechanism

Unfortunately, Chrome Dev Tools doesn’t allow you to see the response body if the URL is absent from the SXG cache. The Preview and Response tabs contain only the following error message (which may be related to this issue):

Failed to load response data: No data found for resource with given identifier

However, you can see it with curl. Just point it to the Google SXG cache URL and set the proper Accept header:

curl -H "Accept: application/signed-exchange;v=b3" https://www-planujemywesele-pl.webpkgcache.com/doc/-/s/www.planujemywesele.pl/muzyka-na-male-wesele/poznan

Assuming the page is not yet present in the Google SXG cache, you will get:

<HTML><HEAD> <meta http-equiv="content-type" content="text/html;charset=utf-8"> <TITLE>Redirecting</TITLE> <META HTTP-EQUIV="refresh" content="0; url=https://www.planujemywesele.pl/muzyka-na-male-wesele/poznan"> </HEAD> <BODY onLoad="location.replace('https://www.planujemywesele.pl/muzyka-na-male-wesele/poznan'+document.location.hash)">

This simple HTML page aims to redirect the user to the original website. This is a fallback mechanism Google uses to ensure users can access the site, even if it's not yet in the SXG cache.

The fallback mechanism has interesting property. It uses client-side redirection that alters referrer.

Typically when someone comes from Google, the referrer should be set to www.google.com or one of Google’s regional domains. But in case the SXG fallback mechanism, you will see visitors coming from your-domain-com.webpkgcache.com.

You should keep that in mind when using your web analytics.

On top of that, measuring SXG traffic in Google Analytics comes with its own challenges. The official documentation suggests using window.isSXG to identify SXG visits, but this method is incomplete. In the 7th part of this series, I explain how to measure SXG traffic correctly.

Ingestion errors

If you looked closely, you might have seen a Warning header.

Google SXG cache uses this header to communicate errors that happen during ingestion. If you wonder how the SXG Validator browser extension knows the cache error message, it gets it from the value of this header.

In our example, we got the following error message:

199 - "debug: content has ingestion error: SXG validation failure: Certificate is not valid; bad OCSP status: 2; details: 6; cert trust"

This is a classic example of transient error, at least if you use Cloudflare. It will vanish when you repeat the request a few seconds later.

You may observe other errors. Most of them could be eliminated by meeting all the requirements described in the previous parts.

Note that fixing the error won’t be reflected immediately in the Warning header, as caching is involved. Try with different URL to get feedback earlier.

If you don’t see the Warning header, it means Google needs more time to fetch the page and store it in the SXG cache.

The actual SXG response

By submitting the URL on the SXG prefetch page, we requested the SXG version of the document from the Google SXG cache. It automatically activated the cache population mechanism, ultimately retrieving the page and storing it in the cache.

Now, let’s clear the requests in Chrome Dev Tools and submit the same URL as previously. This time, the result looks different:

You may notice there are a lot of HTTP responses. We will get to them, but let’s move from the top as previously. Ignore the first 2 responses like before and focus on the third one.

In the Type column, you can see “signed-exchange / Redirect” instead of the “text/html” you saw earlier. When you click on the third row, you will see details:

This time you won’t see the Location header and the Content-Type header is set to application/signed-exchange;v=b3 (previously it was set to text/html; charset=UTF-8). Those things tell you the SXG of the main document has been correctly generated and stored in the Google SXG cache.

If your document uses subresources, the Link header will include URLs of copies stored in the Google SXG cache.

If the list doesn’t include one or more of your subresources, or it there is no Link header at all it means Cloudflare couldn’t include some or all of your subresources. Most probably it is caused by hosting subresources on a different (sub)domain. The other cause is if you have more than 20 subresources—you will miss anything beyond that limit. See the previous part for details.

Notice the structure of URLs in the Link header:

https://domain-com.webpkgcache.com/sub/XXX/s/domain.com/path/file.jpg

Same as with HTML pages, they point to the webpkgcache.com domain. However, the path is constructed differently. The most important difference is the presence of the first few characters of the integrity hash of the subresource (marked with XXX above). This way, the location of a subresource is closely linked to its content.

When you open the Preview tab you will see the decoded SXG response:

The most useful fields are:

• Signature Date. It tells you when the SXG was generated. Remember that the time you see there has to be adjusted by adding 1 hour5. From the debugging perspective, it will tell you if the SXG is fresh or not.

• Response headers. This section lists all the HTTP headers (along with values) included in the SXG when they were generated by Cloudflare Automated Signed Exchanges (ASX). Headers are critical for prefetching subresources; you will understand why after reading the next part of the series.

SXG subresource response

The remaining requests of type “signed-exchange / Redirect” are related to SXG subresources. When you examine them, they will look similar. The most important differences in the Preview tab are:

• Content type (in Response headers field). Your main document is likely HTML, while subresources are styles, images, etc. This header will reflect that.

• Signature Date and Signature Expires. Each subresource has independent signature dates and typically should have higher expiration times than the main document.

Apart from that, you won’t find a Link header in the HTTP response. That’s because subresource nesting is not allowed: subresources can’t have sub-subresources.

Every HTTP response for an existing entry you get from the SXG cache has a Content-Type header set to “application/signed-exchange;v=b3”. The actual content type of the subresource is encoded in the body of the response. It can be examined in the Preview tab.

I like to think of SXG as a package containing the complete HTTP response. This package must be delivered to the browser through a separate HTTP response.

CORS errors

Now, that you are equipped with the debug tool and understand how to use it, let’s examine the most common error you will encounter when you inspect network requests in Chrome Dev Tools:

If you switch to console view, you may see:

⨂ Access to link prefetch resource at 'https://www-yourdomain-com.webpkgcache.com/sub/d3_L0zvunVqT/s/www.yourdomain.com/file.jpeg' from origin 'https://www.google.com' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.  ⨂ GET https://www-yourdomain-com.webpkgcache.com/sub/d3_L0zvunVqT/s/www.yourdomain.com/file.jpeg net::ERR_FAILED 200 (OK)

Why do those errors occur?

When I first encountered this error I was confused.

CORS (Cross-Origin Resource Sharing) errors typically occur when performing cross-origin requests with invalid or missing CORS policy. However, the subresources are stored in the Google SXG cache and only Google has control over its CORS policy. Is it possible Google set an invalid CORS policy on its own website?

Let’s examine the problematic request:

There are no CORS headers in the response. In contrast, when examining valid response, the Access-Control-Allow-Origin header is clearly there:

So this is the reason for the error. But…

Why is the CORS policy different for this particular file?

When you compare the error with a valid response again, you will notice other differences. The most critical HTTP headers that differ in the invalid response include:

• Content-Type. It’s set to “text/html; charset=UTF-8”. But it is supposed to be an SXG-wrapped image…

• Location. It is set to the actual image location on the origin server.

Also, when navigating to the Preview tab this error message shows up:

Failed to load response data: No data found for resource with given identifier

Seems familiar… Wait. It’s the fallback mechanism Google SXG cache uses when asked for non-existent SXGs of HTML documents! It appears Google reuses it for subresources too. As those responses don’t contain CORS headers, CORS errors occur.

So, if you ask Google SXG cache for a non-existing image (or any other file type), instead of this image (or file) you'll receive an HTML response. The HTTP response code is 200, but don't let that fool you. In reality:

Try it yourself

If you want to experience the errors, try a demo page that contains two subresources—one that successfully prefetches and another that fails to prefetch.

Go to the SXG prefetching page and paste the following URL. Remember to change the NUMBER to a random number between 1 and 10000:

https://www.planujemywesele.pl/sxg-tests/good-bad-subresource/NUMBER

After providing the URL, start repeatedly hitting the Submit button.

Watch the process unfold:

• First, you'll need to wait for the Google SXG cache to cache the page.

• Then wait a few more seconds while the Google SXG cache downloads the subresources. CORS errors will appear for both subresources.

• After this process completes, you'll observe that one of the subresources successfully prefetches (good.css) while the other still fails with a CORS error (bad.css).

Google search behavior

The same CORS error occurs when the Google SXG cache lacks a copy of the main document for a specific page. Such errors may occur in actual Google search results, as demonstrated below:

The SXG prefetch page used for debugging employs different mechanics to invoke prefetching, so the CORS error message won't be displayed there. However, when using this tool, you can still see the warnings described in the Ingestion errors section. Therefore, the SXG prefetch page should suffice in debugging and fixing errors in the main SXG document.

Other errors

Sometimes you may observe an error similar to the following:

In the Status column you can see:

(failed) net::ERR_INVALID_SIGNED_EXCHANGE

When you inspect the request details and open the Preview tab, you may see something like this:

The error message says:

Content type of cert-url must be application/cert-chain+cbor. Actual content type: text/html Failed to fetch the certificate.

Also, you may notice the Signature→Certificate URL field is marked in red.

In this case, the signed exchange is invalid because the certificate is inaccessible. The browser tried to download it but received an HTML page instead. You probably already know what this means. Yes, this HTML page is the fallback page served by Google SXG cache when the file is missing. It seems Google uses it for missing certificates too6.

Conclusion

Now you can diagnose SXG errors and know that most of them are caused by missing entries in the Google SXG cache.

• But why are subresources sometimes not present in the SXG cache?

• How does Google decide which subresources to cache and which not?

• Most importantly: how do you convince Google to perform the caching of all subresources used by the page?

I will explain it in the next parts of the series:

Thank you for reading!

Context (Oct 2025): After deep work with SXG, I suspected changes might be coming—just not this quickly. Cloudflare announced SXG deprecation, and I observed SXG stop working around Sept 19, 2025.

If you still want SXG, the current path is to self-host—but the tooling looks abandoned, and setup is non-trivial. Also, Google may follow Cloudflare by shutting down the SXG cache and removing SXG support in Chrome.

I will explain how to drastically improve your website's loading time for Google-referred users using a little-known technology called Signed Exchanges (SXG).

This is the second post in a series. I assume you read the previous part, which contains the fundamental knowledge required to understand and implement the techniques described here. If you don’t or need a refresher, I strongly suggest you click here.

Things to prefetch

You learned how to use SXG to prefetch HTML on Google search results. It was an essential step, but further work is required to fully utilize SXG and make your website load in a fraction of a second. If—apart from HTML—you prefetch also:

• stylesheets, the user can immediately start reading the text and see the final page layout without images,

• images, preferably the above-the-fold ones, the actual improvements to LCP will materialize,

• a custom font (if you use it), the user will experience stable-looking text, potentially improving CLS; on pages with a lot of text above the fold, LCP should improve, too,

• javascript, you will make your page interactive1 the moment the user clicks the Google result link.

Fully prefetched website experience

If you implement prefetching of all of the above, your website will load instantaneously and become fully interactive, despite the connection quality, given it had enough time to prefetch on the Google results page.

Below, I demonstrate how it works while offline. You just can’t make your connection slower than that!

This is a vertical video, so if you are using mobile device, click here to open it in the YouTube app for best experience.

I began by ensuring the browser cache was empty, simulating a first-time visitor experience.

Next, I entered a search phrase and received results from Google. I enabled airplane mode to simulate the worst-case scenario for the connection.

After clicking the link to the website, the browser immediately displayed a fully rendered page. I quickly tested the interactive UI elements, confirming that the JavaScript had loaded correctly.

When I scrolled below the fold, it became clear that the images there were not prefetched. In real-world conditions, network performance should be better than offline, and images below the fold would load in the background as the user interacts with the page.

SXG subresources

In the SXG nomenclature, the assets you choose to prefetch along with the main HTML document are called subresources. Most of the time, these are above-the-fold images, styles, fonts, and critical scripts.

This post focuses on them in contrast to the lower-priority assets that can be fetched later, such as below-the-fold images, non-critical Javascript, etc.

Basic requirements and limits

Cacheability

The subresources have to be cachable, just like the main HTML document. You must set max-age/s-maxage directives in the Cache-Control header to a value greater than 120 seconds. Given that subresources are mainly static, immutable files, 1 year is a much better option. It will be truncated to 7 days for SXG, but other caches will use the longer period.

For assets included with the application, I used the following nginx configuration snippet in locations containing static files:

location ~* \.(?:css|js|gif|png|jpeg|jpg|ico|ttf|woff|woff2|svg)$ {
  expires 1y;
  add_header Cache-Control "public";
  passenger_enabled on; # Passenger is used as an app server
}

No more than 20

As far as I know, SXG spec doesn’t restrict the number of subresources. However, Google SXG cache limits them and Cloudflare ASX honors this limit.

If your website depends on more than 20 subresources, the SXG experience won’t be as good as it could be because those over the limit won’t be prefetched. For example, not all images will be immediately visible or the user won’t be able to fully interact with the website until the missing scripts are downloaded.

It’s therefore a good idea to keep the number of subresources below or equal to 20.

Scripts and CSS files

You may remember my website combines Ruby on Rails and Next.js. I will use those frameworks to illustrate how to deal with SXG challenges. Even if you use a different framework, the challenges may be similar.

Rails gives you full control over the HTML and therefore over all the <link> tags for preloading subresources, so keeping the number of script and CSS files low is easy.

In the case of Next.js apps, the chunking of the javascript and CSS is delegated to the framework (and handled in the background by webpack). If the framework or webpack decides to generate too many chunks, you are out of luck.

In my case, I found the number of generated CSS chunks reasonable. However, the number of javascript chunks was way too high.

By default, Next.js splits JavaScript into chunks to optimize loading. If page A shares some code with page B, Next.js will create three chunks:

1. Code used only on page A.

2. Code used only on page B.

3. Code shared between both pages.

When loading the pages, page A will load chunks #1 and #3, while page B will load chunks #2 and #3. This way pages avoid loading code they won’t use.

To solve the too-many-chunks problem I disabled this optimization on pages receiving the most of the Google traffic. This means users may download the same code again when accessing other pages on the website. I believe the performance impact is minimal since when the user decides to visit another page, most of the non-javascript parts of the page are already present in the browser cache.

To prevent Next.js from chunking javascript on specific pages you can extend your next.config.js using the following template:

function skipChunking(config, pages) {
  const originalChunks = config.optimization.splitChunks.chunks;
  config.optimization.splitChunks.chunks = (chunk) => {
    // Dissallow chunking of specified pages because too many
    // chunks break SXG experience. For the full context see:
    // https://www.pawelpokrywka.com/p/subresources-prefetching-with-signed-exchanges
    if (pages.includes(chunk.name)) return false;

    return originalChunks(chunk);
  }
}

const nextConfig = {

  // Your current config options

  webpack: (config, options) => {
    // Replace with your own pages array.
    if (!options.isServer) skipChunking(config, ['pages/page1', ...]);

    // Your current webpack customizations go here

    return config;
  }
}
 
module.exports = nextConfig

Icons

Remember CSS sprites, popular when the majority of the web’s HTTP traffic used version 1.1? The idea was to put many images into one and use CSS to extract them on the client side.

HTTP/2 exorcised us from CSS sprites. That’s because HTTP/2 multiplexing allows us to perform many requests for small assets efficiently.

But if you have multiple icons on the page and want all of them to be prefetched with SXG, you have to:

• embed them into HTML (increasing the size of your page), or

• invite sprites back to your life.

You can also use an icon font, but it’s just a different implementation of the same idea.

You will find many online resources on how to use sprites, so I won’t rehash this subject here.

Cookies, HSTS & the rest

There are other requirements, but I think most of them should be already met. You don’t typically configure your web server so that static files set cookies, do you? And I assume you properly set the HSTS header, as I explained in the previous part.

The remaining requirements:

• adjusting the main document to include subresources properly and

• respecting the same-origin rule

are not that trivial and will be explained below.

Early hints

Before I explain how to adjust the HTML to prefetch subresources, let's take a step back and look at a different, more generic prefetching technique. It's called Early Hints, and it allows the browser to start fetching the assets required for the page before the application generates the HTML, enabling the page to load much faster.

Let's say your server takes a moment to generate HTML because it needs to fetch data from a slow database and process it using an overloaded CPU. With Early Hints, this time can be used by the browser to download CSS, fonts, and images. When the HTML is finally delivered, the browser has everything it needs to render the page instantly.

Technically, the server sends two responses: one with the URLs of the assets and another with the actual HTML content. This challenges the common expectation that a single HTTP request should result in only one response. It likely posed a challenge for developers working on browsers and web servers, as they had to adapt to handling multiple responses for a single request.

The server supporting Early Hints will check if your HTTP response contains a Link header:

Link: <https://www.example.com/style.css>;rel=preload;as=style

If found, the server generates a separate HTTP response containing this header, followed by the actual HTTP response generated by the app when it's ready.

Early Hints is a generic mechanism, not tied to SXG. Therefore, it applies to all page loads, not just SXG prefetches. It's beneficial to have it enabled. In Cloudflare, you can do it in the Speed / Optimization / Content Optimization:

Standard prefetching

Even if you don’t use Early Hints, the Link header can be useful for standard prefetching.

If you observe that 95% of users landing on page A go to page B next, you can prefetch page B on page A. This way, those 95% of users will experience instant loading of page B (the remaining 5% will prefetch the page and won’t use it, screw them).

Having a Link header in the prefetched page will instruct the browser to prefetch assets mentioned there too, improving the user experience.

SXG-compatible HTTP header

To prefetch a SXG subresource, the browser needs it to be included in a Link header of the HTTP response with your HTML document. The header must point to the subresource and include the file's integrity hash (to ensure the subresource is not altered by malicious SXG cache). Here is an example entry for a stylesheet:

Link: <https://www.example.com/style.css>;rel=preload;as=style,<https://www.example.com/style.css>;rel=allowed-alt-sxg;header-integrity="sha256-H/W6sQAAk1YIBi/NE86aUkNQjVHcYjo6B7Rg3PQ0vDM="

If you wonder why the subresource URL has to be duplicated, it’s because of the compatibility with responsive images mentioned later in this post.

To calculate the integrity hash, you must transform subresource content and its final HTTP headers2. Good luck with doing this in your application, especially given some resources are stored far away on the CDN!

Automating the generation of the Link header

HTTP Header

Fortunately, both Cloudflare Automated Signed Exchanges (ASX) and the SXG module for nginx offer a much easier solution. The only thing your app has to do is output the standard Link header in the HTTP response (the same one used to implement Early Hints):

Link: <https://www.example.com/style.css>;rel=preload;as=style

As you can see, you can skip the integrity hash. ASX/nginx will download the required assets, calculate integrity hashes, generate the SXG-compatible Link header, and use it to replace the original one in the HTTP response.

HTML-only

This doesn’t work for the nginx SXG module, but when using Cloudflare, instead of setting the Link header as described above, you can put the <link> tag inside your HTML. Of course, this tag doesn’t need to include the integrity hash:

<link rel="preload" href="/style.css" as="style">

Cloudflare ASX will parse the HTML, find those tags, download the assets, calculate integrity hashes, and finally generate the Link header and include it in the HTTP response to the SXG request.

This solution seems more elegant because you keep all asset references in the HTML. However, the drawback of not setting the Link header is you don’t get Early Hints and standard prefetching with subresources, because ASX doesn’t do it for non-SXG requests.

Prefetching of SXG subresources in Rails

Ruby on Rails automatically creates the Link header when you use helpers such as stylesheet_link_tag, javascript_include_tag, and preload_link_tag. I believe the original motivation was to activate Early Hints when available, but it helps3 with SXG too. Therefore, for most Rails apps, SXG prefetching will work automatically for assets loaded with these helpers.

You probably already use stylesheet_link_tag and javascript_include_tag helpers for CSS and scripts. Ensure you also add preload_link_tag for above-the-fold images and for fonts.

Prefetching of SXG subresources in Next.js

First, let’s clarify the goal. We want to:

• prefetch SXG subresources and

• benefit from Early Hints and standard prefetching with subresources.

Next.js doesn’t have a mechanism (yet) to set the Link header automatically for styles and javascript chunks. Unfortunately, the framework doesn’t give the developer access to the URLs of those assets either, so the developer can’t preload them manually. On the bright side, the framework preloads the styles using the <link> tag.

It means various subresource classes have varying support:

• Fonts and images: SXG prefetching works using <link> tags in HTML, Early Hints/standard prefetching requires setting the Link header manually,

• Stylesheets: SXG prefetching works out of the box, the Early Hints/standard prefetching mechanism is unsupported,

• Javascript: neither SXG prefetching nor Early Hints/standard prefetching are supported.

In summary, most of the things don’t work.

How to fix it?

Adding support for the required features in Next.js and contributing the changes upstream is likely the best long-term solution. However, my team wasn’t deeply familiar with the framework’s internals, so this approach could take some time. I needed a quicker solution to address the issue.

The next idea was to read the HTTP response of the app, parse the HTML, and add the Link header using middleware. I considered:

• Next.js middleware: at the current form it’s very basic and not capable of performing the task needed.

• One of the nginx modules for transforming HTTP responses using high-level languages such as Lua or Javascript. It adds complexity to the server configuration and comes with the maintenance cost of supporting a custom nginx module.

Instead, I’ve decided to use a Cloudflare worker.

What is a Cloudflare worker?

A Cloudflare worker is a piece of software that runs on every request at the Cloudflare data center closest to the user. You deploy it at a specific URL prefix, and from that point on, it handles all matching requests.

The worker can generate responses on its own or make requests to your application, acting as middleware. You can find the documentation here.

For me, it was important, the solution:

• uses Javascript, so it’s easy to write and extend,

• has great performance and low latency,

• doesn’t come with maintenance costs, because it’s hosted on Cloudflare,

• has a negligible financial cost.

Worker requirements

The primary requirement was to enhance Next.js HTTP responses to support SXG prefetching and Early Hints/standard prefetching, without modifying responses generated by Rails.

To differentiate between responses that required adjustments and those that didn’t, I chose to check for the presence of the Link header. The worker was configured to modify only responses where this header was absent.

SXG subresources are used in documents, so the worker only had to process HTTP responses with HTML content. No need to alter images, CSS, etc.

The worker had two responsibilities:

1. Parse the HTML to find:

   1. All <link> tags with the rel attribute set to preload, enabling Early Hints/standard prefetching for stylesheets (generated by the framework) and developer-specified subresources, such as images and fonts.

   2. All <script> tags with the src attribute to enable Early Hints/standard prefetching and SXG prefetching for scripts, while ignoring inline scripts.

2. Use this information to add the appropriate Link header to the response.

Link-adder worker implementation

Let’s create the worker:

npm create cloudflare@latest -- link-adder

The wizard will ask a few questions:

├ In which directory do you want to create your application?
│ dir ./link-adder
│
├ What would you like to start with?
│ category Hello World example
│
├ Which template would you like to use?
│ type Hello World Worker
│
├ Which language do you want to use?
│ lang JavaScript
│

...

│
├ Do you want to use git for version control?
│ yes git
│

...

│
├ Do you want to deploy your application?
│ no deploy via `npm run deploy`

Now, let’s paste the following code into the src/index.js file:

export default {
  async fetch(request) {
    const response = await fetch(request);

    // Skip non-html responses and responses containing Link header
    const headers = response.headers;
    if (!(headers.get('content-type') || '').includes('text/html')) {
      return response;
    }
    if (headers.has('link')) return response;

    // Parse the HTML and find all the URLs to preload.
    // A detailed explanation can be found at:
    // https://www.pawelpokrywka.com/p/subresources-prefetching-with-signed-exchanges
    const subresources = [];
    const rewriter = new HTMLRewriter()
      .on('link[rel*="preload"]', {
        element: link => (
          subresources.push({
            href: link.getAttribute('href'),
            as: link.getAttribute('as')
          })
        )
      })
      // No need to preload scripts:
      // - external to the site,
      // - intended for legacy browsers not supporting ES Modules.
      .on('script[src^="/"]:not([nomodule])', {
        element: script => (
          subresources.push({
            href: script.getAttribute('src'),
            as: 'script'
          })
        )
      });

    // Make sure to wait until parsing is complete
    const responseText = await rewriter.transform(response).text();

    // Prepare Link header value.
    // Escaping is left as an exercise for the reader.
    const linkText = subresources
      .map(({ href, as }) => `<${href}>; rel=preload; as=${as}`)
      .join(',');

    // Prepare mutable headers object and add the Link header
    const newHeaders = new Headers(headers);
    if (linkText) newHeaders.set('link', linkText);

    // Prepare modified response
    const newResponse = new Response(responseText, {
      headers: newHeaders,
      status: response.status,
      statusText: response.statusText
    });

    return newResponse;
  }
};

And deploy the worker:

npm run deploy

If doing this for the first time, you will be asked to authorize at Cloudflare.

We have the worker ready, now it’s time to set up the routing. Go to your website configuration in the Cloudflare admin panel and enter the Worker Routes section.

Let’s add the route by clicking the Add route button:

In the example above, I provided a test URL and selected a worker from the drop-down menu. In a real deployment, once everything is working as expected, the route should be updated to cover the entire site or at least the sections handled by Next.js.

After clicking the Save button, the worker should be fully functional.

To check if the worker does what it should visit the URL specified in the route with Chrome DevTools opened on the Network tab. You can compare the original app response to the one processed by the worker. Here is how the processed HTTP response looks like:

You can see above, the URLs for preload links and scripts were included in the Link HTTP header.

When comparing the responses you can also compare timing and see that the worker overhead is negligible.

From now on, the pages generated by Next.js will support Early Hints/standard prefetching and SXG subresource prefetching.

This worker is not specific to Next.js, it could be used to fix responses of any framework with similar issues.

Same-origin requirement

Issue with CDNs

It’s common practice to use a Content Delivery Network (CDN) for asset hosting, with images being probably the most frequently hosted type of content. The idea is to leverage the CDN’s high-performance, global network of servers, which are optimized for efficiently hosting static files. This reduces the load on your own infrastructure, allowing your servers to focus on running the application more effectively.

In these scenarios, your website at https://www.your-domain.com/ might reference assets from one of the following URL types:

• CDN endpoint: https://s3.eu-west-1.amazonaws.com/your-bucket/path/to/file.jpg

• Custom subdomain: https://assets.your-domain.com/path/to/file.jpg

However, neither approach will work with SXG. If your page preloads an asset from a different origin than the one the page is loaded from—even if it's a subdomain—Cloudflare won’t include it as a subresource in the SXG version of your page.

One day, Cloudflare may start supporting the prefetching of subresources from subdomains. I’ve created a test to check its current support status, but as of the date of writing this post, that feature is not yet available.

Offloading your assets to CDN, a method meant to make your site load faster does the opposite in the context of SXG!

Proxy-based URL rewriting

No need to move your files back from the CDN to your server just yet!

What about using good old URL rewriting? This technique allows you to serve assets under a new, SXG-compatible URL.

If done on your server, it would function similarly to the previous solution, with the proxy gradually copying files into the local cache. But what if someone else could handle that for us? You guessed it—we'll use Cloudflare Workers again!

The Worker will leverage Cloudflare's cache, so if you're paying for egress traffic (like with Amazon Web Services), you'll also reduce your costs as an added bonus.

URL-rewriter worker implementation

Our goal is to send the user the contents of the https://cdn.com/your-bucket/image.jpg file to the user requesting the https://www.your-domain.com/cdn-proxy/image.jpg URL.

Create another worker project:

npm create cloudflare@latest -- url-rewriter

Now, paste the following code into the src/index.js file:

const MAP = {
  'https://www.your-domain.com/cdn-proxy/':
  'https://cdn.com/your-bucket/',
  // You may add more mappings such as:
  // 'external-sxg-prefetchable-url': 'cdn-url'
}

export default {
  async fetch(request) {
    const url = request.url;
    // Find a mapping and apply to the url.
    // A detailed explanation can be found at:
    // https://www.pawelpokrywka.com/p/subresources-prefetching-with-signed-exchanges
    for (let prefix in MAP) {
      if (url.indexOf(prefix) === 0) {
        const newUrl = request.url.replace(prefix, MAP[prefix]);
        request = new Request(newUrl);
        break;
      }
    }
    return fetch(request);
  },
};

Finally, deploy the worker:

npm run deploy

And add a route in the Cloudflare admin panel, so that the https://www.your-domain.com/cdn-proxy/* prefix is handled by the url-rewriter worker.

Try to access https://www.your-domain.com/cdn-proxy/image.jpg to see if it works.

Now, update your app to reference new URLs and you are ready to go! If the previously mentioned requirements are met, SXG prefetching for your assets should work correctly.

Interesting subresource types

Prefetching scripts or styles is simple. Some subresource classes, however, deserve further discussion.

Custom fonts

The general advice is to avoid custom fonts to improve performance, but it’s not always possible. There are at least three ways to implement custom fonts on your website:

• Self-hosting: Treat font files as other assets and reference them from your CSS. In addition to the WOFF2 format, support for legacy browsers requires using the WOFF1 simultaneously. In the past, it was necessary to provide multiple file formats to ensure compatibility with older browsers. Things improved!

• CSS embedding: Include reference to an external CSS file in your HTML which downloads the fonts in the format best suited for the user’s browser from the external font provider CDN. Examples: embedded Google Fonts, Web Fonts by Hoefler&Co.

• Web font loader: Include a javascript snippet in your HTML that does the same as above. Example: Adobe Fonts when using the dynamic subsetting feature.

Avoid anything other than self-hosted fonts, because loading fonts from an external CDN doesn’t work with SXG prefetching. Also, CSS embedding and JavaScript loaders are terrible from a performance standpoint:

• CSS method blocks font downloading until the CSS file is fetched from the font provider. The CSS file is hosted on an external domain, so it can’t be prefetched using SXG unless a proxying Cloudflare worker is used as a workaround. But it won’t work even then, because its content has to be dynamically generated depending on the User-Agent header. Caching it in Google SXG cache would interfere with this browser-sniffing mechanism.

• The web font loader is even worse because it adds JavaScript loading and execution, slowing things down even more.

To self-host a font you need a @font-face declaration:

@font-face {
  font-family: "My custom font";
  src: url("/fonts/custom.woff2") format("woff2"),
       url("/fonts/custom.woff") format("woff");

  /* other properties, such as font-weight, etc. */
}

To enable SXG prefetching, place the following in the HTML head:

<link rel="preload" href="/fonts/custom.woff2" as="font" type="font/woff2" crossorigin="anonymous" />

I added the type attribute to ensure that browsers without WOFF2 support won’t prefetch it.

The above snippet prefetches only the WOFF2 format for the reasons below:

• Prefetching the legacy WOFF1 format could improve performance for a small number of users with older browsers (likely without SXG support). However, this would come at the cost of reduced performance for the majority of users with modern browsers. These browsers support both WOFF1 and WOFF2 formats, and prefetching both would result in downloading two files when only one is needed.

• Also, adding another file to prefetch would consume one of 20 valuable SXG subresource slots.

Responsive images

The responsive images technique is about serving different images depending on the screen dimensions. A user owning a 32-inch, 8K-grade screen may appreciate a horizontal, high-resolution image. On the other hand, a low-end phone user on a metered connection would prefer a vertical, low-resolution image.

Here is an example of a responsive image:

<img src="wolf_400px.jpg" srcset="wolf_400px.jpg 400w, wolf_800px.jpg 800w, wolf_1600px.jpg 1600w" sizes="50vw">

It will load the wolf_400px.jpg image on a small screen, wolf_800px.jpg on a larger screen, and wolf_1600px.jpg on the largest. The image specified in the href attribute is used by legacy browsers that do not support responsive images. The sizes attribute tells how much screen space is available for an image—in this case, it’s 50% of the screen width.

Let’s forget about SXG for a minute. If you want to speed up the loading of an image, you can prefetch it by placing a <link> tag in the HTML head. This way the browser will download the file very early, before the rest of the HTML is processed.

But which version of the image should be preloaded? To enable preloading for responsive images, you must specify additional attributes on the <link> preload tags that mimic srcset and sizes attributes on the <img> tags: imagesrcset and imagesizes accordingly (I used examples from this guide):

<link rel="preload" as="image" href="wolf_400px.jpg" imagesrcset="wolf_400px.jpg 400w, wolf_800px.jpg 800w, wolf_1600px.jpg 1600w" imagesizes="50vw">

If your website uses responsive images, the good news is they can be optimally prefetched with SXG. The Google SXG cache stores all versions of the image and the user’s browser chooses which one to prefetch on the Google results page.

The first step to enable SXG prefetching of responsive images is to put the <link> tag in the HTML of your page as described above. This link will be translated into the following Link header in the SXG response by Cloudflare ASX:

Link: <https://example.com/wolf_400px.jpg>;rel=preload;as=image;imagesrcset="wolf_400px.jpg 400w, wolf_800px.jpg 800w, wolf_1600px.jpg 1600w";imagesizes="50vw",<https://example.com/wolf_400px.jpg>;rel=allowed-alt-sxg;header-integrity="sha256-mfpQfImL1YSp8DM3HW0y235K5of+vJAdM0pbIh9MAnI="

I found Cloudflare ASX has a very strict HTML parser. If you include a new line in the attribute (such as imagesrcset) value, it won’t include this attribute in the resulting SXG response.

The Chrome browser parser is less strict, so in your local tests, everything will be fine. Be sure to minify your HTML before deployment or avoid newlines in <link> tags attributes.

However, when you try to prefetch your page you will get the following message in the Warning HTTP header of the SXG response from the Google SXG cache:

199 - "debug: content has ingestion error: SXG ingestion failure: Invalid link preload subresources; validating headers"

That’s because Google SXG cache couldn’t find the integrity hash for all image versions. Cloudflare ASX generated it only for the wolf_400px.jpg (the version specified in the href attribute).

The solution is to prepare one <link> tag for each image version:

<link rel="preload" as="image" href="wolf_400px.jpg" imagesrcset="wolf_400px.jpg 400w, wolf_800px.jpg 800w, wolf_1600px.jpg 1600w" imagesizes="50vw">
<link rel="preload" as="image" href="wolf_800px.jpg" imagesrcset="wolf_400px.jpg 400w, wolf_800px.jpg 800w, wolf_1600px.jpg 1600w" imagesizes="50vw">
<link rel="preload" as="image" href="wolf_1600px.jpg" imagesrcset="wolf_400px.jpg 400w, wolf_800px.jpg 800w, wolf_1600px.jpg 1600w" imagesizes="50vw">

As you can see above, all of the tags look identical except the href attribute. As a result, Cloudflare ASX will generate integrity hashes for all versions:

Link: <https://example.com/wolf_400px.jpg>;rel=preload;as=image;imagesrcset="wolf_400px.jpg 400w, wolf_800px.jpg 800w, wolf_1600px.jpg 1600w";imagesizes="50vw",<https://example.com/wolf_400px.jpg>;rel=allowed-alt-sxg;header-integrity="sha256-mfpQfImL1YSp8DM3HW0y235K5of+vJAdM0pbIh9MAnI=",
<https://example.com/wolf_800px.jpg>;rel=preload;as=image;imagesrcset="wolf_400px.jpg 400w, wolf_800px.jpg 800w, wolf_1600px.jpg 1600w";imagesizes="50vw",<https://example.com/wolf_800px.jpg>;rel=allowed-alt-sxg;header-integrity="sha256-Xtd1ha7j8bRbUE/z7IULqPzPN4ZuUvapKBAEZ5XVvg8=",
<https://example.com/wolf_1600px.jpg>;rel=preload;as=image;imagesrcset="wolf_400px.jpg 400w, wolf_800px.jpg 800w, wolf_1600px.jpg 1600w";imagesizes="50vw",<https://example.com/wolf_1600px.jpg>;rel=allowed-alt-sxg;header-integrity="sha256-d0Pusf0qpsEUmIF+dTbvYa8pFmi4BmNt7/HzO0pHDiY="

As you can see, there is a lot of duplication here. It would be enough to have something like below, but I couldn’t find a way to achieve that with Cloudflare ASX other than preparing the Link header all by myself (including integrity hashes):

Link: <https://example.com/wolf_400px.jpg>;rel=preload;as=image;imagesrcset="wolf_400px.jpg 400w, wolf_800px.jpg 800w, wolf_1600px.jpg 1600w";imagesizes="50vw",<https://example.com/wolf_400px.jpg>;rel=allowed-alt-sxg;header-integrity="sha256-mfpQfImL1YSp8DM3HW0y235K5of+vJAdM0pbIh9MAnI=",
<https://example.com/wolf_800px.jpg>;rel=allowed-alt-sxg;header-integrity="sha256-Xtd1ha7j8bRbUE/z7IULqPzPN4ZuUvapKBAEZ5XVvg8=",
<https://example.com/wolf_1600px.jpg>;rel=allowed-alt-sxg;header-integrity="sha256-d0Pusf0qpsEUmIF+dTbvYa8pFmi4BmNt7/HzO0pHDiY="

Nonetheless, the Google SXG cache happily ingests the document with the verbose Link header value, and the warning about the SXG ingestion failure disappears. I prepared a demo page showcasing the correct and incorrect approaches to prefetching responsive images with SXG.

On the downside, older browsers will prefetch all the versions wasting the bandwidth. At the time of writing, this is 5% of desktop and 6% of mobile users, but it will decline. There may be ways to improve the experience for users of legacy browsers, but I haven’t researched them.

If—apart from SXG prefetching—you want to use responsive images with Early Hints I have bad news. At the time of writing responsive images are not supported with Early Hints.

However, this may change in the future. Additionally, remember that support for responsive images already exists in standard prefetching. Therefore, to fully benefit from responsive images in Next.js, you must adjust the link-adder worker (the first one mentioned in this post). Along with the href and as attributes, the worker must also support the imagesrcset and imagesizes attributes in cases they are set. The change should be fairly easy to implement, I’m sure you can handle it!

Images combined with a responsive layout

Responsive websites use different layouts depending on the screen dimensions. The common practice is to use a 1-column layout for mobile phones and 2+ columns on desktops.

If the columns contain graphics, the initial viewport on the phone will typically include 1 or 2 images, while on the desktop, the user will see more of them.

The efficient preloading/prefetching strategy avoids loading images that are not visible in the initial viewport. To save bandwidth, you want to load all the pictures the user sees when the page loads and no more than that.

The dilemma this time is not which version of the image, but which images to choose.

The first thing that comes to mind is using the media attribute on the <link> tag:

<link rel="preload" as="image" href="1.png">
<link rel="preload" as="image" href="2.png" media="(min-width: 800px)">

If the browser loads above HTML it will preload 2 images if the screen width is 800px or more. If it’s less, then only the first image will be preloaded.

However, things become different in the context of SXG prefetching. Cloudflare ASX correctly puts the media attribute in the Link header, but my tests show the browser ignores it. As a result, both images are loaded regardless of screen size.

I solved it by abusing adjusting the imagesrcset and imagesizes attributes on the <link> tags:

<link rel="preload" as="image" href="1.png">
<link rel="preload" as="image" href="2.png" imagesrcset="1.png 2w, 2.png 500w" imagesizes="(max-width: 799px) 1px">

The first <link> tag preloads 1.png unconditionally. However, the second <link> tag preloads 2.png only if the screen is 800px wide or more. Otherwise, it preloads 1.png again (the browser is smart enough to not perform a second request) and 2.png is not preloaded which meets the requirements.

It works because in the second <link> tag, the browser is being told:

• 2.png is 500 px wide,

• 1.png is 2px wide (not true),

• for screen widths below 800px, there is only 1px room for the image (also not true).

The <link> tag is responsible for preloading, not rendering, therefore it doesn’t matter if some things are lies. Our goal is to preload the files. They will be rendered later using a <img> tag with a correct srcset and sizes attributes.

If the screen width is below 800px, the browser recognizes the space is limited (1px), so it checks which image has the closest width. As the browser thinks 1.png is 2px wide, it preloads it.

This approach may be combined with multiple image versions from the previous section. It requires introducing another image that is always preloaded and has only 1 version. You can mix different image formats, so an SVG logo may be a good candidate - it’s used everywhere and is scalable, so one file is enough:

<link rel="preload" as="image" href=1st-tiny.png" imagesrcset="1st-tiny.png 500w, 1st-big.png 1000w" imagesizes="(max-width: 750px) 100vw, 50vw">
<link rel="preload" as="image" href="1st-big.png" imagesrcset="1st-tiny.png 500w, 1st-big.png 1000w" imagesizes="(max-width: 750px) 100vw, 50vw">

<link rel="preload" as="image" href="logo.svg">

<link rel="preload" as="image" href="2nd-tiny.png" imagesrcset="logo.svg 2w, 2nd-tiny.png 500w, 2nd-big.png 1000w" imagesizes="(max-width: 750px) 1px, 50vw">
<link rel="preload" as="image" href="2nd-big.png" imagesrcset="logo.svg 2w, 2nd-tiny.png 500w, 2nd-big.png 1000w" imagesizes="(max-width: 750px) 1px, 50vw">

This approach works with SXG-prefetching. You can see it in action on the demo page.

Be careful about exceeding your subresource limit

It’s worth noting that depending on the number of images you want to prefetch and the number of versions of each image, it may quickly fill up all available subresource slots:

images × versions + logo + styles + javascripts + fonts + icons <= 20

Even if your page uses only 1 CSS file, 1 javascript, 4 fonts (normal, bold, italic, bold italic), 1 icon, and 1 logo you are left with 12 slots, which allows you to preload:

• 12 images (all in 1 version),

• 6 images in 2 versions each,

• 4 images in 3 versions each,

• 3 images in 4 versions each,

• 2 images in 6 versions each (probably doesn’t make sense),

• 1 image in 12 versions (seems like an overkill).

It’s up to you to decide which subresources to prefetch for the best user experience.

Verifying prefetching of SXG subresources

Now, after you’ve applied all the hints you found here and in the previous part of the series, it’s time to check if your website is being correctly prefetched with subresources.

Instead of waiting until Google indexes your site, head over to the prefetch testing tool, type the URL of the page you want to test, and hit the Submit button to make the Google SXG cache aware of your page.

1. Give Google SXG cache time to fetch the page and all the subresources. In my experience, it should take no more than a minute, typically about 15 seconds.

2. Then, hit Submit again to prefetch everything.

3. Go offline in Chrome Dev Tools or disconnect your Internet connection.

4. Click the link to target link.

You should see your page instantly rendered with all the subresources you specified. If this is what you see—congrats, you did it! But don’t relax just yet—there’s more to come. For now, you can take a moment to admire this beautiful fresco:

However, if you see a “no internet” message or something similar to this instead:

Either you haven't waited long enough for the SXG cache, made an error somewhere, or… the SXG gods don't like you!

Troubleshooting

Don’t worry. Before you hit the Submit button in the prefetch testing tool, please open the Network tab in Chrome Dev Tools. You may see red CORS error messages after pressing this button.

Those errors like to appear randomly. Some pages may work correctly, some may not, while others only on Tuesdays. So even if your first test was successful, beware.

I was there, banging my head against the wall. I met every tiny requirement I could find in all the SXG documentation available online. Still, no luck!

Finally, I began researching this problem on my own. It took some time, but I believe I've identified the causes and the solutions for the most common issues. I’ll dive deeper into this topic in the next part. Spoiler alert: CORS is not to blame.

Thanks!

I hope this post will help you make your website load faster. Thank you for reading—you’re the best! :)

If you enjoyed the post, sharing it with your friends or co-workers is the best way to show your appreciation. I would really be grateful for that!

Share

Originally published December 2023, substantially updated January 2025.

Context (Oct 2025): After deep work with SXG, I suspected changes might be coming—just not this quickly. Cloudflare announced SXG deprecation, and I observed SXG stop working around Sept 19, 2025.

If you still want SXG, the current path is to self-host—but the tooling looks abandoned, and setup is non-trivial. Also, Google may follow Cloudflare by shutting down the SXG cache and removing SXG support in Chrome.

Do most of your users come from Google? You can maximize the performance of your website by using the techniques presented in this and further posts.

The speed of your website is considered good if the value of the Largest Contentful Paint (LCP) metric is below 2500 ms. I’m going to show you how I took it below 350 ms on a dynamic, production website with a lot of images, even on a slow connection.

If you wonder what is the point read this. Now, take a look at what it feels like (if you prefer, here is a YouTube version):

On the recording, you can see I started by clearing Chrome browsing data. This ensures the cache is empty, which resembles a first-time visitor.

I typed in a search phrase and got results from Google. I downgraded the connection speed and visited the website. Even on slow 3g, it took 140 ms to load the page (LCP). In my observations, the results may vary between 100-350 ms.

If you need a comparison, perform the above steps for any other website. As of 2025, most will give you 10+ seconds to load.

How to achieve LCP below 350ms?

How was it possible to load the entire page including CSS, custom font, and images in a fraction of a second on a slow connection? Have I used the Pied Piper compression algorithm?

If you haven’t guessed already, I will tell you that the page would behave exactly the same if instead of throttling the connection I cut it entirely off.

This leaves us with only one possibility. The website HTML and all critical resources such as stylesheets and images have to be prefetched by the browser on the Google results page. After clicking on the link, even when offline, the page can be displayed almost instantly from the browser cache.

Now, you see, the demo wasn't entirely honest. I arranged things so that the preloading hasn’t been throttled. However, I believe in normal circumstances (when the connection is not extremely slow) the time users spend on Google results is enough for the preload to complete, so the effect on LCP will remain the same as in demonstration video.

How do I make Google prefetch my website?

You need to:

1. Have a page that frequently appears in Google search results.1

2. Properly implement Signed Exchanges. This and further posts will focus mainly on that.

What are Signed Exchanges?

Signed Exchanges or shortly SXG is being developed by the Web Incubator Community Group which is a community group of the W3C that incubates new web platform features. In the context of IETF standardization, SXG is still a constantly updated draft.

Despite it being not 100% finished, it works in Chrome-based browsers, there is an open-source server implementation, and Cloudflare offers it for paying customers.

Basically, it works as standard prefetching, but with a twist: the website being prefetched can’t tell who does it and when. This is important for user privacy and critical on the Google results page because we don’t want website owners to track our searches.

How does SXG work in the context of Google search?

1. The Googlebot visits the website. It tells the website it understands the SXG format by setting the appropriate Accept HTTP header.

2. The website serves the SXG version of a page instead of a raw HTML. SXG encapsulates HTML and is signed with a website private key.

3. Google saves the SXG to its cache. The cache is located at the webpkgcache.com domain.

4. Later, the user visits the Google website and types the search phrase.

5. Google responds with the results page.

6. The results page instructs the user’s browser to request the SXG version of the website from Google cache…

7. …download it…

8. …and store into the browser’s prefetch cache.

9. The user finally decides to visit the page. There is no need to request the website via the network because the page is prefetched.

10. The browser extracts HTML out of SXG and begins to render the page.

Even if the website is physically loaded from Google, the URL bar in the browser shows the actual website URL. The browser can display it because the SXG is signed, so it can be trusted like the original website.

There is a downside. Your page has to be cached. If your page changes after being downloaded by Googlebot, the user will see the old version.

So here’s the catch, you may think. I agree, that there are use cases that disqualify SXG. But I believe many websites, not only static ones, will work perfectly fine. Mine is the perfect example. And there are ways to minimize the damage of stale content being served.

How to generate Signed Exchanges?

You can generate SXG on your own or use a service that does it for you such as Cloudflare2. I chose the second option because I’m already a Cloudflare customer.

Ok, probably the true reason is I’m lazy and Cloudflare makes turning on SXG a single click (assuming the website is already proxied through Cloudflare).

As you can see, SXG is a global configuration switch for the entire website. To make it more granular, you can create a Configuration Rule that disables SXG if the URL/subdomain doesn’t match the part of the website you want SXG to be enabled. This may be handy for testing SXG without impacting the entire site.

Is that it?

This post wouldn’t make sense if all I needed to do was enable SXG in the Cloudflare panel. There were certain decisions I needed to make, and requirements my website had to meet.

This and upcoming posts will show you my path of making the existing website to become fully SXG-enabled. I will include a lot of details and workarounds for various quirks I discovered during hours of debugging and black-box testing.

I was striving to write in a way that is understandable for a web developer interested in performance optimization. However, this text focuses on practical implementation. Therefore, most discussions and code examples use Ruby on Rails and Next.js, the frameworks used by my website. If you are not interested, you can skip those parts, they are clearly marked.

Without further ado, let’s get started!

Content generation vs content distribution

First, I had to give up the idea that I had 100% control over my website traffic. From now on, the pages generated by my server will be distributed to the users by the infrastructure I can control only partially if at all.

Google supports SXG cache purging, but it is asynchronous, from my observations slow and you can’t purge more than one, specific URL at once. Compare it to Cloudflare where you can purge cache for example by URL prefix3, or even purge the entire cache in one step. You can achieve similar results with the nginx cache4.

And nothing prevents someone from downloading the SXG version of my website and hosting it anywhere. When the browser visits it, it will still behave exactly, as it was on my website. And I won’t see the request coming to my server.

The HTTP request becomes decoupled from the HTTP response which has a life of its own. Just like a mobile app - after the user installs it, you lose control over it.

I will be able to see only a subset of requests from users, the rest will be handled by an opaque cache hosted by Google. My logs won’t reflect all the traffic my website gets. So no server-side page view counting and other request-based analytics.

The only thing I can control is the amount of time the SXG version of a given URL is valid.

No server-side personalization

Given the above, the HTML returned for a given URL should be the same for all visitors, no matter what’s included in the HTTP request5. I needed to serve the same page regardless of the login status of the user, the cookies6, device and browser, the country the user came from, and the language he/she had set.

This becomes a problem because the user expects the page to look different depending on the above variables.

The obvious solution is to customize the page client-side, in the browser. Javascript has to query API endpoints, maintain state using browser storage, and adjust the page. CSS has to be used for responsiveness, as serving separate versions of the page depending on the User-Agent header or redirecting mobile users to a subdomain is not an option.

Caching

The basic rule of making the app SXG-friendly is to make it cache-friendly because the SXG version of the page will eventually land in Google SXG cache.

To be cacheable, it is best if the HTTP response includes a Cache-Control header.

A note on Cloudflare cache

By default, Cloudflare caches static assets but doesn’t cache HTML responses, even if they set the Cache-Control header to allow that.

SXG will work perfectly fine without changing this configuration. Actually, it’s easier to implement SXG without caching HTML in the Cloudflare cache because you don’t need to worry about accidentally caching a private user’s state.

However, I decided to utilize Cloudflare cache for HTML to maximize the performance benefits. The rest of the steps in this and the following posts assume the HTML will be cached this way.

Cache configuration

The Cache-Control header, among other things, allows you to set the maximum amount of time the page is considered fresh and can be kept in the cache. Particularly max-age and s-maxage directives can be used for that. Here is an example:

Cache-Control: public, max-age=600, s-maxage=3600

It tells the browser to cache the page for 600 seconds, or 10 minutes (max-age directive).

It also tells so-called shared caches (for instance Cloudflare cache or Google cache) to cache the response for 3600 seconds, or 1 hour (s-maxage directive). In the absence of s-maxage, shared caches would use max-age.

Note the difference in spelling: max-age vs s-maxage. I experienced quite a bit of frustration while debugging why s-max-age doesn’t work!

Helpful tip: Both directives are written with exactly one hyphen.

You will find a detailed explanation of all Cache-Control directives here.

Google won’t cache SXG if max-age/s-maxage is less than 120 seconds. You can encounter the following warning in SXG Validator (debugging tool described later) or directly in the HTTP response generated by SXG cache:

199 - "debug: content has ingestion error: Error fetching resource: Content is not cache-able or cache-able lifetime is too short"

The solution is to increase the value of the max-age/s-max-age directive within the Cache-Control HTTP header to something higher than 120 seconds (in real life, much higher; otherwise, the cached version will hardly be used).

Apart from that, you can’t use private, no-store, and no-cache directives, otherwise, SXG won’t be generated by Cloudflare. Cloudflare SXG generator doesn’t allow you to set the maximum age to a value higher than 7 days7.

The default Cache-Control header set by Rails, Next.js, and other frameworks typically prevents HTML caching, which consequently disables SXG. This is actually beneficial, as it lets you selectively enable SXG/caching for specific sections of your application while maintaining safety for the rest of your codebase.

Choosing cache expiration times

I had to decide how long to cache HTML pages. It was tough. Naturally, we all want to get the freshest data possible. But we also want to get it immediately. Those two needs are opposite to each other.

The content on my website doesn’t change very often, and I’ve observed that Google typically refreshes its cache more frequently than the Cache-Control header value suggests.

After carefully considering all business requirements, I decided to set the cache duration to 24 hours. This value is also recommended in a post by Devin Mullins, one of the SXG implementers at Google, and it strikes a good balance between minimizing staleness and optimizing performance.

As a result, I configured the s-maxage to 1 day.

There are advanced ways to further minimize users’ exposure to stale data, especially when utilizing Cloudflare cache. However these methods are highly application-specific. Sharing them publicly would likely benefit only competing websites, so I’ve chosen to keep them private.

Caching in Rails

I added the following code in every Rails action I wanted to enable for SXG:

response.headers['Cache-Control'] =
  "s-maxage=#{24.hours.to_i}, public"

Caching in Next.js

To set the required header in Next.js I used the context parameter passed by the framework to the getServerSideProps():

context.res.setHeader(
  "Cache-Control", `s-maxage=${24 * 60 * 60}, public`
);

Keep in mind that Next.js will override the Cache-Control header value in development. The changes can be observed only in production.

HTTP headers to avoid

Cloudflare provides a full list of SXG requirements. The important part is a list of disallowed response HTTP headers, such as Set-Cookie. If your app responds with one of those, SXG won’t be generated for a given page. The intention is to prevent making potentially private information public.

I was unable to provide a real-world example of vulnerability prevented by this safeguard. That’s because of additional safety measures coming in.

This is the only thing that came to my mind. Let’s assume Set-Cookie is allowed when generating SXG. Imagine the following scenario:

1. User submits login information, the page reloads and sets the session cookie.

2. The page gets in Cloudflare cache along with Set-Cookie header. From now on, all users visiting this page get immediately logged as the user from previous step. That’s already a disaster.

3. Now Googlebot fetches the SXG version of this cached page (including cookies) and publishes it in its SXG cache. Everyone can download it and extract session cookies of the unfortunate user. It’s possible even after fixing website and purging Cloudflare cache.

Of course, the correct way of mitigating this vulnerability is to invalidate all session cookies after fixing the page. But the point is Cloudflare tries to limit the damage by stopping the private information from leaking further.

Fortunately, the above scenario won’t happen in real-life. That’s because of additional Cloudflare safeguard that excludes pages with Set-Cookie header from being cached. I suspect Google also won’t cache SXG with cookies, but I haven’t verified that.

If you have a real-world example of the vulnerability Cloudflare prevents with this measure, leave a comment below.

Leave a comment

As most websites use cookies, you may think it disqualifies SXG usage. I strongly believe that’s not the case.

You can use Google Analytics and other cookies set by the browser. Only cookies returned with server-side generated HTML pages are prohibited. And there are ways to handle those cases too.

Session cookie

When you generate a page server-side, typically a session cookie is set. This depends on the framework you use, so you need to check by yourself. For example, Next.js doesn’t use session cookies while they are used extensively in the Rails.

As mentioned above, SXG won’t be generated when your response contains cookies. Therefore I needed to adjust my Rails app.

In every SXG-enabled controller action, I needed to make sure the Rails session was not being used. That’s because loading it automatically sets a cookie in the response. Searching the code for the “session” string allowed me to pinpoint all the places I needed to update.

CSRF protection

In the Cross-Site Request Forgery (CSRF) the attacker tricks the user into performing certain actions in the vulnerable web application.

Modern frameworks using cookie-based authentication implement appropriate protections. In some cases, those mechanisms may prevent SXG from being generated. Adjustments may be necessary.

Warning

Interfering with security mechanisms should be done with great care. Developer doing so should be familiar with computer security, and understand CSRF attack and prevention methods.

Flawed implementation may lead to serious vulnerabilities.

On the other hand, come on, it's not rocket science! Understanding CSRF is fairly easy; you don't need a PhD.

Rails CSRF approach

If you use a different framework, you can safely skip this and the following section.

Rails uses unique, session-bound CSRF tokens which are embedded into HTML and later sent in every unsafe request (using methods other than GET or HEAD), such as form submission or AJAX. After being received by the server, tokens are checked against the user session, and requests containing invalid tokens are rejected.

This mechanism is very effective but doesn’t work well with caching. The cached page would include CSRF tokens. They were valid for the user visiting the page before it was cached. But for every subsequent user, those tokens will be invalid because their sessions don’t match those tokens. Users won’t be able to perform any operations, as all their requests will be rejected.

How to make Rails CSRF protection cacheable

I fixed it by transmitting the CSRF token as a cookie using a separate API endpoint that’s not cached. The front end performs an API request and uses a CSRF token from the cookie to patch the HTML of the current page.

Before proceeding, CSRF tokens must be removed from the HTML. This is necessary for two reasons:

1. It makes the HTML cleaner.

2. More importantly, CSRF tokens are tied to sessions. Their presence triggers session loading, which in turn sets cookies—exactly what we're trying to avoid.

All the form tags used on SXG-enabled pages have to include the authenticity_token option set to an empty string. This way forms will include empty tags, which could be updated later by the front end.

<%= form_with model: article, authenticity_token: '' do |form| %>
  ...
<% end %>

The other place containing the CSRF token is the csrf_meta_tags helper in the layout. It is used by Rails javascript to add an X-Csrf-Token HTTP header to AJAX requests. It should be replaced with static HTML to be updated later:

<meta name="csrf-param" content="authenticity_token" />
<meta name="csrf-token" content />

The API controller simply sets the cookie and returns a 200 status code:

# app/controllers/api/cookies_controller.rb
module Api
  class CookiesController < ActionController::API
    include ActionController::Cookies
    include ActionController::RequestForgeryProtection

    def index
      # Calling #form_authenticity_token loads the session
      # automatically. So apart from the csrf_token cookie,
      # the response will also include the session cookie.
      cookies[:csrf_token] = form_authenticity_token
      head :ok
    end
  end
end

To make it accessible, the following route should be added to config/routes.rb:

namespace :api do
  resources :cookies
end

Then some javascript needs to be executed in the front end:

fetch('/api/cookies').then(() => {
  const cookie = document.cookie.split('; ')
    .find(row => row.startsWith('csrf_token=')) || '=';
  const token = decodeURIComponent(cookie.split('=')[1]);
  document.querySelector(
    'meta[name="csrf-token"]'
  ).setAttribute('content', token);
  document.querySelectorAll(
    'input[name="authenticity_token"]'
  ).forEach(input => { input.value = token; });
})

The above implementation is simplified for clarity. The production-grade javascript could be optimized to perform the request only once in a while, not on every page load. Also, it should gracefully handle cases where the CSRF token may become obsolete, such as login and logout.

Currently logged user

The common pattern in server-side rendered web applications is to keep the ID of the currently logged user in the session. This way user information may be fetched from the database and included in the response. In the context of caching, this approach has 2 problems:

1. The resulting HTML is different for every user.

2. The session is used, therefore the response sets a cookie.

The solution is to use JavaScript to perform an API request to the endpoint that returns current user information and then update the page client-side.

First impression optimization

One of the decisions I had to make was to optimize for the first-time user. The first impression he/she gets greatly contributes to him/her staying on the page or leaving.

That’s why all the pages of my website use server-side rendering. This way user gets the page quicker compared to rendering it client-side.

When the user is logged in, the section in the screen's upper-right corner on the desktop shows his/her name and profile picture. Otherwise, the user section is replaced with a “Login / Register” link.

Initially, the above space remains empty and only after the user's state is known, it is filled. On a slow connection, it may take a while to perform the API request which slows down the appearance of the user section.

That’s why I assume a first-time user is not logged in and the front end doesn’t need to perform the above API request at all. This way “Login / Register“ could be displayed immediately.

After login, browser storage is used to mark the user as logged. This way, the browser can quickly check if performing an API request during the next page load is worth it.

You may think it’s just a detail. But there is also SEO aspect of this approach.

Most crawlers behave as first-time users for every request they make. That’s probably because they don’t keep state between requests, especially don’t store cookies.

In the presented case we saved 1 non-cache’able request per page, which quickly adds up given number of pages crawlers visits. Performant page saves bot’s crawling budget, which could be spent on crawling more pages.

Other cookies

I also had to make sure no other cookies were being set server-side. It was easy to search for all the places in the code setting a cookie. The hard part was to decide how to avoid using it. There were two options: make the front end responsible or remove given cookie usage entirely by solving the problem differently.

The final check is to use Chrome Developer Tools, preferably with cleared cookies or using incognito mode to simulate Googlebot and examine the app response. If the Set-Cookie header shows up, you need to go back to code.

HSTS

HTTP Strict Transport Security (HSTS) is a security policy ensuring browsers will always use a secure connection when connecting to your website.

Years ago, I enabled it in the nginx configuration. The web server was setting the Strict-Transport-Security HTTP header on all responses. By default, Rails sets this header in production too.

Unfortunately, the Strict-Transport-Security header prevents the SXG from being generated. I had to revert the changes I made to the Nginx configuration. In Rails case, the only change needed was to set config.force_ssl to false.

If you use a different stack, you may need to check the documentation of your web server and framework to find a way to disable HSTS.

Fortunately, HSTS can still be used by setting a flag in Cloudflare configuration.

It sets the HSTS header, but in a SXG-compatible way, so that:

• the standard response includes the header,

• the SXG response includes it too,

• the standard response encapsulated in SXG does not include it.

Writing tests

It’s always good to have tests, and in this particular case, it was especially important. That’s because it is very easy to make a code change that breaks SXG, and the effects will be invisible in development and hard to spot in production.

During future development, I expect things like adding new cookies and accessing the session directly or indirectly would be the most common causes of SXG breakage.

That’s why I implemented tests checking the responses for required/prohibited headers according to the Cloudflare specification. Here is a spec for the Rails app:

require 'rails_helper'

RSpec.describe ReplaceWithYourController, type: :request do
  before(:context) do
    # Add app-specific initialization logic before making a request
    get "/replace-with-your-sxg-enabled-page"
  end

  let :cache_control_headers do
    %w[
       Cache-Control Cdn-Cache-Control
       Cloudflare-Cdn-Cache-Control Surrogate-Control
      ]
  end

  let :forbidden_headers do
    %w[
        Authentication-Control Authentication-Info Clear-Site-Data
        Optional-WWW-Authenticate Proxy-Authenticate WWW-Authenticate
        Proxy-Authentication-Info Public-Key-Pins Sec-WebSocket-Accept
        Set-Cookie Set-Cookie2 SetProfile Strict-Transport-Security
      ]
  end

  let :forbidden_directives do
    %w[private no-store no-cache max-age=0]
  end

  it 'does not use forbidden HTTP headers' do
    expect(headers).not_to include(*forbidden_headers)
  end

  it 'does not use forbidden cache directives' do
    cache_control_headers.each do |header|
      next unless headers[header]
      expect(headers[header]).not_to include(*forbidden_directives)
    end
  end
end

In addition, I created tests for application-specific requirements, for example, if s-maxage is being set to the correct value.

Caching the HTML

At this point, the application was ready for Cloudflare to begin caching its HTML content.

The easiest way to accomplish that is to define the cache rule. To do that, go to Caching → Cache Rules and hit the Create rule button. Then provide a name for the rule, choose All incoming requests, and Eligible for cache. The form contains a lot of other settings, they don’t need to be changed. I removed them for readability in the screenshot below:

⚠️ WARNING: This cache rule enables HTML caching site-wide. To test safely, start by applying it to a limited set of URLs using a custom filter expression. This allows you to verify everything functions as expected before broader deployment.

When you feel comfortable with the rule, hit the Deploy button to activate it.

Testing on production (!)

After implementing the above changes and making sure tests passed, I was ready for the production deployment to validate if Cloudflare generates SXG correctly.

Well, actually it wasn’t that simple.

This topic was new to me, there were many unknowns and a lot of learning and experimentation in the process. Also, at the same time I was implementing other performance optimization features. It wasn’t a linear process, like described here. I needed to deploy often and test - on production.

I decided to do this because the risk of something going wrong was minimal. The worst that could happen was Cloudflare failing to generate SXG. And this was my starting point - not a big deal.

When testing SXG I used the following tools and techniques:

SXG Validator

This Chrome extension allows you to quickly check if the current page has an SXG version available. Also, it triggers Googlebot to fetch the page and put it into the Google SXG cache if it’s not already there. It then tells you the status of the page in the cache and displays errors if Google for some reason rejected the page.

As I update this post, the extension's metrics have grown slightly: from 837 users and 1 review in December 2023 to 1,000 users (though still with just 1 review).

It tells a lot about SXG's popularity on the web. But fear not! Most websites don’t use SXG, but it doesn’t mean the tech is to blame. The tech is solid, and the websites using it outperform others.

SXG prefetch page

A simple page that prefetches a specified URL the same way Google does. Very useful, because you don’t need to wait until your page is indexed and presented in search results, on high positions.

When combined with the Network tab in Chrome Developer Tools, you can easily see exactly what HTTP requests are being performed after prefetching is triggered. To see only the SXG traffic, use the Other filter.

Curl

This well-known tool allows you to see the raw, HTTP-wrapped SXG response. Just specify the appropriate Accept HTTP header:

$ curl -siH "Accept: application/signed-exchange;v=b3" \
    https://www.planujemywesele.pl/zespoly/opole | less

HTTP/2 200
date: Wed, 06 Dec 2023 18:55:40 GMT
content-type: application/signed-exchange;v=b3
content-length: 212278
vary: accept,amp-cache-transform
x-content-type-options: nosniff
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=BE7oFSLfFhdPe1PYy6NGP%2BVvj9qeh4ZekE30ydqjTrxITTTxbDkzWHPm3LfvackhcPx0jE3XnWOreho%2Bl6cqUAYRl07dB8qLaCtdPHKyulebCrDnmhqFtuCPOcKtyeAVFQq2mq8dfhQ%3D"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
strict-transport-security: max-age=31536000; includeSubDomains; preload
server: cloudflare
cf-ray: 8316be935976632e-LHR
alt-svc: h3=":443"; ma=86400

sxg1-b3^@^@ … binary data follows

Dump-signedexchange

While curl outputs the raw binary SXG format, which humans can decode manually, there's a more convenient option: a command-line tool written in Go that's part of a larger SXG utilities package. This tool simplifies the decoding process significantly.

Here is an example output. I removed the Link header contents and the signature for readability:

$ dump-signedexchange -payload=false -uri https://www.planujemywesele.pl/zespoly/opole

format version: 1b3
request:
  method: GET
  uri: https://www.planujemywesele.pl/zespoly/opole
  headers:
response:
  status: 200
  headers:
    Etag: "d1xui95j634kzc"
    Status: 200 OK
    Content-Type: text/html; charset=utf-8
    Cf-Cache-Status: HIT
    Content-Encoding: mi-sha256-03
    X-Content-Type-Options: nosniff
    Link: ...
    X-App-Id: 2
    Accept-Ranges: bytes
    Cache-Control: s-maxage=86400, max-age=1800, public
    Date: Wed, 08 Jan 2025 17:27:48 GMT
    Cf-Ray: 8fede682c14702a0-WAW
    Digest: mi-sha256-03=4Sh+fNgY++2HoZ4CzZwUTk4TvTfQKfBJvTOn586Ld7g=
    X-Frame-Options: SAMEORIGIN
    Age: 0
    Server: cloudflare
    Content-Length: 215609
signature: ...
header integrity: sha256-R9wSEb/HOpGyeJM9Inzr9Pz6+Zw2CV03sH6GT1ElH3s=

Please keep in mind both curl and dump-signedexchange won’t work when you use Cloudflare bot protection mechanisms, as those are not apps humans use for everyday browsing. They’ll be recognized as bots and blocked.

So either disable bot protection when you perform tests or use the next tool.

Headers-altering extension

There are many browser extensions allowing you to manipulate HTTP request headers. One I used was Requestly. Similarly to curl, you just need to set the Accept header to:

application/signed-exchange;v=b3

It runs in the browser, so it can be used with Cloudflare bot protection enabled.

It works best when combined with Chrome Developer Tools, so you can see SXG response, nicely decoded with headers and stuff.

Google Search Console

When debugging, it's valuable to view your site as Google sees it. The Google Search Console provides a URL inspection tool with a Live test feature that lets you fetch any URL using Googlebot and examine the results.

When testing an SXG-enabled URL, Googlebot will show you the decoded SXG response, similar to what you'd see using the dump-signedexchange tool:

HTTP/1.1 200 OK
accept-ranges: bytes
cache-control: s-maxage=86400, max-age=1800, public
cf-cache-status: HIT
cf-ray: 8fedfc8990c86806-DFW
content-encoding: mi-sha256-03
content-length: 215601
content-type: text/html; charset=utf-8
date: Wed, 08 Jan 2025 17:42:51 GMT
digest: mi-sha256-03=H6SDmuJOXIr/v8Wcfjl++ilQBmaDOpj53i9CCVpdXkI=
etag: "7y1kp1nupp4kz4"
link: ...
server: cloudflare
status: 200 OK
x-app-id: 2
x-content-type-options: nosniff

Results

Implementing the above changes made it possible to generate the SXG version of the page HTML. Therefore prefetching it in Google results should make the LCP drop by… the time it takes to download HTML. On my connection, it’s probably somewhere between 100-600 ms, depending on how quickly the server responds.

But how does it compare to that 350 ms I promised and demonstrated earlier?

Well, it’s just the beginning. We have a solid foundation for further optimizations. The next step is to prefetch not only HTML, but the so-called subresources - stylesheets, images, and fonts. And this is where the LCP begins to drop significantly!

Thanks!

I would like to express my gratitude to my co-workers, Michał and Besufekad, for their support and assistance in debugging.

Thank you for reading this post. I really appreciate it, because it was a long read!

If you know someone struggling with high LCP, especially if the website in question gets a lot of traffic from Google, then please share this post. I would be grateful :)

See you in the next part of this series!

For the impatient

If you know what Zeitwerk is, understand the problem, and just want to do what the title of this post promises, jump straight into the tutorial.

Otherwise, keep on reading.

The problem

If you work on a non-Rails Ruby project, managing code loading may be challenging. Following the Single Responsibility Principle means having many specialized classes instead of one doing all the work. And if you keep each class in a separate file you quickly end up with a bunch of files that need to be loaded. That’s a lot of sad require statements hiding in your files. You can almost hear their mischievous giggles when you rename a class or move it into a different namespace. It may sound similar to:

cannot load such file -- my_perfect_class (LoadError)

Those exceptions make me angry. And Ruby was meant to be optimized1 for the programmer’s happiness, right?

Enter Zeitwerk

Well, there are solutions. One of the best, in my opinion, is Zeitwerk, described as an “Efficient and thread-safe code loader for Ruby”.

You probably heard about it, as it is used by Rails and Hanami. Zeitwerk is the reason you don’t need to worry about require statements when working with Rails apps. Just follow the intuitive convention of naming files the same as the classes they contain.

Using Zeitwerk in your own gem

The good news is: Zeitwerk is not limited to Rails. You can use it in any Ruby project, including any gem you create.

Just put it into Gemspec, and add a few lines of code. That’s it, you are now free to focus on features instead of code loading.

The bad news is: your gem now depends on Zeitwerk. This means each project using your gem depends on it too. Some people may not like it and decide to not use your awesome gem!

That’s because it’s generally a good practice to minimize dependencies. The benefits include security, maintainability, and stability.

Does it mean you have to choose between ease of development and dependency minimalization?

Have your cake and eat it too

I asked myself this question while working on cryptreboot, a gem that allows a machine with an encrypted disk to reboot by requesting a passphrase beforehand, rather than after the reboot.

I was not able to find an answer on Google. ChatGPT also failed. Therefore I decided to do it by myself.

In this tutorial, I will show you how to create a gem from the ground up. It will use Zeitwerk in development, while not depending on it in runtime. And it will still work.

Conventions

The first character of a line in code blocks has a special meaning:

• $ means the remaining part should be executed in the shell,

• > means the remaining part should be executed in IRB.

In other cases, the block contains the actual code or result of an action you performed.

I assume you use a Unix-based OS such as Linux distribution (may be run in WSL), *BSD, or macOS. If you run Windows, you may need to adjust some shell commands.

Start with an empty gem

We will call our gem zeitgeist. Execute in the console:

$ bundle gem zeitgeist
$ cd zeitgeist

Now adjust the Gemspec to make the gem buildable. Open zeitgeist.gemspec file and:

1. Fill summary, homepage, source_code_uri, and changelog_uri.

2. Delete the line containing the description.

The results should look similar to this (I stripped comments):

require_relative "lib/zeitgeist/version"

Gem::Specification.new do |spec|
  spec.name = "zeitgeist"
  spec.version = Zeitgeist::VERSION
  spec.authors = ["Pawel"]
  spec.email = ["pepawel@users.noreply.github.com"]

  spec.summary = "Awesome gem using Zeitwerk only for development."
  spec.homepage = "https://github.com/pepawel/zeitgeist"
  spec.license = "MIT"
  spec.required_ruby_version = ">= 2.6.0"

  spec.metadata["allowed_push_host"] = "TODO: Set to your gem server 'https://example.com'"

  spec.metadata["homepage_uri"] = spec.homepage
  spec.metadata["source_code_uri"] = spec.homepage
  spec.metadata["changelog_uri"] = "#{spec.homepage}/blob/master/CHANGELOG.md"

  spec.files = Dir.chdir(__dir__) do
    `git ls-files -z`.split("\x0").reject do |f|
      (f == __FILE__) || f.match(%r{\A(?:(?:bin|test|spec|features)/|\.(?:git|travis|circleci)|appveyor)})
    end
  end
  spec.bindir = "exe"
  spec.executables = spec.files.grep(%r{\Aexe/}) { |f| File.basename(f) }
  spec.require_paths = ["lib"]
end

You should spend some more time on Gemspec if you work on a real gem. The above example minimizes changes needed for building, but it’s not production-ready.

Install dependencies:

$ bundle config set --local path .bundle/gems # for easy cleanup
$ bundle install

Let’s commit the changes and check if you can build the gem:

$ git add .
$ git commit -m 'Initial commit'
$ rake build

It should succeed. However, if there is an error it should be self-explanatory. Most probably you will need to adjust Gemspec.

Add some logic

Place the following code in lib/zeitgeist/programmer.rb:

module Zeitgeist
  class Programmer
    def happy?
      # Ruby was created in 1993
      Time.now.year >= 1993 ? 'yes' : 'no'
    end
  end
end

To check if it works, run IRB in the context of the gem by executing in the terminal:

$ bin/console

When you try to access Zeitgeist::Programmer, you will see it’s not loaded:

> Zeitgeist::Programmer
(irb):1:in `<main>':
  uninitialized constant Zeitgeist::Programmer (NameError)
  from bin/console:15:in `<main>'

To make it work, you will need to manually require the correct file:

> require 'zeitgeist/programmer'
=> true
> Zeitgeist::Programmer
=> Zeitgeist::Programmer

But we want it to be required automatically by Zeitwerk. In our example above, it is enough to simply require the correct file. But with your codebase becoming larger, the benefits of using auto-loader will become more and more visible.

Enable Zeitwerk

Add the Zeitwerk to your Gemfile and install it:

$ bundle add zeitwerk

Gems added to the gem’s Gemfile are available in development only, so Zeitwerk won’t become a runtime dependency.

The next step is to make sure your code uses Zeitwerk. Adjust lib/zeitgeist.rb to include the following lines before the Zeitgeist module definition:

require "zeitwerk"
loader = Zeitwerk::Loader.for_gem
loader.setup

Now run the console again and check if your code is being auto-loaded:

$ bin/console
> Zeitgeist::Programmer.new.happy?
=> "yes"

It works, so we can commit our code changes:

$ git add lib Gemfile*
$ git commit -m 'Add business logic and setup autoloader'

However, it won’t work when installed on a system without Zeitwerk. The gem will install normally but will raise an exception on first use because it silently depends on Zeitwerk.

Basic loader

We need a lightweight loader for runtime. Its task would be to load all the Ruby files in gem’s lib/ directory. Why not loop over the files and simply require each one? It’s because the order of loading files matters. In the loop approach, we may end up loading file B depending on file A before file A is loaded.

To make sure we use the correct loading order, let’s create the loader file manually in lib/basic_loader.rb:

# Load every project file in one place
require 'zeitgeist/programmer'

Runtime vs development

We want to use a basic loader for runtime and Zeitwerk for development. Therefore we need to distinguish between those two.

As Gemfile is used only in development, we could use it for this purpose. Add the following code to the beginning of that file:

module ::Zeitgeist # :: is used to escape from Gemfile's scope
  AUTOLOADERS = []
end

It simply sets a constant in the main module of our gem. It will be defined only in development.

You may notice it duplicates the module definition from the zeitgeist.rb. It’s perfectly fine as the modules in Ruby can be reopened as many times as you like.

Now, let’s adjust lib/zeitgeist.rb to decide which loader to use based on the constant we defined:

require 'zeitgeist/version'

if defined? Zeitgeist::AUTOLOADERS
  require 'zeitwerk'
  Zeitgeist::AUTOLOADERS << Zeitwerk::Loader.for_gem.tap do |loader|
    loader.ignore("#{__dir__}/basic_loader.rb")
    loader.setup
  end
else
  require 'basic_loader'
end

module Zeitgeist
  class Error < StandardError; end
  # Your code goes here...
end

The basic_loaded.rb file doesn’t match the Zeitwerk convention - it doesn’t define the BasicLoader constant. Therefore, to avoid warnings we add it to the ignored files as you see above.

The other change was to save the Zeitwerk instance into AUTOLOADERS constant. We will use it later.

Let’s commit our changes, build the gem, and install it locally:

$ git add lib Gemfile
$ git commit -m 'Fix code loading'
$ rake build
$ gem install --user pkg/zeitgeist-0.1.0.gem

As our gem doesn’t contain any executables, you can safely ignore the following warning if it appears:

WARNING:  You don't have /home/user/.gem/ruby/3.0.0/bin in your PATH,
          gem executables will not run.

Now we can test if the gem works:

$ irb
> require 'zeitgeist'
> Zeitgeist::Programmer.new.happy?
=> "yes"
> defined? Zeitwerk
=> nil

It works, and the last line tells us Zeitwerk is not used.

Now let’s verify if development works too:

$ bin/console
> Zeitgeist::Programmer.new.happy?
=> "yes"
> defined? Zeitwerk
=> "constant"

It works too, but uses Zeitwerk just as we like.

However, we need to manually update lib/basic_loader.rb each time we add, remove, rename, or move the files. Why not automate it?

Autogenerate basic loader

Zeitwerk does its magic using Module#autoload. It guarantees the loading order is valid. We can use Zeitwerk to produce lib/basic_loader.rb file containing all the require statements in the correct order.

To generate that file, we will use Zeitwerk’s on_load callback which gets called every time file is loaded. We need to get a list of all files, therefore we will force Zeitwerk to load the entire codebase by using #eager_load method.

Put the following into bin/loader:

#!/usr/bin/env ruby
# frozen_string_literal: true

require 'pathname'
require 'stringio'

class LoaderGenerator
  def call
    writer do |out|
      out.puts <<~HEADER
        # frozen_string_literal: true

        # File generated automatically, do not edit

      HEADER
      yield.each do |auto_loader|
        auto_loader.on_load do |_cpath, _value, abspath|
          next if abspath !~ /.rb$/i # skip directories

          out.puts "require '#{path_to_requirement(abspath)}'"
        end
        auto_loader.eager_load
      end
    end
    true
  end

  private

  def writer(&block)
    output ? block.call(output) : File.open(loader_path, 'w', &block)
  end

  def path_to_requirement(abspath)
    relative_path_from(loader_dir, abspath).sub(/.rb$/i, '')
  end

  def relative_path_from(base_dir, target_path)
    target = Pathname.new(target_path)
    base = Pathname.new(base_dir)
    target.relative_path_from(base).to_s
  end

  attr_reader :loader_path, :loader_dir, :output

  def initialize(loader_path, output = nil)
    @loader_path = loader_path
    @loader_dir = File.dirname(loader_path)
    @output = output
  end
end

class LoaderValidator
  def call(&block)
    StringIO.new.tap do |buffer|
      LoaderGenerator.new(loader_path, buffer).call(&block)
      buffer.rewind
    end.read == current
  end

  private

  def current
    File.read(loader_path)
  end

  attr_reader :loader_path

  def initialize(loader_path)
    @loader_path = loader_path
  end
end

require 'bundler/setup'

loader_path = File.join(__dir__, '..', 'lib', 'basic_loader.rb')

klass = ARGV[0] == 'generate' ? LoaderGenerator : LoaderValidator
action = klass.new(loader_path)
result = action.call do
  require 'zeitgeist'
  Zeitgeist::AUTOLOADERS
end
exit 1 unless result

Now make the file executable and run it:

$ chmod +x bin/loader
$ bin/loader generate

It will:

1. Initialize Zeitwerk and the main module of our gem.

2. Configure Zeitwerk to add require line to the basic loader each time the Ruby file is loaded.

3. Tell Zeitwerk to eagerly load all the code. As a result, lib/basic_loader.rb becomes populated.

The file lib/basic_loader.rb should look like this:

# frozen_string_literal: true

# File generated automatically, do not edit

require 'zeitgeist/programmer'

Ensure the built gem contains current files

If you read the bin/loader code above carefully, you probably noticed LoaderValidator class. It gets activated if generate argument was not specified. It generates a basic loader in memory and compares it with the current. If they don’t match, the script exits with an error.

We will use it to protect us from forgetting to regenerate the basic loader before building the gem.

Let’s add those lines to the end of Rakefile:

Rake::Task.define_task :validate_loader do
  abort "Basic loader is stale, run `bin/loader generate` to fix" unless system("bin/loader validate")
end

Rake::Task[:build].enhance [:validate_loader]

We create a new task called validate_loader which raises an exception if the basic loader is stale. Afterward, we add it as a dependency to the build task, so it gets called before.

From now on running rake build with stale basic loader will fail with the message:

Basic loader is stale, run `bin/loader generate` to fix

Tests: which loader to use?

It makes more sense to use the basic loader instead of Zeitwerk for tests. What if we generate the basic loader incorrectly due to some error? If tests use Zeitwerk, the basic loader is bypassed and we won’t see any problems.

As we use RSpec for our gem, let’s add this line to the beginning of spec/spec_helper.rb:

raise "Failed to regenerate basic loader" unless system "bin/loader generate"

This way before each test, the basic loader will be regenerated automatically.

If you prefer to decide when the basic loader file changes, you can use this instead:

raise "Basic loader is stale, run `bin/loader generate` to fix" unless system "bin/loader validate"

You will be notified when the basic loader needs to be updated. This approach allows you to have full control over updating this file.

Run the tests

We should have a working infrastructure, let’s run the tests:

$ rake
... 
Zeitgeist
  has a version number
  does something useful (FAILED - 1)
...

The second test failed. It’s an example test that is meant to fail. Let’s change it to something related to our logic by replacing spec/zeitgeist_spec.rb with:

RSpec.describe Zeitgeist do
  it "has a version number" do
    expect(Zeitgeist::VERSION).not_to be nil
  end

  it "checks if programmer is happy" do
    expect(Zeitgeist::Programmer.new.happy?).to eq("yes")
  end
end

Run the tests again:

$ rake
...
Zeitgeist
  has a version number
  checks if programmer is happy
...

All green! Let’s commit our changes, build the gem and install it:

$ git add lib spec bin/loader Rakefile
$ git commit -m 'Make sure basic loader is autogenerated'
$ rake build
$ gem install --user pkg/zeitgeist-0.1.0.gem

Final run

We can check if the gem works after installation:

$ irb
> require 'zeitgeist'
> Zeitgeist::Programmer.new.happy?
=> "yes"
> defined? Zeitwerk
=> nil

It works, and it still doesn’t require Zeitwerk.

Now let’s test if Zeitwerk works in development. First, let’s create a random constant:

$ echo "Zeitgeist::Pi = 3.1337" > lib/zeitgeist/pi.rb

This file is not required anywhere. Let’s check if it autoloads:

$ bin/console
> Zeitgeist::Pi
=> 3.1337

It works. From now on, you can easily add new files to your code base and do not worry about require statements. Congratulations :)

Cleanup

To restore your environment to its previous state, you will need to uninstall the gem:

$ gem uninstall zeitgeist

You can also remove the directory containing the repository you were working on. Apart from the code, all the gems you installed including Zeitwerk are located there.

Limitations

Compared to using Zeitwerk, this approach has some limitations. I believe in most cases, minor code adjustments would do. As with every refactoring, good test coverage will help. However, if your project is large and/or you use advanced code-loading features, implementation of this approach may be challenging.

Those limitations were identified by the author of Zeitwerk, Xavier Noria. He gave me awesome feedback which resulted in a major update to the article (the old version could be found here). Thank you, Xavier!

Here are the limitations I’m currently aware of:

Conditional code loading

An example would be to use one module on Linux, and another on macOS:

module Foo
  include Mac if mac?
  include Linux if linux?
end

If OS-specific modules are defined in separate files, the basic loader will be generated differently depending on the developer’s OS.

Effectively gem developed on Linux will crash on macOS and vice-versa.

Delayed code loading

The developer may choose to delay the loading of some code parts. As the method presented in this post depends on eager loading, it won't include those parts. This will lead to crashes in runtime.

Circular references

Let’s assume we have foo.rb:

module Foo
  include Bar
end

And foo/bar.rb:

module Foo::Bar
end

Zeitwerk can handle this, while the generated basic loader will crash.

Implicit namespace definitions

Zeitwerk allows you to define Zeitgeist::Foo::Bar class without defining Zeitgeist::Foo module first. If you want to use the approach presented in this post, you need to define it explicitly.

Final notes

You learned how to use Zeitwerk in gem development while not adding it as a runtime dependency. Here you will find the repository with the gem we just built together.

I encourage you to apply this knowledge to your own project. The code I published here uses a permissive MIT license, so you can use it even in your commercial work.

Also, there are nearly 450 gems depending on Zeitwerk. I think many of them would accept pull requests minimizing runtime dependencies. If you feel brave, go contribute to your favorite gem to make the Open Source world a better place!

The code used in this tutorial was extracted from the cryptreboot gem. This little project is very close to my heart, so forgive me I mentioned it for a second time in this post. If you use a Linux system with an encrypted disk, give cryptreboot a try :)

And thank you for reading this post :) You are the best!

If you have any feedback, please leave it in a comment below.

As an Ethereum solo staker, I want my Linux staking box to be as secure as feasible. It applies to physical security too. If the attacker gains access to the validator key stored on disk, it will allow him/her to do malicious things such as intentionally exposing the validator to the slashing penalties.

My machine isn’t placed in a data center guarded 24/7, therefore to protect it from the physical attacker I used cryptography.

Encryption inconvenience

I decided to use LUKS disk encryption which can be enabled easily during the Linux installation. I set a long passphrase and I rest assured my data is well protected.

I need to provide the passphrase each time I turn the machine on. This way the volume key used to unlock the disk, can be derived from the text I enter. It requires a keyboard and physical presence during startup, therefore it may be inconvenient for headless systems.

There are many methods to make it easier. Though I use one of them, I find them either complex, costly, unreliable, or compromising security. I don’t want to discuss them here, because this topic deserves a separate post.

The important thing is, once started, the node should stay online. And with UPS protection, turning it off doesn’t happen almost at all.

Maintenance

To ensure there are no security bugs I perform regular software updates. Modern Linux distributions make it really easy to automate.

However, kernel updates are not that simple. To use a new kernel, the system needs to be rebooted. It’s just a short downtime which is often acceptable.

But when using disk encryption, the disk needs to be unlocked. How to achieve that when using ssh connection from a remote location? Given the fact kernel updates can be frequent, it becomes an issue.

One way to approach it is to try to avoid reboots.

Live patching…

…is a technique to apply changes to the running kernel, without a reboot. There are many proprietary implementations provided as services. Preparing a binary patch ready to be applied to a running kernel seems to me a complex task that requires a lot of testing. Probably that’s why most suppliers charge for this service. In the case of Ubuntu Livepatch, before the patch is pushed to the customers, it is tested on the production systems of non-paying users. That’s the cost of using it for free!

But still, when your kernel gets a live patch it is recommended to reboot in the nearest maintenance window.

I like to use open-source solutions instead of proprietary ones. Therefore I needed something else.

No rebooting at all?

In some environments and threat models, most vulnerabilities in the kernel are not a big deal. If you trust your local users, expose one or a few regularly updated services, and firewall the rest of the ports, then it’s just unlikely for the attacker to succeed. He/she will need a remote exploit in the tried and tested TCP/IP stack. I believe those types of bugs are extremely rare.

In the above scenario, one may choose to keep running with the same kernel, avoiding reboots. For the record: I don’t recommend this approach.

That said, if the only reason for ignoring established good security practices is to prevent reboots because disk encryption makes it inconvenient, then I think we should do better.

We should make reboots convenient.

How I wish it would work

I would like to reboot to the new kernel without entering the passphrase again. It was entered once: can’t it be reused for a new kernel?

Well, we have kexec. It will be 18 this year, but it is not popular amongst people. If you haven’t heard about it, it allows you to replace the currently running kernel with a new one. And it does it without involving hardware reset and executing the boot loader:

$ sudo kexec -l new_kernel.img   # load kernel into memory
$ sudo kexec -e                  # boot the loaded kernel

While being cool, it’s not the same as live patching. The new kernel has to boot the entire system, filesystems need to be mounted and services have to start. The disk has to be unlocked, so the passphrase is required again.

What if it was possible to preserve the volume encryption key?

Linux keeps the keys for disk encryption in the kernel keyring. It is a special, secure place in memory. Even the root can’t retrieve volume encryption keys from there!1

$ sudo dmsetup table
dm_crypt-0: 0 36284416 crypt aes-xts-plain64 :64:logon:cryptsetup:edd435cc-b51a-4530-aef3-5010caeee165-d0 0 252:3 32768

As you can see above, only key reference is provided. The actual key is kept securely in kernel memory.

If the user could tell kexec to keep the given key intact, then the new kernel could reuse it. The newly booted system could unlock the disk without asking for a passphrase. I imagine kexec adds a new option allowing to specify a key to preserve:

$ sudo kexec -l new_kernel.img --keep-key=my-key-reference

We are not there yet

Most of the work related to the volume key preservation feature would need to be done on the kernel side, but kexec and cryptsetup would also need to implement support. From my limited perspective, it seems doable, but I cannot estimate the amount of work required.

In the meantime, let’s try to resolve this issue using the tools we already have.

Initramfs

Kexec allows to pass 2 things to the new kernel:

• kernel command line containing various options for kernel and userspace,

• initramfs which is run by a new kernel to initialize various things, including disk encryption.

We could include a volume key in the kernel command line and later use it to set up disk encryption. It would work, but the kernel command line is visible to the users, which disqualifies this concept.

So we are left with initramfs. This is a file located typically in an unencrypted /boot partition. We definitely don’t want to store the volume key on an unencrypted disk, because it defeats the purpose of encryption.

But we don’t need to modify the original initramfs file permanently. It’s enough to modify it temporarily and pass it to kexec.

Introducing cryptreboot

This led me to create a tool that:

1. Asks the user for a passphrase to derive the volume key.

2. Copies original initramfs into memory and patches it to include the volume key.

3. Uses kexec to load patched initramfs and kernel into memory.

4. Initiates standard system shutdown to stop services and unmount filesystems.

5. On shutdown completion, a kernel with patched initramfs is executed. This step is done by systemd.

Here is what it looks like compared to the standard reboot. If you prefer, you can check the identical YouTube version.

As you can see, the user is still asked for a passphrase, but it’s done before reboot, not after. This way the user could execute a reboot remotely. Physical presence during startup is not required anymore.

From a security standpoint, it should be almost as secure as the standard setup:

• secrets are not persisted anywhere, everything is done in memory,

• volume key touches userspace for just a brief moment; the rest of the time it stays safely in the kernel keyring.

If you have a different perspective, please leave a comment below.

Leave a comment

Cryptreboot is an easy-to-use, drop-in solution written in Ruby and released under the MIT license. You can find it on GitHub.

What’s next

Here are the features I plan to implement:

• boot loader configuration parsing; currently, cryptreboot relies on symlinks or the user to pick the kernel and initramfs,

• optionally persist the volume key on an encrypted disk in a file accessible to root only; this way user accepting a slight security trade-off won’t be prompted for the passphrase at all,

• integration with systemd, to make cryptreboot the default reboot handler,

• deb package to make installation easier,

• support for non-Debian Linux distributions.

If you use disk encryption in Linux, please give cryptreboot a try and share your feedback!

Thank you for reading this post. If you want to stay in the loop, subscribe and star the project on Github, so you won’t miss new cryptreboot releases.

Subscribe now

3 months ago I received an email that got my attention. I subsequently wrote a blog post about it. To make a long story short I suspected the disposable email address I used exclusively for Ledger Card leaked somehow because I got a suspicious email on that address. Later on, I found out the source of the problem lay in Baanx, Ledger's business partner.

Baanx informed the email I got had been sent by a legitimate project, which is a partner of Baanx. I neglected to check the project carefully — that’s on me!

However, it’s only one part of the issue.

The email message came unsolicited to the address which shouldn’t be used for this type of communication. There is no doubt Baanx failed to handle my email address properly. While Baanx might see it differently, I believe this specific mailing campaign reached others as well. However…

The breach probably did not happen

We exchanged a few messages and had video calls with Baanx. I learned a lot of details, which I won’t provide here. The common part of our conclusions is that there was a human error during the handling of marketing email campaigns.

Most importantly, Baanx said they investigated the case, fixed a software bug, and cleaned up internal processes. Here is part of the email I got from Baanx:

There definitely were some "human error" learnings here on our internal processes that were cleaned up, so I thank you for reporting the issue. No other customers or waiting list participants contacted us, so it appears to be a single edge case - at least from reports.

While I think they could do a bit better, I believe them.

The end of this story is positive. Something that looked like a data breach turned out to be a relatively harmless human error. That’s a relief!

A special thanks to Scott Carlson from Baanx for his diligent management of the situation.

2023-07-05 update

I got an official response from CL Card support on Reddit. I also spoke to a person from Baanx. It seems that Anrk is Baanx’s partner. They're conducting an internal investigation. They say there was no data breach. I'm still not 100% clear why I got this email message. I will update this post when more data becomes available.

Here comes the original text

Do you remember how 1 million records leaked from Ledger?

Today I received this funny-looking email message:

Seems similar to Ankr. But if you check it closely you will notice a typo.

Recipient address

“Oh, just another scam attempt” - I thought1. However, when I checked the recipient address of this message, I found this address looks similar to the one, I use exclusively for my Ledger CL card.

By exclusively I mean, I don’t provide this address in any other place. Every time I register somewhere, I use a different, unique email address. This is my method of localizing sources of data leaks (if it was my data that leaked).

However, in this particular case, I used the same email address twice. I used it to contact CL Card support regarding the issues with my card. After closer inspection, I found that I made a typo in my email address when contacting CL Card support. To sum things up, I used:

• address #1 as a login for the CL Card system,

• address #2 for contacting CL Card support2.

The scam message was delivered to email address #2.

Source of the leak

So it seems, my address leaked from the support department and not the CL card system itself. That’s a big relief!

But still, the data we have now leads to many questions. Here are some of them:

• When the leak happened?

• How many other addresses leaked?

• Apart from email addresses, what other data leaked?

• Was it a corrupt or careless employee?

• Was it a CRM database leak?

• Is CRM self-hosted by Ledger or outsourced? Maybe they outsource support entirely?

• Was CRM data transferred into another system (for example for marketing purposes), and the leak happened there?

I can answer the first question. I sent the first message using address #2 on 8th December 2022. So it seems the leak happened between 2022-12-08 and 2023-07-04.

Also, I know Have I Been Pwned doesn’t report my email as leaked. I checked the same day I wrote this post.

I don’t know other answers. But I wonder how many people noticed this too. If you were affected, please leave a comment below.

Leave a comment

My data leaked - what now?

Unfortunately, if your data leaks, you can’t leak it back. It helps if you provide disposable addresses, but your main address will leak someday too. As always you need to be careful with incoming mail.

That’s it

Thank you for reading this short post.

For interested parties, I pasted the headers of the message and the first part of the text below. I obfuscated my email address and some other data which may potentially lead to revealing private information.

Delivered-To: address-2@obfuscated.com
Received: by 2002:ab3:1c15:0:b0:238:9402:e3c6 with SMTP id u21csp4866874lth;
        Tue, 4 Jul 2023 03:00:09 -0700 (PDT)
X-Google-Smtp-Source: APBJJlGIpdLNd8AIX7qcHMNUTlkl5SwX0AIeWFPH3NbG0NmWonNOfJbO6NP308UQD2NXNBzLQOO5
X-Received: by 2002:a17:902:d501:b0:1b8:3936:7b64 with SMTP id b1-20020a170902d50100b001b839367b64mr20459348plg.1.1688464809144;
        Tue, 04 Jul 2023 03:00:09 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; t=1688464809; cv=none;
        d=google.com; s=arc-20160816;
        b=vppot55BTC0Nh7GIdbYXbKkW+isDHpv7D0N4QQeYwg6qUShM6SncDiiDxRpZGylRNB
         vYll7SyfV7nMe1RkojIwcsGocbl6o8FGgQVgj/sMqZ7bIJNWu0wx+mkQRYOZ5VD/j5pC
         eHPFaD4AadeMlQTtWBrqeZx6lRovpHPBrXJnFQ7BriNOfbINnuxgQgHuAnt9vP8As/Rm
         oij0SvOVoOsXW0wRWOFiWebZ+jmGRUMNWRmkLSsAGnSSAcC7x3+VStBjpg9KMDH37iLr
         WL5MJ7IvbnzmooomKMDeVwrUEKUFVEULpELb7LCF9lM6qAas9bz4XNn8rtgQ0doIfuxh
         mH8A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816;
        h=feedback-id:from:list-unsubscribe:subject:reply-to:mime-version
         :date:message-id:to:dkim-signature;
        bh=8yLsdori4r6GtMEEET+tAfTrVGhEDhyYgbLw94d8CEQ=;
        fh=tDNHxWpIqxviIVEMkK8CTTH5nJteJnWyRzljgxDe6kw=;
        b=Oh3dOJKFdCq5hF0TPRn4pTSwnLdpqhkQbu8ePsrpsa2942ygYm8oqKn8vo1FTIHAqC
         OGWcB6odmBIZxTnE2UktQ13Cxhn7sIWvoxJh4OpGDiF2VmQrvYXvyjrJvAb/xQnVNxYj
         b8YZIK4Q9JTKIhLllhAA2P0Tvo9jY9maEBpz4bs/oq1lFenOrDcuVFwKsJM0AFVtNsBC
         7zjsqMzyPEnVwAeTG1XDr7SgQeq84TESIG/M5j5icR2s40qktBUGIjYnkpCnH5XtW84C
         68W4+/h6Ph2g3gD+t2ze/Fbeg7jdVJfyczvXSUR4/P5YvB4dxeJPvvbq/vZsqPYFnYx6
         qKeg==
ARC-Authentication-Results: i=1; mx.google.com;
       dkim=pass header.i=@anrkprotocol.com header.s=scph0623 header.b=qQ7u47oI;
       spf=pass (google.com: domain of msprvs1=1954918sqetfe=bounces-280172@sparkpostmail.com designates 192.174.87.92 as permitted sender) smtp.mailfrom="msprvs1=1954918SQEtfE=bounces-280172@sparkpostmail.com"
Return-Path: <msprvs1=1954918SQEtfE=bounces-280172@sparkpostmail.com>
Received: from mta-174-87-92.smtp-out.sparkpostmail.com (mta-174-87-92.smtp-out.sparkpostmail.com. [192.174.87.92])
        by mx.google.com with ESMTPS id kb14-20020a170903338e00b001b6ae9f8bb1si12371885plb.75.2023.07.04.03.00.08
        for <address-2@obfuscated.com>
        (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
        Tue, 04 Jul 2023 03:00:09 -0700 (PDT)
Received-SPF: pass (google.com: domain of msprvs1=1954918sqetfe=bounces-280172@sparkpostmail.com designates 192.174.87.92 as permitted sender) client-ip=192.174.87.92;
Authentication-Results: mx.google.com;
       dkim=pass header.i=@anrkprotocol.com header.s=scph0623 header.b=qQ7u47oI;
       spf=pass (google.com: domain of msprvs1=1954918sqetfe=bounces-280172@sparkpostmail.com designates 192.174.87.92 as permitted sender) smtp.mailfrom="msprvs1=1954918SQEtfE=bounces-280172@sparkpostmail.com"
X-MSFBL: OWV7igvEEXPJb+7ZJrrlYsThoPLaO1ot/hRxyPVy+gU=|eyJtZXNzYWdlX2lkIjo iNjQ5Y2E1ZWRhMzY0YTExMDVjOWQiLCJzdWJhY2NvdW50X2lkIjoiMCIsImN1c3R vbWVyX2lkIjoiMjgwMTcyIiwidGVuYW50X2lkIjoic3BjIiwiciI6ImNsLWNhcmR zLmNvbS4xMi4xMi4yMDIxLnN5c3RlbUBjcnlwdG9uaXgub3JnIn0=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=anrkprotocol.com; s=scph0623; t=1688464805; i=@anrkprotocol.com; bh=8yLsdori4r6GtMEEET+tAfTrVGhEDhyYgbLw94d8CEQ=; h=To:Message-ID:Date:Content-Type:Subject:List-Unsubscribe:From:
	 From:To:Cc:Subject; b=obfuscated
To: address-2@obfuscated.com
Message-ID: <obfuscated@jp.mta1vrest.cc.prd.sparkpost>
Date: Tue, 04 Jul 2023 10:00:05 +0000
Content-Type: multipart/alternative; boundary="_----MvvyzH+M+eka7ub4N8/3Kw===_61/D9-38980-5ADE3A46"
MIME-Version: 1.0
Reply-To: anrk@anrkprotocol.com
Subject: On-chain card spending, self-custody and beyond! 🚀
X-Campaign-ID: 7184983
List-Unsubscribe: <https://links.iterable.com/e/encryptedUnsubscribe?obfuscated>,<mailto:unsubscribe+obfuscated@unsubscribe.iterable.com>
From: anrkprotocol <anrk@anrkprotocol.com>
X-Message-ID: obfuscated
X-Feedback-ID: obfuscated:iterable
Feedback-ID: obfuscated:iterable

--_----MvvyzH+M+eka7ub4N8/3Kw===_61/D9-38980-5ADE3A46
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset="UTF-8"

Connect to your Web3 wallet to an open source wallet network



 <https://anrkprotocol.com/on-chain-transactions/>=E2=80=8A

Connect Your Metamask, Ledger, Phamtom or Web3 Wallet To The X Card And Spe=
nd=20
On-Chain!


We are building anrkprotocol - an open-source wallet network that allows yo=
u=20
to connect your Web3 wallet to our Mastercard, enabling on-chain spending a=
nd=20
giving you complete custody over your assets. With anrkprotocol you keep=20
control of your funds at all times and you eliminate the need for trust in=
=20
custodians or financial institutions. Join Waitlist!=20
<https://anrkprotocol.com/on-chain-transactions/>=20

I came across an article whose author described the process of creating a smart contract and deploying it to the Ethereum mainnet. He also published the source code.

I didn’t have experience with smart contract development and blockchain analysis, but after reading the article I began to wonder:

Is it possible to use this public information in order to find Ethereum address of this person?

It would be fun to see how much money he holds ;-)

TL;DR

Given the smart contract’s source code it is possible to find it using blockchain analysis. I’ve used Google Big Query to search for specific function signatures. In the end, I give some hints on how to avoid deanonymization.

Disclaimer

I don’t want to disclose the personal details of the smart contract’s author. Let’s just call him John. I contacted John and he approved this article before publication.

Don’t use the knowledge you got here (or anywhere else) to hurt others.

Challenge

I can define a challenge as entering unknown territory with a clear goal in mind. In the area of computer security and privacy, this feeling boosts my creativity in problem-solving, forces me to learn new things fast, perform reverse engineering and try to get into the state of mind of a given system creator.

It gives me the satisfaction of a deep understanding of the system and a thrill, both at the same time. With that motivation, I started researching the problem.

Examine the code

My first attempt was to check for any hardcoded Ethereum addresses in John’s code. The common practice is to set contract owner address dynamically during deployment instead of hardcoding it, and it was also the case here. But it costs nothing to check.

constructor() {
  owner = msg.sender; // set owner to address of caller
}

Compare bytecode

Every smart contract’s bytecode is available publicly on the blockchain. Therefore my second attempt was to compile John’s code and then use some tool to compare it with every smart contract on Ethereum.

I started by finding a way to query the blockchain for the information I wanted. It appears you can’t search by bytecode in popular blockchain explorers such as Etherscan or Etherchain. Googling bytecode is not a good idea, because Google doesn’t allow such large queries.

BigQuery

However, I’ve found Google created a BigQuery public dataset for Ethereum and updates it daily. From the user’s perspective BigQuery is just a large SQL database similar to Postgresql or Mysql used routinely by developers. Its usage is free (within limits) and there is a web-based query tool called BigQuery Console.

I found it perfect for the job. Looking at schema, there is a contracts table and it contains a bytecode column. To see bytecode related to each one of random 10 transactions made on the last day, I could execute the following SQL statement:

SELECT bytecode
FROM
  bigquery-public-data.crypto_ethereum.contracts
WHERE
  block_timestamp >
  CAST(DATE_SUB(CURRENT_DATE(), INTERVAL 1 DAY) AS TIMESTAMP)
LIMIT 10

Timestamp constrain is optional but recommended. This table is huge. Restricting time interval prevents Google from processing a lot of non-relevant data. I skipped this constrain on my first trials which quickly ate all of my quota.

Knowing the exact bytecode and blog post’s publication date I could find the contract address by executing the following statement:

SELECT address
FROM
  bigquery-public-data.crypto_ethereum.contracts
WHERE
  block_timestamp > {month before post publication date}
AND
  bytecode = {compiled bytecode of smart contract}
LIMIT 10

address is a column containing contract address.

Now I only needed the bytecode of this smart contract.

Compilation

I decided to use a free web-based IDE called Remix because it doesn’t require setting up the development environment.

I’ve created a .sol file in Remix and pasted the code of the contract. Then I opened Solidity compiler tab and set Solidity version to match the version specified in the code. Each file has the following line at the top:

pragma solidity SPEC

In place of SPEC there is a version specification.

Using the same compiler is important because every version may produce different bytecode and I was trying to generate an exact copy of the bytecode deployed by John.

The code was compiled successfully, and I was able to get bytecode.

Blockchain search

To my surprise, SQL query mentioned above didn’t return any values. I modified WHERE statement to use SQL LIKE, which allows finding partial matches:

bytecode LIKE '%{part of compiled bytecode}%'

Then I experimented with passing different parts of bytecode. However, there were no meaningful results neither.

Optimization

Then it hit me. Solidity compiler optimizes code, to reduce code size (contract deployment is cheaper) and execution cost (users have to pay less for gas).

Optimization level is controller by a parameter specified during compilation.

This parameter is a number from 0 to 4294967295. I suspect each number produces a slightly different bytecode.

I didn’t know what optimization parameter had been used. I didn’t want to compile the code and execute SQL queries against the blockchain 4294967296 times. After trying to guess common values few times, I gave up. There has to be another way.

Function signatures

I dug into Ethereum dataset. One of the columns in SQL schema got my attention: function_sighashes.

After some research, I found that, in order for the smart contract’s functions to be called by users or other smart contracts, there needs to be some kind of mapping between the function name and an address in bytecode where the function logic resides.

But what if someone likes to use veryLongFunctionNames? Does it mean the name of the function has to be embedded in smart contract bytecode, increasing blockchain size and eating precious gas?

This may be one of the reasons Ethereum uses the concept called function signature. The function signature is created by hashing the prototype string and discarding anything after the first 4 bytes. The hash algorithm used is Keccak256 which you may know from SHA-3. However SHA-3 uses a version with slightly different parameters, therefore the output is different.

For example, to get function signature for a function myFunction which receives 1 argument of type uint256:

1. Prepare function prototype, let’s say myFunction(uint256).

2. Find an online Keccak256 generator (for example this one) and paste the function prototype there.

3. You will get 50628c969c386d878aac8a993492e42110c19ba346d377fec055d2d56124b695.

4. Remove anything after the first 4 bytes and add 0x prefix to make it clear we are dealing with a hexadecimal number.

5. The result is 0x50628c96.

Those four bytes along with the mapping method will show you where the function resides in a compiled contract (it is a simplification to make you grasp the general idea, check limitations below).

Back to BigQuery

I’ve calculated function signatures for every function of John’s smart contract. Now I was armed with the information needed to find the address of this contract:

SELECT address
FROM
  bigquery-public-data.crypto_ethereum.contracts
WHERE
  '0xf68deb93' IN UNNEST(function_sighashes)
AND
  block_timestamp > {month before post publication date}
LIMIT 10

This simplified example contains only one function signature (0xf68deb93). To find John’s contract I had to add more of them to the WHERE condition.

Yay! The query gave exactly one result. Is it the contract address I was looking for?

Verification

I used Etherscan to get more information about this address. Etherscan confirmed this is a contract address and allowed me to decompile it using a build-in online tool. The resulting code looked similar to the original code published by John. I found it!

Limitations

Function signatures are extracted from contract bytecode using heuristics. In case the contract’s bytecode doesn’t follow conventions, it may be hard or even impossible to obtain function signatures. Therefore some function signatures may not be available in the BigQuery dataset.

For more details, check out how the Ethereum dataset was built and how those heuristics work.

This is how I understand why some function signatures are not available in the dataset. If you have a more accurate explanation, please share it below.

Leave a comment

Extracting information

Let’s see what information I can get knowing the contract address.

The most important data is the Ethereum account address which interacted with the contract. I was able to found it easily in the list of the contract’s transactions on Etherscan. There was only one, therefore I could safely assume it was the address of John.

The rest is simple: when you have someone’s address you can get his entire transactions history and balance. It’s like a bank statement, but public.

Now I could contact John and tell him the exact amount of Ether he holds. His reaction was worth the time I’ve put into this task :)

If you like this story then I can send you my future articles right after publication. This is a privacy-oriented blog - there will be no spam and you can unsubscribe anytime.

Subscribe now

How to avoid being doxxed on Ethereum?

Ethereum transactions are pseudonymous, which means the user performing them is private as long as his address (pseudonym) can’t be linked to his real identity. John runs a blog, therefore his identity is public. Smart contract source code is also public and linked to the blog. Therefore anyone who is able to perform the steps I described above, could deanonymize him.

If you want to deploy a contract which source code is public and associated with you, here are some ideas to make connecting your real identity with your main Ethereum address harder:

• obfuscate source code before deployment (example obfuscator) - this makes finding the smart contract more difficult,

• separate contract deployment from contract usage: deploy contract from address specially generated for this purpose and don’t use it for anything else - others could see who created the contract, but won’t be able to easily connect this information with your transactions (which in most cases reveal your financial situation), especially if you follow next advice,

• if the contract is meant to be used by other people - don’t be first to use it, wait for others to transact - hide in a crowd; break up a transaction into many smaller ones and execute them at irregular intervals from many different, unrelated accounts, preferable with different transaction history; this method is similar to techniques employed by anonymity mixers such as Tornado.cash,

• make sure all addresses you use (for deployment and transactions) are anonymously funded and not linked to your main account.

  I plan to write dedicated article about maintaining anonymity on Ethereum blockchain. I will link it here. Subscribe to get it right after publication.

  Subscribe now

Final word

Thank you for reading this little deanonymization story, I hope you enjoyed it :)

In case you would like to add something, have a question, or found an error, please comment down below.

Privacy is important. If you think this article will be useful to others, please spread the word.

Share

2011-01-18 update

I received information that the vulnerability had already been known earlier, in July 2008. The issue was discovered by Józef and described in this forum thread.

2011-01-12 update

On January 12, 2011, Freeconet's management issued an official statement regarding the vulnerability described in this article.

Freeconet statement

64.3KB ∙ PDF file

Download

Download

The platform owners reacted immediately—on the day the article was published, the “Add external operator” feature was disabled.

Introduction

Freeconet is a VoIP platform that offers a range of internet telephony services to its users.
At the end of September 2010, I discovered a vulnerability that allows an attacker to intercept calls from Freeconet users to numbers chosen by the attacker.
The attacker has full control over the calls: they can terminate, record, join, and manipulate them in any way.
To demonstrate some of the possible attacks, I created the VOIPROX tool (see below).

Immediately after discovering the vulnerability, I passed detailed information to the platform’s owner.
Unfortunately, despite assurances from Freeconet, the vulnerability was not removed.
That’s why I’ve decided to publish full details about the issue.
I believe in the magic of Full Disclosure—the vulnerability will be eliminated, and Freeconet users will soon be safe.

However, after the fix, the platform’s owner should be urged to investigate whether such abuse has already occurred.
If so, the affected individuals—as well as law enforcement agencies, if necessary—should be notified.
To ensure the credibility of such an investigation, it should be conducted by an external, trusted body, e.g., the Urząd Komunikacji Elektronicznej.

This matter is also important to me personally, as I use this otherwise innovative and probably the most technologically advanced VoIP platform in Poland.

The vulnerability

The platform allows you to assign additional phone numbers to a VoIP account—numbers from external, non-Freeconet operators.
The idea is to increase user convenience while reducing call costs.

Let’s assume users A and B are registered on Freeconet.
User A also has a number from a traditional operator and can associate it with their account.
If user B dials that number, the platform sets up an internal network connection, so the call is free.
If the number wasn’t registered, the platform would use traditional telephony, which would result in charges for user B.

Note: It’s user A’s VoIP terminal that rings, not the phone connected to the traditional landline.
This issue is typically solved by using a hybrid gateway or a VoIP phone that can handle both IP and traditional line connections.
That way, a single device handles both types of calls.

The external operator definition feature is available in the Freeconet control panel under Configuration » Operators » External.
Clicking “Add external operator” opens a form where the number and area code are entered in separate fields.
Clicking “Add” links the number to the account—provided the number format is correct and it’s not already linked to another account.

There might be other validation checks, but it’s clear a crucial one is missing: verifying the user’s right to the entered number.
An attacker can enter any number—not owned by them.
From then on, all calls to that number are no longer routed to the actual owner—they go directly to the attacker!

Scope of the threat

Freeconet has between several thousand and several hundred thousand subscribers, depending on the source.

I believe the vulnerability has existed since Freeconet launched in September 2006.
So, for over four years, users have been vulnerable to attacks like those described below.

Attack scenarios

Denial of Service (DoS)

The simplest attack type.
The attacker doesn’t answer calls, hangs up, or simulates a busy line.
This effectively blocks access to the targeted number.

Eavesdropping

As a call is initiated, the attacker starts a parallel call to the same number using traditional telephony (via or outside of Freeconet).
Once the recipient answers, the attacker connects the audio streams of both calls and records them.

The attack is harder to detect if the attacker hides their number or (more difficultly) spoofs the caller’s number.
This way, the callee won’t notice that the call came from a different number.

The attacker can then use fragments of the recorded call to construct misleading or self-serving statements—for example, to carry out the next attack.

Call modification

This builds upon the eavesdropping attack.
The attacker can manipulate the call in real-time—altering pitch, adding echo, or even injecting their own speech.
More advanced versions may target interactive phone systems (IVRs), such as banking lines.

For example, after the victim passes authentication via phone, the attacker disconnects them and continues the call impersonating the user.

How to protect yourself?

Until the vulnerability is fixed, users can protect themselves by changing call routing rules to avoid internal Freeconet calls.
This will increase call costs for calls to other Freeconet users but ensures your calls aren't intercepted.

To do this, go to Configuration » Calling » Routing and set external routing rules for all calls.

VOIPROX tool

To demonstrate the vulnerability, I created a tool called VOIPROX (from VoIP proxy—or, for fun, VoIP rocks!).

The tool logs into two SIP accounts—an input and an output one. It works similarly to a proxy. Incoming calls to the number(s) assigned to the input account (hereafter referred to as intercepted numbers) are received and forwarded to the real recipient via the output account. The incoming call is only answered once the person being called picks up. This prevents a suspicious pause that could alert the caller.

Each intercepted call is automatically saved to a separate WAV file. While the call is ongoing, the user can interact with it in real time via a simple console.

VOIPROX was written in Python. I chose this language because the VoIP library I selected is only available for C and Python.

VOIPROX was written and tested on Linux. However, I see no reason it wouldn't work on other operating systems supported by Python and PJSIP.

To test the tool conveniently, you'll need a separate VoIP terminal, gateway, or softphone (e.g., the web softphone provided by Freeconet). Note: the softphone must run on a different machine than VOIPROX. VOIPROX requires access to a sound card (even if it doesn’t use it) and cannot share it with other software.

Dependencies

Install all dependencies needed to compile PJSIP. You can find detailed instructions here. Pay special attention to the ALSA libraries. If they are not detected, VOIPROX will not support local audio.

Download the PJSIP library (I tested version 1.8.10):

$ wget http://www.pjsip.org/release/1.8.10/pjproject-1.8.10.tar.bz2

Unpack the archive and compile the library without installing it:

$ tar jxf pjproject-1.8.10.tar.bz2
$ cd pjproject-1.8.10
$ ./configure
$ make dep
$ make

Install Python and the necessary packages to compile extensions. You can find all the required information here.

Compile and install the PJSIP extension for Python:

$ cd pjsip-apps/src/python
$ sudo make

Check installation:

$ python -c 'import pjsua' && echo 'OK'

VoIP accounts

You will need three SIP accounts:

• Input account: Receives calls to one or more intercepted numbers.

• Output account: Used by VOIPROX to make calls to actual recipients.

• Test account: Used for testing. If you already have a Freeconet account, you can skip creating this one.

Although in theory, the output account can be registered with any VoIP provider, VOIPROX currently supports only Freeconet accounts. You can create all accounts using the Freeconet registration form.

Assign a phone number, provide your details, email, and login for each account. You can use the same email for all. To distinguish accounts, include the type in the login (e.g., -in, -out, -test).

Account configuration

Log in using the credentials sent to your email and configure:

• Output account:

  • Top up your balance: Payments » Top Up.

  • Deactivate Freeconet internal calls (see "How to Protect Yourself").

  • Optionally, hide caller ID: Configuration » Users » Your user » Presentation » Hide.

• Input account:

  • Configuration » Operators » External » Add external operator.

  • Enter the number to intercept and click Add. For mobile numbers, provide the first three digits (area code) without the leading zero in the first field and the rest in the second.

  • Note: The number must not be from Freeconet.

  • Do not use popular numbers (e.g., company hotlines), as intercepting random calls can have legal consequences. Use your own landline or mobile number.

• Test account:

  • If you already have a Freeconet VoIP phone, use it for testing and skip ahead.

  • Otherwise, after logging in, use the web client under "Call from Website". Try dialing 901 (for free account status info) or Freeconet’s support at 801 009 500.

  • If calls don’t work, try another SIP client. Configuration info is in the email from Freeconet.

  • Do not make test calls from the same machine running VOIPROX. Use a separate VoIP device or run a softphone on another computer.

Launching the program

Download VOIPROX:

$ git clone https://github.com/pepawel/voiprox.git
$ cd voiprox

Start VOIPROX with SIP credentials:

$ ./voiprox input_login:password1 output_login:password2

A short sound confirms correct sound card communication.

If issues occur, run with -v for verbose mode.

First test

Make a call from the test account (on a different device) to the intercepted number set in the input account. The test call is free for the test account but will charge the output account.

After the call, a .wav file will appear in the VOIPROX directory. Congrats—you’ve just intercepted your first phone call. ;-)

Further capabilities

VOIPROX has an interactive console. Type help for commands:

>> help
Possible commands:
  list, show       - show all active connections
  disconnect [[caller|callee] [from]] connection
    - disconnect caller or calee from connection given by
      connection id; if caller/callee not given entire
      connection is terminated
  attach [mic|speaker] [to] connection
    - attach local microphone or speaker to given connection;
      if mic/speaker keyword not given both will be attached
  detach [mic|speaker] [from] connection - analogical to attach
  play file.wav [to] connection
                   - play wav file (no spaces allowed in file name)
  help             - show this help
  quit, exit or ^D - terminate all active connections and quit

Call and don’t hang up. Use the list command to see active calls:

>> list
1: 123456789 > 0987654321

Attach mic/speaker:

>> attach to 1

Detach mic to listen only:

>> detach mic from 1

Disconnect the caller to impersonate:

>> attach mic to 1
>> disconnect caller from 1

Play a WAV file:

>> play test.wav to 1

Enjoy—and play nice!

Disclaimer

VOIPROX is a proof-of-concept tool. It may contain bugs. I’ve tested intercepting two calls simultaneously, but not more. Tested on Ubuntu 9.10.

All calls during development were initiated by me and used only for testing.

Use VOIPROX at your own risk. Do not break the law. Freeconet logs all connections—logs may be used as evidence in court.

I hope bad actors won’t use this tool. Public release should pressure Freeconet to patch the flaw quickly, denying criminals time to abuse it. Anyone already secretly exploiting it will lose that opportunity.

Summary

TP Internet DSL, based on ADSL technology, is a service from Telekomunikacja Polska (TP) mainly targeted at businesses. The DSL modem management system contains a serious security flaw. In this post, I will explain how I discovered the flaw, what risks it involves, and propose countermeasures.

Background

It all started when my DSL modem broke. I had been using TP’s DSL Internet Access service for a while and was mostly satisfied. The connection was provided by a Siemens Speedstream 5660 ADSL modem. The modem worked well, though it occasionally suffered from stability issues1. However, one thing always annoyed me: TP didn’t allow users to configure the modem themselves—it was password-protected, and neither the customer nor even the technician knew the password.

TP claimed this restriction was for security reasons2. Ironically, it actually compromised security.

Since the modem broke on a weekend, I knew I’d be waiting a while for a replacement. A friend lent me a Planet ADSL modem, and I tried configuring it manually. The DSL link connected fine, but I couldn’t get past the PPP login—I didn’t have the username or password, and TP didn’t provide this information to customers. After some persistence, a helpful technician gave me the PPP credentials, which I greatly appreciated.

During that call, I learned something interesting: by entering a special service login and password, a script would automatically configure the modem. Unfortunately, this only worked with Speedstream modems, so I had to configure everything manually.

How it works

The service login and password are a clever trick for installers. The installer resets the modem to factory settings, connects it to the DSL line, and enters the login konfiguracja@konfiguracja with the password konfiguracja. A script then completes the configuration remotely.

This setup prevents rogue installers from keeping access to modems or sharing it with customers. TP controls a centralized database of logins and passwords, allowing them to enforce strong password policies.

But as I realized, this special pair of login and password shouldn't be public—because it can be exploited. And anyone can learn them—all it takes is carefully watching the technician’s hands during modem configuration.

The experiment begins

I began to wonder whether this automation could be used to gain access to the modem.

I downloaded the modem manual from the manufacturer’s website. After studying it, I concluded that the modem is most likely configured remotely via Telnet, since it has a built-in Telnet server. If I could intercept the Telnet session, I might be able to extract some interesting information. But how? The transmission takes place over the DSL line. A DSL sniffer? It was beyond my reach.

Eventually, I came up with an idea. An attacker equipped with two DSL modems and some additional hardware can build a network like the one shown below:

The attacker configures the Planet modem to forward (NAPT) traffic coming from the Internet on administrative ports (Telnet, WWW, SNMP, FTP) to an internal host—the Speedstream modem. The Speedstream is reset to factory settings—this enables Telnet access to its configuration. The last step is to set the default route on the Speedstream so that it goes through the Planet modem.

I entered the service login and password into the Planet modem's PPP configuration, then started a sniffer on my PC...

First success

After a moment, the Planet modem establishes a connection. It receives an address from the pool of service addresses. After a short while, the attacker notices activity on the Ethernet network. The traffic comes from a provisioning server that initiates a Telnet connection to the Planet modem, which forwards it to the Speedstream modem, allowing interception of the session. After recording and analyzing the transmission, I learned, among other things, confidential information about the modem!

Type "?" at the command prompt for a list of commands.
Type "help" at the command prompt for general help.
For detailed help on a specific command, type command name
followed by a "?",  for instance, "show ?".

Command-> show

--- General Router Information
 System Mode          - Router
 System Type          - SpeedStream 5660-R:ENI
 System HW Version    - 0
 System Up Time       - 0 Days 0 Hours 22 Minutes 36 Seconds
 Software Version     - 2.3.0(8) Jan 13 2003 09:29:38
 Factory MAC Address  - 00:11:22:33:44:55
 DSL Phy Description  - Motorola 850 SAR Alcatel/RT Adapter
 DSL Phy Version      - 
 DSL Interface State  - Down
 Host Name            - SpeedStream
 Domain Name          - domain.invalid
 IP Gateway           - 10.0.0.2
 Ethernet Interface   - 10.0.0.1/255.0.0.0
 DSL Interface        - /
 RIP Mode             - Disabled
 DNS Server           - Enabled
 DHCP Server          - Enabled
 NAPT Mode            - Enabled
 IP Filter Mode       - Disabled

Command-> set pppauth xxxxxxxx@internetdsl xxxxxxxx
Command-> set napt disable
Command-> set ethip x.x.x.x x.x.x.x

Implement IP changes now? default: n [y,n] n

Changes will not take effect until modem is rebooted!

Command-> set snmpcfg xxxxxxxx a a a 10.10.10.10 10.10.10.10
Command-> set password
Setting user password:

New password : ********
New password : ********

Password updated
Command-> reboot

Are you sure? default: n [y,n] y

System rebooting as requested!!!!

The provisioning script connects, displays the modem’s data (using the show command), and sets configuration parameters, passwords, and the IP address. Finally, it reboots the modem so the new configuration can be loaded. As a side effect of the reboot, even if the attacker had initiated an administrative session (via Telnet or serial console using the factory password) before the script started configuring the modem, that session would be terminated by the reboot. If the reboot didn’t occur, the attacker would remain connected and could retrieve the newly set passwords.3

For a moment, I wondered why the script ran the show command, but the answer quickly became clear: this is how the script identifies the modem to assign it the correct configuration. The only piece of data likely to differ between modems is the MAC address…

The masquerade

I wrote and ran a simple Perl script. The script, when hooked up to inetd on port 23, emulates the behavior of the modem’s Telnet server after a factory reset. I set up a network similar to the previous one, but without the Speedstream modem, since it’s no longer needed.

To check whether it’s possible to obtain passwords for any modem, I configured the script to display the MAC address of another existing device.

Indeed, the provisioning script connected to my script and handed over the configuration data of a different modem!

I could repeat this experiment successfully using the MAC address of a modem from another city. To obtain confidential configuration data for any modem in Poland, all you need is its MAC address!

Packet filter

I gained Telnet access to modems to which I had physical access. However, an attempt to log in to a modem in another city failed. Connections to the Telnet port were hanging—there was no response from the remote modem. Meanwhile, TCP connections to other ports were actively rejected (the modem sent a TCP packet with the RST flag set). This was a clear sign of packet filtering.

Using the tcptraceroute tool, I determined where the packets were being filtered. It turned out the filtering was done by the DSLAM located just before the remote modem. A DSLAM (DSL Access Multiplexer) is a central hub to which many DSL modems are connected. It provides connectivity between the modems and the external network. In the case of TP, these are most likely Lucent’s Stinger devices.

The packet filters and “firewalls” in the operating systems of such devices usually have little in common with professional solutions (like Cisco PIX, GNU/Linux, or *BSD systems). Instead, they function more like add-ons that “sort of work.”

Based on this assumption, one can try to bypass the DSLAM’s blocking rules. Simple packet filters may, by default, allow all fragmented packets through. I needed to structure the connection attempt to the modem so that at least the initial connection packet is fragmented. For this, I used the fragroute tool. It turns out a small patch was necessary because the program doesn’t fragment packets with the SYN flag set, which are precisely the packets that need to be fragmented.

Filter bypassed!

The filter allowed the fragmented packets through, and the attacker gained remote access to the modem.

Another method of bypassing the filter is IP spoofing. For the attack to succeed, one would need to find an inactive IP address that’s on the DSLAM’s list of “trusted” addresses. These IPs should be searched for “near” the IP address of the server running the modem configuration script.

If such a trusted, inactive IP were found, the attack—despite the obvious challenge of performing it without feedback—would be relatively simple because, according to Nmap, the modem’s ISN (Initial Sequence Number) generator is predictable:

Starting nmap x ( http://www.insecure.org/nmap/ ) at xxxx-xx-xx xx:xx CEST
Host x (x.x.x.x) appears to be up ... good.
Initiating SYN Stealth Scan against x (x.x.x.x) at xx:xx
Adding open port 1723/tcp
Adding open port 21/tcp
Adding open port 80/tcp
Adding open port 23/tcp
The SYN Stealth Scan took 6 seconds to scan 1659 ports.
For OSScan assuming that port 21 is open and port 1 is closed and neither are firewalled
Interesting ports on x (x.x.x.x):
(The 1655 ports scanned but not shown below are in state: closed)
PORT     STATE SERVICE
21/tcp   open  ftp
23/tcp   open  telnet
80/tcp   open  http
1723/tcp open  pptp
Device type: firewall|switch|WAP
Running: SonicWall embedded, Enterasys embedded, Cisco embedded
OS details: SonicWall SOHO firewall, Enterasys Matrix E1, or Accelerated Networks VoDSL, or Cisco 360 Access Point
TCP Sequence Prediction: Class=64K rule
                         Difficulty=1 (Trivial joke)
IPID Sequence Generation: Incremental

Nmap run completed -- 1 IP address (1 host up) scanned in 12.380 seconds

What’s next?

Full access to the modem gives a malicious actor significant capabilities.

In the simplest case, the attacker could disconnect the subscriber from the Internet by disabling the LAN interface; they could change the modem’s access password, preventing TP technicians from reaching the device; or they could attempt to break into the subscriber’s internal network, which until now was protected by the modem’s NAT.

There is an officially undocumented SET PRIV command that provides full access to the operating system (VxWorks). Full system access gives the attacker virtually unlimited possibilities—they could upload and run any program on the modem. Such a program could turn the modem into a so-called “zombie” used in DDoS attacks; it could allow the attacker to eavesdrop on traffic and perform man-in-the-middle (MitM) attacks. Finally, the program could reprogram the modem’s FLASH memory to the point that it stops functioning entirely and would require service reprogramming.

The described threats become more serious when one considers the number of modems in use in Poland.4

The problem of obtaining MAC addresses

To gain control over a modem, one must know its MAC address. It wasn’t possible to obtain this address based on the IP of the Speedstream 5660 modem.

After analyzing a transmission dump between the script and the modem using p0f, it turned out that the server running the configuration script most likely operates under Solaris 8.

p0f - passive os fingerprinting utility, version 2.0.2
(C) M. Zalewski <lcamtuf@coredump.cx>, W. Stearns <wstearns@pobox.com>
p0f: listening (SYN) on 'filtered.dump', 193 sigs (9 generic), rule: 'all'.
80.50.248.71:14396 - Solaris 8 (1) (NAT!) 
  -> 10.0.0.1:23 (distance 8, link: unknown-1472)
[+] End of input file.

If an attacker were able to gain control of this server, they would also have access to the MAC address database (and most likely passwords and other confidential data as well).

The server is protected by a firewall that likely performs packet defragmentation, so the previous trick with packet fragmentation wouldn’t work here. I did not attempt direct intrusion, but I tested the provisioning script by feeding it incorrect input. It proved to be resistant—at least to the inputs it was tested with.

While testing the buffer capacity of the remote side, it became clear what kind of firewall was in use. When my program sent a large amount of data, the script on the other side disconnected and closed the connection. The firewall interpreted this as a terminated connection, while my program kept sending data. As a result, the packets sent by the program were reset—but in a very specific way. The RST packets weren’t empty—they contained a payload that was a copy of the packet being reset. This is a known bug in Cisco PIX.

With other methods blocked, brute-forcing MAC addresses was the remaining option. Of course, it’s not necessary to scan the entire MAC address space. While there are 2⁴⁸ possible MAC addresses, which might sound discouraging, the first three bytes are identical for all the modems. Additionally, the first half of the fourth byte is often the same. That narrows it down to 2²⁸ = 268,435,456 possibilities.

It’s reasonable to assume that TP purchased modems in batches. Devices from the same batch have almost identical MAC addresses. If the attacker “hits” one valid MAC address, finding others from the same batch becomes easy.

During testing, I observed that TP’s script has timeouts for modem configuration. If the modem doesn’t respond within a certain time, the script disconnects and reconnects—this cycle can repeat multiple times. This behavior can be exploited to speed up password collection.

With enough time, it would be possible to create a password-harvesting program that intelligently scans MAC addresses and extracts passwords from the database at a rate of one every few seconds—provided it “hits” a valid series. The wait time could be longer if the MAC address is a miss, but likely no more than 2–3 minutes.

New modems

After identifying and initially describing the vulnerability, it turned out that, in addition to the Speedstream 5660 modems, TP was also providing subscribers with newer modems—devices from the same manufacturer, but marked with the model number 5100.

Like the 5660, the Speedstream 5100 also features a web interface for managing and monitoring the modem. However, unlike the 5660, which protects the entire interface with a password, the 5100 allows users to monitor the modem's status without authentication. This allows subscribers to diagnose connection issues without needing full access.

The main page of the web interface also includes basic information about the device. From an attacker’s perspective, the key piece of information is… the MAC address.

The problem of obtaining MAC addresses—solved

This is a major convenience for an attacker. Knowing the IP address of a remote modem is enough to connect to the modem’s homepage (again using fragmented packets) and retrieve its MAC address.

An attacker, using a tool to scan all IP ranges assigned to TP and looking specifically for 5100 modems (which are easy to identify remotely), could very quickly collect the MAC addresses of all such modems in Poland. Needless to say, all of the previously described threats also apply to these devices.

Default password

An attempt to retrieve authentication data for the 5100 using the spoofing script failed. The provisioning script disconnected, just as it does when it receives an invalid response from the modem. It’s likely because the CLI interface was different.

To adapt the spoofing script to “work” with the 5100, I recreated the test setup—this time using the newer modem. Analysis of the captured transmission provided the necessary details and revealed an amusing fact: to prevent subscribers from tampering with the configuration, the factory password had been changed. This means even someone with the modem's manual—including the default password—cannot gain access to the device after a factory reset.

Unfortunately, by analyzing the transmission between the script and the modem, an attacker can easily learn this password.

Countermeasures

It's difficult for me to suggest what actions TP could take to secure the DSL system as I’m certainly unaware of all its features and the requirements it must meet. I also don't know the system's internal architecture or the capabilities of the devices that comprise it. Still, I will briefly discuss some simple remedies that wouldn't require major changes to the system design.

The most reliable solution would be to abandon automatic modem configuration entirely. However, I realize this could increase the cost of installing new modems and maintaining existing ones.

If the packet filters on DSLAMs allow it, a simple measure could be to block fragmented packets. However, this would serve only as a temporary fix, as it doesn’t prevent credential theft from the central database nor stop the use of those credentials when the attacker has physical access to the modem.

A more balanced solution might be to limit automatic modem provisioning to a few trusted locations—for example, one in each major city. In those locations, technicians would configure the modems before installing them at customer premises.