DOCS: Update README and system.xml with Varnish/Fastly configuration …

[JaJuMa]

…details for Back/Forward Cache support

[rhoerr]

Thank you Oli. I'm going to tentatively approve this, but leave it open a bit for anyone else that has comments.

[rhoerr]

Possible alternate bfcache config section, subject to testing and approval:

Back/Forward Cache

...

If you use Varnish FPC

For the Back/Forward Cache feature to work with Varnish Full Page Cache, you must modify your VCL file's vcl_deliver subroutine by updating the existing Cache-Control header logic.

sub vcl_deliver {
  # Find the existing line that sets Cache-Control, like:
  set resp.http.Cache-Control = "no-store, no-cache, must-revalidate, max-age=0";
  
  # Replace it with:
  if (resp.http.Cache-Control ~ "public") {
      set resp.http.Cache-Control = "no-cache, must-revalidate, max-age=0";
  } else {
      set resp.http.Cache-Control = "no-store, no-cache, must-revalidate, max-age=0";
  }
}

• This modification requires manual VCL file editing and Varnish service restart
• Test thoroughly in a staging environment before deploying to production
• Consider using elgentos/magento2-varnish-extended for a more complete enhanced Varnish configuration

If you use Fastly (including Adobe Commerce Cloud)

For Fastly CDN, create a custom VCL snippet through the Magento admin panel:

Step 1: Access VCL Snippets

1. Navigate to Stores > Settings > Configuration > Advanced > System
2. Expand Full Page Cache > Fastly Configuration > Custom VCL Snippets
3. Click Create Custom Snippet

Step 2: Configure Snippet 1

• Name: bfcache-preserve-public-private
• Type: fetch
• Priority: 1
• VCL Content:

if (beresp.http.Cache-Control) {
    if (beresp.http.Cache-Control ~ "public") {
        set beresp.http.X-MageOS-Bfcache = "public";
    } else {
        set beresp.http.X-MageOS-Bfcache  = "private";
    }
}

Save the snippet, then Click Create Custom Snippet again

Step 3: Configure Snippet 2

• Name: bfcache-remove-ccns
• Type: deliver
• Priority: 100
• VCL Content:

if (fastly.ff.visits_this_service == 0 && req.restarts == 0) {
    if (resp.http.X-MageOS-Bfcache == "public") {
       set resp.http.Cache-Control = "no-cache, must-revalidate, max-age=0";
    }
}

unset resp.http.X-MageOS-Bfcache;

Save the snippet

Step 4: Deploy

Click Upload VCL to Fastly, and Activate the uploaded VCL

I can test and verify the Fastly snippet for a client tomorrow. Tested, works

I am concerned about the HIT check though. Would like to investigate that further. I don't like that it would mean FPC cache misses aren't eligible for bfcache (if I interpret correctly). Updated with revisions per comments below (6 Oct)

Updated with @convenient 's latest snippets (8 Oct)

[convenient]

I am concerned about the HIT check though. Would like to investigate that further. I don't like that it would mean FPC cache misses aren't eligible for bfcache (if I interpret correctly).

@rhoerr From the comment from @JaJuMa here #2 (comment)

The module uses the Cache-Control: public header as the primary “flag” to signal to Varnish whether a page is eligible for bfcache.

In the case of this module, it seems like the cache-control header can be trusted enough to vary?

[rhoerr]

Okay, in that case the Fastly snippet could potentially be simplified to ?:

# BFCache optimization: Allow bfcache for public pages
if (resp.http.Cache-Control ~ "public") {
  set resp.http.Cache-Control = "no-cache, must-revalidate, max-age=0";
}

(setting aside whether pages can/should be edge cacheable in general -- out of scope for these purposes for now, and bfcache ignores that either way).

[rhoerr]

To my last comment: no, because that value would already have been overwritten by deliver.vcl (priority 50) to remove any public or private.

This seems to work instead, and avoids the HIT constraint.

    # If the page is tagged as cacheable, but has no-store header, remove it to allow bfcache.
    if (resp.http.Pragma ~ "^cache"
        && resp.http.Cache-Control ~ "no-store"
        && req.http.X-Requested-With !~ "XMLHttpRequest") {
        set resp.http.Cache-Control = "no-cache, must-revalidate, max-age=0";
    }

It goes by the Pragma response header, which is more or less a proxy for the public Cache-Control flag. If Magento sent pragma cache, it sent either private or public response -- but private is used only for core private data AJAX requests (ESI-type system), which (1) haven't been used or recommended in core since 2.0 or so, and (2) are excluded separately here via the XMLHttpRequest check. So I think this is safe. Relatively. Even in the event that a request was private, that should preclude caching (-> no-cache), but not temporary local storage (-> no-store). Which is what we're doing here. The risk is viewing a private-tagged request, logging out, then going back, but the existing JaJuMa code will check for that client-side after each restore anyway.

From a practical standpoint, the difference between Cache-Control public vs private is whether intermediates (like Varnish/Fastly) are allowed to store the content. https://stackoverflow.com/questions/3492319/private-vs-public-in-cache-control

https://github.com/mage-os/mageos-magento2/blob/e43e73cba662ea2df31cc0530c15fb5eb66e8230/lib/internal/Magento/Framework/App/Response/Http.php#L134-L172

An alternative solution would be to add two separate snippets, one before the standard deliver to set whether it is public, and then one like this after to update Cache-Control if it is, since the core rule overwrites the Cache-Control value and any public flag it contained.

Thoughts? @convenient

---

While looking into this, I found in passing: https://github.com/fastly/fastly-magento2/blob/master/Documentation/Guides/CUSTOM-VCL-SNIPPETS.md#automated-custom-vcl-snippets-deployment

It would be theoretically possible for us to automatically make the snippet available to Fastly by writing it to var/vcl_snippets_custom/deliver_60_allow_bfcache.vcl. We could do that with a composer file map. But even then it would require the user to push updated VCL to Fastly, so it's probably best left to README instructions regardless.

[JaJuMa]

Relying on the Pragma header is a nice idea that could work.
But since Pragma is technically deprecated and its use discouraged by Google (even if some browsers, incl. Chrome, still consider it), maybe we should be careful to depend on it long term. It might just disappear at some point.

The two-snippet approach though, maybe slightly hacky, but imho a smart workaround given Fastly’s limitations.
At least until there’s a cleaner way to hook in before/after their default rule.

Maybe something like this could be worth testing?

Snippet 1: Pre Fastly default rule (priority 40)
Store Magento’s original cacheability intent before Fastly overrides it:

# Preserve Magento's original cacheability intent
if (resp.http.Cache-Control ~ "public") {
    set resp.http.X-MageOS-Bfcache = "public";
} else {
    set resp.http.X-MageOS-Bfcache = "private";
}

Snippet 2: Post Fastly default rule (priority 60)
Apply the final Cache-Control logic for bfcache:

# Adjust final Cache-Control for bfcache logic after Fastly default rule
if (resp.http.X-MageOS-Bfcache == "public") {
    # Allow bfcache: keep no-cache but remove no-store
    set resp.http.Cache-Control = "no-cache, must-revalidate, max-age=0";
} else {
    # Enforce no-store for truly private content
    set resp.http.Cache-Control = "no-store, no-cache, must-revalidate, max-age=0";
}

# Clean up internal marker
unset resp.http.X-MageOS-Bfcache;

By this we can still have it work consistently work with Magento/Varnish default, based on the original Cache-Control: public while avoiding the dependency on HIT logic.

Would be great if someone could give this a spin on a Fastly setup and confirm it behaves as expected.

---

Another alternative/suggestion:
We could detect within the module if Fastly is in use and if so, add an additional, explicit response header
(e.g., X-MageOS-Bfcache: public)
This custom header should reliably pass through to the vcl_deliver stage and would allow to simplify the VCL action down to only one single snippet (like the 2nd one above):

Single Snippet - Post Fastly default rule (priority 60)

# Adjust final Cache-Control for bfcache logic
if (resp.http.X-MageOS-Bfcache == "public") {
    set resp.http.Cache-Control = "no-cache, must-revalidate, max-age=0";
}
# Clean up internal marker
unset resp.http.X-MageOS-Bfcache;

This moves the "hack" from the VCL layer into the PHP module, where it's more self-contained and doesn't require users to manage two separate VCL snippets.
Just a thought for a possible improvement, what do you think @rhoerr or others?

[convenient]

I will try and test the dual fastly snippet approach today, I do have a new site launch this week so am pretty stacked but will try and squeeze it in

[convenient]

@JaJuMa @rhoerr I need a bit more time to test fastly on staging.

I was attempting to do similar to what was suggested with a multi snippet approach

# Fetch priority 1
    if (beresp.http.Cache-Control) {
        set beresp.http.X-Luker-Orig-Cache-Header = beresp.http.Cache-Control;
        if (beresp.http.Cache-Control ~ "public") {
            set beresp.http.X-Luker-Bfcache = "public";
        } else {
            set beresp.http.X-Luker-Bfcache  = "private";
        }
    }

I was having X-Luker-Orig-Cache-Header display odd values or appear missing in some cases. I think I need to gate this so it operates only at the edge node once.

Once I can pass this header through to the "deliver" section then it will be similar to my HIT functionality, and I believe that will work okay. I just need to verify the pingpong between these two VCL snippets and the distributed fastly system.

[rhoerr]

If we have to deal with reliability across Fastly nodes with multiple snippets, and don't want to rely on Pragma, maybe @JaJuMa's suggestion of a Magento-driven header flag for Fastly is the way to go here. That would be reliable, consistent, and easy to configure. The only risk I see is the value reaching the user if the snippet is never configured, but that shouldn't be an issue for security or performance even if so.

[convenient]

@rhoerr TBH I believe I can get the above dual snippet approaching working. I just need a little longer to poke at it. The debug cycle to upload a snippet and wait for it to become active seems to be around 3-4 mins, so its just a bit of a process. I nabbed a few minutes today at EOD, I should be able to do the same tomorrow

[convenient]

@JaJuMa @rhoerr the dual snippet approach worked okay I believe, as a fastly magento2 module user I would be happy with this approach.

Name: bfcache-preserve-public-private
Type: fetch
Priority: 1
----------------------
if (beresp.http.Cache-Control) {
    if (beresp.http.Cache-Control ~ "public") {
        set beresp.http.X-MageOS-Bfcache = "public";
    } else {
        set beresp.http.X-MageOS-Bfcache  = "private";
    }
}

Name: bfcache-remove-ccns
Type: deliver
Priority: 100
----------------------
if (fastly.ff.visits_this_service == 0 && req.restarts == 0) {
    if (resp.http.X-MageOS-Bfcache == "public") {
       set resp.http.Cache-Control = "no-cache, must-revalidate, max-age=0";
    }
}

unset resp.http.X-MageOS-Bfcache;

Test script

Click to view test script

#!/usr/bin/env bash
set -euo pipefail

BASE_URL="${1:-https://mcstaging.example.com/}"
KEY="${2:-t}"

pick_headers() { grep -iE '^(x-cache|cache-control):'; }
line() { printf '%s\n' "------------------------------------------------------------------------------"; }

assert_cache_control() {
  local headers="$1"
  local expectation="${2:-do_not_expect_no_store}"
  local has_no_store="false"
  local expect_no_store="false"

  if [[ "$expectation" == "expect_no_store" ]]; then
    expect_no_store="true"
  fi

  if echo "$headers" | grep -qi "no-store"; then
    has_no_store="true"
  fi

  if [[ "$expect_no_store" == "$has_no_store" ]]; then
    echo "PASS"
  else
    echo "FAIL"
  fi
}

force_hit() {
  local path="${1:-/}"
  local expectation="${2:-do_not_expect_no_store}"
  local token url headers
  token="$(date +%s)"
  url="${BASE_URL%/}/${path#/}"
  curl -sI "${url}?${KEY}=${token}" >/dev/null
  sleep 1
  headers="$(curl -sI "${url}?${KEY}=${token}")"
  echo "HIT: ${url}?${KEY}=${token}"
  printf '%s\n' "$headers" | pick_headers
  assert_cache_control "$headers" "$expectation"
  line
}

force_miss() {
  local path="${1:-/}"
  local expectation="${2:-do_not_expect_no_store}"
  local token url headers
  token="$(date +%s%N)"
  url="${BASE_URL%/}/${path#/}"
  headers="$(curl -sI "${url}?${KEY}=${token}")"
  echo "MISS: ${url}?${KEY}=${token}"
  printf '%s\n' "$headers" | pick_headers
  assert_cache_control "$headers" "$expectation"
  line
}

line
echo "no-store header should not be present"
line

echo "Test homepage cache HIT"
line
force_hit "/" do_not_expect_no_store

echo "Test homepage cache MISS"
line
force_miss "/" do_not_expect_no_store

echo "Test HIT CMS delivery-info"
line
force_hit "/delivery-info" do_not_expect_no_store

echo "Test MISS CMS delivery-info"
line
force_miss "/delivery-info" do_not_expect_no_store

line
echo "no-store header expected"
line

echo "Test 404"
line
force_hit "/this-is-a-404" expect_no_store

echo "Test cart"
line
force_hit "/checkout/cart" expect_no_store

echo "Test customer/account/login"
line
force_hit "/customer/account/login/" expect_no_store

------------------------------------------------------------------------------
no-store header should not be present
------------------------------------------------------------------------------
Test homepage cache HIT
------------------------------------------------------------------------------
HIT: https://mcstaging.example.com/?t=1759912507
x-cache: MISS, MISS, HIT
cache-control: no-cache, must-revalidate, max-age=0
PASS
------------------------------------------------------------------------------
Test homepage cache MISS
------------------------------------------------------------------------------
MISS: https://mcstaging.example.com/?t=1759912508N
x-cache: MISS, MISS, MISS
cache-control: no-cache, must-revalidate, max-age=0
PASS
------------------------------------------------------------------------------
Test HIT CMS delivery-info
------------------------------------------------------------------------------
HIT: https://mcstaging.example.com/delivery-info?t=1759912509
x-cache: MISS, MISS, HIT
cache-control: no-cache, must-revalidate, max-age=0
PASS
------------------------------------------------------------------------------
Test MISS CMS delivery-info
------------------------------------------------------------------------------
MISS: https://mcstaging.example.com/delivery-info?t=1759912511N
x-cache: MISS, MISS, MISS
cache-control: no-cache, must-revalidate, max-age=0
PASS
------------------------------------------------------------------------------
------------------------------------------------------------------------------
no-store header expected
------------------------------------------------------------------------------
Test 404
------------------------------------------------------------------------------
HIT: https://mcstaging.example.com/this-is-a-404?t=1759912511
x-cache: MISS, MISS, MISS
cache-control: no-store, no-cache, must-revalidate, max-age=0
PASS
------------------------------------------------------------------------------
Test cart
------------------------------------------------------------------------------
HIT: https://mcstaging.example.com/checkout/cart?t=1759912513
cache-control: max-age=0, must-revalidate, no-cache, no-store
x-cache: MISS, MISS
PASS
------------------------------------------------------------------------------
Test customer/account/login
------------------------------------------------------------------------------
HIT: https://mcstaging.example.com/customer/account/login/?t=1759912515
cache-control: max-age=0, must-revalidate, no-cache, no-store
x-cache: MISS, MISS, MISS
PASS
------------------------------------------------------------------------------

[convenient]

Updated suggested doc reads well to me @rhoerr