Explicitly dedupe prefetches and prerenders

prefetch.bs

@@ -137,6 +137,10 @@ spec: resource-timing; urlPrefix: https://w3c.github.io/resource-timing/
   type: dfn; for: PerformanceResourceTiming; text: delivery type; url: dfn-delivery-type
 </pre>
 
+<style>
+dfn var { font-style: italic; } /* override bad base.css */
+</style>
+
 <h2 id="concepts">Concepts</h2>
 
 In light of <a href="https://privacycg.github.io/storage-partitioning/">storage partitioning</a>, this specification defines prefetch for navigations which would occur within the same partition (for example, top-level navigations within the same site) and for navigations which would occur in a separate partition (for example, top-level navigations to a different site).
@@ -299,6 +303,47 @@ The user agent may [=prefetch record/cancel and discard=] records from the [=Doc
     <p class="note">The reasoning for setting the cutoff time *after* waiting for a prefetch record to finish is to allow for flexibility in selecting a prefetch to serve the navigation while still guaranteeing falling back to a non-prefetched navigation in the case of repeated prefetch failures. We allow blocking on prefetch attempts which started before we see an attempt fail, but we don't block on subsequent attempts. Notably, this approach: does not finalize the set of prefetches to block on at the start of the navigation; allows a prefetch which started and completed after the navigation started to serve the navigation; avoids the use of a fixed timeout, which would be arbitrary and detrimental to the use of prefetch with slower servers; and blocks on, at most, two nearly-consecutive prefetches before falling back to a conventional navigation.</p>
 </div>
 
+<div algorithm="has a matching prefetch record">
+  A {{Document}} |document| <dfn export>has a matching prefetch record</dfn> given a [=prefetch record=] |recordUnderConsideration| and an optional boolean <dfn for="has a matching prefetch record" export>|checkPrerender|</dfn> (default false) if the following algorithm returns true:
+
+  1. [=list/For each=] |record| of |document|'s [=Document/prefetch records=]:
+    1. If |record|'s [=prefetch record/state=] is "`canceled`", then [=iteration/continue=].
+
+    1. If |checkPrerender| is true and |record|'s [=prefetch record/prerendering traversable=] is null, then [=iteration/continue=].
+
+    1. Let |recordNVS| be null.
+
+    1. If |record|'s [=prefetch record/response=] is not null, then set |recordNVS| to the result of [=obtaining a URL search variance=] given |record|'s [=prefetch record/response=].
+
+    1. Otherwise, set |recordNVS| to |record|'s [=prefetch record/No-Vary-Search hint=].
+
+    1. If |recordUnderConsideration|'s [=prefetch record/No-Vary-Search hint=] is not equal to |recordNVS|, then [=iteration/continue=].
+
+    1. If |recordUnderConsideration|'s [=prefetch record/URL=] and |record|'s [=prefetch record/URL=] are [=equivalent modulo search variance=] given |recordUnderConsideration|'s [=prefetch record/No-Vary-Search hint=], then return true.
+
+  1. Return false.
+
+  <div class="note" id="note-require-equivalent-nvs">
+    <p>The requirement that the No-Vary-Search values be equivalent is somewhat strict. It means that some cases which could theoretically be treated as matching, are not treated as such, and thus redundant prefetches could happen.
+
+    <p>However, allowing more lenient matching makes the check no longer an equivalence relation, and thus prohibits implementation strategies based on normalized URL keys. For this reason, we require equivalent No-Vary-Search values. This is in line with the best practices for server operators demanded in [[NO-VARY-SEARCH#section-6]].
+
+    <p>In practice, we do not expect this to cause redundant prefetches, since server operators and the corresponding speculation rules-writing web developers will follow best practices and use static No-Vary-Search values.
+  </div>
+
+  <div class="example" id="example-non-matching-nvs">
+    <p>Consider three [=prefetch records=]:
+
+    * |A| has a [=prefetch record/URL=] of `https://example.com?a=1&b=1` and a [=prefetch record/No-Vary-Search hint=] parsed from `params=("a")`.
+    * |B| has a [=prefetch record/URL=] of `https://example.com?a=2&b=1` and a [=prefetch record/No-Vary-Search hint=] parsed from `params=("b")`.
+    * |C| has a [=prefetch record/URL=] of `https://example.com?a=2&b=2` and a [=prefetch record/No-Vary-Search hint=] parsed from `params=("a")`.
+
+    <p>With the current definition of [=has a matching prefetch record=], none of these records match each other.
+
+    <p>A definition which did not require equivalent No-Vary-Search values could consider |A| and |B| to match (using |A|'s [=prefetch record/No-Vary-Search hint=]), and |B| and |C| to match (using |B|'s [=prefetch record/No-Vary-Search hint=]). But it could not consider |A| and |C| to match, so it would not be transitive, and thus not an equivalence relation.
+  </div>
+</div>
+
 <div algorithm>
     To <dfn export>create navigation params from a prefetch record</dfn> given a [=navigable=] |navigable|, a [=document state=] |documentState|, a [=navigation id=] |navigationId|, a {{NavigationTimingType}} |navTimingType|, a [=request=] |request|, a [=prefetch record=] |record|, a [=target snapshot params=] |targetSnapshotParams|, and a [=source snapshot params=] |sourceSnapshotParams|, perform the following steps.
 
@@ -666,6 +711,7 @@ The <dfn>list of sufficiently strict speculative navigation referrer policies</d
 <div algorithm>
     To <dfn export>prefetch</dfn> given a {{Document}} |document| and a [=prefetch record=] |prefetchRecord|, perform the following steps.
 
+    1. If |document| [=has a matching prefetch record=] given |prefetchRecord|, then return.
     1. Let |sourceSnapshotParams| be the result of [=snapshotting source snapshot params=] given |document|.
     1. Let |targetSnapshotParams| be the result of [=snapshotting target snapshot params=] given |document|'s [=node navigable=].
     1. Set |prefetchRecord|'s [=prefetch record/source partition key=] to the result of [=determining the network partition key=] given |document|'s [=relevant settings object=].

prerendering.bs

@@ -92,8 +92,11 @@ spec: nav-speculation; urlPrefix: prefetch.html
     text: supports prefetch; url: supports-prefetch
     text: list of sufficiently-strict speculative navigation referrer policies
     text: wait for a matching prefetch record; url: wait-for-a-matching-prefetch-record
+    text: has a matching prefetch record
     for: prefetch record
       text: prerendering traversable
+    for: has a matching prefetch record
+      text: checkPrerender
 spec: RFC8941; urlPrefix: https://www.rfc-editor.org/rfc/rfc8941.html
   type: dfn
     text: structured header; url: #section-1
@@ -291,6 +294,8 @@ Every {{Document}} has an <dfn for="Document">activation start time</dfn>, which
 
      <p class="note">Currently, cross-site prerendering is not specified or implemented, although we have various ideas about how it could work in this repository's explainers.
 
+  1. If |referrerDoc| [=has a matching prefetch record=] given |prefetchRecord| with <i>[=has a matching prefetch record/checkPrerender=]</i> set to true, then return.
+
   1. [=Assert=]: |prefetchRecord|'s [=prefetch record/prerendering traversable=] is "`to be created`".
 
   1. Let |prerenderingTraversable| be the result of [=creating a new top-level traversable=].

speculation-rules.bs

@@ -627,6 +627,28 @@ A <dfn>prerender candidate</dfn> is a [=struct=] with the following [=struct/ite
   <p class="note">A user agent's heuristics for enacting non-eager candidates could incorporate a [=prefetch candidate/No-Vary-Search hint=]. For example, a user hovering over a link whose URL and a candidate's [=prefetch candidate/URL=] are [=equivalent modulo search variance=] given the candidate's [=prefetch candidate/No-Vary-Search hint=] could indicate to the user agent that enacting it would be useful.</p>
 
   Notwithstanding the above, user agents should prioritize user preferences (express and implied, such as a low-data-usage mode) over eagerness expressed by the author.
+
+  <div class="example" id="example-multiple-matching-rules">
+    <p>It's possible for multiple speculation rules to generate "the same" speculative navigation request, represented in the above algorithm by multiple similar candidates. For example, consider
+
+    <pre highlight="json">
+    {
+      "prefetch": [
+        {
+          "urls": ["next.html"]
+        },
+        {
+          "urls": ["next.html"],
+          "referrer_policy": "no-referrer"
+        }
+      ]
+    }
+    </pre>
+
+    <p>Because [=prefetching=] or [=start referrer-initiated prerendering|prerendering=] is a no-op if there is already a matching request in progress, following the above algorithm will generally result in the first rule winning. So in our example, the request will be made with the default referrer policy, not <code>[="no-referrer"=]</code>.</p>
+
+    <p>Because of the "<span class="allow-2119">may</span>" step in the algorithm, user agents could technically instead choose to let the second rule win, by skipping the first candidate for [=implementation-defined=] reasons, but this is expected to be rare. Usually, a user agent would either skip none or all of the matching rules, at least within a given [=prefetch candidate/eagerness=] level.</p>
+  </div>
 </div>
 
 <p class="issue">