./wpt run firefox runs Firefox release instead of Nightly

[mbrodesser-Igalia]

According to ./wpt run --help, Nightly is the default channel.

The same happens when explicitly setting the channel to nightly:

./wpt run --channel=nightly --webdriver-arg="-vv" --log-mach - --log-mach-level debug firefox trusted-types/block-string-assignment-to-Element-setAttribute.html 
 0:00.00 INFO Set tests_root to /home/mirko/work/code/wpt
 0:00.00 INFO Set metadata_root to /home/mirko/work/code/wpt
 0:00.00 INFO Set manifest_update to True
 0:00.00 INFO Set manifest_download to True
 0:00.02 INFO Can't find certutil, certificates will not be checked.
Consider installing certutil via your OS package manager or directly.
 0:00.02 INFO Using webdriver binary /snap/bin/geckodriver
 0:00.18 WARNING Supplied channel doesn't match binary, using supplied channel
 0:00.18 INFO Using cached test prefs from /home/mirko/work/code/wpt/_venv3/profiles/nightly/122.0
 0:00.18 INFO Running in headless mode, pass --no-headless to disable
 0:00.47 vcs DEBUG git rev-parse --show-toplevel
 0:00.47 vcs DEBUG git rev-parse --show-cdup
 0:00.47 vcs DEBUG git rev-parse HEAD
 0:02.36 wptserve INFO Starting http server on http://127.0.0.1:8003
 0:02.36 wptserve INFO Starting http server on http://127.0.0.1:8001
 0:02.36 wptserve INFO Starting http server on http://127.0.0.1:8002
 0:02.36 wptserve INFO Starting http server on http://127.0.0.1:8000
 0:02.36 wptserve INFO Starting https server on https://127.0.0.1:8443
 0:02.37 wptserve INFO Create socket on: (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 8889))
 0:02.37 wptserve INFO Bind on: (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 8889))
 0:02.37 wptserve INFO Listen on: (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 8889))
 0:02.40 wptserve INFO Starting http2 server on https://127.0.0.1:9000
 0:02.41 wptserve INFO Starting https server on https://127.0.0.1:8445
 0:02.41 wptserve INFO Starting https server on https://127.0.0.1:8444
 0:02.41 wptserve INFO Starting https server on https://127.0.0.1:8446
 0:02.41 wptserve INFO Create socket on: (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 8888))
 0:02.41 wptserve INFO Bind on: (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 8888))
 0:02.41 wptserve INFO Listen on: (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 8888))
 0:02.44 wptserve INFO Starting WebTransport over HTTP/3 server on 127.0.0.1:11000
 0:03.35 SUITE_START: web-platform-test - running 1 tests
 0:03.35 INFO Using 1 child processes
 0:03.36 DEBUG Dispatch start_init
 0:03.36 DEBUG Got new test group subsuite: test_type:testharness
 0:03.36 DEBUG new state: initializing
 0:03.36 DEBUG Dispatch init
 0:03.38 INFO --certutil-binary not supplied; Firefox will not check certificates
 0:03.38 DEBUG Init called, starting browser and runner
 0:03.38 DEBUG Starting browser with settings {'check_leaks': False, 'lsan_disabled': False, 'lsan_allowed': set(), 'lsan_max_stack_depth': None, 'mozleak_allowed': False, 'mozleak_thresholds': False, 'special_powers': None}
 0:03.38 DEBUG Starting Firefox
 0:03.38 INFO Application command: /usr/bin/firefox --marionette about:blank -profile /tmp/tmpgjy6kv3i

[mirko@mirko-ThinkPad-P1-Gen-5 wpt (TrustedTypesSpec_425_import_srcdoc_attr_node_which_throws)]$ which firefox
/usr/bin/firefox
[mirko@mirko-ThinkPad-P1-Gen-5 wpt (TrustedTypesSpec_425_import_srcdoc_attr_node_which_throws)]$ which firefox-nightly 
/usr/local/bin/firefox-nightly
[mirko@mirko-ThinkPad-P1-Gen-5 wpt (TrustedTypesSpec_425_import_srcdoc_attr_node_which_throws)]$

CC @whimboo

[whimboo]

@jgraham could you please check what's wrong here on Linux?

[jgraham]

Right, it doesn't really know how to find different binaries for different Firefox channels, and I don't really know how we'd solve that. We could make WARNING Supplied channel doesn't match binary, using supplied channel into a fatal error instead, but it would still be on the user to provide a --binary argument pointing to the correct Firefox to use (or to use --install-browser to download a one-off binary).

[mbrodesser-Igalia]

A fatal error would be better than the current state, since especially in headless mode, one might not notice that release instead of Nightly was used.