Simplify FuzzySearch test (avoid dependency on /usr/share/dict/words)

[jhrcek]

Trying to address #4530
Saw this in Haskell Weekly newsletter 😄

[fendor]

@jhrcek Thanks for taking care of this! You should only have to add this in test.yml. The bindist and release.yaml jobs are only relevant for the release.

[jhrcek]

Ok, /usr/share/dict/words file is now present on ubuntu in CI.
What remains now is to fix (stabilize?) those tests 😄

[fendor]

Didn't expect that 😯 Welp, so sounds like the reference impl is not the same as the FuzzySearch impl...

[jhrcek]

I can reproduce that failure locally
cabal test ghcide-tests --test-options='-p"/match works as expected on the english dictionary/"'

Seems to fail quite often and every time there seems to be uppercase letters involved.

I don't know how it's supposed to work, but the main implementation seems to expect that the first parameter consists of lowercase letters (with possible exception of the first letter):

haskell-language-server/ghcide/src/Text/Fuzzy/Parallel.hs

Line 32 in fcf8a7f

match :: T.Text -- ^ Pattern in lowercase except for first character

but there are many words in the words file that have uppercase letters..

[jhrcek]

Now it's clear that reference impl. doesn't agree with the parallel impl.
Sometimes they assign different scores (reference results on the left, parallel on the right):

Input:  "Elys\233e"
[("Elys\233e",246),("Elys\233e's",246)] /= [("Elys\233e",501),("Elys\233e's",501)]

Sometimes reference doesn't return any matches:

Input: "TelePrompte"
          [] /= [("TelePrompter",8177),("TelePrompter's",8177)]
          
Input:  "HR"
          [] /= [("HR",10),("HRH",10)]

[Bodigrim]

Sometimes they assign different scores (reference results on the left, parallel on the right):

That's probably expected, because the "parallel" implementation cuts corners with regards to non-ASCII characters.

Sometimes reference doesn't return any matches:

These violate a precondition that only the first character might be upper case.

[jhrcek]

@Bodigrim you seem to have some context on this 😸 (EDIT: I see you implemented the parallel yourself in #2639). Would you say it would make more sense to

1. filter out the words from /usr/share/dict/words that don't satisfy the precondition or that contain non-ascii characters (or alternatively map non-first chars of each word to lowercase / filter out the non-ascii chars from each word)?
2. Invest time into making the reference impl. to be have more closely to the parallel impl?
3. Disable this flaky unit test, because it doesn't add too much value / adds too much time to CI?

[Bodigrim]

I'd put a few unit tests to check that HLS fuzzy search can order a few alternatives in a reasonable way, but that's it. The reference implementation is apparently fragile as are additional system dependencies. Ditch them, there is bigger fish to fry.

[jhrcek]

I removed all the stuff using words and replaced if with few unit tests + a small property test.
Let me know if you think some more complicated scenarios should be tested.

[fendor]

LGTM, thanks!