Core: Migrate from `qs` to `picoquery` #28315

43081j · 2024-06-23T00:11:48Z

Moves the manager-api, preview-api and ui/manager to use picoquery instead of qs - a much smaller and faster alternative.

Note that we still have qs in our overall monorepo tree, but that can be tackled in a follow-up PR.

Checklist for Contributors

Testing

The changes in this PR are covered in the following automated tests:

stories
unit tests (existing tests should suffice)
integration tests
end-to-end tests

Manual testing

Existing stories and tests should continue to function correctly.

Documentation

Add or update documentation reflecting your changes
If you are deprecating/removing a feature, make sure to update
MIGRATION.MD

Checklist for Maintainers

When this PR is ready for testing, make sure to add ci:normal, ci:merged or ci:daily GH label to it to run a specific set of sandboxes. The particular set of sandboxes can be found in code/lib/cli/src/sandbox-templates.ts
Make sure this PR contains one of the labels below:
Available labels
- bug: Internal changes that fixes incorrect behavior.
- maintenance: User-facing maintenance tasks.
- dependencies: Upgrading (sometimes downgrading) dependencies.
- build: Internal-facing build tooling & test updates. Will not show up in release changelog.
- cleanup: Minor cleanup style change. Will not show up in release changelog.
- documentation: Documentation only changes. Will not show up in release changelog.
- feature request: Introducing a new feature.
- BREAKING CHANGE: Changes that break compatibility in some way with current major version.
- other: Changes that don't fit in the above categories.

nx-cloud · 2024-06-23T00:24:40Z

☁️ Nx Cloud Report

CI is running/has finished running commands for commit 26659e4. As they complete they will appear below. Click to see the status, the terminal output, and the build insights.

📂 See all runs for this CI Pipeline Execution

✅ Successfully ran 1 target

nx run-many -t build --parallel=3

Sent with 💌 from NxCloud.

Moves the `manager-api`, `preview-api` and `ui/amanger` to use `picoquery` instead of `qs` - a much smaller and faster alternative. Note that we still have `qs` in our overall monorepo tree, but that can be tackled in a follow-up PR.

shilman · 2024-06-23T14:28:33Z

Looks like there are a few extra qs references that are causing linting to fail!

43081j · 2024-06-23T14:38:16Z

whoops my mistake! i forgot to run --task check i think

looks like you got it building though. let me know if you need any changes or help

shilman · 2024-06-23T14:40:00Z

I swapped one of them over, but didn't go the extra distance to do the other two, which are a little more involved 🙈

43081j · 2024-06-23T14:42:26Z

no worries, its probably the one i left out on purpose

happy to tackle that in its own PR as long as the rest of the packages build ok

sure means we temporarily have both in the dependency tree but its small enough we'll be ok i think, and i can probably sort it fairly soon

Uses the `foo.bar[0]` syntax picoquery provides, rather than dot-syntax (`foo.bar.0`).

43081j · 2024-06-24T17:28:43Z

note there's still one usage left i need to tackle

i have failing tests locally for it but will get to it at some point 👍

Moves the preview api to use picoquery. Note that tests will not pass yet.

ghengeveld

I'm happy to move from qs to picoquery, but only if it involves no functional changes. Unfortunately, it seems picoquery urlencodes args, which aren't currently encoded. This would be a breaking change and make the URLs look really ugly, so I'm opposed to changing that behavior.

code/lib/preview-api/src/modules/preview-web/UrlStore.ts

code/lib/router/src/utils.test.ts

ghengeveld · 2024-06-25T12:37:15Z

code/lib/router/src/utils.ts

+export const queryFromLocation = (location: Partial<Location>) =>
+  queryFromString(location.search ? location.search.slice(1) : '');


Suggested change

export const queryFromLocation = (location: Partial<Location>) =>

queryFromString(location.search ? location.search.slice(1) : '');

export const queryFromLocation = (location: Partial<Location>) =>

queryFromString(location.search.slice(1));

ghengeveld · 2024-06-25T12:41:59Z

code/ui/manager/src/components/preview/utils/stringifyQueryParams.tsx


 export const stringifyQueryParams = (queryParams: Record<string, string>) =>
-  qs.stringify(queryParams, { addQueryPrefix: true, encode: false }).replace(/^\?/, '&');


I think we need some unit tests to verify stringifyQueryParams doesn't change between the two implementations.

43081j · 2024-06-25T13:25:44Z

I'm happy to move from qs to picoquery, but only if it involves no functional changes. Unfortunately, it seems picoquery urlencodes args, which aren't currently encoded. This would be a breaking change and make the URLs look really ugly, so I'm opposed to changing that behavior.

this is true, however it is doing the right thing here while qs/storybook hasn't been

the URI spec defines it this way, and is why URLSearchParams also encodes them

so we're blocked here i guess. i understand the URLs looked nicer, but they're non-standard and its unlikely pq will ever support that (since it is trying to conform to the standard and be interchangeable with URLSearchParams in those cases)

the dependency tree & package size reduction is what i've been chipping away at for months now. it'd be a shame to back off from that for nice looking URLs, especially given qs is the last piece before we shed a bunch of weight (the last connection to a deep subtree of deps)

alternatively, we could have code inside storybook which literally does replace('%3B', '[') etc. i'm not a fan but at least it doesn't result in veering away from standards in PQ and can live here instead

ghengeveld · 2024-06-25T14:30:07Z

alternatively, we could have code inside storybook which literally does replace('%3B', '[') etc. i'm not a fan but at least it doesn't result in veering away from standards in PQ and can live here instead

That works for me. It's hacky but at least it's consistent. If being noncompliant turns out to be an issue then we can always change it in a major version release (in which case I'd opt for a different/custom array encoding to avoid %5B / %5D).

43081j · 2024-06-25T14:48:29Z

no worries, that works for me too

we already do some transforms on the parsed and stringified queries so it won't be much to add to that

Decodes certain chars we want to keep in our URIs (e.g. `[`).

43081j · 2024-06-25T16:49:05Z

we have some failures ill have to get to at some point another day 👍

basically what to do about invalid keys since pq will deal with them differently than qs did (for perf mostly)

e.g. a[b:val will likely result in {a: []}

code/core/src/router/utils.ts

ndelangen · 2024-09-25T13:52:16Z

code/core/src/preview-api/modules/preview-web/parseArgsParam.test.ts

-    const args = parseArgsParam('key:!date(2001-02-03T04:05:06.789+09:00)');
+    const args = parseArgsParam('key:!date(2001-02-03T04:05:06.789%2B09:00)');


Are all these updated tests not indications that this is backwards incompatible, and we should delay merging this until the next major version @JReinhold ?

I don't agree with this updated format to begin with. + is valid in a URL. Moreover, this is fully client-side URL parsing so we don't even have to adhere to the specs necessarily.

this instance is what I'm commenting on here:
https://github.com/storybookjs/storybook/pull/28315/files#r1763000236

as for the other tests, it looks to me that they were testing that invalid paths were still "working", and it's fine to me that they don't work the same. I don't see how they could possibly be constructed by us, so the user must have manually written an invalid query param, and I don't feel the need to support that at all.

elsewhere, we translate spaces to + so maybe I also missed a place where we need to do the same

…eArgsParam.test.ts

…detect uppercase color prefixes

JReinhold

🎉

it was resolved

43081j · 2024-10-01T07:42:09Z

Great work landing this 🙌

If you run into anything in the pre release, let me know and I'll try help out

JReinhold · 2024-10-01T12:16:22Z

Thanks for the help @43081j ❤️

feat: migrate from qs

050044f

Moves the `manager-api`, `preview-api` and `ui/amanger` to use `picoquery` instead of `qs` - a much smaller and faster alternative. Note that we still have `qs` in our overall monorepo tree, but that can be tackled in a follow-up PR.

43081j force-pushed the pq branch from fd1e618 to 050044f Compare June 23, 2024 09:40

43081j marked this pull request as ready for review June 23, 2024 10:57

shilman changed the title ~~feat: migrate from qs~~ Core: Migrate from qs to picoquery Jun 23, 2024

shilman added dependencies core ci:normal and removed ci:normal labels Jun 23, 2024

shilman added 2 commits June 23, 2024 21:52

Use destructured imports

2b8156a

qs => picoquery

23f28dc

Merge branch 'next' into pq

a8b3f91

43081j mentioned this pull request Jun 23, 2024

remove qs from storybook es-tooling/ecosystem-cleanup#30

Open

yannbf assigned shilman Jun 24, 2024

feat: use js syntax in query strings

3c04a1e

Uses the `foo.bar[0]` syntax picoquery provides, rather than dot-syntax (`foo.bar.0`).

feat (preview-api): move to picoquery

4474a04

Moves the preview api to use picoquery. Note that tests will not pass yet.

ghengeveld requested a review from ndelangen June 25, 2024 11:50

ghengeveld previously requested changes Jun 25, 2024

View reviewed changes

43081j added 2 commits June 25, 2024 17:20

Merge branch 'next' into pq

c5f7409

fix: decode some known query string chars

fcce3bb

Decodes certain chars we want to keep in our URIs (e.g. `[`).

JReinhold mentioned this pull request Sep 11, 2024

[Tracking] Reduce install footprint 🐾 #29038

Open

Merge branch 'next' into pq

5e9b06e

JReinhold reviewed Sep 17, 2024

View reviewed changes

code/core/src/router/utils.ts Show resolved Hide resolved

JReinhold assigned JReinhold and 43081j and unassigned shilman Sep 17, 2024

Merge branch 'next' into pr/43081j/28315

a770d1a

ndelangen reviewed Sep 25, 2024

View reviewed changes

ndelangen and others added 11 commits September 25, 2024 15:53

rename variable to indicate we're not using qs anymore

a76e060

dedupe

4468a15

Discard changes to code/core/src/preview-api/modules/preview-web/pars…

f8c9172

…eArgsParam.test.ts

fix parseArgsParams and tests

b45ea68

format

099be6b

Merge branch 'next' into pr/43081j/28315

87bc10d

fix

c08542b

Fixes by not re-using query-options meant for args&globals use only

df1aad2

make our color examples lowercase always & fix a bug where we didn't …

6066c64

…detect uppercase color prefixes

fix

1f43562

Merge branch 'next' into pr/43081j/28315

be07741

ndelangen requested a review from JReinhold September 30, 2024 13:59

ndelangen approved these changes Sep 30, 2024

View reviewed changes

JReinhold approved these changes Sep 30, 2024

View reviewed changes

Merge branch 'next' into pq

26659e4

ndelangen enabled auto-merge September 30, 2024 19:32

ndelangen merged commit 148fa0f into storybookjs:next Oct 1, 2024
52 checks passed

github-actions bot mentioned this pull request Oct 1, 2024

Release: Prerelease 8.4.0-alpha.3 #29248

Merged

7 tasks

43081j deleted the pq branch October 1, 2024 07:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Core: Migrate from `qs` to `picoquery` #28315

Core: Migrate from `qs` to `picoquery` #28315

43081j commented Jun 23, 2024 •

edited

Loading

nx-cloud bot commented Jun 23, 2024 •

edited

Loading

shilman commented Jun 23, 2024

43081j commented Jun 23, 2024

shilman commented Jun 23, 2024

43081j commented Jun 23, 2024

43081j commented Jun 24, 2024

ghengeveld left a comment

ghengeveld Jun 25, 2024

ghengeveld Jun 25, 2024

43081j commented Jun 25, 2024 •

edited

Loading

ghengeveld commented Jun 25, 2024

43081j commented Jun 25, 2024

43081j commented Jun 25, 2024

ndelangen Sep 25, 2024

ghengeveld Sep 25, 2024

JReinhold Sep 25, 2024

43081j Sep 27, 2024

JReinhold left a comment

43081j commented Oct 1, 2024

JReinhold commented Oct 1, 2024

		export const queryFromLocation = (location: Partial<Location>) =>
		queryFromString(location.search ? location.search.slice(1) : '');


		export const stringifyQueryParams = (queryParams: Record<string, string>) =>
		qs.stringify(queryParams, { addQueryPrefix: true, encode: false }).replace(/^\?/, '&');

		const args = parseArgsParam('key:!date(2001-02-03T04:05:06.789+09:00)');
		const args = parseArgsParam('key:!date(2001-02-03T04:05:06.789%2B09:00)');

Core: Migrate from qs to picoquery #28315

Core: Migrate from qs to picoquery #28315

Conversation

43081j commented Jun 23, 2024 • edited Loading

Checklist for Contributors

Testing

The changes in this PR are covered in the following automated tests:

Manual testing

Documentation

Checklist for Maintainers

nx-cloud bot commented Jun 23, 2024 • edited Loading

☁️ Nx Cloud Report

shilman commented Jun 23, 2024

43081j commented Jun 23, 2024

shilman commented Jun 23, 2024

43081j commented Jun 23, 2024

43081j commented Jun 24, 2024

ghengeveld left a comment

Choose a reason for hiding this comment

ghengeveld Jun 25, 2024

Choose a reason for hiding this comment

ghengeveld Jun 25, 2024

Choose a reason for hiding this comment

43081j commented Jun 25, 2024 • edited Loading

ghengeveld commented Jun 25, 2024

43081j commented Jun 25, 2024

43081j commented Jun 25, 2024

ndelangen Sep 25, 2024

Choose a reason for hiding this comment

ghengeveld Sep 25, 2024

Choose a reason for hiding this comment

JReinhold Sep 25, 2024

Choose a reason for hiding this comment

43081j Sep 27, 2024

Choose a reason for hiding this comment

JReinhold left a comment

Choose a reason for hiding this comment

43081j commented Oct 1, 2024

JReinhold commented Oct 1, 2024

Core: Migrate from `qs` to `picoquery` #28315

Core: Migrate from `qs` to `picoquery` #28315

43081j commented Jun 23, 2024 •

edited

Loading

nx-cloud bot commented Jun 23, 2024 •

edited

Loading

43081j commented Jun 25, 2024 •

edited

Loading