feat: duckdb-wasm query UI [DRAFT] #39

mattrothenberg · 2022-05-12T17:08:36Z

This PR is a rough pass at #38, providing an interface for querying the current dataset via duckdb-wasm. cc @LoneRifle, I welcome any and all thoughts you may have here.

Todos

Validate whether this "query as you type" UI/UX is helpful or awful. I kind of like that there's no discrete Run Query button
Handle transient DB connection errors that seem to happen when switching tabs
Figure out a UI/UX for toggling between the flat-ui component and the query interface
Hook up the flat-ui component to the results section of the query interface so that we can better visualize results
Make the code editor better (more robust column name and type detection)

src/components/repo-detail.tsx

tailwind.config.js

src/api/index.ts

jaked

really cool!

I don't totally get why it's necessary to include all the duckdb code explicitly in public. I wonder if there is a way to pass through the contents of a package somehow in the build (so use the code in the duckdb package dep instead of duplicating it explicitly); access it via the built code instead of via HTTP (probably hard to do with WASM); or maybe to point to the code on a CDN instead.

I get that this a spike, none of this is criticism, just curiosity.

mattrothenberg · 2022-05-12T17:50:28Z

@jaked Great questions! I can answer a few of these for you.

There are in fact docs for installing the library in a vite environment and they simply did not work for me. I ran into weird, esoteric issues that you can reproduce by following the steps in the linked README.

As for the CDN approach, this is one option. The drawback is that jsdelivr doesn't seem to like connections from localhost:3000, so I couldn't load the bundles that way.

EDIT: this is fixed in 99055d6. Turns out we were using an outdated version of vite which didn't seem to like the URL imports.

vite.config.ts

LoneRifle

@mattrothenberg, thanks for giving me the chance to look at this first cut of the feature, it looks really promising!

Query as you type is very helpful, and the debounce seems to be tuned just nice.

Couple of specific things that I'll comment in the appropriate file locations

src/components/repo-detail.tsx

src/components/db-explorer.tsx

LoneRifle · 2022-05-16T08:25:04Z

cc @ankoh of @duckdb in case the DuckDB team are interested in tracking this feature

ankoh · 2022-05-16T09:24:13Z

@jaked Great questions! I can answer a few of these for you.

There are in fact docs for installing the library in a vite environment and they simply did not work for me. I ran into weird, esoteric issues that you can reproduce by following the steps in the linked README.

As for the CDN approach, this is one option. The drawback is that jsdelivr doesn't seem to like connections from localhost:3000, so I couldn't load the bundles that way.

Maybe I can help here.
DuckDB-Wasm only requires to treat our WebAssembly modules as plain binary assets.
Bundling from npm should be more convenient for future updates.

This PR is exciting! 🚀

Co-authored-by: LoneRifle <[email protected]>

mattrothenberg · 2022-05-16T11:57:05Z

This is wonderful feedback @LoneRifle and @ankoh, thank you both.

Stay tuned as we work through these updates 😎

src/components/db-explorer.tsx

domoritz · 2022-05-16T14:14:51Z

src/components/db-explorer.tsx

+  const execQuery = async (query: string) => {
+    if (!connectionRef.current) return;
+    const queryRes = await connectionRef.current.query(query);
+    const asArray = queryRes.toArray();


You don't need to convert the result to an array. Arrow tables already behave like arrays of objects if you iterate over them for for.

@domoritz Fantastic! This felt kludgy and I suspected there was a better way.

map and forEach won't work so you may have to refactor your code a bit but it would be worth it since you avoid a lot of upfront conversion.

@domoritz Good call. We're ultimately passing this data down to our flat-ui component , so it needs to be in a shape that this component understands (or we need to refactor the component)

You may need to refactor the component. Shouldn't be a big deal, though, and would be great for interoperability anyway.

src/components/db-explorer.tsx

domoritz · 2022-05-16T14:20:36Z

Looks very exciting.

Matt Rothenberg added 3 commits May 12, 2022 10:16

wasm spike successful

fa9d83a

auto fetch

713488f

wip

4127796

mattrothenberg requested review from idan and jaked May 12, 2022 17:08

mattrothenberg self-assigned this May 12, 2022

mattrothenberg changed the title ~~feat: duckdb-wasm query UI~~ feat: duckdb-wasm query UI [DRAFT] May 12, 2022

mattrothenberg marked this pull request as draft May 12, 2022 17:08

mattrothenberg requested a review from Wattenberger May 12, 2022 17:08

remove janky require shim

6f75755

mattrothenberg commented May 12, 2022

View reviewed changes

src/components/repo-detail.tsx Outdated Show resolved Hide resolved

mattrothenberg commented May 12, 2022

View reviewed changes

tailwind.config.js Show resolved Hide resolved

remove vite config changes

c225595

jaked reviewed May 12, 2022

View reviewed changes

src/api/index.ts Outdated Show resolved Hide resolved

jaked reviewed May 12, 2022

View reviewed changes

Matt Rothenberg added 4 commits May 12, 2022 14:20

error handling

12955b2

fix stale transaction error

e0b17f4

pull in grid component

abc4d0b

rudimentary toggle UI

c66c2a2

Wattenberger reviewed May 12, 2022

View reviewed changes

vite.config.ts Show resolved Hide resolved

LoneRifle reviewed May 16, 2022

View reviewed changes

src/components/repo-detail.tsx Outdated Show resolved Hide resolved

src/components/db-explorer.tsx Outdated Show resolved Hide resolved

mattrothenberg and others added 2 commits May 16, 2022 07:48

Update src/components/repo-detail.tsx

54376d9

Co-authored-by: LoneRifle <[email protected]>

refactor

c558164

Matt Rothenberg added 3 commits May 16, 2022 08:02

remove co-located duckdb package

99055d6

use filename as table name

5efc418

handle SQL errors

c848a11

LoneRifle reviewed May 16, 2022

View reviewed changes

src/components/db-explorer.tsx Show resolved Hide resolved

Matt Rothenberg added 3 commits May 16, 2022 09:20

move SQL out to query param

7c5a885

fix: conditionally insert CSV or JSON

93c654f

tweaks

7fff932

domoritz reviewed May 16, 2022

View reviewed changes

Matt Rothenberg added 2 commits May 16, 2022 10:44

switch statement for db initialization

c83b98c

suppress fade animation for SQL results

62bdc9f

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: duckdb-wasm query UI [DRAFT] #39

feat: duckdb-wasm query UI [DRAFT] #39

mattrothenberg commented May 12, 2022 •

edited

Loading

jaked left a comment

mattrothenberg commented May 12, 2022 •

edited

Loading

LoneRifle left a comment

LoneRifle commented May 16, 2022

ankoh commented May 16, 2022 •

edited

Loading

mattrothenberg commented May 16, 2022

domoritz May 16, 2022

mattrothenberg May 16, 2022

domoritz May 16, 2022

mattrothenberg May 16, 2022

domoritz May 16, 2022

domoritz commented May 16, 2022

feat: duckdb-wasm query UI [DRAFT] #39

Are you sure you want to change the base?

feat: duckdb-wasm query UI [DRAFT] #39

Conversation

mattrothenberg commented May 12, 2022 • edited Loading

jaked left a comment

Choose a reason for hiding this comment

mattrothenberg commented May 12, 2022 • edited Loading

LoneRifle left a comment

Choose a reason for hiding this comment

LoneRifle commented May 16, 2022

ankoh commented May 16, 2022 • edited Loading

mattrothenberg commented May 16, 2022

domoritz May 16, 2022

Choose a reason for hiding this comment

mattrothenberg May 16, 2022

Choose a reason for hiding this comment

domoritz May 16, 2022

Choose a reason for hiding this comment

mattrothenberg May 16, 2022

Choose a reason for hiding this comment

domoritz May 16, 2022

Choose a reason for hiding this comment

domoritz commented May 16, 2022

mattrothenberg commented May 12, 2022 •

edited

Loading

mattrothenberg commented May 12, 2022 •

edited

Loading

ankoh commented May 16, 2022 •

edited

Loading