[TreeView] Add lazy loading for children of tree items #15308

noraleonte · 2024-11-06T17:14:43Z

Fix #9687

Preview: https://deploy-preview-15308--material-ui-x.netlify.app/x/react-tree-view/rich-tree-view/lazy-loading/

TO DO:

add JSDocs
add test

mui-bot · 2024-11-06T17:19:10Z

Deploy preview: https://deploy-preview-15308--material-ui-x.netlify.app/

Updated pages:

Generated by 🚫 dangerJS against 999a3b7

oliviertassinari

Cool exploration 👍

oliviertassinari · 2024-11-12T23:32:43Z

docs/data/tree-view/rich-tree-view/lazy-loading/lazy-loading.md

+waiAria: https://www.w3.org/WAI/ARIA/apg/patterns/treeview/
+---
+
+# Rich Tree View - Lazy Loading Children


Suggested change

# Rich Tree View - Lazy Loading Children

# Rich Tree View - Lazy loading children [<span class="plan-pro"></span>](/x/introduction/licensing/#pro-plan 'Pro plan')

Yep I agree, it should be part of the pro plan
I saw the grid discussing moving some lazy loading to the community package, we should just check with them before moving ours to the pro that we aren't doing a mistake, but AFAIK the feature should be rpo

So why have data grid data source MIT but tree view data source Pro?

In the case of the data grid data: data source allows to maximize the open source adoption (get to a fully working prototype sooner, the fastest we get there, the least we give people time to try something else, I don't think people say I have 10h, I will try as many grids as possible, but instead, give a 2hrs budget to the first grid, if they feel too slow, kill it, try the next one, and so one) and even if you have the feature, you can't do much with it, you still need the Pro plan for their features. So MIT.

In the case of the Tree View: I would expect developers to be able to implement lazy loading using React Query (TanStack/query#413) with our MIT plan and not to have a too terrible experience. I assumed that our new docs page is about doing more than this, but looking closer, it's not so clear. So maybe the right move is:

Make the page MIT, since we likely want to have an example using TanStack Query https://npm-stat.com/charts.html?package=react-dom&package=%40tanstack%2Freact-query&from=2023-11-12&to=2024-11-12?

Add that example

Rely on the fact that once people integrate lazy loading, they are more likely to hit a point where they need virtualization (Pro) or need drag & drop (Pro), and we need Lazy loading to be competitive in the MIT space, e.g. https://ant.design/components/tree#tree-demo-dynamic. This is a better 📈 ramp.

We see if there is space for a more complex data fetching logic. Maybe we don't need a data source, if we do, it goes Pro.

Guesses our where a data source could truly shine for cases where:

the simple lazy loading API isn't enough

you are using a data grid data source, you are used to it, and you want the same abstraction

you are going to use virtualization, and you need to be smart about loading

SWR or React Query have too many limitations (we need to be careful to not rebuild those libraries but to be truly complementary). A quick research on what SWR does that fetch doesn't: https://www.reddit.com/r/nextjs/comments/17603jy/comment/k4j3stp/.

docs/data/tree-view/rich-tree-view/lazy-loading/lazy-loading.md

oliviertassinari · 2024-11-12T23:48:42Z

docs/data/tree-view/rich-tree-view/lazy-loading/BasicLazyLoading.tsx

+import { RichTreeView } from '@mui/x-tree-view/RichTreeView';
+import { TreeViewBaseItem } from '@mui/x-tree-view/models';
+
+const fetchData = async (): Promise<


Suggested change

const fetchData = async (): Promise<

async function fetchData(): Promise<

oliviertassinari · 2024-11-12T23:49:57Z

docs/data/tree-view/rich-tree-view/lazy-loading/BasicLazyLoading.tsx

+
+export default function BasicLazyLoading() {
+  return (
+    <Box sx={{ width: '300px' }}>


Suggested change

<Box sx={{ width: '300px' }}>

<Box sx={{ minHeight: 320, minWidth: 300 }}>

oliviertassinari · 2024-11-12T23:54:54Z

docs/data/tree-view/rich-tree-view/lazy-loading/BasicLazyLoading.js

+        treeViewDataSource={{
+          getChildrenCount: (item) => item?.childrenCount,
+          getTreeItems: fetchData,
+        }}


Not super clear to me if:

This should be part of MUI X and not Toolpad. But maybe [data grid] Provide ready to use data sources #14731 is another layer of abstraction, on top of this one

How much code should be shared with the data grid

For now we are following a DX that is close to the one of the grid.
Once we have an initial version fully working, we will see with the grid and the charts if there are things to share with the grid. Sharing DX is obvious (to me at least), sharing code might be more complex since it's very tight to the internals of each set of components.

flaviendelangle · 2024-11-12T08:49:29Z

packages/x-tree-view/src/internals/plugins/useTreeViewItems/useTreeViewItems.types.ts

@@ -9,6 +9,12 @@ export interface TreeViewItemToRenderProps {
  children?: TreeViewItemToRenderProps[];
 }

+export type AddItemsParams<R> = {


Suggested change

export type AddItemsParams<R> = {

export type AddItemsParameters<R> = {

oliviertassinari · 2024-11-13T17:19:05Z

packages/x-tree-view/src/RichTreeView/RichTreeView.tsx

@@ -172,6 +172,7 @@ RichTreeView.propTypes = {
  experimentalFeatures: PropTypes.shape({
    indentationAtItemLevel: PropTypes.bool,
    labelEditing: PropTypes.bool,
+    lazyLoading: PropTypes.bool,


I don't understand this one. If lazing loading can be detected with the prop. What's the advantage of this? We could be doing:

Suggested change

lazyLoading: PropTypes.bool,

and then: unstable_treeViewDataSource, unstable_treeViewDataSourceCache. I thought the point of experimentalFeatures is for things that we can't change with a prop.

I have the same confusion with Next.js: https://nextjs.org/docs/app/building-your-application/upgrading/version-15#serverexternalpackages, I don't understand why they handle this this way.

We went for consistency here, each new major feature is hidden under an experimentalFeature.
For me this has two main benefits:

If you have several props, you don't need to prefix each one of them (plus the API methods, plus the utility hooks etc...)

Users don't have a breaking change when the feature becomes stable. They can remove there experimentalFeature but there app will work fine even if they keep it. With the unstable_ prefix, there is a breaking change that we have to care about (by exporting two version with and without the prefix for some time) and that is still a cost for our users.

Some feature have to be flagged as experimentalFeatures because as you said we can't do it with props (because they are opt-out, like the aggregation). So I prefer to be consistent and not have the user learn a new way of enabling an experimental feature for each new feature.

@flaviendelangle The limits I see:

It's not clear what's stable and what's unstable. This would be clearer

<TreeView experimentalFeatures={{ treeViewDataSource: treeViewDataSourceCache: }}

The terminology is confusing, we call the rest unstable. "experimental" sounds likely that it could get dropped, "unstable" feels like a higher degree of commitment from MUI, more likely to land as stable.

<TreeView unstableFeatures={{ treeViewDataSource: treeViewDataSourceCache: }}

Users don't have a breaking change when the feature becomes stable

The API could change when it becomes stable though. We also don't have to make it a breaking change, we can have deprecations.

The terminology is confusing, we call the rest unstable. "experimental" sounds likely that it could get dropped, "unstable" feels like a higher degree of commitment.

On the other hand, our assumption was that "unstable" was more repelling than "experimental" which is why we called this prop "experimentalFeatures" instead of "unstableFeatures" on the grid.
Both have benefits for sure.

The API could change when it becomes stable though.

But most of the time it doesn't on the experimental features we had in the past.

@oliviertassinari

It's not clear what's stable and what's unstable. This would be clearer

experimentalFeatures={{ treeViewDataSource: treeViewDataSourceCache: }}

I agree, we should add both props that are introduced to the experimentalFeatures list (or unstableFeatures if we decide to rename) 👌.
We weren't sure about the exact dx we wanted here and how similar it should be to the one the grid has when I flagged this as experimental (did it right at the beginning of my exploration). So the inconsistency between what's in experimental features and the names of the props is due to the fact that this is still WIP. 🙈

If we think experimentalFeatures={{ treeViewDataSource: true }} is not clear enough to tell that the cache is experimental as well, I'm fine splitting the two.
I felt like both were close enough to keep them together (which is what the grid did I think) but that's debatable.

React uses both, unstable and experimental https://github.com/facebook/react/blob/main/packages/react/index.js#L63. Confusing perceptions in https://www.linuxquestions.org/questions/debian-26/unstable-vs-experimental-331158/.

No a strong view, but unless there are major cons with this, going like this seems good enough:

stick to a single term in code, and in docs, it feels unlikely that we can reliability communicate: unstable > experimental or unstable < experimental.

we could introduce a pre GA prefix, effective what we use unstable_ for today so that we can then use unstable for really unstable things like AI integration (we might kill it). But do we really need this level of precision/complexity? Until we have too many examples where critical feedback isn't left on the features during the non stable phase and is only left after stable, no, let's be lazy.

with a single term, use unstable because it's shorter, a bit closer to what React does?

prefer to flatten unstable props rather than grouping them, our API docs generation struggles in documenting nested props. Not sure about the value of grouping in the first place, I don't see why Next.js group them.

have deprecations when promoting a unstable API to stable.

make all components follow the same policy, Base UI, etc.

oliviertassinari · 2024-11-13T17:31:53Z

docs/data/tree-view/rich-tree-view/lazy-loading/lazy-loading.md

+waiAria: https://www.w3.org/WAI/ARIA/apg/patterns/treeview/
+---
+
+# Rich Tree View - Lazy Loading Children


So why have data grid data source MIT but tree view data source Pro?

In the case of the data grid data: data source allows to maximize the open source adoption (get to a fully working prototype sooner, the fastest we get there, the least we give people time to try something else, I don't think people say I have 10h, I will try as many grids as possible, but instead, give a 2hrs budget to the first grid, if they feel too slow, kill it, try the next one, and so one) and even if you have the feature, you can't do much with it, you still need the Pro plan for their features. So MIT.

In the case of the Tree View: I would expect developers to be able to implement lazy loading using React Query (TanStack/query#413) with our MIT plan and not to have a too terrible experience. I assumed that our new docs page is about doing more than this, but looking closer, it's not so clear. So maybe the right move is:

Make the page MIT, since we likely want to have an example using TanStack Query https://npm-stat.com/charts.html?package=react-dom&package=%40tanstack%2Freact-query&from=2023-11-12&to=2024-11-12?

Add that example

Rely on the fact that once people integrate lazy loading, they are more likely to hit a point where they need virtualization (Pro) or need drag & drop (Pro), and we need Lazy loading to be competitive in the MIT space, e.g. https://ant.design/components/tree#tree-demo-dynamic. This is a better 📈 ramp.

We see if there is space for a more complex data fetching logic. Maybe we don't need a data source, if we do, it goes Pro.

Guesses our where a data source could truly shine for cases where:

the simple lazy loading API isn't enough

you are using a data grid data source, you are used to it, and you want the same abstraction

you are going to use virtualization, and you need to be smart about loading

SWR or React Query have too many limitations (we need to be careful to not rebuild those libraries but to be truly complementary). A quick research on what SWR does that fetch doesn't: https://www.reddit.com/r/nextjs/comments/17603jy/comment/k4j3stp/.

…g-poc

github-actions · 2024-11-15T14:05:18Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

…g-poc

github-actions · 2024-12-13T17:30:10Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

…g-poc

github-actions · 2024-12-23T08:09:20Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

…g-poc

github-actions · 2024-12-29T06:17:38Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

…g-poc

github-actions · 2025-01-20T11:13:16Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

…g-poc

github-actions · 2025-01-27T06:24:30Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

…g-poc

github-actions · 2025-02-03T11:04:36Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

flaviendelangle

Sorry for the huge delay 😢

docs/package.json

docs/data/data-grid/list-view/components/ListCell.js

flaviendelangle · 2025-02-06T07:43:46Z

docs/data/tree-view/rich-tree-view/editing/editing.md

+
+## Editing lazy loaded children
+
+In order to register changes for item labels on your server or update them in the cache use the `onItemLabelChange` callback function. Visit the dedicated page for [lazy loading](/x/react-tree-view/rich-tree-view/lazy-loading/#lazy-loading-and-label-editing) to read more.


Suggested change

In order to register changes for item labels on your server or update them in the cache use the `onItemLabelChange` callback function. Visit the dedicated page for [lazy loading](/x/react-tree-view/rich-tree-view/lazy-loading/#lazy-loading-and-label-editing) to read more.

In order to register changes for item labels on your server or update them in the cache use the `onItemLabelChange()` callback function.

See [Rich Tree View Pro—Lazy Loading Children]((/x/react-tree-view/rich-tree-view/lazy-loading/#lazy-loading-and-label-editing) for more details.

And I would just add a small code snippet between the two sentences of the prop usage.
Something super basic to make this section sufficient for basic use cases

flaviendelangle · 2025-02-06T07:44:28Z

docs/data/tree-view/rich-tree-view/lazy-loading/BasicLazyLoading.tsx

+  childrenCount?: number;
+}>;
+
+function valuetext(value: number) {


Suggested change

function valuetext(value: number) {

function getSliderAriaValueText(value: number) {

flaviendelangle · 2025-02-06T07:52:17Z

packages/x-data-grid-pro/src/hooks/features/dataSource/utils.ts

-        loading: {
-          ...state.dataSource.loading,
-          ...loadingIds,
+    this.api.setState((state) => {


Not related to this PR

flaviendelangle · 2025-02-06T08:29:43Z

packages/x-tree-view/src/internals/plugins/useTreeViewItems/useTreeViewItems.tsx

    itemMetaLookup[id] = {
      id,
      label,
      parentId,
      idAttribute: undefined,
-      expandable: !!item.children?.length,
+      expandable: !!item.children?.length || isItemExpandable,


Would it make sense to do the following?

Suggested change

expandable: !!item.children?.length || isItemExpandable,

expandable: getChildrenCount ? (getChildrenCount(item) > 0) : !!item.children?.length,

flaviendelangle · 2025-02-06T08:31:23Z

packages/x-tree-view/src/internals/plugins/useTreeViewItems/useTreeViewItems.tsx

+    [store],
+  );
+
+  const setTreeViewLoading = (isLoading: boolean) => {


useEventCallback on all the new API methods 🙏

flaviendelangle · 2025-02-06T08:32:58Z

packages/x-tree-view/src/internals/plugins/useTreeViewItems/useTreeViewItems.tsx

+      getItemId: params.getItemId,
+      getItemLabel: params.getItemLabel,
+    }),
+    loading: false,


Can we move loading and error inside updateItemsState so that it handles the full state?

flaviendelangle · 2025-02-06T08:35:32Z

packages/x-tree-view/src/internals/plugins/useTreeViewItems/useTreeViewItems.tsx

+    getChildrenCount,
+  }: AddItemsParameters<TreeViewBaseItem>) => {
+    if (items) {
+      const newState = updateItemsState({


I feel like we should slit updateItemsState into 2:

A new method that handles the shared logic that you use in updateItemsState and in addItems

updateItemsState that is only used for the already existing use cases.

Because the method stats to be quite complex 😬

flaviendelangle · 2025-02-06T08:36:37Z

packages/x-tree-view/tsconfig.json

-  "include": ["src/**/*"]
+  "include": [
+    "src/**/*",
+    "../x-tree-view-pro/src/internals/plugins/useTreeViewLazyLoading/useTreeViewLazyLoading.types.ts"


😬 This dependency is really not great

…g-poc

github-actions · 2025-02-07T17:13:56Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

noraleonte added 7 commits October 28, 2024 17:42

add getChildrenCount

0918085

wip

45779ba

wip

a29e2d0

wip

765c2a5

wip

7a3fad1

docs wip

80bcc27

wip docs

3e1e022

noraleonte added the component: tree view TreeView, TreeItem. This is the name of the generic UI component, not the React module! label Nov 6, 2024

noraleonte self-assigned this Nov 6, 2024

oliviertassinari added the plan: Pro Impact at least one Pro user label Nov 12, 2024

oliviertassinari reviewed Nov 12, 2024

View reviewed changes

flaviendelangle reviewed Nov 13, 2024

View reviewed changes

oliviertassinari reviewed Nov 13, 2024

View reviewed changes

noraleonte added 2 commits November 14, 2024 14:09

wip

ea5a955

Merge branch 'master' of github.com:noraleonte/mui-x into lazy-loadin…

cf18e16

…g-poc

github-actions bot added the PR: out-of-date The pull request has merge conflicts and can't be merged label Nov 15, 2024

noraleonte added 12 commits November 20, 2024 15:56

Merge branch 'master' of github.com:noraleonte/mui-x into lazy-loadin…

ef26a40

…g-poc

add selector

1aff38e

wip

5b9f5f8

wip

addcdb8

fix expansion and adjust initial state

c99a9dd

add loading and error for tree view

620578e

update error demo

26ebc5a

Merge branch 'master' of github.com:noraleonte/mui-x into lazy-loadin…

0d54f50

…g-poc

rerun selection if needed

aeefe05

check usability of dnd

081d8a3

rename treeViewDataSource -> dataSource

c31a456

fix initial state and error demos

ce5f0c5

Merge branch 'master' of github.com:noraleonte/mui-x into lazy-loadin…

a9bcc3d

…g-poc

github-actions bot removed the PR: out-of-date The pull request has merge conflicts and can't be merged label Dec 16, 2024

noraleonte changed the title ~~[POC] Add Lazy loading for children of tree items~~ [TreeView] Add lazy loading for children of tree items Dec 19, 2024

github-actions bot added the PR: out-of-date The pull request has merge conflicts and can't be merged label Dec 23, 2024

Merge branch 'master' of github.com:noraleonte/mui-x into lazy-loadin…

25ce29d

…g-poc

github-actions bot removed the PR: out-of-date The pull request has merge conflicts and can't be merged label Dec 23, 2024

github-actions bot added the PR: out-of-date The pull request has merge conflicts and can't be merged label Dec 29, 2024

Merge branch 'master' of github.com:noraleonte/mui-x into lazy-loadin…

70581b0

…g-poc

github-actions bot removed the PR: out-of-date The pull request has merge conflicts and can't be merged label Jan 7, 2025

github-actions bot added the PR: out-of-date The pull request has merge conflicts and can't be merged label Jan 20, 2025

noraleonte requested a review from flaviendelangle January 20, 2025 11:17

Merge branch 'master' of github.com:noraleonte/mui-x into lazy-loadin…

8c96d04

…g-poc

github-actions bot removed the PR: out-of-date The pull request has merge conflicts and can't be merged label Jan 20, 2025

github-actions bot added the PR: out-of-date The pull request has merge conflicts and can't be merged label Jan 27, 2025

Merge branch 'master' of github.com:noraleonte/mui-x into lazy-loadin…

476dc72

…g-poc

github-actions bot added PR: out-of-date The pull request has merge conflicts and can't be merged and removed PR: out-of-date The pull request has merge conflicts and can't be merged labels Jan 27, 2025

flaviendelangle reviewed Feb 6, 2025

View reviewed changes

noraleonte added 2 commits February 6, 2025 14:05

Merge branch 'master' of github.com:noraleonte/mui-x into lazy-loadin…

ec1c2e7

…g-poc

update query library

651127a

github-actions bot removed the PR: out-of-date The pull request has merge conflicts and can't be merged label Feb 6, 2025

remove random line

999a3b7

github-actions bot added the PR: out-of-date The pull request has merge conflicts and can't be merged label Feb 7, 2025

	# Rich Tree View - Lazy Loading Children
	# Rich Tree View - Lazy loading children [<span class="plan-pro"></span>](/x/introduction/licensing/#pro-plan 'Pro plan')

	const fetchData = async (): Promise<
	async function fetchData(): Promise<

	<Box sx={{ width: '300px' }}>
	<Box sx={{ minHeight: 320, minWidth: 300 }}>

	export type AddItemsParams<R> = {
	export type AddItemsParameters<R> = {


		## Editing lazy loaded children

		In order to register changes for item labels on your server or update them in the cache use the `onItemLabelChange` callback function. Visit the dedicated page for [lazy loading](/x/react-tree-view/rich-tree-view/lazy-loading/#lazy-loading-and-label-editing) to read more.

	In order to register changes for item labels on your server or update them in the cache use the `onItemLabelChange` callback function. Visit the dedicated page for [lazy loading](/x/react-tree-view/rich-tree-view/lazy-loading/#lazy-loading-and-label-editing) to read more.
	In order to register changes for item labels on your server or update them in the cache use the `onItemLabelChange()` callback function.
	See [Rich Tree View Pro—Lazy Loading Children]((/x/react-tree-view/rich-tree-view/lazy-loading/#lazy-loading-and-label-editing) for more details.

	function valuetext(value: number) {
	function getSliderAriaValueText(value: number) {

	expandable: !!item.children?.length \|\| isItemExpandable,
	expandable: getChildrenCount ? (getChildrenCount(item) > 0) : !!item.children?.length,

[TreeView] Add lazy loading for children of tree items #15308

Are you sure you want to change the base?

[TreeView] Add lazy loading for children of tree items #15308

Conversation

noraleonte commented Nov 6, 2024 • edited Loading

mui-bot commented Nov 6, 2024 • edited Loading

oliviertassinari left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

flaviendelangle Nov 13, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oliviertassinari Nov 13, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oliviertassinari Nov 14, 2024 • edited Loading

Choose a reason for hiding this comment

flaviendelangle Nov 14, 2024 • edited Loading

Choose a reason for hiding this comment

noraleonte Nov 14, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oliviertassinari Nov 14, 2024 • edited Loading

Choose a reason for hiding this comment

oliviertassinari Nov 13, 2024 • edited Loading

Choose a reason for hiding this comment

github-actions bot commented Nov 15, 2024

github-actions bot commented Dec 13, 2024

github-actions bot commented Dec 23, 2024

github-actions bot commented Dec 29, 2024

github-actions bot commented Jan 20, 2025

github-actions bot commented Jan 27, 2025

github-actions bot commented Feb 3, 2025

flaviendelangle left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Feb 7, 2025

noraleonte commented Nov 6, 2024 •

edited

Loading

mui-bot commented Nov 6, 2024 •

edited

Loading

flaviendelangle Nov 13, 2024 •

edited

Loading

oliviertassinari Nov 13, 2024 •

edited

Loading

oliviertassinari Nov 14, 2024 •

edited

Loading

flaviendelangle Nov 14, 2024 •

edited

Loading

noraleonte Nov 14, 2024 •

edited

Loading

oliviertassinari Nov 14, 2024 •

edited

Loading

oliviertassinari Nov 13, 2024 •

edited

Loading