Cache and reuse render resources for external images

[nithinp7]

@sanjeetsuhag brought to my attention that CesiumJS saw some significant performance improvements in tilesets outputted by the CDB to 3D Tiles converter by introducing an in-memory cache for external images. Apparently the converter makes extensive use of external images and often shares the same external images amongst several tiles. I have particularly noticed frequent freezes in Cesium for Unreal / Cesium Native that can last multiple seconds when viewing the Aden, Yemen tileset (an OGC CDB dataset converted to 3D Tiles Next), with excessive texture memory being the almost guaranteed culprit. We should modify our image API to make it possible to cache external image renderer resources.

[nithinp7]

This came up in a recent forum post and I thought some more about the details:

Currently, resource request urls (e.g., external image urls) are looked-up in the cache before making a network request - this means when there are multiple tiles using the same image, they should currently only be making a single network request and later reusing the response data from the cache. We could go further by adding a “resource table lookup” before checking the cache, which checks if the image is actually already loaded and resident in GPU memory. If so, we can give the new tile a shared reference to the already loaded image resource. When there are no tiles referencing the shared image, the reference count should go to 0 and it should be removed from the resource table and evicted from GPU memory.

There are a lot of implementation details to sort out with this though. Resource ownership has to be decoupled from the lifetime of individual tiles. We need to restructure the image representations in Cesium Native to accommodate the fact that they can be external and “shared” (currently we assume they are embedded in the gltf after loading). There might be tricky corner cases like tiles referencing the same image but disagreeing on sampler filter modes, wrapping, etc, it might be unclear what the shared image should look like in such a situation.

And I'll mention again that CesiumJS has something like this, so it's worth taking inspiration from their approach.

[kring]

when there are multiple tiles using the same image, they should currently only be making a single network request and later reusing the response data from the cache

I suspect this won't actually work when two tiles request the same image at close to the same time. They'll both do a cache lookup, both get a cache miss, and both start a real network request. It's possible Unreal's HttpManager will be smart enough to avoid the redundant request, but I wouldn't count on it.

[nithinp7]

They'll both do a cache lookup, both get a cache miss, and both start a real network request

@kring Hm I think you're right. Maybe the resource table (which should get checked before the cache) could contain "future" resources. On a resource table "miss", a shared future resource could be inserted into the table. That way, other tiles requesting the same resource will just build continuations on the existing future in the table instead of issuing their own network request, even before the network request resolves.

[kring]

Yeah something like that makes sense to me. Might be sufficient and more performant to keep the record of in-progress requests in memory rather than in the DB.

[nithinp7]

Yup the resource table does not have to be in the DB, it's just a map of currently resident (or soon to be resident) resources.

[olivierpascal]

In my case, glTF tiles are embedding textures and do not use any external images. Ideally, they probably should share same external images but I don't know how to do that through Ion (if that's even possible). Anyway.

Could we imagine, and how hard would be to implement a basic resource table lookup in cesium-unreal::CesiumGltfComponent, without touching to cesium-native for the moment, as @nithinp7 described:

adding a “resource table lookup” (...), which checks if the image is actually already loaded and resident in GPU memory. If so, we can give the new tile a shared reference to the already loaded image resource. When there are no tiles referencing the shared image, the reference count should go to 0 and it should be removed from the resource table and evicted from GPU memory.

That would saves me. At least, any information that would lead me in the right direction would be extremely helpful to me. I spent a good amount of time trying to understand cesium-unreal::CesiumGltfComponent but the road is still long!

[kring]

@olivierpascal if each tile has an embedded texture, then what resource are you trying to share? Cesium for Unreal should already only be loading such embedded images exactly once.

[olivierpascal]

@kring let's say for example I have a million buildings using the same facade texture. My asset is consisting of one CityGML file, referencing a single external image file. When I import it through Cesium Ion, my asset is tiled and each b3dm tile is embedding the same texture data. Ideally all tiles should reference the same external image file, but I don't know if that's even possible. Again, anyway.

You're right, Cesium for Unreal is loading the same texture (+ mipmaps) in the GPU exactly once per loaded tile. But at the end, as there are a lot of tiles, the GPU memory is flooded by thousands (or more) copies of the same texture (+ mipmaps).

So, the first level of optimization I see would be a basic resource table lookup in the glTF component of Cesium for Unreal to be able to share the same texture in GPU, which would already be a huge improvement. From what I saw, b3dm files does not contains any texture identifier (like the name of the original external referenced file) that could be used to deduplicate embedded textures. So I tried to use the image size as lookup key (yes, this is dirty because it could theoretically exist some collisions but it should do the job). But I struggle at some points.

The second level of optimization would be the ability in the Ion tiling pipeline to share external referenced files, if that's possible (but from what I read in the Cesium for Unreal source code, this is possible). That would also be a huge improvement in term of data size. Then it would be possible to extend and generalize the resource table lookup to external images.

I hope that it's more clear, and that I did not say too many nonsense.

[kring]

@olivierpascal I under the use-case with external images, but you said "In my case, glTF tiles are embedding textures and do not use any external images." So I don't think I understand your use case.

[olivierpascal]

I've edited my comment to precise my use-case. Is that more clear now?

[kring]

I think I understand better after your edit. It sounds like the problem is that Cesium ion is embedding the same texture in every tile, right? Which would indicate a problem in Cesium ion, but I don't think we can paper over that in Cesium for Unreal for the reasons you mentioned. Reliably de-duplicating embedded textures would be expensive, and even then we'd still waste time downloading the duplicated data. Better to fix the problem at the source.

[olivierpascal]

@olivierpascal I under the use-case with external images, but you said "In my case, glTF tiles are embedding textures and do not use any external images." So I don't think I understand your use case.

I have no hand on the Cesium Ion tiling pipeline which is embedding my external images into the glTF tiles. Even if it wasn't the case, the problem of duplicate textures loaded in GPU would be the same.

[olivierpascal]

I think I understand better after your edit. It sounds like the problem is that Cesium ion is embedding the same texture in every tile, right? Which would indicate a problem in Cesium ion, but I don't think we can paper over that in Cesium for Unreal for the reasons you mentioned. Reliably de-duplicating embedded textures would be expensive, and even then we'd still waste time downloading the duplicated data. Better to fix the problem at the source.

You are right. I was just looking for a quick fix I could eventually do myself right now at my level because I'm a little bit stucked because of that.

[olivierpascal]

@kring let's say for example I have a million buildings using the same facade texture. My asset is consisting of one CityGML file, referencing a single external image file.

In reality this is 20x worst because I use a set of ~20 facade textures for all my dataset. So at worst, a tile can embed up to 20 textures. And independently of this tiling problem, this is anyway 20x more duplicate textures uploaded to the GPU. Which make my GPU memory usage to sky rocket overflow.

[kring]

Another request on the forum: https://community.cesium.com/t/optimizing-texture-loads-in-tilesets/23707

[javagl]

It's probably already on the radar, but a very simple, artificial, focussed test data set for this could be the one from CesiumGS/cesium#11897

[kring]

This was implemented in #926.