Improved UBO/SSBO implementation

[riccardobl]

This fixes some bugs on the current UBO/SSBO implementation and extends/rewrites part of it to be more user friendly.

Usage:

1. Make a class that implements Struct and defines the fields

public static class MyStruct implements Struct<Void>{ 
        public final StructField<Vector3f> a=new StructField<Vector3f>(0 /*position of the field in the struct*/,new Vector3f()/*default value*/);
        public final StructField<Float> b=new StructField<Float>(1,0f);

       @Override public Void get() {return null;}
}

2. Create an instance and set the fields

MyStruct s=new MyStruct();
s.a.getValueForUpdate().set(0,0,0); /* Get the current value and mark for update, this is useful to modify mutable objects*/
s.b.setValue(1f); /* Set value and mark for update*/
//...

3. Create the buffer object

BufferObject  bo=new BufferObject(StructuredBufferSTD140Layout.class);
bo.updateData(s);

4. Define in j3md

MaterialDef TestMat {
    MaterialParameters {    
        UniformBufferObject MyBufferObject
    }
....
}

5. Define in the shader

layout (std140) uniform m_MyBufferObject {
    vec3 a;
    float b;
}

6. Create and configure the material

Material mat=...;
mat.setUniformBufferObject("MyBufferObject",bo);

7. Update the buffer object as many times as you want, it will automatically take care of updating only the fields that changed and the memory used by them

...
bo.updateData(s);
s.b.setValue(2f);
bo.updateData(s);
...

[MeFisto94]

Suggestion about the API:
What if we would get rid of StructField and just use annotations, like:

class MyStruct implements BufferStruct {
    @BufferIndex(index = 0)
    public float a;
    @BufferIndex(index = 1)
    protected Vector3f b;
}

That way you don't need a wrapper or "complex" API if you want to update values.
The downside is that you need to keep track of what has been changed manually, i.e.

bo.updateAllData(s);

bo.updateData(s, 0);
bo.updateData(s, 0, 1); // equivalent to updateAllData in this example

Yes it shifts "responsibility" down to the user to know what values have been changed this frame, but wouldn't the user know anyway?
If s is used globally throughout the game then updating the whole buffer is probably not much worse than differenting, and even then the user could keep track of it manually (e.g. with nullable types)

I just feel accessing fields directly feels a lot more native than having wrapper classes and having to call .get() and .set() on them.
If we really wanted, we could probably use some hashCode on Struct to detect changed fields, given that Struct can be a baseClass and not an interface

[riccardobl]

Mhh.. you would need to remember the order of all the fields for that, the StructField is there to provide an api for someone who is not familiar with the internals.

I believe the lower level should use the StructFields since they provide a more flexible and solid api, however, we could implement those "unwrapped" structs as an abstraction layer on top of it, maybe with a StaticStruct interface that assumes fields will never change and then wrap the fields internally into StructFields with reflection, it should be pretty trivial to extend the StructUtils.getFromClass to do that.

---

Another thing we can do to improve this is getting rid of the get() method that might be confusing, it was originally there as a convenience method to return

BufferObject  bo=new BufferObject(StructuredBufferSTD140Layout.class);
bo.updateData(s);

but i think it probably makes more sense for the developer to do it outside of the "struct", in the code that is binding it to the material, that's why i didn't include it in the small "how to".

[stephengold]

Is this a candidate for v3.5.0?

[MeFisto94]

I think this should essentially be two commits, because the fixes for the Buffer implementation should probably be ported into 3.4 (if we even do a maintenance release, that is), because that's fundamentally broken, but different from a new buffer API. For instance, the parameters that are passed to the methods are just wrong.

Regarding the features of this PR, I've found two more things in my usecase: Arrays and nested objects. Consider:

struct PointLight {
    vec3 Position;
    vec4 Color;
    float Radius;
};

const int NR_POINTLIGHTS = 32;
uniform PointLight m_PointLights[NR_POINTLIGHTS];

A) Would this be one UBO with an array or would this be 32 UBOs of PointLight?
B) What if I had a enclosing struct Light (to save Uniforms, in case that array would otherwise be 32 uniforms or maybe just for organizing reasons), I guess we'd need to make the code flexible enough?

Another thing I thought about though, is: Does a good change detection and updateBufferRange even make sense, considering UBOs are typically smaller than 64 kIB and in the worst case of changing every second (interlaced) element, we'd have one update call per element, instead of just swapping the whole (maximum) 64 kiB at once.

Anyway, other ideas I had were using a reflection proxy together with java records to hook the set methods, but ultimatively comparing the hashcode of every field would be even better, because that way we can, say, set the vector to be camera.getTranslation and it will automatically update the buffer with the latest camera position.

[riccardobl]

Resurrecting this PR.
The complexity of the dirty region detection is there to make the UBOs usable without worrying about performances degradation, with this implementation they are always an improvement over passing separated uniforms to shaders both for performances and readability:

    public static class CameraStruct  implements Struct<Void>{ 
        public final StructField<Vector2f> frustumNearFar=new StructField<Vector2f>(0,new Vector2f());
        public final StructField<Vector2f> resolutionInverse=new StructField<Vector2f>(1,new Vector2f());
        public final StructField<Vector2f> resolution=new StructField<Vector2f>(2,new Vector2f());
        public final StructField<Matrix4f> viewMatrix=new StructField<Matrix4f>(3,new Matrix4f());
        public final StructField<Matrix4f> projectionMatrix=new StructField<Matrix4f>(4,new Matrix4f());
        public final StructField<Matrix4f> viewProjectionMatrix=new StructField<Matrix4f>(5,new Matrix4f());
        public final StructField<Matrix4f> viewMatrixInverse=new StructField<Matrix4f>(6,new Matrix4f());
        public final StructField<Matrix4f> projectionMatrixInverse=new StructField<Matrix4f>(7,new Matrix4f());
        public final StructField<Matrix4f> viewProjectionMatrixInverse=new StructField<Matrix4f>(8,new Matrix4f());
        public final StructField<Vector4f> viewPort=new StructField<Vector4f>(9,new Vector4f());

        public final StructField<Float> aspect=new StructField<Float>(10,0f);

        public final StructField<Vector3f> cameraPosition=new StructField<Vector3f>(11,new Vector3f());
        public final StructField<Vector3f> cameraDirection=new StructField<Vector3f>(12,new Vector3f());
        public final StructField<Vector3f> cameraLeft=new StructField<Vector3f>(13,new Vector3f());
        public final StructField<Vector3f> cameraUp=new StructField<Vector3f>(14,new Vector3f());
        @Override public Void get() {return null;}
    }


    public static class TimerStruct  implements Struct<Void>{
        public final StructField<Float> speed=new StructField<Float> (0,0f);
        public final StructField<Float> time=new StructField<Float> (1,0f);
        public final StructField<Integer> intTime=new StructField<Integer> (2,0);
        public final StructField<Float> tpf=new StructField<Float> (3,0f);
        public final StructField<Float> frameRate=new StructField<Float> (4,0f);
        public final StructField<Vector2f> deltaTime=new StructField<Vector2f> (5,new Vector2f());
        @Override public Void get() {return null;}
    }


    public static class GeometryStruct  implements Struct<Void>{
        public final StructField<Matrix4f> worldMatrix=new StructField<Matrix4f> (0,new Matrix4f());
        public final StructField<Matrix4f> worldViewMatrix=new StructField<Matrix4f>(1,new Matrix4f());
        public final StructField<Matrix3f> normalMatrix=new StructField<Matrix3f>(2,new Matrix3f());
        public final StructField<Matrix3f> worldNormalMatrix=new StructField<Matrix3f>(3,new Matrix3f());
        public final StructField<Matrix4f> worldViewProjMatrix=new StructField<Matrix4f>(4,new Matrix4f());
        public final StructField<Matrix4f> worldMatrixInv=new StructField<Matrix4f>(5,new Matrix4f());
        public final StructField<Matrix3f> worldMatrixInvTrsp=new StructField<Matrix3f>(6,new Matrix3f());
        public final StructField<Matrix4f> worldViewMatrixInv=new StructField<Matrix4f>(7,new Matrix4f());
        public final StructField<Matrix4f> worldViewProjMatrixInv=new StructField<Matrix4f>(8,new Matrix4f());
        public final StructField<Matrix3f> normalMatrixInv=new StructField<Matrix3f>(9,new Matrix3f());
        @Override public Void get() {return null;}
    }

#extension GL_ARB_explicit_attrib_location : enable

#define bindUBO(type,name)  layout (std140) uniform bo_##name { \
    type name;\
} 

#define bindSSBO(type,name)  layout (std140) buffer  bo_##name { \
    type name;\
} 


struct Camera{
    vec2 frustumNearFar; 
    vec2 resolutionInverse; 
    vec2 resolution; 
    mat4 viewMatrix; 
    mat4 projectionMatrix; 
    mat4 viewProjectionMatrix; 

    mat4 viewMatrixInverse; 
    mat4 projectionMatrixInverse; 
    mat4 viewProjectionMatrixInverse; 

    vec4 viewPort; 
    float aspect; 
    
    vec3 cameraPosition; 
    vec3 cameraDirection; 
    vec3 cameraLeft; 
    vec3 cameraUp;  
};


struct Geometry{
    mat4 worldMatrix;
    mat4 worldViewMatrix;
    mat3 normalMatrix;
    mat3 worldNormalMatrix;
    mat4 worldViewProjMatrix;
    mat4 worldMatrixInv;
    mat3 worldMatrixInvTrsp;
    mat4 worldViewMatrixInv;
    mat4 worldViewProjMatrixInv;
    mat3 normalMatrixInv;
};
#if defined INSTANCING
    in mat4 inInstanceData;
#endif

mat4 Geometry_getWorldMatrix(in Geometry geo){
    #if defined INSTANCING
        mat4 worldMatrix = mat4(vec4(inInstanceData[0].xyz, 0.0),
        vec4(inInstanceData[1].xyz, 0.0),
        vec4(inInstanceData[2].xyz, 0.0),
        vec4(inInstanceData[3].xyz, 1.0));
        return worldMatrix;
    #else
        return geo.worldMatrix;
    #endif
}


vec3 Geometry_transformWorldNormal(in Geometry geo,in vec3 normal){
    #if defined INSTANCING
        vec4 quat = vec4(inInstanceData[0].w, inInstanceData[1].w,inInstanceData[2].w, inInstanceData[3].w);
        return normal + vec3(2.0) * cross(cross(normal, quat.xyz) + vec3(quat.w) * normal, quat.xyz);
    #else
        return normalize(geo.worldNormalMatrix*normal);
    #endif 

}





struct Timer{ 
    float speed;
    float time;
    int intTime;
    float tpf;
    float frameRate;
    vec2 deltaTime;
};

Would this be one UBO with an array or would this be 32 UBOs of PointLight?

Arrays of structs are not implemented in this pr

[riccardobl]

I reworked this and made a new PR: #1782