[core] Correctly report size of variable shaped TensorArray

[BitPhinix]

Currently, TensorArray reports a grossly incorrect size for variable-shaped arrays. This causes ray data to overfetch blocks, causing spilling and eventually a crash due to running out of disk space. This PR contains a simple fix for the issue, specifically handling this case in nbytes.

Simple reproduction:

import numpy as np
import pandas as pd
from ray.data.extensions import TensorArray

test_data = [np.random.rand(i, i) for i in range(1000)]

df = pd.DataFrame({"tensor": TensorArray(test_data)})

print("reported size", df.memory_usage(index=True, deep=True).sum()) # reported size 8128
print("actual size", sum(t.nbytes for t in test_data)) # actual size 2662668000

[BitPhinix]

Any news on this?