[Data] Error when reading JSONL of lists

[scottjlee]

What happened + What you expected to happen

For the following sample.jsonl file:

{"messages": [{"role": "system", "content": "You are a helpful assistant"}, {"role": "user", "content": [{"type": "text", "text": "What is this?"}, {"type": "image", "image": ["data:image/jpeg;base64,aaa"]}]}]}

When we read with ray.data.read_json:

import ray
ray.data.read_json("sample.jsonl")

We get the following error:

File "/home/ray/anaconda3/lib/python3.11/site-packages/ray/anyscale/data/_internal/readers/file_reader.py", line 157, in read_paths
    yield from _read_paths(paths)
  File "/home/ray/anaconda3/lib/python3.11/site-packages/ray/anyscale/data/_internal/readers/file_reader.py", line 143, in _read_paths
    for batch in self.read_stream(file, path):
  File "/home/ray/anaconda3/lib/python3.11/site-packages/ray/anyscale/data/_internal/readers/json_reader.py", line 48, in read_stream
    yield from self._read_with_python_json(buffer)
  File "/home/ray/anaconda3/lib/python3.11/site-packages/ray/anyscale/data/_internal/readers/json_reader.py", line 119, in _read_with_python_json
    for k, v in row.items():
                ^^^^^^^^^
AttributeError: 'str' object has no attribute 'items'

We would expect to be able to read the file successfully, since the following works:

import json
with open("sample.jsonl", "r") as f:
    for line in f:
        json.loads(line)

Versions / Dependencies

ray master

Reproduction script

See above

Issue Severity

Medium: It is a significant difficulty but I can work around it.