[Bug]: Direct UNION ALL over partially compressed chunks fetches only uncompressed data

[natalya-aksman]

What type of bug is this?

Incorrect result

What subsystems and features are affected?

Compression

What happened?

If we directly query UNION ALL of partially compressed chunks, the plan retrieves only uncompressed parts (and doesn't retrieve compressed parts), leading to wrong result.

We will get incorrect plan like the one below for partially compressed chunks, only fetching uncompressed data. It leads to incorrect result as data stored in compressed chunks is not fetched.

explain 
SELECT * FROM _timescaledb_internal._hyper_48_67_chunk 
UNION ALL SELECT * FROM _timescaledb_internal._hyper_48_66_chunk;
                QUERY PLAN
-------------------------------------------------
 Append 
   ->  Seq Scan on _hyper_48_67_chunk
   ->  Seq Scan on _hyper_48_66_chunk 

TimescaleDB version affected

2.18

PostgreSQL version used

17.4

What operating system did you use?

Ubuntu 24.04

What installation method did you use?

Source

What platform did you run on?

On prem/Self-hosted

Relevant log output and stack trace

How can we reproduce the bug?

CREATE TABLE dist_null(time timestamptz not null, device text, value float);
SELECT table_name FROM create_hypertable('dist_null','time',chunk_time_interval:='1 year'::interval);

ALTER TABLE dist_null SET (tsdb.compress, tsdb.compress_segmentby='device', tsdb.compress_orderby='time');

-- Set up partially compressed chunk
INSERT INTO dist_null SELECT '2025-01-01', NULL, 0.1;
INSERT INTO dist_null SELECT '2025-01-02', NULL, 0.1;
INSERT INTO dist_null SELECT '2025-01-02', 'd', 0.1;
SELECT count(compress_chunk(ch)) FROM show_chunks('dist_null') ch;
INSERT INTO dist_null SELECT '2025-01-02', 'd', 0.1;

select show_chunks as chunk1 from show_chunks('dist_null') limit 1 \gset

-- Wrong plan
postgres=# explain select * from :chunk1 UNION ALL select * from :chunk1;
                                            QUERY PLAN
--------------------------------------------------------------------------------------------------
 Append  (cost=0.00..52.10 rows=2140 width=48)
   ->  Seq Scan on _hyper_50_71_chunk  (cost=0.00..20.70 rows=1070 width=48)
   ->  Seq Scan on _hyper_50_71_chunk _hyper_50_71_chunk_1  (cost=0.00..20.70 rows=1070 width=48)
(3 rows)


postgres=# select * from :chunk1;
          time          | device | value
------------------------+--------+-------
 2025-01-02 00:00:00-05 | d      |   0.1
 2025-01-01 00:00:00-05 |        |   0.1
 2025-01-02 00:00:00-05 |        |   0.1
 2025-01-02 00:00:00-05 | d      |   0.1
(4 rows)

-- Wrong result
postgres=# select * from :chunk1 UNION ALL select * from :chunk1;
          time          | device | value
------------------------+--------+-------
 2025-01-02 00:00:00-05 | d      |   0.1
 2025-01-02 00:00:00-05 | d      |   0.1
(2 rows)