v0.4.5

What's Changed 🚀

💥 Breaking Changes

• refactor!: split column expression into unresolved and resolved types @kevinzwang (#3804)

✨ Features

• feat(connect): daft.pyspark module @universalmind303 (#3861)
• feat: Emit children of join before shuffle + add stats to explain analyze @colin-ho (#3852)
• feat: Stageify plan on shuffle boundaries @colin-ho (#3781)
• feat(sql): adds session sql for leveraging attached catalogs @rchowell (#3860)
• feat(catalog): Cutover deprecated APIs to use session, catalog, table abstractions [3/3] @rchowell (#3830)
• feat(connect): read csv/parquet/json options @universalmind303 (#3791)
• feat(sql): select from multiple joins @kevinzwang (#3842)
• feat(catalog): Integrate session and catalog actions alongside existing APIs [2/3] @rchowell (#3825)
• feat(catalog): Prepare existing catalog APIs for integration [1/3] @rchowell (#3820)
• feat(sql): supports schemas in read_json, read_csv, read_parquet @rchowell (#3836)
• feat(sql): supports array of paths in read_ table-value functions @rchowell (#3835)
• feat: Add a daft dashboard to display queries plans and stats @raunakab (#3790)

🐛 Bug Fixes

• fix: sql round without precision @universalmind303 (#3863)
• fix: pypi publish workflow @kevinzwang (#3862)
• fix: build wheel Github action inputs @kevinzwang (#3858)
• fix: protocol in iceberg writes @colin-ho (#3851)
• fix: LogicalPlan::get_schema_for_alias should stop when it hits any alias @kevinzwang (#3848)
• fix: Reduce number of nodes in random join graph test @desmondcheongzx (#3839)
• fix: Add excludes to broken link checker @colin-ho (#3834)
• fix: Grab Daft config from environment variables for new contexts @desmondcheongzx (#3832)
• fix: create series of np.datetime64['D'] @rchowell (#3829)

🚀 Performance

• perf(optimizer): Infer additional join graph edges during join reordering @desmondcheongzx (#3807)

♻️ Refactor

• refactor!: split column expression into unresolved and resolved types @kevinzwang (#3804)

📖 Documentation

• docs: respect daft analytics env var @ccmao1130 (#3856)
• docs: Update configuration docs to show set_runner_native @colin-ho (#3833)

🔧 Maintenance

• chore: replace anaconda with S3 for nightly build publish @kevinzwang (#3857)
• chore: minor cleanup to table-value functions @rchowell (#3854)
• chore: remove accidental printlins @universalmind303 (#3845)

Full Changelog: v0.4.4...v0.4.5

v0.4.4

What's Changed 🚀

• build: update python-publish workflow @ccmao1130 (#3797)
• build(docs): fix docgen failed workflow @ccmao1130 (#3766)

✨ Features

• feat: Adds .summarize() to compute statistics @rchowell (#3810)
• feat(sql): SELECT without FROM @rchowell (#3814)
• feat: Simplify is ins to an OR chain of eqs @colin-ho (#3800)
• feat(session): Adds session class to python @rchowell (#3809)
• feat(session): Replaces direct usage of DaftCatalog with Session @rchowell (#3794)
• feat: Sequentially materialize left and right sides during hash join @colin-ho (#3735)
• feat(connect): add temporal functions @universalmind303 (#3799)
• feat: nulls first kernels @universalmind303 (#3789)
• feat(table): implement list_unique and Set aggregation @f4t4nt (#3710)
• feat: add functions to daft-connect @universalmind303 (#3780)
• feat(catalog): Defines a session for connection state @rchowell (#3782)
• feat: implement bool_and and bool_or @f4t4nt (#3754)
• feat(catalog): Defines an identifier for use across catalogs @rchowell (#3763)
• feat(optimizer): Brute force join ordering @desmondcheongzx (#3688)
• feat(swordfish): Properly buffer unordered scan tasks @colin-ho (#3751)
• feat: better sql datatype support @universalmind303 (#3750)
• feat: Adds list constructor to Expression and SQL APIs @rchowell (#3737)
• feat: spark connect set operations @universalmind303 (#3739)
• feat: add spark explain @universalmind303 (#3741)

🐛 Bug Fixes

• fix: unity managed table reads @pmogren (#3806)
• fix: boolean casts to strings and null propagation @rchowell (#3770)
• fix: catalog table names @universalmind303 (#3760)

🚀 Performance

• perf(swordfish): Parallel expression evaluation @colin-ho (#3593)
• perf: Use parquet metadata from schema inference for accurate scan task statistics @desmondcheongzx (#3784)

♻️ Refactor

• refactor: rename table to recordbatch @universalmind303 (#3771)
• refactor: port DaftContext to rust side @universalmind303 (#3767)
• refactor: renames to_struct to just struct @rchowell (#3755)

📖 Documentation

• docs: fix readthedocs build @ccmao1130 (#3824)
• docs: add scarf analytics @ccmao1130 (#3773)
• docs: Update distributed docs to add byoc mode, change name to daft cli @jessie-young (#3768)
• docs: update README.rst diagram @ccmao1130 (#3803)
• docs: update links in readme @ccmao1130 (#3779)
• docs: add footer and update broken links @ccmao1130 (#3764)

👷 CI

• ci: Allow TPCH benchmarks to use ARM cluster profile @desmondcheongzx (#3777)
• ci: Record info for TPCH benchmarks @desmondcheongzx (#3729)
• ci: send slack notification for broken links @ccmao1130 (#3742)

Full Changelog: v0.4.3...v0.4.4

v0.4.3

What's Changed 🚀

• build(docs): Adds make docs phony target @rchowell (#3693)

✨ Features

• feat: Add a new dashboard UI to Daft @raunakab (#3738)
• feat(shuffles): Determination logic for pre shuffle merge @colin-ho (#3674)
• feat: Limit number of sources in merged scan task @colin-ho (#3695)
• feat: Expose parquet chunk size to swordfish reads @colin-ho (#3714)
• feat: add LiteralValue::Int8 and Int16 @ugoa (#3736)
• feat: with_column(s)_renamed expression for DataFrame @jessie-young (#3732)
• feat: Explain for swordfish @colin-ho (#3667)
• feat: Adds .describe() to DataFrame and DESCRIBE to SQL @rchowell (#3720)
• feat: Add column format option to iter rows @colin-ho (#3681)
• feat(core): make micropartition streamable over tables @universalmind303 (#3709)
• feat(iceberg): Adds support for read_iceberg with metadata_location to Daft-SQL @rchowell (#3701)
• feat: Overwrite partitions mode @colin-ho (#3687)
• feat(docs): Adds copy-to-clipboard to code samples @rchowell (#3702)
• feat(connect): sql @universalmind303 (#3696)
• feat: add binary string operations (length and concatenation) @f4t4nt (#3646)
• feat(sql): Adds url_download and url_upload to daft-sql @rchowell (#3690)
• feat(connect): distinct + sort @universalmind303 (#3677)
• feat(core): Implement null-safe equality operator @f4t4nt (#3663)
• feat(sql): Adds JsonScanBuilder to daft-scan and read_json to daft-sql @rchowell (#3683)
• feat: support using S3Config.credentials_provider for writes @kevinzwang (#3648)
• feat(sql): Adds FROM source check for string paths @rchowell (#3679)
• feat(connect): Rust ray exec @universalmind303 (#3666)

🐛 Bug Fixes

• fix: Set filter selectivity estimate lower bound @colin-ho (#3694)
• fix(join): joining on different types @kevinzwang (#3716)
• fix: to_cnf and to_dnf functions @kevinzwang (#3728)
• fix: pushdowns for unpivot @universalmind303 (#3724)
• fix(optimizer): Fix issues with join graph construction @desmondcheongzx (#3668)
• fix: Run filter null join key optimization once @colin-ho (#3657)

🚀 Performance

• perf: Track accumulated selectivity in logical plan to improve probe side decisions @desmondcheongzx (#3734)
• perf: simplify boolean expression rules @kevinzwang (#3731)
• perf(shuffles): Incrementally retrieve metadata in reduce @colin-ho (#3545)
• perf: Improve stats for join side determination @colin-ho (#3655)

♻️ Refactor

• refactor: remove eyre from daft-connect @universalmind303 (#3719)
• refactor(execution): NativeExecutor refactor @universalmind303 (#3689)
• refactor: logical op constructor+builder boundary @kevinzwang (#3684)
• refactor(connect): internal refactoring to make connect code more organized & extensible @universalmind303 (#3680)

📖 Documentation

• docs: linked mkdocs & api docs @ccmao1130 (#3703)
• docs: higher quality daft diagram for readme @ccmao1130 (#3697)
• docs: add daft launcher docs to docs v2 @ccmao1130 (#3678)

👷 CI

• ci: skip tests during publishing of release @jaychia (#3744)
• ci: Allow upstream git refs to be used for benchmarking @desmondcheongzx (#3730)
• ci: Remove daft tracing @raunakab (#3692)
• ci: Add new benchmarking cluster profile @raunakab (#3665)

🔧 Maintenance

• chore: Pin sql server version in docker compose @colin-ho (#3715)
• chore(connect): better error propagation & handling @universalmind303 (#3675)
• chore(connect): consolidate multiple files in tests/connect @universalmind303 (#3676)

Full Changelog: v0.4.2...v0.4.3

v0.4.2

What's Changed 🚀

• build: Publish A Long Term Support CPU Release of Daft @samster25 (#3650)

✨ Features

• feat(connect): printSchema @andrewgazelka (#3617)
• feat: Allow building probe table for either side of anti semi joins @colin-ho (#3643)
• feat(optimizer): Add join reordering as an optimizer rule @desmondcheongzx (#3642)
• feat(swordfish): Memory manager @colin-ho (#3599)
• feat(scantask-2): Implement new module for splitting Parquet ScanTask @jaychia (#3628)
• feat(scantask-1): add a config flag for new scantask splitting algorithm @jaychia (#3615)
• feat: Support intersect all and except distinct/all in DataFrame API @advancedxy (#3537)
• feat: support new PyIceberg IO properties and custom IOConfig in write_iceberg @kevinzwang (#3633)
• feat(expressions): Extend Expression.url.upload() to support row-specific URLs @desmondcheongzx (#3518)

🐛 Bug Fixes

• fix: special characters in GCS urls @kevinzwang (#3651)
• fix(swordfish): Track future poll times for explain analyze @colin-ho (#3511)

👷 CI

• ci: Improve visualization of tpcds + tpch benchmarking outputs @raunakab (#3654)

🔧 Maintenance

• chore: update PyO3 version to 0.23 @kevinzwang (#3647)
• chore: Fix parquet benchmark test @colin-ho (#3632)
• chore: Clean up join order iteration @desmondcheongzx (#3638)

⬆️ Dependencies

• build(deps-dev): bump moto[s3,server] from 5.0.21 to 5.0.26 @dependabot (#3640)

Full Changelog: v0.4.1...v0.4.2

v0.4.1

What's Changed 🚀

✨ Features

• feat(optimizer): Implement naive join ordering @desmondcheongzx (#3616)
• feat(connect): add more unresolved functions @andrewgazelka (#3618)
• feat(connect): with_columns_renamed @andrewgazelka (#3386)
• feat(connect): read/write → csv, write → json @andrewgazelka (#3361)

🐛 Bug Fixes

• fix: unity catalog import from write_deltalake @jaychia (#3630)

🚀 Performance

• perf(optimizer): convert filter predicate to CNF to push through join @kevinzwang (#3623)

📖 Documentation

• docs: daft documentation v2 @ccmao1130 (#3595)

✅ Tests

• test(connect): verify show()output @andrewgazelka (#3610)

👷 CI

• ci: Output results in a CSV format @raunakab (#3625)
• ci: Add build step to run-cluster @raunakab (#3606)

🔧 Maintenance

• chore: Build progress bar only on first update @colin-ho (#3626)
• chore: Fix csv benchmark test @colin-ho (#3631)

Full Changelog: v0.4.0...v0.4.1

v0.4.0

What's Changed 🚀

• build: Use uv for maturin builds instead @raunakab (#3540)

💥 Breaking Changes

• feat: Default native runner @colin-ho (#3608)
• chore!: upgrade Ray pins and pyarrow pins @jaychia (#3612)
• chore!: drop support for Python 3.8 @kevinzwang (#3592)
• chore!: remove pyarrow-based file reader @kevinzwang (#3587)

✨ Features

• feat: Default native runner @colin-ho (#3608)
• feat(swordfish): Progress Bar @colin-ho (#3571)
• feat(connect): df.show @universalmind303 (#3560)
• feat(connect): support DdlParse @andrewgazelka (#3580)
• feat(swordfish): Optimize grouped aggregations @colin-ho (#3534)
• feat(swordfish): Enable left/right joins to build probe table on either side @colin-ho (#3548)
• feat: Add DataType inference from Python types @jaychia (#3555)
• feat(shuffles): Locality aware pre shuffle merge @colin-ho (#3505)
• feat: Implement count-distinct for sql @raunakab (#3553)
• feat(connect): add drop support @andrewgazelka (#3345)
• feat: support for basic subquery execution @kevinzwang (#3536)
• feat(connect): add df.filter @andrewgazelka (#3346)
• feat: Make serialization code not unwrap and panic on failures @raunakab (#3546)
• feat: Unity Catalog writes using daft.DataFrame.write_deltalake() @anilmenon14 (#3522)
• feat(connect): add parquet support @andrewgazelka (#3360)
• feat: Add iterators to more types @raunakab (#3539)
• feat(optimizer): Add scaffolding to create join graphs from logical plans @desmondcheongzx (#3501)
• feat(tpcds-benchmarking): Add basic tpcds benchmarking for local testing @raunakab (#3509)
• feat(list): add fixed-size list support for value_counts @andrewgazelka (#3521)
• feat(parquet): Limit parallel tasks in remote parquet reader @colin-ho (#3490)
• feat(parquet): Target parquet writes by size bytes instead of rows @colin-ho (#3457)
• feat: cross join @kevinzwang (#3437)
• [FEAT] connect: remove excessive warnings from spark connect @universalmind303 (#3499)
• [CHORE] connect, test: df.withColumn @andrewgazelka (#3359)
• [FEAT]: expr simplifier @universalmind303 (#3393)
• [FEAT] shuffle testing @raunakab (#3492)
• [FEAT]: add coalesce to dataframe and SQL @universalmind303 (#3482)
• [FEAT] add register-table helper to sql-catalog @chuanlei-coding (#2837)
• [FEAT] Respect resource request for projections in swordfish @colin-ho (#3460)
• [FEAT] Enable Actor Pool UDFs by default @kevinzwang (#3488)
• [FEAT] connect: add modulus operator and withColumns support @andrewgazelka (#3351)
• [FEAT] connect: createDataFrame @andrewgazelka (#3363)
• [FEAT] Support parquet RLE decoding for booleans @desmondcheongzx (#3477)
• [FEAT] Cap parallelism on local parquet reader @colin-ho (#3310)
• [FEAT] connect: add binary operators @andrewgazelka (#3350)
• [FEAT] connect: support basic column operations @andrewgazelka (#3362)
• [FEAT] extend build-commit workflow to support different compile-archs @raunakab (#3459)
• [FEAT] Add count-distinct aggregation @raunakab (#3455)

🐛 Bug Fixes

• fix(udf): udf call with empty table and batch size @kevinzwang (#3604)
• fix: use arrow's schema instead of spark's for local rel @universalmind303 (#3602)
• fix: guard concurrent extension datatype setting with a lock @jaychia (#3589)
• fix(parquet): Fix parquet reads of required fields nested within optional fields @desmondcheongzx (#3598)
• fix: boolean and/or expressions with null @kevinzwang (#3544)
• fix(run-cluster-workflow): Add null check when parsing metadata @raunakab (#3507)
• fix(tpcds): fix bugs in tpcds datagen script @universalmind303 (#3495)
• [BUG] Fix build commit workflow @raunakab (#3487)
• [BUG]: dont panic on count(distinct) @universalmind303 (#3481)
• [BUG] Block on parquet schema future in estimate_size_bytes @colin-ho (#3484)

🚀 Performance

• perf: filter null join key optimization rule @kevinzwang (#3583)
• perf: lazily import pyiceberg and unity catalog if available @jaychia (#3565)

♻️ Refactor

• refactor: allow InMemory to take in non python based entries @universalmind303 (#3554)
• refactor: create a rust based PartitionSet @universalmind303 (#3515)
• refactor(swordfish): Generic broadcast state bridge @colin-ho (#3508)

📖 Documentation

• docs: update tpch benchmark link @ccmao1130 (#3542)
• docs: Enable Linting of docstrings @samster25 (#3506)
• [FEAT] Enable Actor Pool UDFs by default @kevinzwang (#3488)

✅ Tests

• test(connect): add more tests for createDataFrame @andrewgazelka (#3607)
• test: Add more size estimation tests from our s3 bucket @jaychia (#3514)

👷 CI

• ci: Always download logs @jaychia (#3588)
• ci: Add ability to array-ify args and run multiple jobs @raunakab (#3584)
• ci: Add "build" label type to accepted PR titles @raunakab (#3541)
• ci: add a tool to launch workloads on cluster @jaychia (#3516)
• ci(release-drafter): use conventional commit labels @andrewgazelka (#3503)

🔧 Maintenance

• chore!: upgrade Ray pins and pyarrow pins @jaychia (#3612)
• chore: add warning for native runner @jaychia (#3613)
• chore!: drop support for Python 3.8 @kevinzwang (#3592)
• chore!: remove pyarrow-based file reader @kevinzwang (#3587)
• chore: Fix ordering in sql tests + pin docker images in read_sql tests @colin-ho (#3596)
• chore: move symbolic and boolean algebra code into new crate @kevinzwang (#3570)
• [CHORE] use conventional commits @andrewgazelka (#3493)
• [CHORE] connect, test: df.withColumn @andrewgazelka (#3359)
• [CHORE] Add tests for parquet size estimations @jaychia (#3405)
• [CHORE] Move all python wrapping logic to separate module @raunakab (#3458)

Full Changelog: v0.3.15...v0.3.16

v0.3.15

Changes

✨ New Features

• [FEAT] run cluster on commit @raunakab (#3461)
• [FEAT]: Support .clip function @conradsoon (#3136)
• [FEAT] Add cluster profiles @raunakab (#3426)
• [FEAT] add pyiceberg 0.8.0 support @rongfengliang (#3448)
• [FEAT] migrate schema inference → async, block at py boundary @andrewgazelka (#3432)
• [CHORE] connect: df.schema @andrewgazelka (#3353)
• [CHORE] connect test: df.get_attr @andrewgazelka (#3349)
• [FEAT] Get native execution enablement from DAFT_RUNNER @desmondcheongzx (#3409)
• [FEAT] Add ability to download log files from ray-cluster @raunakab (#3406)
• [FEAT] Add ability to run arbitrary command on a set working directory @raunakab (#3404)
• [FEAT] Add steps to spin up, submit job, and spin down ray clusters @raunakab (#3403)
• [CHORE] connect: add tests for df.take() method @andrewgazelka (#3385)
• [FEAT] Create new run workflow @raunakab (#3402)
• [FEAT] Enable group by keys in aggregation expressions @kevinzwang (#3399)
• [FEAT] Build release python wheels and upload to AWS S3 @raunakab (#3398)
• [FEAT] connect: Add support for select @andrewgazelka (#3344)
• [FEAT] connect: add df.limit and df.first @andrewgazelka (#3309)
• [FEAT] connect: to_daft_* use ref instead of value @andrewgazelka (#3355)
• [FEAT] connect: add alias support @andrewgazelka (#3342)
• [FEAT] Filter predicates in SQL join @kevinzwang (#3371)
• [FEAT] connect: collect @andrewgazelka (#3326)

🚀 Performance Improvements

• [PERF] Improve hash table probe side decisions for Swordfish @desmondcheongzx (#3327)

👾 Bug Fixes

• [BUG] Fix extension type display @jaychia (#3456)
• [BUG] Remove enum imports from match statements @raunakab (#3436)
• [BUG] Explicitly set IO config in unity catalog load table @colin-ho (#3453)
• [BUG] Include storage options in lance write commit @colin-ho (#3451)
• [BUG] Replace semicolons in filenames with underscore @raunakab (#3430)
• [BUG] Terminate nodes instead of stopping them @raunakab (#3427)
• [BUG] Fix run-cluster passing in environment variables wrongly @jaychia (#3422)

📖 Documentation

• [FEAT]: Support .clip function @conradsoon (#3136)
• [DOCS] Shorten union of Literals @desmondcheongzx (#3449)
• [DOCS] Add missing list expression entries @desmondcheongzx (#3428)

🧰 Maintenance

• [CHORE] Add warning in PyRunner to switch to Native @colin-ho (#3472)
• [CHORE] Address comments on previous PR @raunakab (#3473)
• [CHORE] Write tpch parquet files one at a time @colin-ho (#3396)
• [CHORE] Remove CountMode and ResourceRequest from public API @desmondcheongzx (#3429)
• [CHORE] Add schemas for remaining local plan ops @colin-ho (#3446)
• [CHORE] Put empty table when building probe table @colin-ho (#3445)
• [CHORE] Explain block_on function in common-runtime @colin-ho (#3442)
• [CHORE] connect: df.schema @andrewgazelka (#3353)
• [CHORE] Update execution config to turn on Ray tracing @jaychia (#3431)
• [CHORE] connect test: df.get_attr @andrewgazelka (#3349)
• [CHORE] Cleanup ExprResolver @kevinzwang (#3401)
• [CHORE] connect: add tests for df.take() method @andrewgazelka (#3385)
• [CHORE] Change IOConfig to be serialized into binary instead of JSON @kevinzwang (#3400)
• [CHORE] Pin PyIceberg version to <0.8 @kevinzwang (#3391)
• [CHORE] Add TPC-H queries in SQL @kevinzwang (#3392)
• [CHORE] connect: Optimize plans in connect @colin-ho (#3378)
• [CHORE] delete empty file xyz @andrewgazelka (#3370)

⬆️ Dependencies

14 changes

• Bump orjson from 3.10.11 to 3.10.12 @dependabot (#3464)
• Bump grpcio from 1.67.0 to 1.68.1 @dependabot (#3465)
• Bump arrow-buffer from 51.0.0 to 53.3.0 @dependabot (#3467)
• Bump regex-syntax from 0.7.5 to 0.8.4 @dependabot (#3468)
• Bump memmap2 from 0.9.4 to 0.9.5 @dependabot (#3470)
• Bump image from 0.25.4 to 0.25.5 @dependabot (#3471)
• Bump bytes from 1.7.1 to 1.8.0 @dependabot (#3411)
• Bump astral-sh/setup-uv from 3 to 4 @dependabot (#3410)
• Bump serde_json from 1.0.124 to 1.0.133 @dependabot (#3413)
• Bump sample-arrow2 from 0.1.0 to 0.17.2 @dependabot (#3414)
• Bump chrono-tz from 0.8.6 to 0.10.0 @dependabot (#3415)
• Bump azure-storage-blob from 12.17.0 to 12.24.0 @dependabot (#3416)
• Bump opencv-python from 4.8.1.78 to 4.10.0.84 @dependabot (#3417)
• Bump sqlalchemy from 2.0.25 to 2.0.36 @dependabot (#3418)

v0.3.14

Changes

✨ New Features

• [FEAT]: sql HAVING @universalmind303 (#3364)
• [FEAT] consolidate Spark session fixture into conftest.py @andrewgazelka (#3341)
• [FEAT]: allow for implicit coercion between str & date @universalmind303 (#3337)
• [FEAT] daft-connect range use python generator @andrewgazelka (#3308)
• [FEAT] Monotonically Increasing Id for Swordfish @colin-ho (#3180)
• [FEAT] Support for correlated subqueries in SQL (not yet executable) @kevinzwang (#3304)
• [FEAT]: SQL read_csv @itzhakstern (#3255)
• [FEAT] Daft Catalog API @jaychia (#3036)
• [FEAT]: allow is_in to take in Vec<Expr> instead of Expr @universalmind303 (#3294)
• [FEAT] Lance writes for swordfish @colin-ho (#3299)
• [FEAT] Support for aggregation expressions that use multiple AggExprs @kevinzwang (#3296)
• [FEAT] SQL union/union all and sql intersect @universalmind303 (#3274)

👾 Bug Fixes

• [BUG] Implement deserialize for Python objects serialized as sequences @kevinzwang (#3339)
• [BUG]: tbl alias with join @universalmind303 (#3333)
• [BUG] Fixes regexp_replace expression @ConeyLiu (#3306)
• [BUG] Fix ray wait in RayPartitionSet @jaychia (#3251)
• [BUG] Partially qualified joins join a.x = y and join x = b.y @universalmind303 (#3290)
• [BUG] Check env in benchmarking script @colin-ho (#3297)
• [BUG] Fix writes for empty dataframes if target directory does not exist @colin-ho (#3278)
• [BUG]: panic in sql subquery @universalmind303 (#3291)

📖 Documentation

• [DOCS] Fix typo in limit example @colin-ho (#3303)
• [DOCS] Update incomplete SQL doc pages @willvo2004 (#3298)

🧰 Maintenance

• [CHORE]: prepare for nulls first/last kernels @universalmind303 (#3301)
• [CHORE] Fix join alias test @kevinzwang (#3335)
• Bump bytemuck from 1.16.3 to 1.19.0 @dependabot (#3171)
• [CHORE]: move utf8 functions from daft-dsl to daft-functions @ConeyLiu (#3101)
• [CHORE] Swordfish refactors @colin-ho (#3256)
• [CHORE]: better subquery handling @universalmind303 (#3295)
• [CHORE] Possibility to create environment with system installed uv @maruschin (#3281)
• [CHORE]: defer Expr subquery error until eval @universalmind303 (#3272)
• [CHORE] Fix style in workflow file @jaychia (#3284)

⬆️ Dependencies

12 changes

• Bump bytemuck from 1.16.3 to 1.19.0 @dependabot (#3171)
• Bump psycopg2-binary from 2.9.9 to 2.9.10 @dependabot (#3174)
• Bump codecov/codecov-action from 3 to 5 @dependabot (#3318)
• Bump slackapi/slack-github-action from 1.27.0 to 2.0.0 @dependabot (#3317)
• Bump lxml from 5.1.0 to 5.3.0 @dependabot (#3172)
• Bump moto[s3,server] from 5.0.2 to 5.0.21 @dependabot (#3312)
• Bump async-stream from 0.3.5 to 0.3.6 @dependabot (#3313)
• Bump unicode-normalization from 0.1.23 to 0.1.24 @dependabot (#3314)
• Bump tikv-jemallocator from 0.5.4 to 0.6.0 @dependabot (#3316)
• Bump sysinfo from 0.30.13 to 0.32.0 @dependabot (#3168)
• Bump pretty_assertions from 1.4.0 to 1.4.1 @dependabot (#3169)
• Bump lz4 from 1.26.0 to 1.28.0 @dependabot (#3170)

v0.3.13

Changes

✨ New Features

• [FEAT] Pre Shuffle Merge Strategy @colin-ho (#3191)
• [FEAT] Minimal indices dtype for FixedShapeSparseTensors @sagiahrac (#3149)
• [FEAT] implement range operation and data streaming @andrewgazelka (#3267)
• [FEAT] Support intersect as a DataFrame API @advancedxy (#3134)
• [FEAT] Adds a read_generator method that reads tables from a generator @colin-ho (#3258)
• [FEAT] Add initial Spark Connect support @andrewgazelka (#3261)

👾 Bug Fixes

• [BUG] Always run telemetry codepath @jaychia (#3275)
• [BUG] Cleanup context side-effects @jaychia (#3270)
• [BUG]: bad merge from intersect PR @universalmind303 (#3273)

📖 Documentation

• [FEAT] Minimal indices dtype for FixedShapeSparseTensors @sagiahrac (#3149)
• [DOCS] Use an absolute path for the canonical link @desmondcheongzx (#3271)
• [DOCS] Set canonical link @desmondcheongzx (#3269)

🧰 Maintenance

• [CHORE] Allow manual launch of release-drafter.yml @jaychia (#3283)
• [CHORE] add daftrunner env to install and test step @colin-ho (#3279)
• [CHORE] Remove the concept of runner configs @jaychia (#3276)
• [CHORE] Expose read_sql partition bound strategy and default to min-max @colin-ho (#3246)
• [CHORE]: remove daft-table dependency from daft-logical-plan @universalmind303 (#3265)
• [CHORE] Remove daft-scan dependency from planning crates @kevinzwang (#3250)

v0.3.12

Changes

✨ New Features

• [FEAT]: Sql joins with duplicate cols @universalmind303 (#3241)
• [FEAT] Add tracing for runner @jaychia (#3113)
• [FEAT] add spark-connect protocol @andrewgazelka (#3189)

🚀 Performance Improvements

• [PERF] Harden GCP Retries @samster25 (#3253)

👾 Bug Fixes

• [BUG]: orderby with aggs @universalmind303 (#3190)

📖 Documentation

• [DOCS] Fix typo in write_parquet's parameters @desmondcheongzx (#3252)
• [DOCS] Changing docs for UDF @jaychia (#2880)

🧰 Maintenance

• [CHORE] implement mean and stddev for decimal @samster25 (#3159)