Add support for Microsoft SQL (take2)

[jtjeferreira]

Continues the work of #29

Current status: 2 tests failing

[lihaoyi]

nice work @jtjeferreira , looks like we may be reaching the finish line!

[jtjeferreira]

I forgot to push yesterday commits 🤦 . Can you run CI again?

[lihaoyi]

@jtjeferreira done

[jtjeferreira]

I fixed/ignored a few more tests and only 2 tests are failing:

• scalasql.mssql.OptionalTests.sorting.roundTripOptionalValues, when inserting a None, I get Operand type clash: varbinary is incompatible with float. I think this happens because the mssql jdbc driver fallbacks to binary. Might also be related to Fix option null values failing to put #42
• scalasql.mssql.DataTypesTests.nonRoundTrip due to dates/timezones/etc

[jtjeferreira]

@lihaoyi I noticed scalasql.mssql.DataTypesTests scalasql.mssql.DataTypesTests.constant is only failing in scala3 and not scala2, because DataTypes.select is generating the following query:

SELECT
data_types0.my_tiny_int AS my_tiny_int, 
data_types0.my_small_int AS my_small_int, 
data_types0.my_int AS my_int, 
data_types0.my_big_int AS my_big_int, 
data_types0.my_double AS my_double, 
CASE WHEN data_types0.my_boolean THEN 1 ELSE 0 END AS my_boolean, 
data_types0.my_local_date AS my_local_date, 
data_types0.my_local_time AS my_local_time, 
data_types0.my_local_date_time AS my_local_date_time, 
data_types0.my_util_date AS my_util_date, 
data_types0.my_instant AS my_instant, 
data_types0.my_var_binary AS my_var_binary, 
data_types0.my_uuid AS my_uuid, 
data_types0.my_enum AS my_enum 
FROM data_types data_types0

which triggers the error An expression of non-boolean type specified in a context where a condition is expected, near 'THEN'.

My suspicion is that this due to different implementation of scalasql.query.TableMacros#initTableMetadata... Let me know if you have any hint...

[lihaoyi]

only 2 more failures. Seems like we're almost there!

scalasql[3.4.2].test.test 2 tests failed: 
  scalasql.mssql.DataTypesTests scalasql.mssql.DataTypesTests.constant
  scalasql.mssql.OptionalTests scalasql.mssql.OptionalTests.sorting.roundTripOptionalValues

[jtjeferreira]

@lihaoyi what do you think about this?

[lihaoyi]

If the underlying JDBC driver doesn't support it, there's nothing we can do, and we can just conditionally disable that line. But we should make sure we understand and document what the consequences are: does SqlServer not support savepoints at all? Or can you create them but not release them? So what does that mean for users?

[jtjeferreira]

I don't have any hints haha, Scala 3 support was contributed by @mrdziuban a while back. Maybe he has some ideas, but if not we'll have to investigate this and ourselves and figure out what's going on

Here are my (unsolicited) thoughts,

• The common expectation is that savepoint-related resources be released after savepoint delineation. Thus, I'd argue for this behavior to be default. (Even SQLite supports releasing savepoints :)
• My research also confirms that SQLServer doesn't support releaseSavepoint(). As such, we should only conditionally call releaseSavepoint() to avoid failure with SQLServer.

@aboisvert I guess you wanted to reply here, so I will reply here.

I can try to implement that conditional call to releaseSavepoint(), but I suspect at the time I faced this it was not trivial. But meanwhile I gained a bit more knowledge about scalasql. I will try again.

However what worries me is that no test failed when I commented this line. I also need to investigate that...

[lihaoyi]

I don't know why no tests failed when commented out, I thought we did have pretty good tests for a variety of transaction/savepoint scenarios in https://github.com/com-lihaoyi/scalasql/blob/main/scalasql/test/src/api/TransactionTests.scala, but maybe I missed a spot

[jtjeferreira]

I can try to implement that conditional call to releaseSavepoint(), but I suspect at the time I faced this it was not trivial.

I revisited this today, and indeed I can't just write dialect.isInstanceOf[MsSqlDialect] because in core I don't have access to MsSqlDialect.

Shall I add an def supportSavepointRelease: Boolean to DialectConfig? Shall I add a try/catch and swallow the exception?

[lihaoyi]

let's go with def supportSavepointRelease: Boolean

[jtjeferreira]

done in 63c538a

[jtjeferreira]

only 2 more failures. Seems like we're almost there!

scalasql[3.4.2].test.test 2 tests failed: 
  scalasql.mssql.DataTypesTests scalasql.mssql.DataTypesTests.constant
  scalasql.mssql.OptionalTests scalasql.mssql.OptionalTests.sorting.roundTripOptionalValues

Those are related to this comment. I think the code generated by the scala3 macro is calling this code, but not the scala2 generated code. Do you have any hint?

The scala 2.x tests are green...

[lihaoyi]

I don't have any hints haha, Scala 3 support was contributed by @mrdziuban a while back. Maybe he has some ideas, but if not we'll have to investigate this and ourselves and figure out what's going on

[aboisvert]

I don't have any hints haha, Scala 3 support was contributed by @mrdziuban a while back. Maybe he has some ideas, but if not we'll have to investigate this and ourselves and figure out what's going on

Here are my (unsolicited) thoughts,

• The common expectation is that savepoint-related resources be released after savepoint delineation. Thus, I'd argue for this behavior to be default. (Even SQLite supports releasing savepoints :)
• My research also confirms that SQLServer doesn't support releaseSavepoint(). As such, we should only conditionally call releaseSavepoint() to avoid failure with SQLServer.

[jtjeferreira]

I don't have any hints haha, Scala 3 support was contributed by @mrdziuban a while back. Maybe he has some ideas, but if not we'll have to investigate this and ourselves and figure out what's going on

No worries. I left this for the end because troubleshooting macros is always a pain...

[mrdziuban]

@lihaoyi I noticed scalasql.mssql.DataTypesTests scalasql.mssql.DataTypesTests.constant is only failing in scala3 and not scala2, because DataTypes.select is generating the following query:

...
CASE WHEN data_types0.my_boolean THEN 1 ELSE 0 END AS my_boolean, 
...

I've started looking into this and can reproduce it by loading a console in the test project:

./mill -i scalasql[3.4.2].test.console

then running this:

import scalasql._
import scalasql.dialects.MsSqlDialect._

case class Foo[T[_]](b: T[Boolean])
object Foo extends Table[Foo]

scalasql.example.MsSqlExample.mssqlClient.renderSql(Foo.select)
// SELECT CASE WHEN foo0.b THEN 1 ELSE 0 END AS b FROM foo foo0

What I've learned is that the CASE WHEN statement is generated here: https://github.com/jtjeferreira/scalasql/blob/mssql-joao/scalasql/src/dialects/MsSqlDialect.scala#L335

That class is constructed by implicit def ExprQueryable here: https://github.com/jtjeferreira/scalasql/blob/mssql-joao/scalasql/src/dialects/MsSqlDialect.scala#L125

And ExprQueryable is resolved by this summonInline[Queryable.Row[t2, t]] in the Scala 3 macros: https://github.com/jtjeferreira/scalasql/blob/mssql-joao/scalasql/query/src-3/TableMacro.scala#L67

I don't yet understand why ExprQueryable isn't resolved in Scala 2 by this (I thought equivalent) implicitly call: https://github.com/jtjeferreira/scalasql/blob/mssql-joao/scalasql/query/src-2/TableMacro.scala#L53

Instead Scala 2 is resolving this other implicit def ExprQueryable: https://github.com/jtjeferreira/scalasql/blob/mssql-joao/scalasql/core/src/Expr.scala#L40

So @jtjeferreira, should class ExprQueryable actually be generating the CASE WHEN statement? If so, could you change it to add the 1 = check, e.g. sql"CASE WHEN 1 = $q THEN 1 ELSE 0 END"?

[jtjeferreira]

thx @mrdziuban, your comment was very useful. I am still experimenting a few alternatives...