In-memory SQLite database and testing

[ogarcia]

I tell you my case that has me a bit “crazy”. The source code of the project I am going to talk about is here.

The problem is that when I do the Rocket tests for some reason it seems to mix one test with another and, possibly because of a race condition, it fails me. It is something that happens from time to time, sometimes never, sometimes several times in a row.

If we look at this code we see that what it has is a rocket function that raises the service, by default as it is not passing any configuration it does it in :memory:.

Rocket function (code fragment)

#[launch]
fn rocket() -> _ {
    let figment = Figment::from(rocket::Config::default())
        .merge(Serialized::defaults(RockpassConfig::default()))
        .merge(Serialized::default("databases.rockpass.url", ":memory:"))
        .merge(Toml::file("/etc/rockpass.toml").nested())
        .merge(Toml::file("rockpass.toml").nested())
        .merge(Env::prefixed("ROCKPASS_").global())
        .select(Profile::from_env_or("ROCKPASS_PROFILE", "release"));

    rocket::custom(figment)
        .attach(fairings::Cors)
        .attach(fairings::ForceContentType(ContentType::JSON))
        .attach(RockpassDatabase::fairing())
        .attach(AdHoc::config::<RockpassConfig>())
        .attach(AdHoc::on_ignite("Database Migrations", database_migrations))
        .mount("/", routes![
               routes::options_auth_users,
               routes::post_auth_users,
               routes::options_auth_users_me,
               routes::get_auth_users_me,
               routes::delete_auth_users_me,
               routes::options_auth_users_set_password,
               routes::post_auth_users_set_password,
               routes::options_auth_jwt_create,
               routes::post_auth_jwt_create,
               routes::options_auth_jwt_refresh,
               routes::post_auth_jwt_refresh,
               routes::options_passwords,
               routes::get_passwords,
               routes::post_passwords,
               routes::options_passwords_id,
               routes::put_passwords_id,
               routes::delete_passwords_id
        ])
}

So to test this what I do is to raise a client with let client = Client::tracked(rocket()).await.unwrap(); and from there I start making requests, it is supposed to give me a client connected to an in-memory DB that is clean and independent, but it seems that it does not because sometimes it fails. For example here, here, here, here and here.

Whether it is asynchronous or synchronous should not have any influence since each test is supposed to be independent and in fact it is designed to be so. Can you think of any idea why it might be failing? Could it be a rare bug?

[the10thWiz]

I would guess that you are not selecting an in-memory database during test. You use the following code to select a profile: Profile::from_env_or("ROCKPASS_PROFILE", "release"). Which profile is this selecting during test? I would guess release, since you don't appear to be setting the ROCKPASS_PROFILE env var in your tests. Second, do you have a rockpass.toml or /etc/rockpass.toml in your test env?

You can check what db your tests are using by running println!("{}", figment.extract_inner::<String>("databases.rockpass.url").unwrap()); during startup.

Once you've determined that it actually is using an independent in-memory db for each test, we can take a look at what else might be causing these issues. It's also possible that some tests have internal race conditions, and some kind of cleanup got executed before the test actually completed.

[ogarcia]

I had already done those checks before to rule out that a .sqlite file was being used (which would perfectly explain the failures) and yes, it is using the release profile and the in-memory DB (in fact the profile can already be seen in the previous tests).

In any case I have done the test again adding the line that prints where the DB is and as you can see it is in memory.

Dartabase location: :memory:
Configured for release.

I don't believe that each test has internal race conditions, I could be wrong, but I made sure that the operations were done in the specific order. Moreover, the failures are random, in the sense that I run the tests several times and in the first run fail the second and the third, in the second the first, the fourth and the fifth... etc. It is even possible that none of them fail. This can be seen in the jobs related to the first one, this one, this one and this one.