Parallel database calls

[Narayanbhat166]

Hi, I have a use case where multiple tables needs to be queried ( or inserted into ). Querying them can be done independently. A simple parallel database call would look like this

async fn get_table1() -> Result<Table1, DatabaseError> {}
async fn get_table2() -> Result<Table2, DatabaseError> {}

let (table1, table2) = join!(get_table1(), get_table2())

where the join!() is from future utils

Currently the application dependencies looks like

• actix for api layer ( with single threaded runtime, total 8 workers )
• diesel with postgres
• async-bb8-diesel for async connection pooling.

The async-bb8-diesel crate mentions that

This crate prefers to use tokio::spawn_blocking function. This function moves the requested operation to an entirely distinct thread where blocking is acceptable.

Because of this, when the parallel calls are made, those many threads are created for the blocking operation ( 2 threads in this case ). Wouldn't this lead to increased memory usage, if there are 100 requests per second, then there will be 200 threads at a moment.

I also looked into pipelining which supports executing multiple independent queries in the same network call, is there anything like this supported by diesel? In the end, what I want to know is, will it actually benefit if the database call futures are run in parallel, maybe by using a different connection pooling library which supports true async. Will using diesel_async be of any help here? Any information is appreciated, Thanks!

[weiznich]

See the documentation: https://docs.rs/diesel-async/0.3.1/diesel_async/struct.AsyncPgConnection.html