mysql_async icon indicating copy to clipboard operation
mysql_async copied to clipboard

Statement leaks

Open pimeys opened this issue 4 years ago • 3 comments

Hi. For some reason we haven't been getting this error until now, when user reported crashes due to reaching max_prepared_stmt_count. We typically set the LRU to 1000 statements for everybody, and up until now, nobody except one user have been able to trigger the bug.

I'm able to get the same error, but only when disabling the cache completely:

use mysql_async::{self as my, prelude::Queryable as _};

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let config = my::OptsBuilder::default()
        .stmt_cache_size(Some(0))
        .ip_or_hostname("localhost")
        .tcp_port(3308)
        .user(Some("root"))
        .pass(Some("prisma"))
        .db_name(Some("prisma"));

    let mut conn = my::Conn::new(config).await?;

    loop {
        let stmt = conn.prep("SELECT 1").await?;
        let _: Vec<my::Row> = conn.exec(&stmt, my::Params::Empty).await?;
    }
}

This code will increment the Prepared_stmt_count in show global status like '%stmt%'; very fast, and eventually crash the program when reaching the limit. If setting the statement cache size to 1, the bug doesn't happen.

But. There is a very probable chance that even when enabling the cache, we still leak statements in certain scenarios. In our user's case, the value is 1000 and there are probably thousands of different statements, we again reach the maximum and crash. I cannot reproduce the second case.

Here's more information in a separate issue: https://github.com/prisma/prisma/issues/6872

pimeys avatar May 31 '21 15:05 pimeys

This fixes the case where we disable the cache, but still use statements:

https://github.com/blackbeam/mysql_async/pull/159

The other leak I haven't yet found.

pimeys avatar May 31 '21 15:05 pimeys

The user reports how these problems started after our January 5th release. I can read from our commits some changes after that how we use mysql_async:

  • Upgrade from 0.24.2 to 0.26.0 (and later 0.27.1)
  • Upgrade to Tokio 1.0
  • We stopped using the internal pool of this crate in favor of mobc

I'm sorry this is a bit hard to reproduce...

pimeys avatar May 31 '21 16:05 pimeys

Ok, reopening this if there's a bug in the mysql_async LRU. I'm definitely just utilizing the LRU we use for tokio_postgres, so that's a good way for me to deal with this now.

pimeys avatar Jun 01 '21 08:06 pimeys