long2ice
Packet sequence number wrong
Running wrk against a web page with a simple select caused my logs to fill up with errors. Aiomysql has no problem with the same exact code. I'll create a simple test later to reproduce this.
Unhandled exception:
<class 'asyncmy.errors.InternalError'>
InternalError('Packet sequence number wrong - got 102 expected 1')
Unhandled exception:
<class 'asyncmy.errors.InternalError'>
InternalError('Packet sequence number wrong - got 112 expected 1')
Using create_pool
await aiomysql.create_pool( autocommit=True, minsize=4, maxsize=20)
The page handler is just a simple select
async with app.mysql_pool.acquire() as conn:
async with conn.cursor() as cur:
await cur.execute("SELECT * from table where ... ");
The wrk cmd
wrk -t2 -c8 -d2s http://localhost/select
@trim21 it's a concurrcy bug that not easy to repoduce all the time. It could be happen when your program in high loads.
@trim21 it's a concurrcy bug that not easy to repoduce all the time. It could be happen when your program in high loads.
I understand, can you provide a minimal example?
@trim21 I think the first example could reproduce this issue. But you may need try it multiple times
@trim21 I think the first example could reproduce this issue. But you may need try it multiple times
sadly I can't.
import asyncmy # 0.2.9
import fastapi
app = fastapi.FastAPI()
@app.on_event("startup")
async def startup_event():
app.mysql_pool = await asyncmy.create_pool(
autocommit=True,
minsize=4,
maxsize=20,
...
)
@app.get("/test")
async def test():
async with app.mysql_pool.acquire() as conn:
async with conn.cursor() as cur:
await cur.execute("...")
with uvicorn a:app --no-access-log and hey -z 3m -c 16 http://127.0.0.1:8000/test on win10/ubuntu
In my case using SQLAlchemy and FastAPI it always throws this error for this specific handler when querying for a large amount of rows. This still works locally for small dataset (dev database). Code obfuscated for legal reasons:
async def public_sitemap():
async with get_async_session() as session:
query = (
select(
SAC.name, # type: ignore
SAC.s.label('cs'), # type: ignore
SAC.ca.label('cca'), # type: ignore
SAC.ua.label('cua'), # type: ignore
SA.s.label('as'), # type: ignore
SAR.y,
SA.ca,
SA.ua
)
.select_from(SAC)
.join(SACM)
.join(SA)
.join(SAR)
.where(filter_cond())
)
categories = {}
for row in (await session.execute(query)).fetchall():
if row.cs not in categories:
categories[row.cs] = {
"n": row.n,
...
"as": []
}
categories[row.cs]["as"].append({
"t": row.t,
...
})
return {
"categories": list(categories.values())
}
While other handlers with similar or even heavier SQLAlchemy usage perform without any issue. I switched to aiomysql and it doesn't seem to error out anymore.
