bun
bun copied to clipboard
SegmentationFault when running async function containing `bun:sqlite`
What version of Bun is running?
Bun v0.2.0
What platform is your computer?
Linux 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00:30:59 UTC 2022 x86_64 x86_64
What steps can reproduce the bug?
The following async function should run at a set interval but sometimes results in a SegmentationFault at 0x7000000000000000
and does not run at all.
Since there are many moving parts to the code, I have not created a reproducible example for now.
Code
One second Interval that calls the getHistory
method on the minute mark:
setInterval(function() {
if ( new Date().getSeconds() === 0 ) environment.getHistory(optimalPortfolio);
},1000);
Statements to create the database tables:
db.exec(
`CREATE TABLE IF NOT EXISTS assets_history (timestamp TEXT NOT NULL, symbol TEXT NOT NULL, open REAL, high REAL, low REAL, close REAL, volume INTEGER, number INTEGER, vw_avg REAL, ema_30 REAL, ema_60 REAL, PRIMARY KEY (timestamp, symbol))`
);
db.exec(
`CREATE TABLE IF NOT EXISTS portfolio_history (timestamp TEXT NOT NULL, open REAL, high REAL, low REAL, close REAL, volume INTEGER, number INTEGER, vw_avg REAL, ema_30 REAL, ema_60 REAL, PRIMARY KEY (timestamp))`
);
Async function to retrieve raw data, transform it and insert it into the database.
async getHistory(portfolio): Promise<void> {
const sltBtmLast = db.query(
`SELECT * FROM assets_history WHERE symbol = ? ORDER BY timestamp DESC LIMIT 1;`
);
const sltBtmSixty = db.query(
`SELECT * FROM assets_history WHERE symbol = ? limit 60`
);
const insAssHistRaw = `INSERT INTO assets_history
(timestamp, symbol, open, high, low, close, volume, number, vw_avg)
VALUES ($timestamp, $symbol, $open, $high, $low, $close, $volume, $number, $vw_avg);`;
const insAssHistInds = `INSERT INTO assets_history
(timestamp, symbol, open, high, low, close, volume, number, vw_avg, ema_30, ema_60)
VALUES ($timestamp, $symbol, $open, $high, $low, $close, $volume, $number, $vw_avg, $ema_fast, $ema_slow);`;
const insPflHist = `INSERT INTO portfolio_history
(timestamp, open, high, low, close, volume, number, vw_avg, ema_30, ema_60)
VALUES ($timestamp, $open, $high, $low, $close, $volume, $number, $vw_avg, $ema_fast, $ema_slow);`;
let symbols: string[] = portfolio.map((asset) => asset.symbol);
//Get raw data and insert into assets_history
try {
await this.restClient
.getLatestMultiBars({
symbols,
})
.then((res) => res.json())
.then((json) => json["bars"])
.then((bars) => {
Object.entries(bars).forEach(([symbol, bar]: [string, any]): void => {
let emaSlow: number[];
let emaFast: number[];
const nyTimestamp = subtractHours(4, new Date(bar["t"]));
const closes = sltBtmSixty.all(symbol).map((row) => row.close);
// console.log(`symbol: ${symbol}\ncloses: ${closes}`); //for testing
if (closes.length < 60) {
console.log(
`Not enough data points to calculate EMA for ${symbol}`
);
db.run(insAssHistRaw, {
$timestamp: nyTimestamp,
$symbol: symbol,
$open: bar["o"],
$high: bar["h"],
$low: bar["l"],
$close: bar["c"],
$volume: bar["v"],
$number: bar["n"],
$vw_avg: bar["vw"],
});
}
if (closes.length === 60) {
console.log(`calculating EMA for ${symbol}...`);
emaFast = ema({
period: 30,
values: closes,
reversedInput: false,
});
emaSlow = ema({
period: 60,
values: closes,
reversedInput: false,
});
db.run(insAssHistInds, {
$timestamp: nyTimestamp,
$symbol: symbol,
$open: bar["o"],
$high: bar["h"],
$low: bar["l"],
$close: bar["c"],
$volume: bar["v"],
$number: bar["n"],
$vw_avg: bar["vw"],
$ema_fast: emaFast[emaFast.length - 1],
$ema_slow: emaSlow[emaSlow.length - 1],
});
}
});
});
} catch (err) {
if (err.message === "constraint failed")
console.log(`Duplicate key: Unable to insert row into assets_history`);
}
//Calculate portfolio history
try {
let pflBar = portfolio
.map((asset) => {
const query = sltBtmLast.all(asset["symbol"]);
console.log(
`query[0] & asset inside portfolio.map (line 153)\nquery[0]: ${query[0]}\nasset: ${asset}`
);
return query[0];
})
.forEach((asset) => {
console.log("asset inside of forEach (line 156): ", asset);
asset.reduce(
(acc, cur) => {
acc["open"] += cur["open"] * asset["w"];
acc["high"] += cur["high"] * asset["w"];
acc["low"] += cur["low"] * asset["w"];
acc["close"] += cur["close"] * asset["w"];
acc["volume"] += cur["volume"] * asset["w"];
acc["number"] += cur["number"] * asset["w"];
acc["vw_avg"] += cur["vw_avg"] * asset["w"];
if (cur["ema_30"] && cur["ema_60"]) {
acc["ema_30"] += cur["ema_30"] * asset["w"];
acc["ema_60"] += cur["ema_60"] * asset["w"];
} else {
acc["ema_30"] = null;
acc["ema_60"] = null;
}
return acc;
},
{
open: 0,
high: 0,
low: 0,
close: 0,
volume: 0,
number: 0,
vw_avg: 0,
ema_30: 0,
ema_60: 0,
}
);
});
const nyTimestamp = subtractHours(4, new Date());
db.run(insPflHist, {
$timestamp: nyTimestamp,
$open: pflBar["open"],
$high: pflBar["high"],
$low: pflBar["low"],
$close: pflBar["close"],
$volume: pflBar["volume"],
$number: pflBar["number"],
$vw_avg: pflBar["vw_avg"],
$ema_fast: pflBar["ema_30"],
$ema_slow: pflBar["ema_60"],
});
} catch (err) {
if (err.message === "constraint failed")
console.log(
`Duplicate key: Unable to insert row into portfolio_history`
);
}
}
How often does it reproduce? Is there a required condition?
Half of the time, the code throws the segfault the first time the getHistory
is called. Other times it works as expected.,
What is the expected behavior?
No segfault.
What do you see instead?
SegmentationFault at 0x7000000000000000
----- bun meta -----
Bun v0.2.0 (b542921f) Linux x64 #1 SMP Wed Mar 2 00:30:59 UTC 2022
AutoCommand: dotenv
Elapsed: 45362ms | User: 925ms | Sys: 566ms
RSS: 0.14GB | Peak: 0.23GB | Commit: 0.14GB | Faults: 0
----- bun meta -----
Additional information
No response
@Electroid I am keen to know if you see any issues resulting from bugs in my code, or whether this is a segfault caused by Bun? Perhaps there is a way I can generate a core file to debug and see if any lines are causing the issue?
By the way, I made some slight updates to the code above for easier readability. I have also updated Bun to v0.2.2
with the same result.
0x7 is “null” in JS so it looks like something internally is being set to null unexpectedly
One thing that sticks out is new Date() - it’s calling toString() on that most likely. If you manually call .toString() on the Date does that workaround the issue?
Can you download a -profile build of Bun? That will include a stack trace with debug symbols so that we can see more detailed information
Thanks, @Jarred-Sumner, for the reply. the new Date()
doesn't seem to be the issue. I tried swapping it for a string constant, and the same issue occurred.
The following code should work as far as I can tell, however it also crashes with the same segfault:
setInterval(async function () {
if (new Date().getSeconds() === 0) {
await restClient
.getLatestMultiBars({
symbols: ["AAPL", "TSLA"],
})
.then((res) => res.json())
.then((json) => json["bars"])
.then((bars) => {
Object.entries(bars).forEach(([symbol, bar]: [string, any]): void => {
console.log("bar=", bar);
console.log("symbol=", symbol);
});
});
}
}, 1000);
I don't see any issue with the above code, so I suspect the issue is in Bun, perhaps only for Linux WSL.
Can you download a -profile build of Bun? That will include a stack trace with debug symbols so that we can see more detailed information
I tried to set up the development environment for Bun previously, but to be honest, I failed and gave up. It might be that I don't have the hardware necessary but I'll nevertheless give it another try.
-profile
builds can be downloaded without compiling Bun from the releases and canary page
Here is the one for your OS + arch: https://github.com/oven-sh/bun/releases/download/bun-v0.2.2/bun-linux-x64-profile.zip
Thanks @Jarred-Sumner!
I have tried to run it with the bun-profile but am not able to make it run.
The debugger outputs:
Console is in 'commands' mode, prefix expressions with '?'.
Launching: /home/myuser/project/test/bun-profile /home/myuser/project/src/main.ts
Then a prompt shows up, which I have no idea of what it means:
This is what my launch.json
looks like:
{
"version": "0.2.0",
"configurations": [
{
"type": "lldb",
"request": "launch",
"name": "bun-profile current file",
"program": "/home/myuser/bin/bun-profile",
"args": ["${file}"],
"cwd": "${file}/../../",
"env": {
"FORCE_COLOR": "1"
},
"console":"internalConsole"
}
]
}
According to HERE the message 'A' packet returned error: 8
means:
This is the catch-all message for LLDB's failure to create the debuggee process. Some common reasons include:
- The current working directory ("cwd") does not exist or is not accessible by the current user.
- Debugging inside an unprivileged Docker container with disabled ASLR. While disabling ASLR is generally desirable during debugging (because it may cause non-determinism of program execution), doing so may be disallowed by the default container security profile. Possible solutions: ** Relax container security profile by starting it with --security-opt seccomp=unconfined option. ** Turn off ASLR disabling by adding "initCommands": ["set set target.disable-aslr false"] to your launch configuration.
My file privileges:
drwxr-xr-x 2 myuser myuser 4096 Oct 30 00:06 ./
drwxr-xr-x 16 myuser myuser 4096 Oct 30 00:06 ../
-rw-r--r-- 1 myuser myuser 124193808 Oct 28 01:16 bun-profile
-rw-r--r-- 1 myuser myuser 90 Oct 28 01:16 bun-profile:Zone.Identifier
After changing permissions to write -rwx rx rx
I think I was able to debug.
The debug console shows:
Console is in 'commands' mode, prefix expressions with '?'.
Launching: /home/myuser/bin/bun-profile /home/myuser/project/src/main.ts
Launched process 22203
[0.33ms] ".env"
RestClient instance created
Stop reason: signal SIGSEGV: invalid address (fault address: 0x70)
SegmentationFault at 0xB4560000E8030000
----- bun meta -----
Bun v0.2.2 (6632135e) Linux x64 #1 SMP Wed Mar 2 00:30:59 UTC 2022
AutoCommand: dotenv
Elapsed: 79168ms | User: 273ms | Sys: 35ms
RSS: 69.21MB | Peak: 68.08MB | Commit: 69.21MB | Faults: 0
----- bun meta -----
Ask for #help in https://bun.sh/discord or go to https://bun.sh/issues
Process exited with code 139.
The file @us_socket_context_get_native_handler
shows:
; id = {0x00013ed9}, range = [0x0000000003f1b000-0x0000000003f1b011), name="us_socket_context_get_native_handle"
; Source location: /build/bun/src/deps/uws/uSockets/src/context.c:143
03F1B000: 85 FF testl %edi, %edi
03F1B002: 74 0A je 0x3f1b00e ; <+14> at context.c
03F1B004: 55 pushq %rbp
03F1B005: 48 89 E5 movq %rsp, %rbp
03F1B008: 48 8B 46 70 movq 0x70(%rsi), %rax
03F1B00C: 5D popq %rbp
03F1B00D: C3 retq
03F1B00E: 31 C0 xorl %eax, %eax
03F1B010: C3 retq
I have updated the issue title and description.
Changing .then()
to full await gives the same result; most of the time it throws the segfault but sometimes it gives the expected result.
@jastraberg can you try in bun upgrade --canary
I made some tweaks last night to this code
In canary, both full await and then()
throws a segfault.
Can you try once again?
Sorry Jarred, it still seems to throw the segfault with the latest canary.
SegmentationFault at 0x7000000000000000
----- bun meta -----
Bun v0.2.3 (cb41d77d) Linux x64 #1 SMP Wed Mar 2 00:30:59 UTC 2022
AutoCommand: dotenv
Elapsed: 37065ms | User: 24ms | Sys: 169ms
RSS: 69.21MB | Peak: 63.21MB | Commit: 69.21MB | Faults: 0
----- bun meta -----
I'm going to need a complete repro to fix this
Can you paste a minimal reproduction? Thanks for being patient with all the back-and-forth
No problem @Jarred-Sumner. Here is a minimal reproduction (I will PM you the key and secret):
const apiKey = 'KEY';
const apiSecret = 'SECRET';
async function getLatestBar() {
const path = `AAPL/bars/latest`;
const url = new URL(path, 'https://data.alpaca.markets/v2/stocks/');
try {
return await fetch(url.href, {
method: "GET",
headers: new Headers({
"APCA-API-KEY-ID": apiKey,
"APCA-API-SECRET-KEY": apiSecret,
"Content-Type": "application/json",
}),
redirect: "follow",
}).then((res) => {
if (res.ok) return res;
else {
throw new Error(`status: ${res.status}\nstatusText: ${res.statusText}\nType: ${res.type}`);
}
});
} catch (error) {
console.log(error);
}
}
setInterval(async () => {
if (new Date().getSeconds() === 0) {
await getLatestBar().then((res) => res.json()).then((json) => console.log(json));
}
}, 1000);
sounds good