chatmail
Debug logging, second try
- [X] config value
- [X] delete msg (avoid recursion)
- [X] send update (on event)
- [X] log to status update
- [x] the actuall log viewer webxdc
- [x] ~~benchmarks~~
- [x] show config in UI https://github.com/deltachat/deltachat-android/pull/2292 https://github.com/deltachat/deltachat-desktop/pull/2760 (TODO: iOS, but this can be after merging this PR)
Unfortunately, this currently makes the core about 3% slower even if debug logging is disabled. We probably need to speedup the context.get_config_int(Config::DebugLogging).await, either by improving config caching speed or by saving whether logging is enabled in an extra property.
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 7.0s, or reduce sample count to 70.
create 500 contacts time: [68.702 ms 69.948 ms 71.355 ms]
change: [+12.065% +13.966% +15.964%] (p = 0.00 < 0.05)
Performance has regressed.
Benchmarking create 100 contacts and read it 1000 times: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 19.5s, or reduce sample count to 20.
create 100 contacts and read it 1000 times
time: [192.75 ms 193.25 ms 193.78 ms]
change: [+5.3167% +5.8627% +6.3170%] (p = 0.00 < 0.05)
Performance has regressed.
create 1 account time: [365.37 us 377.88 us 390.31 us]
change: [-10.608% -6.9846% -3.4697%] (p = 0.00 < 0.05)
Performance has improved. (COMMENT: this benchmark seems to be flaky)
Benchmarking chat::get_chat_msgs (load messages from 10 chats): Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 5.1s, or reduce sample count to 90.
chat::get_chat_msgs (load messages from 10 chats)
time: [50.839 ms 50.941 ms 51.047 ms]
change: [-2.9890% -2.4207% -1.9397%] (p = 0.00 < 0.05)
Performance has improved.
Benchmarking chatlist:try_load (Get Chatlist): Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 8.3s, or reduce sample count to 60.
chatlist:try_load (Get Chatlist)
time: [81.984 ms 82.297 ms 82.630 ms]
change: [+1.8212% +2.4065% +2.9372%] (p = 0.00 < 0.05)
Performance has regressed.
Benchmarking Receive messages/Receive 100 simple text msgs: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 17.2s, or reduce sample count to 20.
Receive messages/Receive 100 simple text msgs
time: [149.81 ms 151.28 ms 152.96 ms]
change: [+3.0344% +3.9571% +5.1635%] (p = 0.00 < 0.05)
Performance has regressed.
Benchmarking search hello: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 46.0s, or reduce sample count to 10.
search hello time: [454.15 ms 455.11 ms 456.13 ms]
change: [+1.2180% +1.4849% +1.7638%] (p = 0.00 < 0.05)
Performance has regressed.
Unfortunately, this currently makes the core about 3% slower
great you have that in mind and continue benchmarking 👌
where is the source code of the debugger webxdc?
https://github.com/Septias/webxdc_logging
Edit from Hocuri: https://github.com/webxdc/webxdc_logging now
The current main problem with this is: When I enable debug logging and leave it enabled for some days, then the webxdc doesn't load anymore, probably because there are too many status updates (it stays blank for some time and then just closes itself).
So, we have to somehow improve the webxdc performance for this to be useful.
The problem is easily reproducible with this diff:
diff --git a/src/config.rs b/src/config.rs
index d82c6036..886adffe 100644
--- a/src/config.rs
+++ b/src/config.rs
@@ -341,6 +341,13 @@ pub async fn set_config(&self, key: Config, value: Option<&str>) -> Result<()> {
self.debug_logging
.store(msg_id.to_u32(), atomic::Ordering::Relaxed);
}
+
+ let context = self.clone();
+ async_std::task::spawn(async move {
+ for _ in 1..10_000 {
+ info!(context, "Events stress test");
+ }
+ });
}
_ => {
self.sql.set_raw_config(key, value).await?;
then turn the logging off and on again.
fixed some stuff but can't test it on a real device since I haven't setup android-dev, so you should maybe test it for me @Hocuri Is there already a desktop-version that has a switch to enable logging? @Simon-Laux
Better already, it doesn't crash anymore but I have to wait like 2min and then it actually shows some data. However after closing and opening the webxdc again, it didn't show any data at all anymore ("No data for table" or so).
no switch yet
but I can easily add one when you rebase to master, so that I can just git install the core repo which now contains deltachat-node.
no switch yet
but I can easily add one when you rebase to master, so that I can just git install the core repo which now contains deltachat-node.
Done, you can add the button now
There also si no production-ready (tested on all platforms) webxdc and @link2xt rightfully notes it might be better to simply offer a "get log events" api which is usable by all clients and bots. Probably this is best done then via jsonrpc and would later be a good first example case to add a jsonrpc-API for Android and iOS? Those platforms are in most need of showing more info about the last events from previous runs of the app (apps often get killed).
There also si no production-ready (tested on all platforms) webxdc
What about debuglog @hpk42? Is that what we want to use? I can maybe push it forward
There also si no production-ready (tested on all platforms) webxdc
What about debuglog @hpk42? Is that what we want to use? I can maybe push it forward
just had a small discussion with hocuri and link2xt about it but it wasn't conclusive yet. Main usability issue for webxdc development is being able to post a new .xdc app to "saved messages" and when you click it, it shows you all the past logs (not just the ones since you added the xdc to the chat). This way you could also test an xdc on android while developing it on desktop (just post to saved messages, open on android to look at the last hours worth of logs). This does not currently work as the xdc is hard-wired into the PR, giving very long development cycle (you need android building etc.). We discussed writing the log lines into a separate dedicated sql logging table, and when an xdc with a "debuglog" marker in the manifest is added to "saved messages", starting that app means getting access to the log lines as application updates (so just like it is now). It also means providing a jsonrpc (or CFFI) API is quite easily done.
I rebased this to @Hocuri so we can start working on it again. Maybe try have a look if I missed some stuff... Also, should we create a new clean pr because the git history is pretty messy for this one ?
@hpk42, sounds reasonable and is in line with what I already thought about. Should I help with this or is someone caring about it?
Finished the discussion with @hpk42 and @link2xt. Results:
What has to be done on the PR here:
- If a message is sent into "Saved Messages" AND it's called
webxdc_debug_logging.xdc(if you @Septias like another name better, go for it), the DebugLogging config andcontext.debug_loggingis set to its MsgId. Quoting @hpk42: This way you could also test an xdc on android while developing it on desktop (just post to saved messages, and open on android) This does not currently work as the xdc is hard-wired into the PR, giving very long development cycle (you need android building etc.). - The rest of the things described in @hpk42's comment are not necessary to get this working, so we don't do them (for now).
- All the code around adding the hard-wired webxdc to the device chat (
let data: &[u8] = include_bytes!("../assets/webxdc_logging.xdc");and so on) can be removd, then.
Also, should we create a new clean pr because the git history is pretty messy for this one ?
Sure, just create one, put a link to this one (and maybe a summary of what happened so far, if you feel that it's worth it) into the description, and close the one here.
Should I help with this or is someone caring about it?
Nobody is doing it currently, so, yes, do this :))
It also means providing a jsonrpc (or CFFI) API is quite easily done.
What exactly do you mean by this @hpk42 ?
@Septias that it would be easy to add an ffi get_debug_logs(), if we chose writing the log lines into a separate dedicated sql logging table. But we decided that we won't (for now), so, you don't need to worry about this.