restriced webxdc internet access

[r10s]

in the future, we want some selected, probably internal, webxdc to access the internet, eg. a webxdc showing maps (needs to download tiles) or a video chat (need webrtc).

this pr is about preparing this future:

• offer a way to test webxdc with internet access on real devices
• give the UIs time to implement internet access

for that purpose, the UIs should check the field internet_access from dc_get_webxdc_info() - if that is true, internet access should be allowed for that webxdc.

currently, this flag will be set if the webxdc requests internet access and the webxdc is encrypted and the webxdc is placed in the self chat ("Saved Messages"). we selected the self chat as no data can be leaked there and it is seems sufficient for some first test.
the condition may change in the future to whatever, however, the work for the UIs would probably be roughly the same.

rough idea is also that the "integration" webxdc as map, video chat or an image editor are also placed in self chat, however, will be wired internally differently. that way, development would also be very easy and also ppl could replace the "shipped" webxdc (which would maybe be hidden).

things may change, however this is the current plan as discussed recently with @hpk42 and others :)

[Simon-Laux]

pr looks good, though we should really think about full internet access, so that nothing can be leaked. like mutually exclusive modes:

• internal: special dc api's, but no connectivity (no network nor status updates)
• normal: no dc access and no internet access
• internet-connected: definitively no dc access, maybe status updates in groups

It's really important to come up with some concept here before webxdc devs might talk us into opening up everything just for their convenience even if its not needed at all.

whatever we do we should preserve privacy by default, I would prefer postponing the need for a permission dialog for now.

[r10s]

a permission dialog is not needed currently.

for now, this is only for the very limited usecase as described above and to get UIs ready to listen to the flag. note, that internet access for "normal" webxdc is not planned at all currently.

and sure, on all adaptions and iterations, we need to be super-careful.

[link2xt]

It's possible to create an app requesting internet access, and if user sends it to self-chat, it will connect somewhere, leaking user IP, screen resolution etc?

Another example: user gets a "notes app" somewhere, forwards it to self-chat from some public chat and starts taking notes there, then the app can leak notes to remote server.

[link2xt]

Btw I don't really like the idea of abusing webxdcs for logs (#3296), maps etc., as it assumes the platform has webview, which is not always the case. We already have connectivity view which is unusable in TUI clients.

[r10s]

thanks a lot for the detailed analysis.

in fact, by the self+encrypted approach we target the threat that no other person can send you a webxdc that then can send data outside. if we regard the threat of self-harming as a real-world issue, maybe we could force the summary of these webxdc to display a warning (an additional debug-mode or so would maybe not help much here, also i like to have no extra ui for that as it may be removed by other approaches).

to the usage of webxdc internally: let me note that i personally also prefer native things, however, this is about a pragmatic approach. eg. creating map views natively for android/ios/desktop is a lot of effort, not to speak about the idea of later replacing that by peermaps [^1]. and even if we would do that natively, it would not help TUI or other clients.
to help TUI or other clients, of course, we can add non-webxdc/html api as needed, eg. we could expose connectivity data - and have another layer that generates html as needed. (the connectivity view as such already saved quite some development time, eg. the quota stats were added "just so" to all UIs).

[^1]: peermaps on webxdc is becoming real, currently a hacked desktop is used, however, there is not really an easy way to test that on android/ios at some scale, which is not so good, cc @ralphtheninja

[ralphtheninja]

@r10s Nice to see that we're making progress here. With regards to the hack in desktop, this was mostly done in order to get a POC working for getting data from ipfs. I know this is about internet connectivity in general but I'd just like to remind you that peermaps ideally should get data from p2p sources, even though it can get data from a data source over http as well. So in the long run, I'd love to see support for ipfs (or other p2p protocols for that matter), since this would also allow deltachat to help out serving map data for other deltachat clients/nodes (or other peermaps nodes using that same p2p protocol).

Since I haven't been around for quite some time in the deltachat world I'm having some dificulties seeing what's needed from the peermaps side. What's needed more specifically from a webxdc app in order to get this internet connectivity and what else is missing? E.g. what kind of flag is needed in the manifest, what does it mean for a webxdc app to be encrypted?

Also, what does TUI mean here?

[link2xt]

Also, what does TUI mean here?

Text-based User Interface, e.g. https://github.com/adbenitez/deltachat-cursed

[r10s]

What's needed more specifically from a webxdc app in order to get this internet connectivity and what else is missing? E.g. what kind of flag is needed in the manifest, what does it mean for a webxdc app to be encrypted?

with this pr, the line request_internet_access = true needs to be added to the manifest. internet access is then granted when the app is placed in the self-chat and the app is end-to-end-encrypted. the requirements ensure that no one else can send you an app in your name by forging From: somehow. droping a webxdc to the self-chat in delta chat fulfils both requirements.

native ipfs is not yet supported in the webviews, but who knows, maybe that changes and ipfs makes it to the browsers :) otherwise, we need some specific workaround somewhen later, that's true, however, another effort - and would also require this flag as we probably do not want to give random apps access to ipfs as well.

in the long term, the conditions may change, maybe we require signing, only internal apps or so. but whatever we do, it seems to be clear we want to have a flag where the core says to the UIs that internet access should be granted.

[Simon-Laux]

Core handles already blobs from the zip, nothing prevents us from serving ipfs resources at fake/virtual paths like /ipfs/hash or sth. like that. Though we should make sure those requests are async in all ui's (because of their networked nature). also maybe its possible to get the uis to make core the handler for all webrequests, so core could handle (or also have its own) REST APIs with GET, POST, PUT, DELETE and so on.

[ralphtheninja]

native ipfs is not yet supported in the webviews, but who knows, maybe that changes and ipfs makes it to the browsers :) otherwise, we need some specific workaround somewhen later, that's true, however, another effort

I wouldn't count on any webview supporting this in the coming 10+ years :smile: This is why I made the hack in desktop, so desktop proxies the call to fetch('ipfs://cid'). The webview should only handle what it currently can, render html and run javascript.

and would also require this flag as we probably do not want to give random apps access to ipfs as well.

Probably yes. I'm not too concerned about what we need to prevent, but more what we can enable. Not trying to say that this is unimportant, it definitely is important, I'm just more focused on painting with big brushes and not getting caught up in "details".

Core handles already blobs from the zip, nothing prevents us from serving ipfs resources at fake/virtual paths like /ipfs/hash or sth. like that.

Exactly! :rocket:

The following code https://github.com/deltachat/deltachat-desktop/blob/1290e4fb8309ec6541c8663e8d8141b81415221b/src/main/deltachat/webxdc.ts#L155-L184 is an example of this. It's async and a streaming protocol, so no buffering. Behind the scenes we call out to ipfs cat $cid but this could just as well be a call into deltachat core, assuming core has support for ipfs.

[r10s]

i am a bit confused, not sure if that pr is helpful or not :)

not sure if i got the whole picture, however, if the restricted internet access is not urgent for anything just now, maybe better close this pr for now and put it to resurrection until there is some need :)

for ipfs - i thought this is just working somehow from the browsers with some libraries that we can also use in the webviews, of course, with enabled internet access. but maybe this is not how it works :)

[ralphtheninja]

i am a bit confused, not sure if that pr is helpful or not :)

not sure if i got the whole picture, however, if the restricted internet access is not urgent for anything just now, maybe better close this pr for now and put it to resurrection until there is some need :)

I think it's super helpful. It would allow to run peermaps in the self-chat with full map support (using http storages) on all types of devices. It would still be centralized maps though since we will be getting data from a server somewhere.

[Septias]

Is there a reason on why this hasn't been merged yet? It would be nice to get this in so develepoment on debuglog gets nicer which would also help to close #3296 at some point.

[r10s]

Is there a reason on why this hasn't been merged yet?

no, it just seems not super-urgent, no concrete things are waiting for it, and in summer things seem to be a bit slower :)

It would be nice to get this in so develepoment on debuglog gets nicer which would also help to close https://github.com/deltachat/deltachat-core-rust/pull/3296 at some point.

how does this help with debuglog? imu, debuglog does not need internet access.

[Septias]

Aaaaahh, my bad. When we chatted a while back about adding some manifest parsing to give special rights to webxdcs you linked this issue. I just skimmed it and did't see it doesn't provide a general api to access manifest information. To take this idea a little bit further, wouldn't it be nice to have said api to interact with the manifest? I believe that there might be a range of different features hidden by an access-border in the future. For example access to geolocation, bluetooth, device rotation etc. which could then be requested by the webxdc through the manifest.

[r10s]

It's possible to create an app requesting internet access, and if user sends it to self-chat, it will connect somewhere, leaking user IP, screen resolution etc?

Another example: user gets a "notes app" somewhere, forwards it to self-chat from some public chat and starts taking notes there, then the app can leak notes to remote server.

UIs will print a show now, see commit message for details - this was easy to do and will mitigate these threats.