let web apps register CSP rules

[kulmann]

Description

The current implementation of registering CSP rules is not flexible enough and frequently breaks ocis instances (at least the web ui). Since CSP rules are for the web client it would be good if we'd have a kind of "base CSP rule set" and then let web apps register additional CSP rules, merging them all together. I'll give examples below.

User Stories

• As a web app developer I want my app with external sources to work out of the box so that the installation experience is as simple as possible for the ocis admin.

Value

Easy setup, less failures in the web ui when installing a web app.

Status quo / context / examples

Web apps currently come with a manifest.json which defines the entrypoint .js file and can set default config for an app. Optionally we allow the admin to place a config.json file in the same folder as the manifest.json to define instance specific config for the app which survives an app update. This is already great! Sometimes CSP rules need to be added so that the web app works as desired. See examples below

Examples:

• We have an external-sites app for adding external sites to the app switcher. You can decide to add those embedded into an iframe, which makes really nice integration of websites possible with quite good UX for the users. If you decide to add wikipedia.com as embedded external site you need to add some CSP rules for wikipedia.net and wikipedia.com to your csp.yaml file. This breaks the previously described great admin experience. It would be a nice admin experience if you could add the additional csp rules to the manifest.json (app defaults) and config.json (admin config) where you define the app switcher entry.
• We have an app-store app which fetches a json file from raw.githubusercontent.com. The app-store app was added as default app in ocis 6.3.0. The app-store app is broken for anyone who updates their ocis instance and has a custom csp.yaml file (because we only have the chance to add the csp rule to the default csp.yaml in the ocis proxy). In this case it would be great if the app store app would register csp rules dynamically which are then merged with the admin-provided csp.yaml.

Concerns

• Security: Trusting a web app to only register harmless CSP rules is short sighted. We'd need to make it very very clear to the admin which CSP rules are being added if an admin decides to install a web app.

Acceptance Criteria

• the manifest.json of an app can register additional CSP rules
• the config.json of an app can register additional CSP rules

Definition of ready

• [ ] Everybody needs to understand the value written in the user story
• [ ] Acceptance criteria have to be defined
• [ ] All dependencies of the user story need to be identified
• [ ] Feature should be seen from an end user perspective
• [ ] Story has to be estimated
• [ ] Story points need to be less than 20

Definition of done

• Functional requirements
  • [ ] Functionality described in the user story works
  • [ ] Acceptance criteria are fulfilled
• Quality
  • [ ] Code review happened
  • [ ] CI is green (that includes new and existing automated tests)
  • [ ] Critical code received unit tests by the developer
• Non-functional requirements
  • [ ] No sonar cloud issues
• Configuration changes
  • [ ] The next branch of the ocis charts is compatible

[kulmann]

This story is crucial for the success of an upcoming web app ecosystem.

cc @JammingBen @dschmidt @AlexAndBear @micbar @tbsbdr

[jvillafanez]

Proposal 1

The basic process will use the following steps:

1. The web app will generate a key pair (private + public keys)

2. Web app sends to POST /webapp (or to another endpoint) the public key and the encrypted manifest.json files.

3. The server responds with the status of the app:

   • already installed
   • installation denied
   • installation approved
   • .....

   For the "installation approved" the server will respond with an id (likely random containing app name) and an encrypted symmetric key. In the rest of the cases, there isn't anything else to send other that the status. Note that in those cases, the web app won't proceed further

4. The web app sends the required data (the CSP rules in this case, although there could be more data in the future) encrypted with the symmetric key, to the POST /webapp/{id} endpoint. In addition, the web app sends (as header, for example) the sha256 of the symmetric-key-encrypted manifest.json file.

From a security perspective, it should work:

• For the key pair in step 1, the idea is to generate a new key pair each time the web app starts the process. However, since it could become expensive, the web app can store and reuse a previously generated key pair. If you opt for storing the key pair, you'll need to securely store it, and make sure the key expires at some point.
• For the step 2, encrypting the manifest.json file (with the private key) and sending the public key will give us some benefits:
  • The server will be able to decrypt the manifest.json file, otherwise it can send an error to abort the process.
  • The server can use the public key to communicate to the web app securely, with the guarantee that only the web app will be able to read the message.
  • The server can have some verification on the manifest.json file, such as web app not allowed, newer version already installed, etc.
  • Note that the manifest.json file is expected to have public information. Anyone, not just the server, can decrypt the file and have access to the contents of the manifest.json file.
• During the step 3, if the "installation" is approved, the server sends a symmetric key encrypted with the public key that was sent in step 2. This guarantees that only the web app will know the symmetric key. The symmetric key will have some additional requirements in order to increase the security:
  • It must be randomly generated (not using a static pre-generated key)
  • It must be valid for a specific amount of time, such as 15-30 minutes. Using the symmetric key after that time must return an error even if the key is valid.
• In step 4, the symmetric key is only known by the server (which generated the key randomly) and the web app (which decrypted the symmetric key using its private key). This means that the data transferred in step 4 is secured.
• The sha256 sent during step 4 will act as a proof: the data (CSP rules in this case) is aimed for that manifest file and is encrypted with the server's key, ensuring that the data comes from a known source.

Note that additional verification might be added at steps 2 and 3 in order for the server to decide whether the web app can be "installed" or not, such as requests coming from a specific subnet, or done by admins. The web app might need to send additional data in order to fulfill those extra requirements.

Data persistence in the server

It should be possible to do the whole flow without persisting data, just using the memory.

• All the data can be removed after the process is over.
• Data can be stored in memory, but it should be removed after 30 minutes (when the symmetric key becomes invalid)
  • There could be fixed "data buckets" so the server can reject further requests if all the buckets are filled. This can help to prevent collapsing the server if there are too many "installation" requests
• Process can be restarted any number of times.

However, we might still want to persist some data in order to improve the flow.

• Persisting the manifest.json file along with additional information such as who sent ("installed") the file and when, has a couple of advantages:
  • We know who and when it happens, so we can have some traceability for this feature.
  • It makes easier to respond with "already installed" further requests matching the same manifest.json file, so the flow can be stopped earlier.

Note that we might still want to use a key-value store instead of memory if we want to use multiple replicas of the proxy service (assuming we want to implement this there). In addition, as said above, the whole process should have a 30 minutes limit (or lower), and we should ensure that all the data (except maybe the manifest.json file) is removed from the server after the time limit. This means that we need to consider that some data might not be available (because the request came too late, for example) for the server to send a successful response.

Open considerations

We need a way for the web app to trigger the flow.

Ideally, the web app could make a request to a "CSP protected" endpoint. If the request fails because the CSP rules don't allow the connection, the web app could trigger the flow explained above. This idea should work well on most scenarios because we assume we'll be able to connect where we want, and we can request access if we don't connect. The flow should happen only once because after the web app has been "installed" it should be able to connect wherever it wants. There are a couple of problems though:

• We need to reliably detect the CSP problem (not sure if possible)
• We'll rely on server's behavior on how the CSP is updated across oCIS (we need to deal with services replicas)

Another idea could be to keep track of the "installation" status in the client. Starting from a "not installed" state, the client can start the flow (maybe with some user interaction such as clicking a button) ending up in an "installed" state, which would be saved in the client. The "install" button might still be offered in case there are problems in order to restart the process and "re-install" the web app. The problem with this approach is that every user will trigger the "installation" process because the client doesn't have the current state, so it will need to go through steps 1 and 2 so the server can at least respond with an "already installed" response. We might need to add another endpoint in order to get the state of the app somehow instead of having to send the public key and the manifest.json file.

In theory, just triggering the flow every time the page loads should work, but it would generate unwanted load in the client and the network. This is why it's important to minimize the number of times we trigger the flow.

Dealing with multiple service (proxy) replicas.

Since the main goal is to update the CSP rules, it's expected that the code changes will be in the proxy service (this might change though).

The main problem is that we'll have to ensure all the replicas share the same information. This means that:

• All the transient data must be stored in a key-value store (not in memory) so any replica can access the information
• All persistent data must be stored in a common FS available to all the replicas.

Probably, the easiest option is that the proxy copies the current CSP.yaml file in a different place, updates the contents and then updates the CSP file location in order to point the the modified CSP.yaml file. The problem is that, somehow, it must trigger a config reload in all the replicas in order to update the loaded CSP data, otherwise some requests might get blocked (if the request go through a replica without the updated CSPs). As far as I know, there is no config reload action we could use at the moment. Killing and restarting the service might be an option, but I doubt it works without a proper service termination (right now I think it would kill the whole oCIS, which would be a big problem).

Other alternatives might include communication among replicas, which seem more complex.

Unless we ensure there will be only one replica of the proxy service (or the service implementing this feature), I think this is an open question we need to deal with before starting the implementation.

[kulmann]

@jvillafanez I think there is a misunderstanding here. We already have a mechanism for "installing" web apps into the web service. This issue is ONLY about an easier way to register CSP rules per app but staying within the bounds of the filesystem based app installation concept for now (see below).

Installing an app is currently file system based: apps are distributed as .zip file with a) a manifest.json file and b) all assets of the app (.js-bundle, images, etc). You can find examples by clicking any of the Download buttons in the new app store app. Those .zip files can be extracted into a custom apps (or whatever you want to name it) folder which you mount into ocis and let the web service know via the env var WEB_ASSET_APPS_PATH that this is the folder where apps reside. On each web service restart all apps inside that folder are being loaded from the filesystem, the manifest.json files get interpreted. The result is that the apps are being appended to the external_apps array within the payload of the config.json endpoint of the web service.

This issue here is ONLY about also allowing to register additional CSP rules for the apps. I.e. if I "install" the external-sites app into my ocis by extracting the external-sites-0.1.0.zip file into my apps folder and restarting ocis, I want the CSP rules which are needed for an embedded link to wikipedia.com to be loaded from within the external-sites app folder. Having to append them manually to the csp.yaml file is not so nice.

What you have been writing down here is already two steps further in the process. :-) Of course we want to have an endpoint for installing an app from the app store, but that's not relevant right now.

[jvillafanez]

I don't think I'm too far of the mark... maybe "installing" isn't the right word in my case.

I mean, if I understood correctly, what we have at the moment is, basically:

1. web client has app A and B
2. web client does its magic to install app C
3. now web client has app A, B and C, although C doesn't work properly due to CSP rules.

My proposal should be able to follow the next steps to let the web client send the CSP rules for app C to the server in a secure way.

In any case, I think my proposal just covers the communication between client and server. Creating / updating server files and manipulating services (likely just restarting or reloading the configuration) isn't defined yet. The data persistence section mentioned in the proposal is for the client-server communication, not for whatever changes the client wants to do in the server.

[kulmann]

2. web client does its magic to install app C

That's not correct. The admin extracts a zip file into a dedicated folder and restarts ocis (or only the web service). The web client doesn't do anything so far.

[jvillafanez]

Proposal 1 (part 2)

Communication between client and server is covered above, so let's focus on the server's actions.

We'll have a service hooked up to the "/webapp" and "/webapp/{id}" endpoints. This service might temporary be the proxy service if we don't want to create a new service for now. This service will send specific events (through nats) to notify what the client is requesting. The events must be consumed by all the services and services' replicas that are interested in that event.

Flow example:

1. We receive the CSP rules (and maybe other things) in the request body of the "/webapp/{id}" endpoint
2. We send a "WebAppCSPEv" event, which contains the CSP rules (and maybe other data that could be useful)
3. It's expected that only the proxy service is listening to this event (the rest of the services shouldn't be interested)
4. The proxy service updates the CSP rules with the data coming from that event.

There are some important notes to take into account:

• Unless there is a good reason, all the data should be contained inside the event so the target service doesn't need to lookup for additional data.
• It's up to the service to decide what to do with the event.
  • The target service might decide to just update its running configuration. Restarting the service might lose the configuration changes.
  • The target service might update a configuration file in the FS. It might need a manual restart to apply the configuration.
  • The target service might do both.
  • The target service might update a configuration file and try to reload it by restarting itself.
• The previous point also applies to service's replicas.
  • All service's replicas might send an additional event to decide which one will update the configuration file (assuming all of them share the same FS)
  • Another event might be sent from all the replicas in order to know when they have finished updating.

Advantages of this approach:

• The service hooked up to the "/webapp" and "/webapp/{id}" endpoints might send additional events in the future. For example, a "WebAppInstallPackageEv" that the web service could be listening in order to setup the new webapp.
• Only specific services will listen to specific events. For example, only the proxy service will listen to the "WebAppCSPEv". The rest of the services don't need another connection to the nats service.
• The target service (consuming the event) will have full control on the expected behavior.

Another more complex flow:

1. "InitialService" implementing the webapp endpoints (likely proxy service) receives the CSP rules.
2. "InitialService" sends the "WebAppCSPEv" event with the CSP rules through the nats.
3. The proxy service (which is listening to the "WebAppCSPEv" event) wants to update the CSP rules accordingly.
4. Since the proxy service has multiple replicas, all the replicas have seen and read the "WebAppCSPEv"
   1. Let's assume that all the replicas know how many of them are available. So if we have "replica1", "replica2" and "replica3", all of them know there are 3 replicas (counting themselves).
   2. All the replicas update the CSP rules they have in memory according to the "WebAppCSPEv" event they received.
   3. In order to decide which replica updates the shared csp.yaml file, they send a "ProxyRepConsensusEv" containing a unique id identifying the replica with a random number between 0 and 65000 (or max integer). The id with the highest number will write the changes.
      • We can set a timeout in case one replica is down and can't respond, so we don't wait indefinitely.
      • 2 replicas getting the same random number shouldn't happen, specially with a big number range. If it happens, it's possible to repeat the operation.
   4. All the replicas will send a "ProxyRepCSPDoneEv" event containing the status of the operation for that replica.
   5. The replica chosen in step 3 will gather all the status from all the replicas and send a "WebAppCSPDoneEv" event containing the final status (whether all the replicas updated correctly, or one of them failed to update, or every replica failed).
5. Once the "InitialService" has received the "WebAppCSPDoneEv", it can respond to the "/webapp/{id}" with the correct status.

Note that step 4 is the part where the target service (proxy in this case) can do whatever it wants. It's possible to simplify those inner steps by taking some risks:

• Replicas might want to overwrite the csp.yaml file anyway. Instead of deciding who is going to write the file, they could fight for it and overwrite the contents of needed: replica1 could lock the file, write the contents and unlock the file; if replica2 tries to get the lock, it should fail, so it can assume the file is being written and can skip that step; replica3 could overwrite the file again with the same contents a few moments later. We might need different safety measures for this case though.
• All the replicas might send the "WebAppCSPDoneEv" event, although the "InitialService" might wait only for the first one. This could cause timing issues if the next request ends up in a proxy replica that hasn't updated the CSP yet, or if another replica had problems updating the CSP rules.
• One of the replicas could update the csp.yaml file and send an event to all the replicas in order to force a restart, so all the replicas would have the CSP updated by reloading the csp.yaml file. Note that this "restart" event is only for this service's replicas and shouldn't be used from any other service.
• We could require that there won't be any proxy replica (only one proxy service), so that step 4 should be simplified considerably.

This "InitialService" might grow too big to be inside the proxy, so it might need to become its own service in the future. Even if we implement everything inside the proxy now, implementing the flow as described should help when we decide to create a specific service to handle the modification of service files.

[kulmann]

@jvillafanez I'm going to collapse your comments, they are not actionable for this issue. Please don't invest such time at the moment. There is no endpoint for anything, yet, neither is one planned short term. Everything is done via config files which are being interpreted at service start time and for the moment that's good enough. Of course we could already discuss a new world where everything is done via network communication, but I don't want to think that far ahead.

[kulmann]

Ah, @fschade I forgot you in the initial CC.

[butonic]

AFAIU this issue is about adding a CSP by reading it from the apps manifest.json. The admin can override the CSP with a config.json.

Regarding security I'd prefer the manifest.json and config.json files to only be evaluated when the web service starts. Dynamically reloading the config is possible, but having to restart the service as a dedicated step IMO can prevent admins from installing malicious apps on the fly. We could also read the CSP from the manifest and render it prominently in the appstore.

[butonic]

hm allowing a CSP to raw.githubusercontent.com seems to open a can of worms, doesn't it? anyone can host content there.

[fschade]

i have my concerns too, the manifest.json should not be able implicitly inject a rule... sounds like a big backdoor to me...

but maybe we could..., the manifest can request a required rule and the admin gets a warning that the rule should be added to the proxy manually...

[micbar]

We have a hard time estimating that because we have no idea how to implement without opening the doors.

[butonic]

ok, the URLs can contain path, so we can limit it to our web app.

Still, I would prefer the web ui to show an explanation of what the admin needs to add to the csp.yaml file.

[AlexAndBear]

ok, the URLs can contain path, so we can limit it to our web app.

Still, I would prefer the web ui to show an explanation of what the admin needs to add to the csp.yaml file.

This is not the right place. The web ui is not a documentation document, but we can link to one.

[dragonchaser]

I agree with @butonic, automatically adding csp rules during runtime is IMHO a huge security issue. I think configuring them statically in manifest.json or (preferable) config.json would be the way to go. So the admin has to explicitly enable the needed rules. It may be helpful to create a small warning, in the admin-area, whenever there is an external site defined that is not whitelisted in the CSP rules.

[micbar]

@dragonchaser how would you get that information from the web service to the proxy?

[dragonchaser]

@micbar what about a use-once endpoint on either side?

[butonic]

ok, the URLs can contain path, so we can limit it to our web app. Still, I would prefer the web ui to show an explanation of what the admin needs to add to the csp.yaml file.

This is not the right place. The web ui is not a documentation document, but we can link to one.

In oc10 the web ui shows detected configuration problems in the admin settings. Wes could detect if csps are missing based on the app manifest.json and tell the admin that a CSP is missing and provide a link to the docs?

[kulmann]

I agree with @butonic, automatically adding csp rules during runtime is IMHO a huge security issue.

You mean during runtime of the proxy service? Because for the web service it's on service startup.

I think configuring them statically in manifest.json or (preferable) config.json would be the way to go. So the admin has to explicitly enable the needed rules.

That's what I proposed. :-)

• manifest.json for CSP rules from the app developer
• config.json for CSP rules from the admin

We have the app store content under our control. What about signing apps together with their required CSP rules and only accept them during service startup when the signature is valid.

To be honest as long as we don't have one-click-installation of apps but admins need to extract the content into their apps folder and restart the web service I think the security risk is reasonable.

[kulmann]

ok, the URLs can contain path, so we can limit it to our web app. Still, I would prefer the web ui to show an explanation of what the admin needs to add to the csp.yaml file.

This is not the right place. The web ui is not a documentation document, but we can link to one.

In oc10 the web ui shows detected configuration problems in the admin settings. Wes could detect if csps are missing based on the app manifest.json and tell the admin that a CSP is missing and provide a link to the docs?

Please don't. Config is very cumbersome already. Advertising to do more config is a step back IMO.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 10 days if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 10 days if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 10 days if no further activity occurs. Thank you for your contributions.