server icon indicating copy to clipboard operation
server copied to clipboard

503 Encryption not ready: multikeydecrypt with share key failed

Open CamZie opened this issue 7 years ago • 76 comments

Steps to reproduce

  1. enable encryption
  2. upload and download files

Expected behaviour

Nextcloud should allow downloading of files without any errors.

Actual behaviour

Cannot download some files. User is receiving errors that the server is temporarily unavailable (503) or that the server is in maintenance.

Server configuration

Operating system: Debian 8.10

Web server: NGINX 1.12

Database: MariaDB 10.0

PHP version: PHP 5.6

Nextcloud version: 12.0.2

Updated from an older Nextcloud/ownCloud or fresh install: Updated from an older Nextcloud version.

Signing status:

Signing status
No errors have been found.

List of activated apps:

App list
Enabled:
  - activity: 2.5.2
  - admin_audit: 1.2.0
  - bookmarks: 0.10.1
  - bruteforcesettings: 1.0.3
  - calendar: 1.5.7
  - comments: 1.2.0
  - contacts: 2.0.1
  - dav: 1.3.0
  - encryption: 1.6.0
  - federatedfilesharing: 1.2.0
  - files: 1.7.2
  - files_pdfviewer: 1.1.1
  - files_sharing: 1.4.0
  - files_texteditor: 2.4.1
  - files_trashbin: 1.2.0
  - files_versions: 1.5.0
  - files_videoplayer: 1.1.0
  - firstrunwizard: 2.1
  - gallery: 17.0.0
  - logreader: 2.0.0
  - lookup_server_connector: 1.0.0
  - mail: 0.7.9
  - nextcloud_announcements: 1.1
  - notes: 2.3.2
  - notifications: 2.0.0
  - oauth2: 1.0.5
  - password_policy: 1.2.2
  - provisioning_api: 1.2.0
  - qownnotesapi: 17.5.0
  - serverinfo: 1.2.0
  - sharebymail: 1.2.0
  - systemtags: 1.2.0
  - tasks: 0.9.5
  - theming: 1.3.0
  - twofactor_backupcodes: 1.1.1
  - updatenotification: 1.2.0
  - workflowengine: 1.2.0
Disabled:
  - federation
  - files_external
  - survey_client
  - user_external
  - user_ldap

Nextcloud configuration:

Config report
    "system": {
        "instanceid": "ocpom4ncgfhghkwru",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "***REMOVED SENSITIVE VALUE***"
        ],
        "datadirectory": "\/mnt\/***REMOVED SENSITIVE VALUE***\/data",
        "overwrite.cli.url": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "12.0.2.0",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "localhost",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "logtimezone": "Europe\/Zurich",
        "installed": true,
        "theme": "***REMOVED SENSITIVE VALUE***",
        "enable_previews": true,
        "memcache.local": "\\OC\\Memcache\\APCu",
        "enable_avatars": false,
        "logdateformat": "Y-m-d_H:i:s",
        "updatechecker": false,
        "log_type": "errorlog",
        "logfile": "",
        "loglevel": 2,
        "customclient_desktop": "***REMOVED SENSITIVE VALUE***",
        "maintenance": false,
        "trashbin_retention_obligation": "auto,90",
        "activity_expire_days": 90,
        "preview_max_scale_factor": 1,
        "preview_max_filesize_image": 10,
        "skeletondir": "***REMOVED SENSITIVE VALUE***",
        "mail_from_address": "no-reply",
        "mail_smtpmode": "php",
        "mail_smtpauthtype": "LOGIN",
        "mail_domain": "***REMOVED SENSITIVE VALUE***"}

Are you using encryption: yes

Client configuration

Browser: Operating system: Nextcloud-iOS/2.19.2

Logs

Nextcloud log (data/nextcloud.log)

Nextcloud log
2018/02/10 04:14:07 [error] 32243#32243: *2115256 FastCGI sent in stderr: "PHP message: [owncloud]
[webdav][4] Exception: {"Exception":"Sabre\\DAV\\Exception\\ServiceUnavailable","Message":"Encryption
 not ready: multikeydecrypt with share key failed:error:0906D06C:PEM routines:PEM_read_bio:no start 
line","Code":0,"Trace":"#0 \/var\/www\/nextcloud\/3rdparty\/sabre\/dav\/lib\/DAV\/CorePlugin.php(85): 
OCA\\DAV\\Connector\\Sabre\\File->get()\n#1 [internal function]: Sabre\\DAV
\\CorePlugin->httpGet(Object(Sabre\\HTTP\\Request), Object(Sabre\\HTTP\\Response))\n#2 \/var\
/www\/nextcloud\/3rdparty\/sabre\/event\/lib\/EventEmitterTrait.php(105): call_user_func_array(Array, 
Array)\n#3 \/var\/www\/nextcloud\/3rdparty\/sabre\/dav\/lib\/DAV\/Server.php(479): Sabre\\Event
\\EventEmitter->emit('method:GET', Array)\n#4 \/var\/www\/nextcloud\/3rdparty\/sabre\/dav\/lib\/DAV
\/Server.php(254): Sabre\\DAV\\Server->invokeMethod(Object(Sabre\\HTTP\\Request), 
Object(Sabre\\HTTP\\Response))\n#5 \/var\/www\/nextcloud\/apps\/dav\/appinfo\/v1\/webdav.php(71): 
Sabre\\DAV\\Server->exec()\n#6 \/var\/www\/nextclo" while reading response header from upstream, client: 
***REMOVED SENSITIVE VALUE***, server: ***REMOVED SENSITIVE VALUE***, request: "GET 
/remote.php/webdav/Photos/2018/01/18-01-19%2018-37-42%200433.jpg HTTP/2.0", upstream: 
"fastcgi://unix:/var/run/php5-fpm.sock:", host: "***REMOVED SENSITIVE VALUE***"

CamZie avatar Feb 14 '18 08:02 CamZie

Same problem here with two different Nextcloud 12.0.2 installations. One installation is running on Debian 8 and the other is running on Debian 9, for what it's worth. The rest of my set-up is pretty much the same as @CamZie's. Any ideas?

RandieM avatar Feb 20 '18 15:02 RandieM

@schiessle

tflidd avatar Mar 04 '18 11:03 tflidd

Same problem on 13.0.2. Happens on sharing encrypted directories / files. Also: php occ encryption:migrate throws a lot of errors "An unhandled exception has been thrown: ArgumentCountError: Too few arguments to function OCA\Encryption\Migration::__construct()"

albertogscotti avatar May 02 '18 12:05 albertogscotti

Same problem here too with 13.0.1.

@schiessle what is the status or progress regarding this encryption related bug?

hostingnuggets avatar May 25 '18 14:05 hostingnuggets

I have the same problem on 13.0.2! A lot of files can not be syncronized over dav. This Version of NextCloud is not stable to use in a productive environment!!

How can I get back my files??

berho avatar May 31 '18 20:05 berho

Same problem here on 13.0.4 stable release. Server side encryption activated = impossible to share files. <!> This encryption feature should be disabled on the stable/production releases <!>

m33m33 avatar Jun 16 '18 16:06 m33m33

Is there any updates or news for this issue? This is starting to be a big problem since it is impossible to access the files anymore...

CamZie avatar Jul 09 '18 07:07 CamZie

Is this reproducable with the newest versions eg. 12.0.9? https://github.com/nextcloud/server/wiki/Maintenance-and-Release-Schedule if not it seems rather something for a subscription service - if yes it should get immediate attention, indeed.

Escubaer avatar Jul 09 '18 08:07 Escubaer

We have upgraded our installation to 13.0.1 and this issue still persists. We haven't been able to identify the cause of this problem.

CamZie avatar Jul 09 '18 12:07 CamZie

@Escubaer, I am having trouble understanding your comment, as several other users (refer to comments above) have already reported that they experience this issue in versions 13.0.1, 13.0.2 and 13.0.4. People keep losing, possibly irreversibly, their data; how does such a major issue qualify as a case for a subscription service?

RandieM avatar Jul 09 '18 12:07 RandieM

@RandieM I am not working for the vendor, just to make sure. I think I am trying to say and ask if this can be reproduced with a brand new setup eg. with 13.0.4 and with which steps or if this is random and rather happened suddenly in people's running environment. IMHO this will make debugging difficult and therefor it seems maybe more for the subscription/support service. Besides that no developer seems to come up with any idea or solution till now here ...

Escubaer avatar Jul 09 '18 12:07 Escubaer

It is also maked as a feature whereas for you guys it sounds like a strong bug ...

Escubaer avatar Jul 09 '18 12:07 Escubaer

@Escubaer, when it comes to programming, I tend not to believe in "random" events. The described problem is triggered by something, which I am currently unable to identify. This also seems to be the case for @CamZie, according to his/her latest comment.

Besides, you do have a point when you say:

It is also maked as a feature whereas for you guys it sounds like a strong bug ...

I believe that this issue has been assigned the wrong label, as it is certainly not a feature, but a bug /cc @tflidd

RandieM avatar Jul 09 '18 15:07 RandieM

I believe that this issue has been assigned the wrong label, as it is certainly not a feature, but a bug /cc @tflidd

It just says that this topic is related to the server-side-encryption. There are different tags for feature requests ;-)

But regarding the number of users reporting this problem, it is probably more than just a single coincidence. I will put a bug-label to it.

tflidd avatar Jul 09 '18 21:07 tflidd

Thanks @tflidd for the explanation and for the assignment of the new label.

RandieM avatar Jul 10 '18 11:07 RandieM

I had this wired error once more today and I tested around but can't get any clue why that happens:

Upload from: ------------> Server Thumbnail Creation --------> Download to Windows Client ----------------------------(View and download with Browser)----------------------------------

iOS App ----------------------------> OK --------------------------------> Fail iOS send To NextCloud--------------> OK --------------------------------> Fail Browser (FF on W7) ------------------> OK -------------------------------> Fail Windows Client ----------------------> OK -------------------------------> (uploaded)

Error in Logfile always: Sabre\DAV\Exception\ServiceUnavailable: Encryption not ready: multikeydecrypt with share key failed:error:0407109F:rsa routines:RSA_padding_check_PKCS1_type_2:pkcs decoding error /htdocs/3rdparty/sabre/dav/lib/DAV/CorePlugin.php - line 88: OCA\DAV\Connector\Sabre\File->get() [internal function] Sabre\DAV\CorePlugin->httpGet(Object(Sabre\HTTP\Request), Object(Sabre\HTTP\Response))

berho avatar Jul 10 '18 14:07 berho

Yet another user has this problem and they keep receiving this error when trying to access their files. multikeydecrypt with share key failed:error:0906D06C:PEM routines:PEM_read_bio:no start line

Any news on this as it is getting more and more critical?

CamZie avatar Jul 31 '18 14:07 CamZie

Issue still present on 13.0.5.

As a workaround, is it safe to follow https://docs.nextcloud.com/server/13/admin_manual/configuration_files/encryption_configuration.html and decrypt files with occ ?

No, or I may do something wrong...

After using : php occ encryption:decrypt-all user1

The files are still encrypted on the storage, and users get a "bad signature" on all files. Better have a good backup.

In nextcloud.log : "Exception: {"Exception":"OCP\\Encryption\\Exceptions\\GenericEncryptionException","Message":"Bad Signature","Code":0,"Trace":"#0 \/mnt\/sd0d\/usr\/pkg\/share\/nextcloud\/apps\/encryption\/lib\/Crypto\/Crypt.php(465)

A decrypted file "About.txt": file data/user1/files/Documents/About.txt data/user1/files/Documents/About.txt: data ===> should be "text"

First few lines of About.txt: " HBEGIN:oc_encryption_module:OC_DEFAULT_MODULE:cipher:AES-256-CTR:signed:true:HEND---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- " Still encrypted...

m33m33 avatar Aug 01 '18 09:08 m33m33

It seems that his behaviour is sometimes triggered by a password change, although I do have users in the same installation that have never changed their password, yet they experience this problem.

Any help would be greatly appreciated, as an increasing number of my users are permanently losing access to their files!

RandieM avatar Aug 02 '18 13:08 RandieM

A clue about this issue: it seems related to public link shared files only:

A. I share a file with a user of my nextcloud instance: the user can open the file. B. I share a file with a public link (url): the link is unusable and throws the multikeydecrypt error message.

m33m33 avatar Aug 10 '18 15:08 m33m33

@m33m33, thanks for posting. Initially, I also thought that this was the case, but, in my experience, it does not only happen with shared files.

RandieM avatar Aug 13 '18 13:08 RandieM

Are there any updates or news for this issue?

Just as @RandieM and @m33m33 mentioned, I have also noticed that these are mostly triggered by a password change or shared files, but some of my users also do not have either of them but are still experiencing this problem. Any help would be greatly appreciated.

CamZie avatar Sep 12 '18 07:09 CamZie

Bug still lives in v14. Exterminate. Exterminate. Exterminate.

Considering total NC removal under users grunts.

" Can't read file multikeydecrypt with share key failed:error:0407109F:rsa routines:RSA_padding_check_PKCS1_type_2:pkcs decoding error "

m33m33 avatar Sep 14 '18 13:09 m33m33

Dropping NC too

albertogscotti avatar Sep 14 '18 13:09 albertogscotti

Another clue about this issue: it seems image format files are not affected.

A. I share a picture (.jpg) with a public link : the destination user can open the link and the image shows in NC viewer. B. I share a document (.pdf, .odt...) with a public link : the link is unusable and throws the multikeydecrypt error message.

m33m33 avatar Sep 14 '18 13:09 m33m33

@m33m33 The behavior you describe in your point A might be the effect of the cache: my assumption here is that image files get cached unencrypted and this picture file you shared with a public link is then accessed directly from the cache, that's why it works.

Have a look at my comment here and the answers below on the nextcloud forum: https://help.nextcloud.com/t/nextcloud-14-focus-on-security-and-compliance/36116/2

In my comment I have asked the nextcloud core team why they don't seem to care about fixing and even replying to all the server-side encryption issues...

hostingnuggets avatar Sep 14 '18 16:09 hostingnuggets

@m33m33 The behavior you describe in your point A might be the effect of the cache: my assumption here is that image files get cached unencrypted and this picture file you shared with a public link is then accessed directly from the cache, that's why it works.

You are right. I am fooled by the preview from cache, if I click on "download" the picture don't show and the multikey failure message appears :(

m33m33 avatar Sep 14 '18 17:09 m33m33

I have the feeling that this issue mixes many potential different problems together. E.g. the original issue says that the user gets a "503 Nextcloud unavailable or in maintenance mode" which I never saw and I don't know how this could be triggered by the server side encryption. The other error messages posted here make more sense but I still struggle to find the necessary information and what all this reports have in common in order to try to reproduce it.

So my request to everyone in this issue. Can someone of you describe a step by step scenario with the latest Nextcloud version (13.0.6 or 14, because they contain some changes to make the file cache updates more robust) where they can reliable reproduce the issue?

If I have something like this I'm happy to give it another try and see if I can reproduce it.

schiessle avatar Sep 17 '18 10:09 schiessle

NC 14, current user with server side encryption enabled.

  1. Upload a new file : picture.jpg

  2. Create a public link

  3. Give public link to an external anonymous user : the preview is OK. Click on "Download" shows "Can't read file multikeydecrypt with share key failed:error:0407109F:rsa routines:RSA_padding_check_PKCS1_type_2:pkcs decoding error"

m33m33 avatar Sep 17 '18 13:09 m33m33

@m33m33 I just tried it and couldn't reproduce it. Does the error happens for you reliable with all files? Is it a fresh installation or a update from a older version. On which version did you enabled encryption? Do you still use per-user keys or the new master key?

schiessle avatar Sep 18 '18 12:09 schiessle