JS SNAPSHOT restore API should not be limited by account's max msg payload or server max_payload setting

[phho]

Defect

• [] Included nats-server -DV output
• [V] Included a [Minimal, Complete, and Verifiable example] (https://stackoverflow.com/help/mcve)

Versions of nats-server and affected client libraries used:

nats-server: 2.7.3 nats cli: 0.0.29

OS/Container environment:

Ubuntu 20.04.2 LTS

Steps or code to reproduce the issue:

1. Configure account max msg payload size

+--------------------------------------------------------------------------------------+
|                                   Account Details                                    |
+---------------------------+----------------------------------------------------------+
| Name                      | acc                                                      |
+---------------------------+----------------------------------------------------------+
...
| Max Msg Payload           | 66 kB (65536 bytes)                                      |
...
+---------------------------+----------------------------------------------------------+

2. Check account payload limitation

nats account info --creds ~/.nkeys/creds/test-operator/acc/testrestore.creds

Connection Information:

...
         Maximum Payload: 64 KiB
...

JetStream Account Information:

Configuration Requirements:

   Requires Max Bytes Set: false

Stream Resource Usage Limits:

      Memory: 0 B of 0 B
     Storage: 0 B of 16 EiB
     Streams: 0 of 0
   Consumers: 0 of 0

3. Create a stream and publish some messages to make the stream size larger than account's max payload setting

nats str add --creds ~/.nkeys/creds/test-operator/acc/testrestore.creds
? Stream Name testbak
? Subjects to consume test
? Storage backend file
? Retention Policy Limits
? Discard Policy Old
? Stream Messages Limit -1
? Per Subject Messages Limit -1
? Message size limit -1
? Maximum message age limit -1
? Maximum individual message size -1
? Duplicate tracking time window 2m0s
? Allow message Roll-ups No
? Allow message deletion Yes
? Allow purging subjects or the entire stream Yes
? Replicas 1
Stream testbak was created

Information for Stream testbak created 2022-03-07T22:51:06+08:00

Configuration:

             Subjects: test
     Acknowledgements: true
            Retention: File - Limits
             Replicas: 1
       Discard Policy: Old
     Duplicate Window: 2m0s
    Allows Msg Delete: true
         Allows Purge: true
       Allows Rollups: false
     Maximum Messages: unlimited
        Maximum Bytes: unlimited
          Maximum Age: unlimited
 Maximum Message Size: unlimited
    Maximum Consumers: unlimited
State:
             Messages: 0
                Bytes: 0 B
             FirstSeq: 0
              LastSeq: 0
     Active Consumers: 0

nats pub test mymsg --count 10000 --creds ~/.nkeys/creds/test-operator/acc/testrestore.creds

10000 / 10000 [======================================================================================================================================================]    3s

nats str info testbak --creds ~/.nkeys/creds/test-operator/acc/testrestore.creds
Information for Stream testbak created 2022-03-07T22:51:06+08:00

Configuration:

             Subjects: test
     Acknowledgements: true
            Retention: File - Limits
             Replicas: 1
       Discard Policy: Old
     Duplicate Window: 2m0s
    Allows Msg Delete: true
         Allows Purge: true
       Allows Rollups: false
     Maximum Messages: unlimited
        Maximum Bytes: unlimited
          Maximum Age: unlimited
 Maximum Message Size: unlimited
    Maximum Consumers: unlimited

State:
             Messages: 10,000
                Bytes: 381 KiB
             FirstSeq: 1 @ 2022-03-07T14:52:10 UTC
              LastSeq: 10,000 @ 2022-03-07T14:52:14 UTC
     Active Consumers: 0

4. Try to backup stream and restore, nats-server will report maximum payload exceeded error

nats str backup testbak testbak --consumers --creds ~/.nkeys/creds/test-operator/acc/testrestore.creds
Starting backup of Stream "testbak" with 381 KiB

227 KiB/s [========================================================================================>-------------------------------------------------------------]  60%

Received 227 KiB compressed data in 2 chunks for stream "testbak" in 6ms, 0 B uncompressed

nats str delete testbak --creds ~/.nkeys/creds/test-operator/acc/testrestore.creds
? Really delete Stream testbak Yes

nats str restore testbak --creds ~/.nkeys/creds/test-operator/acc/testrestore.creds --trace
22:55:26 >>> $JS.API.STREAM.INFO.testbak
null

22:55:26 <<< $JS.API.STREAM.INFO.testbak
{"type":"io.nats.jetstream.api.v1.stream_info_response","error":{"code":404,"err_code":10059,"description":"stream not found"}}

Starting restore of Stream "testbak" from file "testbak"

22:55:26 >>> $JS.API.STREAM.RESTORE.testbak
{"config":{"name":"testbak","subjects":["test"],"retention":"limits","max_consumers":-1,"max_msgs_per_subject":-1,"max_msgs":-1,"max_bytes":-1,"max_age":0,"max_msg_size":-1,"storage":"file","discard":"old","num_replicas":1,"duplicate_window":120000000000,"sealed":false,"deny_delete":false,"deny_purge":false,"allow_rollup_hdrs":false},"state":{"messages":10000,"bytes":390000,"first_seq":1,"first_ts":"2022-03-07T14:52:10.7483214Z","last_seq":10000,"last_ts":"2022-03-07T14:52:14.4103896Z","num_subjects":1,"consumer_count":0}}

22:55:26 <<< $JS.API.STREAM.RESTORE.testbak
{"type":"io.nats.jetstream.api.v1.stream_restore_response","deliver_subject":"$JS.SNAPSHOT.RESTORE.testbak.9fkZEG5x3OEg6WGr9vX719"}

nats: error: restore failed: nats: maximum payload exceeded

Expected result:

Restore successfully even if account's max payload is configured. (Report the same error when configured server max_payload). From the user's point of view, I hope that the snapshot restore API payload size should not be limited by account's max payload setting. Account's max payload only limits actually published message size.

Actual result:

Restore failed

[ripienaar]

There's no way to tell the server some API can bypass the payload limits, we could support adjusting the payload size for backups - and I think it's even in the API fields - but I dont think the server implemented support for that.