Per-User Data and Time Quota Feature

[White-Owl86]

It is requested to add a per-user quota system in Sing-Box to enhance multi-user server management. The feature should include two independent limits:

Data Quota

Each user can be assigned a maximum data usage (e.g., 50GB).

When the user reaches the limit, the client connection is automatically disabled or blocked.

Time Limit

Each user can be assigned an active period (e.g., 30 or 90 days).

When the period expires, the client connection is automatically disabled or blocked.

Proposed Configuration Example (JSON style):

{ "users": [ { "name": "client1", "data_limit": "50GB", "time_limit": "30d", "action_on_limit": "disable" } ] }

Rationale:

This feature is essential for servers hosting multiple clients or resellers, allowing administrators to control usage efficiently.

It prevents overuse of server resources and simplifies subscription management without relying on external scripts or panels.

Current Sing-Box supports bandwidth limits but lacks any mechanism for per-user data or time quota enforcement.

Desired Behavior:

Both limits operate independently; either limit being reached results in user deactivation.

Admin can configure action on limit (disable, block, or notify).

Should be applicable to all supported protocols and outbound types.

[Cyan4973]

This file is synthetic and describes an unlikely scenario that is implausible in real-world cases.

This is evident from the compression ratio, which can reach an extraordinary x10000 ratio, far exceeding the expected parameters.

It is not surprising that behavior becomes unpredictable in these conditions.

The file itself consists largely of the characters a and b.

If the content were random, we would expect a ratio of x8. However, the significantly better ratio suggests that these seemingly nonsensical sequences are actually repeated, with large segments essentially copied and pasted.

With so many matches available, searching becomes very challenging, and finding the "best match" becomes an implausibly costly task. This situation favors different approaches based on probabilistic methods (i.e., random chance), which are more commonly used at lower compression levels. Given that many expectations are defied in this file, it is not surprising that the match-finding algorithms are pushed to their limits, resulting in a scenario where chance plays a disproportionately large role.

[Paladynee]

i understand, thank you for the detailed response and explanation.

[tansy]

One can always find data like that.

 1060424 required_block_states.nbt
   93043 required_block_states.nbt.zst-02
   86999 required_block_states.nbt.zst-01
   80818 required_block_states.nbt.zst-03
   80814 required_block_states.nbt.zst-04
   67911 required_block_states.nbt.zst-05
   60318 required_block_states.nbt.zst-06
   55341 required_block_states.nbt.zst-07
   54478 required_block_states.nbt.zst-08
   54452 required_block_states.nbt.zst-09
   50664 required_block_states.nbt.zst-10
   47866 required_block_states.nbt.zst-12
   47866 required_block_states.nbt.zst-11
   47171 required_block_states.nbt.zst-13
   45346 required_block_states.nbt.zst-18
   45173 required_block_states.nbt.zst-14
   44637 required_block_states.nbt.zst-17
   44404 required_block_states.nbt.zst-16
   44397 required_block_states.nbt.zst-15
   40761 required_block_states.nbt.zst-19


 1008188 e1m7.dem
  229306 e1m7.dem.zst-01
  228970 e1m7.dem.zst-02
  209440 e1m7.dem.zst-04
  208572 e1m7.dem.zst-03
  207954 e1m7.dem.zst-05
  206875 e1m7.dem.zst-07
  206845 e1m7.dem.zst-06
  206112 e1m7.dem.zst-15
  206098 e1m7.dem.zst-12
  206098 e1m7.dem.zst-11
  206090 e1m7.dem.zst-14
  205949 e1m7.dem.zst-13
  205651 e1m7.dem.zst-10
  205615 e1m7.dem.zst-09
  205600 e1m7.dem.zst-08
  203339 e1m7.dem.zst-16
  202717 e1m7.dem.zst-17
  201127 e1m7.dem.zst-18
  200530 e1m7.dem.zst-19

Maybe not that extreme, but I didn't make them artificially mid/long term redundant, which would make them 'behave' even more 'erratic'.

They also have something in common. They are not real real, only game real data.

Anything that would look similar is DNA sequence and though it experiences 'anomalies' it's nowhere near to that.

10652155 DNA_ScPo
 3241541 DNA_ScPo.zst-01
 3224028 DNA_ScPo.zst-02
 3203281 DNA_ScPo.zst-03
 3165842 DNA_ScPo.zst-04
 3117768 DNA_ScPo.zst-05
 3109233 DNA_ScPo.zst-06
 3078787 DNA_ScPo.zst-07
 3078456 DNA_ScPo.zst-08
 3077464 DNA_ScPo.zst-09
 3050357 DNA_ScPo.zst-13
 3048055 DNA_ScPo.zst-10
 3022373 DNA_ScPo.zst-11
 3021637 DNA_ScPo.zst-12
 2926669 DNA_ScPo.zst-14
 2887919 DNA_ScPo.zst-15
 2763037 DNA_ScPo.zst-17
 2758631 DNA_ScPo.zst-16
 2624450 DNA_ScPo.zst-19
 2624307 DNA_ScPo.zst-18

Your case is extreme, as explained by @cyan4973.

[tansy]

To not generate new issue I will tag along this one.

I found this png image data to compress worse with increasing windows size (wlog=X) between 15 and 17 bits. As reference Lzma has the same problem to a lesser extent.

409820 png.idat
 12968 png.idat.bz2
 15521 png.idat.gz-12
 16438 png.idat.gz-9

 14929 png.idat.zst-12-w15
 15142 png.idat.zst-12-w16
 15538 png.idat.zst-12-w17
 11949 png.idat.zst-12-w18
 11947 png.idat.zst-12-w19
 11947 png.idat.zst-12-w20

 14864 png.idat.zst-15-w15
 15085 png.idat.zst-15-w16
 15475 png.idat.zst-15-w17
 11688 png.idat.zst-15-w18
 11686 png.idat.zst-15-w19
 11686 png.idat.zst-15-w20

 14502 png.idat.zst-18-w15
 14587 png.idat.zst-18-w16
 14651 png.idat.zst-18-w17
 11148 png.idat.zst-18-w18
 11143 png.idat.zst-18-w19
 11143 png.idat.zst-18-w20

 13880 png.idat.lzma-6-w15
 13901 png.idat.lzma-6-w16
 13961 png.idat.lzma-6-w17
 10631 png.idat.lzma-6-w18
 10647 png.idat.lzma-6-w19
 10647 png.idat.lzma-6-w20

File (MD5: 1c05f15152130a7077075c9a71989d1b) is decompressed IDAT from actual image, nothing artificial.

I treat it like a curiosity.

[Cyan4973]

The first idea that comes to mind is that smaller window sizes (< 17) produce smaller blocks. In turn, these smaller blocks may produce more adjusted statistics for the data they contain.

This is likely the case here because this data is very compressible, and consists in many cases of repeated character values.

If correct, this effect should be partially compensated by internal block splitting in latest dev branch. Note that this mechanism is not part of any prior release.

@tansy : which version of zstd have you been using in your test?

[tansy]

I used v1.5.5 but v1.5.6 is no different. Here is the comparison.

[Cyan4973]

Difference should appear in dev branch. It will be present in the next release.

[tansy]

Not much have changed. Most files are slightly smaller, but general trend remains.

Unless you want to compress content generated images it's more of an exception than a rule.

[tansy]

FWIW, real example of such 'funny' file.

@ https://lists.gnu.org/archive/html/index.html

1105307 index.html
  84356 index.html.zst-01
  83570 index.html.zst-02
  83532 index.html.zst-03
  83503 index.html.zst-04
  81430 index.html.zst-05
  80915 index.html.zst-16
  80869 index.html.zst-12
  80869 index.html.zst-11
  80523 index.html.zst-18
  80476 index.html.zst-10
  80338 index.html.zst-14
  80290 index.html.zst-13
  80200 index.html.zst-15
  80100 index.html.zst-07
  80093 index.html.zst-08
  80082 index.html.zst-09
  79993 index.html.zst-06
  79287 index.html.zst-17
  71780 index.html.zst-19

And it's not zstd specific - deflate (gzip/zlib/libdeflate) share the same characteristic behaviour.

1105307 index.html
  93703 index.html.gz-1
  91883 index.html.gz-2
  91094 index.html.gz-3
  85955 index.html.gz-7
  85722 index.html.gz-6
  85635 index.html.gz
  85070 index.html.gz-8
  84979 index.html.gz-9
  84964 index.html.gz-4
  84388 index.html.gz-5

Not so extreme but still interesting.