piccolo icon indicating copy to clipboard operation
piccolo copied to clipboard

`Feature`: implemented all remaining `string` functions in stdlib except `dump`

Open reloginn opened this issue 7 months ago • 1 comments

All functions are based on and fully comply with the Lua 5.3 manual (https://www.lua.org/manual/5.3/).

Features

  • Implemented functions string.char, string.byte, string.find, string.match, string.gmatch, string.gsub, string.rep, string.pack, string.unpack, string.packsize.
  • Added tests in string.lua for all new functions.
  • In Cargo.toml, the lsonar crate has been added - an implementation of Lua patterns in Pure Rust, without unsafe and dependencies (https://github.com/reloginn/lsonar).
  • In Cargo.toml, the lformat crate has been added - an implementation of string.format based on the sprintf crate, without unsafe and with (almost) no dependencies (https://github.com/reloginn/lformat).

Caveats

  • In the string.gsub function, the Value::Function(_) case is not implemented (I don't know how to call VM functions from Rust).

reloginn avatar May 02 '25 11:05 reloginn

I've had a chance to read through this now, and it looks reasonable, though it'll need some cleanup and should probably be split into multiple smaller PRs. (The simplest split would be the base string ops; format; pattern matching; pack/unpack.)

  • string.format needs to determine the types of its arguments from the format string, so that it can handle tables with __tostring metamethods (while also supporting %p). I think this should be possible with your approach of forking sprintf, but you'd have to implement string.format over the &[FormatElement] slice directly, rather than using lformat::format.

  • The pattern matching functions need to be able to suspend in the middle of long operations;

    For example, evaluating this match (basically ("aaa", "a?a?a?aaa")) takes exponential time, and can hang the host program if the matching doesn't have suspend points.

    n = 30
    print(string.match(string.rep("a", n), string.rep("a?", n) .. string.rep("a", n)))
    

    Adding suspend points requires using Sequences, though the new-ish async-callback API makes that easier do by using Rust's async blocks / async functions.

  • For string.gsub, here's an example of how to call a Lua function from Rust, using async_sequence: https://github.com/kyren/piccolo/blob/acb88feeb9e847c5165b839cec33e8c2bb66aca5/src/stdlib/table.rs#L374-L396 In this case it's calling the __len metamethod, but the core logic is the same. The difficult part is that the sequence needs to suspend every time it calls a Lua function, so we have to make the Rust code into an async sequence. (Or a normal Sequence, but that takes more work to write.)

  • string.pack and string.unpack look reasonable, but they should be moved to a submodule (ie. stdlib/string/pack.rs), and I'd personally prefer if they were split into a separate PR, given the complexity.

Aeledfyr avatar May 30 '25 22:05 Aeledfyr

I started implementing all your fixes and began splitting this PR into smaller ones: #126: string.pack, string.unpack, string.packsize #127: string.rep #128: string.format #129: pattern matching

I will close this PR.

reloginn avatar Aug 10 '25 17:08 reloginn