jomini icon indicating copy to clipboard operation
jomini copied to clipboard

Add `TextTape::from(Vec<TextToken>, bool)` for Tape construction after token manipulation.

Open KitsuneDev opened this issue 7 months ago • 2 comments

Currently, there is no way to reconstruct a Tape after performing raw token manipulation. A use case (which is what I have in a project I'm working on) is parsing scripted_variables files and replacing their reference tokens on, say, technologies with their actual values.

KitsuneDev avatar Jun 11 '25 20:06 KitsuneDev

Looks good, let me think about if it makes sense to supply the bom marker in the constructor, as that should only surface from initial parsing 🤔

nickbabcock avatar Jun 12 '25 03:06 nickbabcock

A thought ocurred to me, is your use case covered by something like:

// pseudo-code
impl TextTape {
    pub fn tokens_mut(&mut self) -> &mut Vec<TextToken>
    ...
    }
}

A benefit is that you don't need to clone the tokens (that's how I'm assuming you are able to pass in a Vec<TextToken> to the constructor).

If that does work, it would be nice to include a rustdoc example and give the caveat that any modification needs to respect expected invariants or could otherwise risk object / array index corruption.

nickbabcock avatar Jun 12 '25 17:06 nickbabcock