Bug: Memory allocation failure

[fernandocanizo]

Scenario: trying to get ride of some initial lines from an SQL dump file to be able to properly import it on my local Postgres database. File size is around 80GB.

$ sd 'CREATE PROFILE.*' "" db-dump.sql 
memory allocation of 84931776403 bytes failedAborted (core dumped)

An application that tries to emulate or be an easier sed should work with files of any size, by chunks. Not try to load the whole thing in memory. Otherwise I'd just use my favourite editor.

[aiac]

Same here, 4GB RAM, 10GB file

memory allocation of 8589934592 bytes failed
Aborted (core dumped)

[PeterlitsZo]

But the behavior of the application depends on the regex library and it looks like they do not want to deal with stream cases (see https://github.com/rust-lang/regex/issues/425) - and that is what we need. Does anyone have a good idea? Maybe we can create ourself regex library that can help us deal with the edge cases but it ought to be much much hard than we think.

See src/replacer/Replacer/replace for more detail.