Would it be possible to use a special token for separating segments?

[Semisol]

Something like this:

Below is an instruction that describes a task, possibly paired with an input that provides further context. Write a response that appropriately completes the request.
<instruction>Add two numbers
<input>2, 3
<response>

[T-Atlas]

#32

[Semisol]

#32

That seems to be only modification of the dataset.

I'm just thinking it may be best to use a special token instead of text to separate different segments