Custom provenance/metadata

[mcdearman]

I was wondering if there was a way to have custom provenance information or additional metadata on tokens that you can easily get access to. The spans I use for provenance are source offsets (and I'm parsing on a token stream after a lexing pass). I include these spans in the token stream but I don't have a nice way to get them out without altering every combinator. Before, when I was parsing on text, I had a basic wrapper using the builtin getOffset:

withSpan :: Parser a -> Parser (Spanned a)
withSpan p = do
  startPos <- getOffset
  result <- p
  Spanned result . SrcLoc startPos <$> getOffset

But now I can't do this because offsets are for tokens not source. I tried getting the parser state, but the issue is, even though I can get spans for tokens from the state, once I reach end of input I'm not sure how to get the span info from the token because the stream is empty.