sway icon indicating copy to clipboard operation
sway copied to clipboard

Consider only storing interned values of types rather that the whole AstNode

Open JoshuaBatty opened this issue 1 year ago • 0 comments

I guess this would work for most types that are already stored in the Engines. Otherwise we are already storing a copy in the Engines of the type and the same in the TokenMap.

Maybe a better mapping might be:

map: DashMap<span: Span, token_id: usize> 

Jibril mentioned that Joao was thinking about implementing a system where each AstNode could be mapped to an int. This would be really helpful to us to keep the TokenMap memory size as small as possible. @tritao do you have any thoughts or plans for a feature like this?

JoshuaBatty avatar Jul 05 '24 00:07 JoshuaBatty