Crazyhouse / nonstandard notation (Invalid character: @)

[vandenman]

Hi, thanks for making a very cool package!

I was trying to read my own Lichess games but I encountered an issue with the crazyhouse variant. The problem is that dropped pieces are stored as for example "N@g3" to indicate that a knight is dropped at g3. However, when I try to read this file I get:

nested task error: Chess.PGN.PGNException("Invalid character: @")

now, obviously, the "@" is the issue. Is there a recommended workaround for this? I'm not necessarily interested in reading the games with the crazyhouse variant, so I'd be happy to skip this game if that's possible. However, my goal is to read in the larger Lichess datasets, and in those files all variants are mixed.

full example:

Chess.PGN.gamefromstring("""
[Event "Hourly Crazyhouse Arena"]
[Site "https://lichess.org/urHSzup4"]
[Date "2021.04.06"]
[White "vandenman"]
[Black "damatte"]
[Result "0-1"]
[UTCDate "2021.04.06"]
[UTCTime "17:22:01"]
[WhiteElo "1869"]
[BlackElo "2181"]
[WhiteRatingDiff "-4"]
[BlackRatingDiff "+2"]
[Variant "Crazyhouse"]
[TimeControl "300+0"]
[ECO "B02"]
[Termination "Normal"]
1. e4 Nf6 2. e5 Nc6 3. exf6 exf6 4. d4 d5 5. Bb5 Bd6 6. Bxc6+ bxc6 7. N@h5 O-O 8. Nxg7 Kxg7 9. N@h5+ Kg8 10. P@g7 Re8+ 11. Qe2 B@g4 12. Qxe8+ Qxe8+ 13. Be3 Q@d1# 0-1
""")

ERROR: Chess.PGN.PGNException("Invalid character: @")
Stacktrace:
 [1] readtoken(p::Chess.PGN.PGNReader)
   @ Chess.PGN ~/.julia/packages/Chess/oXD5R/src/pgn.jl:351
 [2] readgame(p::Chess.PGN.PGNReader; annotations::Bool)
   @ Chess.PGN ~/.julia/packages/Chess/oXD5R/src/pgn.jl:444
 [3] #gamefromstring#6
   @ ~/.julia/packages/Chess/oXD5R/src/pgn.jl:557 [inlined]
 [4] gamefromstring(s::String)
   @ Chess.PGN ~/.julia/packages/Chess/oXD5R/src/pgn.jl:557
 [5] top-level scope
   @ REPL[29]:1

[fsmosca]

I believe this package does not support crazyhouse variant. It is better to just skip parsing crazyhouse variant games.

Sample code to skip a crazyhouse game.

using Chess
using Chess.PGN


fn = "games.pgn"


for g in gamesinfile(fn)
    variant = headervalue(g, "Variant")

    if variant == "Crazyhouse"
        continue
    end

    # do whatever you like on the game
    println(g)
end

[vandenman]

@fsmosca I'd be happy to skip that, but unfortunately, that does not work. For example:

using Chess, Chess.PGN
filename = "crazyhouse_test.pgn"
println(read(filename, String))

result = SimpleGame[]
for g in gamesinfile(filename)
	variant = headervalue(g, "Variant")

	if variant == "Crazyhouse"
		continue
	end

	push!(result, g)
end

errors with:

ERROR: TaskFailedException
Stacktrace:
 [1] try_yieldto(undo::typeof(Base.ensure_rescheduled))
   @ Base ./task.jl:710
 [2] wait
   @ ./task.jl:769 [inlined]
 [3] wait(c::Base.GenericCondition{ReentrantLock})
   @ Base ./condition.jl:106
 [4] take_unbuffered(c::Channel{Any})
   @ Base ./channels.jl:405
 [5] take!
   @ ./channels.jl:383 [inlined]
 [6] iterate(c::Channel{Any}, state::Nothing)
   @ Base ./channels.jl:465
 [7] top-level scope
   @ ./REPL[5]:6

    nested task error: PGNException("Invalid character: @")
    Stacktrace:
     [1] readtoken(p::PGNReader)
       @ Chess.PGN ~/.julia/packages/Chess/oXD5R/src/pgn.jl:351
     [2] readgame(p::PGNReader; annotations::Bool)
       @ Chess.PGN ~/.julia/packages/Chess/oXD5R/src/pgn.jl:444
     [3] (::Chess.PGN.var"#3#5"{Channel{Any}, Bool, Int64})(io::IOStream)
       @ Chess.PGN ~/.julia/packages/Chess/oXD5R/src/pgn.jl:535
     [4] open(::Chess.PGN.var"#3#5"{Channel{Any}, Bool, Int64}, ::String, ::Vararg{String, N} where N; kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
       @ Base ./io.jl:330
     [5] open
       @ ./io.jl:328 [inlined]
     [6] createchannel
       @ ~/.julia/packages/Chess/oXD5R/src/pgn.jl:522 [inlined]
     [7] (::Base.var"#517#518"{Chess.PGN.var"#createchannel#4"{Bool, Int64, String}, Channel{Any}})()
       @ Base **./channels.jl:132**

Here is the file (rename and drop the .txt). crazyhouse_test.pgn.txt

Now, if I understand the problem correctly, the issue is that gamesinfile always calls skipgame and readgame, which in turn both call readtoken, which errors on the '@'. So gamesinfile will error no matter what. I guess I can copy the source of gamesinfile and read the headers myself, then also skip the game myself but I was hoping there is a better way. That solution would essentially boil down to copying 90% of the existing functionality. At that point, I might as well use a fork that allows '@', but if there is a better solution I also would not mind submitting a PR.

[fsmosca]

Your are right, there is an error.

[romstad]

Sorry for replying late to this. I've had a holiday and a mild burnout.

Yes, this is annoying. I'm not sure what the best solution could be, but I'll think about it. Ideally I'd like to simply support all sorts of variants, but I'm afraid this is too ambitious for now.