"Couldn't import type" exception with circular struct dependencies and typedefs

[olbat]

@watzon just found a bug while trying to compile a binding for OpenSSL's PKCS7 library (see https://github.com/olbat/libgen/issues/48).

The bug can be reproduced using the following code (assuming that OpenSSL's headers are available):

$ crystal src/main.cr <<'EOF'
@[Include("openssl/pkcs7.h", prefix: %w(PKCS7_))]
lib LibPKCS7
end
EOF

Unhandled exception: Couldn't import type: PKCS7_SIGNED (Exception)
  from src/crystal_lib/type_mapper.cr:190:5 in 'map_internal'
  from src/crystal_lib/type_mapper.cr:0:36 in 'map_non_recursive'
  from src/crystal_lib/type_mapper.cr:72:18 in 'map_internal'
  from src/crystal_lib/type_mapper.cr:34:36 in 'map_non_recursive'
  from src/crystal_lib/type_mapper.cr:30:5 in 'map'
  from src/crystal_lib/type_mapper.cr:250:77 in 'expand_pending_structs'
  from src/crystal_lib/type_mapper.cr:30:35 in 'map'
  from src/crystal_lib/type_mapper.cr:250:77 in 'expand_pending_structs'
  from src/crystal_lib/type_mapper.cr:30:35 in 'map'
  from src/crystal_lib/prefix_importer.cr:102:5 in 'map_type'
  from src/crystal_lib/prefix_importer.cr:46:101 in 'process'
  from src/crystal_lib/prefix_importer.cr:5:7 in 'import'
  from src/crystal_lib/lib_transformer.cr:24:19 in 'transform'
  from /usr/lib/crystal/compiler/crystal/syntax/transformer.cr:7:14 in 'transform'
  from /usr/lib/crystal/compiler/crystal/syntax/transformer.cr:23:9 in 'transform'
  from /usr/lib/crystal/compiler/crystal/syntax/transformer.cr:7:14 in 'transform'
  from src/main.cr:5:1 in '__crystal_main'
  from /usr/lib/crystal/crystal/main.cr:97:5 in 'main_user_code'
  from /usr/lib/crystal/crystal/main.cr:86:7 in 'main'
  from /usr/lib/crystal/crystal/main.cr:106:3 in 'main'
  from __libc_start_main
  from _start
  from ???

After a bit of testing, I came up with a simple example to reproduce the bug:

$ cat <<'EOF' > /tmp/bug.h
typedef struct first_st {
	struct second_st *second;
} FIRST_ST;

struct second_st {
	FIRST_ST *first;
};

struct second_st * test_function();
EOF


$ crystal src/main.cr <<'EOF'
@[Include("/tmp/bug.h", prefix: %w(test_))]
lib LibTest
end
EOF

Unhandled exception: Couldn't import type: FIRST_ST (Exception)
  from src/crystal_lib/type_mapper.cr:190:5 in 'map_internal'
  from src/crystal_lib/type_mapper.cr:0:36 in 'map_non_recursive'
  from src/crystal_lib/type_mapper.cr:72:18 in 'map_internal'
  from src/crystal_lib/type_mapper.cr:34:36 in 'map_non_recursive'
  from src/crystal_lib/type_mapper.cr:30:5 in 'map'
  from src/crystal_lib/type_mapper.cr:250:77 in 'expand_pending_structs'
  from src/crystal_lib/type_mapper.cr:30:35 in 'map'
  from src/crystal_lib/prefix_importer.cr:102:5 in 'map_type'
  from src/crystal_lib/prefix_importer.cr:48:5 in 'process'
  from src/crystal_lib/prefix_importer.cr:5:7 in 'import'
  from src/crystal_lib/lib_transformer.cr:24:19 in 'transform'
  from /usr/lib/crystal/compiler/crystal/syntax/transformer.cr:7:14 in 'transform'
  from /usr/lib/crystal/compiler/crystal/syntax/transformer.cr:23:9 in 'transform'
  from /usr/lib/crystal/compiler/crystal/syntax/transformer.cr:7:14 in 'transform'
  from src/main.cr:5:1 in '__crystal_main'
  from /usr/lib/crystal/crystal/main.cr:97:5 in 'main_user_code'
  from /usr/lib/crystal/crystal/main.cr:86:7 in 'main'
  from /usr/lib/crystal/crystal/main.cr:106:3 in 'main'
  from __libc_start_main
  from _start
  from ???

Problem seems to occur with circular struct dependencies that also uses type definitions (without the typedef everything is working fine).

My guess is that second_st is expanded too early (I didn't look at the code yet).

[watzon]

Look at you go! Thanks for pinging me.

[olbat]

It looks like the issue is coming from the parser. I implemented a fix, please let me know if there is anything wrong with it.