Seperate doc tests into separate stages

[hjr3]

When generating documentation tests, the following stages will happen:

• tests will be gathered from the docs as a list of strings
• tests will be saved to target/doc/tests as individual files
• tests will be compiled into a single binary called "rustdoc-test"
• tests will be run by executing the "rustdoc-test" binary

Doc tests are now actual tests. This allows us to take advantage of the existing test infrastructure supplied by rustc.

Fixes #32, #54

[euclio]

Nice, I really like the direction this PR is taking! A couple of things:

• I don't actually see doctests being run as a part of cargo run --release -- --manifest-path=example/Cargo.toml test. Just "running 0 tests".
• Could we hook this into the build --emit infrastructure? Like --emit=tests? I could see a world where we don't actually write out the test files unless explicitly requested.

[hjr3]

Nice, I really like the direction this PR is taking!

Thank you for the review!

I don't actually see doctests being run as a part of cargo run --release -- --manifest-path=example/Cargo.toml test. Just "running 0 tests".

There seems to be a bug where the tests do not get parsed out into a file. I am working on tracking it down.

Could we hook this into the build --emit infrastructure? Like --emit=tests? I could see a world where we don't actually write out the test files unless explicitly requested.

Yes, great idea. I will do this.

[steveklabnik]

this is gonna need a rebase

[hjr3]

@steveklabnik yup. i have also made some changes and am in the process of tracking down a bug where the tests do not get built every time. seems to be there prior to this patch too.

beyond that, are we good with changing the doc tests into actual tests? as @euclio mentioned, it is a change to the way existing rustdoc works.

[steveklabnik]

I'm happy with not worrying about compatibility right now. We can work on that later.

[steveklabnik]

Oh one more thing: it seems that master's tests are failing, dunno whats up with that. If you get CI in the same place, we can :shipit: and fix those issues some other way.

[hjr3]

I pushed some more commits. Give me another pass to make sure this change is sound and then we can merge and work on the edge cases, etc.

[hjr3]

@steveklabnik This change is working pretty well. There are some outstanding issues listed below. If we land this PR, then I will make an issue for each and start working on them. Otherwise, I can try to address those issues in this PR.

Outstanding issues I will open issues for and fix:

• Only the first line of the doc test is captured. This is due to how syn::parse_crate works. This is fixed in a newer version of syn, but the upgrade path will take some work. This is an existing issue in master.
• Only one doc test is in the data.json. When testing against the mio crate, there are definitely more than one doc tests, but only one is in data.json. This is an existing issue in master.
• A doc test that uses the #[macro_use] attribute will fail to compile. This happens because #[macro_use] cannot be specified inside a test. I have to write some code to strip it out and put it in the main.rs file.
• Per the code review, I need to hook this up to the --emit infrastructure.

[hjr3]

Only the first line of the doc test is captured. This is due to how syn::parse_crate works. This is fixed in a newer version of syn, but the upgrade path will take some work. This is an existing issue in master.

Now that https://github.com/dtolnay/syn is on 0.12, this upgrade makes sense.