qsharp icon indicating copy to clipboard operation
qsharp copied to clipboard

Add samples of testing Q# code

Open tcNickolas opened this issue 11 months ago • 5 comments

Testing Q# code is an important part of developing a quantum project with QDK. We do most of our own testing (for example, for libraries) using Rust, so an external user working on their own project has very little information on how they can test their project using Python.

Some things can be very challenging to figure out, for example, dump_operation: https://learn.microsoft.com/en-us/azure/quantum/user-guide/testing-debugging does not have information on it, and the only place I could find info on it was https://github.com/microsoft/qsharp/blob/d6a8f8a71a71bf80168e846c940674595c413a5b/pip/tests/test_qsharp.py#L103 which is helpful but 1) hard to discover without knowing upfront about dump_operation and 2) insufficient for developing practical tests (for example, it doesn't handle the global phase that can show up in the output).

We need to add tests for different scenarios to our samples folder (I'd create a separate folder for it called "testing", to match the naming logic used in "estimation" folder), to help users develop their own tests. Here are some examples I can think of right away:

  • [x] Tests done mostly in Q# with a thin Python wrapper to integrate with pytest
    • verifying that measurement results of basis states are as expected
    • verifying classical computation
    • verifying that two operations are the same up to a global phase using CheckOperationsAreEqual
  • [ ] Test that validates that Q# state is as expected using Python API dump_machine
  • [x] Test that validates that Q# operation is as expected using Python API dump_operation
  • We'll also need to make sure these tests are executed as part of the CI build - #1859

tcNickolas avatar Mar 18 '24 16:03 tcNickolas

@tcNickolas , I would like to work on this issue

Manvi-Agrawal avatar May 07 '24 23:05 Manvi-Agrawal

Sounds great!

There is a bit of information on testing in https://learn.microsoft.com/en-us/azure/quantum/user-guide/testing-debugging, but it's missing dump_operation (and is somewhat outdated, I think - dump_machine returns complex numbers for the past few releases).

I also have some examples of pytest integration and using dump_machine and dump_operation for test at https://github.com/tcNickolas/quantum-programming-in-depth (qsharp-v1 folders), those use QDK 1.3.

tcNickolas avatar May 10 '24 00:05 tcNickolas

Hey @tcNickolas , I would also like to work on this issue.

Pulkit1822 avatar May 10 '24 21:05 Pulkit1822

Sorry, I think I missed the last comment on this issue. @Manvi-Agrawal, @Pulkit1822 - what's the status of this issue? Are either of you or both working on it?

If both of you are working on it, you need to make sure to coordinate, the easiest way is to work on separate tests so as not to have to coordinate on the same branch.

tcNickolas avatar May 30 '24 18:05 tcNickolas

Hmm, life got a bit busy, and I wouldn't get to this for a week ot two. I am happy to relinquish it to @Pulkit1822.

Manvi-Agrawal avatar May 30 '24 18:05 Manvi-Agrawal