pretext icon indicating copy to clipboard operation
pretext copied to clipboard

adding AC problems for runestone testing

Open Tanaquil18 opened this issue 10 months ago • 8 comments

These are the problems (with my commentary) Matt and I had identified 1.5 years ago as problematic or previously problematic and good for testing. I built it as part of a separate project at that time, deployed here, but I haven't tried building either as part of this repository or at all since that time.

Tanaquil18 avatar Feb 25 '25 19:02 Tanaquil18

Thanks for formulating this PR. Very sorry if I missed this contribution in some other format.

The problems that are in the csafranski directory are not going to generally be available. This sample gets tested with lots of servers (different versions, different hosts) so we can't assume prolems placed this way. Nor do we have any control over the stability of their contents.

Should I proceed and just remove those few problems?

rbeezer avatar Feb 28 '25 18:02 rbeezer

When #2336 is merged, local .pg files become usable. As in, local to your computer. I can't think right off top of my head if they are usable with a 2.18-or-below server, but definitely usable with 2.19. So the csafranski problems could come in at that time.

Alex-Jordan avatar Feb 28 '25 19:02 Alex-Jordan

Since Chrissy's questions are on the runestone webwork server and are illustrative of problem areas we experienced it would be good to have them in there.

Could we create a RunestoneAcademy version that would include said questions when built for RA?

Maybe the solution Alex just proposed as I was typing this is better.

bnmnetp avatar Feb 28 '25 19:02 bnmnetp

OK, @bnmnetp wants them all. @Tanaquil18: could you rewtite the six problems from csafranski in PreTeXt source syntax, much like the (big) ajordan_8a-_2_Preview_sine_Taylor problem?

That would really be the best for using this for testing, on and off, Runestone. Versions are a possibility, but come with their own baggage.

rbeezer avatar Feb 28 '25 19:02 rbeezer

If anything is going to be rewritten here anyway...

I would appreciate it if you can tend to the indentation. I have to work with this file a lot and it's helpful to have it logically indented.

Another thing: there are large blocks of content commented out here. (Some solutions, at least one exercise, maybe more.) Would you go through and make decisions to (a) uncomment them (b) delete them, or (c) intentionally decide it is best to keep them commented? There should be a compelling reason for doing (c) that is explained in a comment.

Some "exercise" have a label, some do not. Best to go ahead now and give all of them a label.

Some of these are OPL problem files and I am uncertain about if these belong here. Because in some cases (at minimum, the last two) the issue is not our issue. It's entirely how badly the OPL problem was coded. I don't have advice for what to do about this (leave the OPL problems in this PR versus take them out) but I thought it worth mentioning.

Alex-Jordan avatar Feb 28 '25 19:02 Alex-Jordan

Thanks, @Alex-Jordan. Yes, nothing should be commentd out - it confuses testing when you are looking (grep'ing) for problems.

And, if a problem is in the OPL and not authored in a way PreTeXt can digest it, then it doesn't belong here.

rbeezer avatar Feb 28 '25 20:02 rbeezer

It's okay. I posted on -dev at the time, but no one responded and I didn't follow up.

OK, @bnmnetp wants them all. @Tanaquil18 could you rewtite the six problems from csafranski in PreTeXt source syntax, much like the (big) ajordan_8a-_2_Preview_sine_Taylor problem?

In a word, no. I am not good enough to do that. I invested a bunch of time on my sabbatical in learning to write webwork-in-pretext-source, and I successfully wrote about 15 problems for another project, and then I gave up and went back to using .pg files on a server for all later problems. One of the issues is that my webwork knowledge far outpaces my webwork-in-pretext knowledge; it was easy enough to find examples in the sample webwork chapter to mimic for simple questions with a single number or formula answer expected but I tend to use a lot of multi-part questions, often with custom feedback and answer checkers. I had the Chapter 8a previews that Alex wrote to look at, which were very helpful, but I still had a lot of trouble writing non-simple webwork problems in pretext. I am very happy about the 2.19 PR feature that allows local problems written in .pg, as that seems like a great compromise between completeness and portability of a pretext project and usability in practice.

I think the old OPL problems should be in the test document. As far as I recall, PreTeXt digested those old OPL problems just fine, and that's why they were in use in AC, but the old coding didn't play well with Runestone as far as restoring answers. I think checking answers and giving credit for correct responses worked, but it's been a while now so I'm not positive What if the protocol for the interaction between WebWorK and Runestone changes? Seems to me like the problems should be there for testing since pretext authors can include them. Of course, somebody might update those problems in the OPL, and then they wouldn't be exceptional or useful for testing anymore.

I thought I usually explained in a comment why things were included but commented out. One problem broke compilation of all the webwork problems in my project, even though the same problem was working fine in ac-single with a different webwork server. I believe that was the same webwork problem that recently caused Brad and Alex trouble, and Alex figured out it was due to an image permission issue (I don't understand it, but they did), and so the problem might/should work now if uncommented. In a different problem, a solution Alex wrote was commented out because it made the problem too long, but if 2.19 has changed to a different protocol and problems can be longer, we can try uncommenting it. (Although I don't think Runestone has any way to show solutions to webwork problems that aren't visible from the start, at least as of now.)

I can work on the indentation and labels and revisit/review my commenting decisions, maybe in a week when Spring Break starts.

Chrissy

On Fri, Feb 28, 2025, 3:12 PM Rob Beezer @.***> wrote:

Thanks, @Alex-Jordan https://github.com/Alex-Jordan. Yes, nothing should be commentd out - it confuses testing when you are looking (grep'ing) for problems.

And, if a problem is in the OPL and not authored in a way PreTeXt can digest it, then it doesn't belong here.

— Reply to this email directly, view it on GitHub https://github.com/PreTeXtBook/pretext/pull/2424#issuecomment-2691468238, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB4353JX77J2WZQ7TQSHR3D2SC7MJAVCNFSM6AAAAABX3NDKG6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMOJRGQ3DQMRTHA . You are receiving this because you were mentioned.Message ID: @.***> [image: rbeezer]rbeezer left a comment (PreTeXtBook/pretext#2424) https://github.com/PreTeXtBook/pretext/pull/2424#issuecomment-2691468238

Thanks, @Alex-Jordan https://github.com/Alex-Jordan. Yes, nothing should be commentd out - it confuses testing when you are looking (grep'ing) for problems.

And, if a problem is in the OPL and not authored in a way PreTeXt can digest it, then it doesn't belong here.

— Reply to this email directly, view it on GitHub https://github.com/PreTeXtBook/pretext/pull/2424#issuecomment-2691468238, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB4353JX77J2WZQ7TQSHR3D2SC7MJAVCNFSM6AAAAABX3NDKG6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMOJRGQ3DQMRTHA . You are receiving this because you were mentioned.Message ID: @.***>

Tanaquil18 avatar Mar 01 '25 13:03 Tanaquil18

I fixed what I understood you wanted and am capable of fixing. I deleted unnecessary comments, but left the solutions to the uber-long problem commented out. I uncommented the problem that used to break the entire webwork generation process (I think Brad and Alex know what caused that before and what was needed to fix it, but I don't).

I still haven't tried building it, so I have no idea if the things that were problematic 1.5 years ago are still problematic now, or if they've been fixed, or if new things will break.

I'm not entirely convinced it makes sense to use the same document for the webwork testing needs of PreTeXt and of Runestone, but the csafranski problems are in there, as they are being used in AC on Runestone right now.

Tanaquil18 avatar Mar 14 '25 17:03 Tanaquil18

I'm closing this since it can't be used as-is and there has not been any apparent interest in fixing it up by anybody with a vested interest (or just a good Samaritan). It can be re-opened if several items are addressed.

but left the solutions to the uber-long problem commented out.

Source in comments really friustrates testing and debugging so there is rarely a good reason for it.

I still haven't tried building it

PRs need to be tested before being submitted.

csafranski problems are in there

And then this tie this general example to one particular server, which cannot happen.

rbeezer avatar Jul 22 '25 22:07 rbeezer

That all makes sense. The webwork testing needs of PreTeXt and of Runestone are not the same, and it was probably not a good idea to try to combine them in the first place. Almost all of the WeBWorK sample chapter consists of problems written in pretext source by Alex who is the architect of the integration, and that's simply not how most webwork problems are going to show up in the wild in PreTeXt projects on Runestone.

Runestone's development and troubleshooting timeframe is also vastly different from that of PreTeXt and WeBWorK. When issues arise, Runestone needs to fix things now for instructors and students in currently running courses, while PreTeXt and WeBWorK can each take a longer and more measured approach.

@bnmnetp I would be willing to, for the third time now, volunteer my time and effort in creating what you had originally asked for, a short testing document of troublesome webwork problems for Runestone, as long as it's separate from the webwork sample chapter.

I do want to point out that if the problems Matt and I had identified when we first did this work back in 2023 had been investigated at the time, it might have avoided some of the time-sensitive malfunctions that occurred during courses in 2024 and 2025. Specifically, I had commented out a problem because it stopped the generation of all subsequent webwork, and I reported that on -dev when I was asked for feedback and troublesome problems. https://groups.google.com/g/pretext-dev/c/6TMzp5Hb1Hg/m/vjROJsvWBAAJ

I believe that same identified and commented out problem caused more recent issues and Alex found something wrong with one image's permissions.

But I have plenty of other things to spend my time on if such a project is no longer desired!

Chrissy

Tanaquil18 avatar Jul 23 '25 11:07 Tanaquil18

@Tanaquil18 -- If you made a standalone short book in its own repository with problematic questions that would great as far as I'm concerned. I would put it in the weekly update and check it just like I do the sample book when I'm doing development work.

I think it would be especially helpful in the very near future as I try to get all of Alex's recent changes tested, and/or moving to using the local server for building representations.

bnmnetp avatar Jul 23 '25 15:07 bnmnetp