Livecheck outdated status should be cleared when port is updated

[jmroot]

Currently you can get something like this between when a new port version is committed and livecheck is next run:

[mojca]

Just some brainstorming.

@arjunsalyan: I would probably implement this somewhere inside Port.add_or_update: https://github.com/macports/macports-webapp/blob/932ff73243a849205ef3400f7f46e5dedf8ab532/app/port/models.py#L91-L92

We could check whether the version or revision have changed, and if any of those two fields did, schedule the livecheck via some async/non-blocking job (we basically only need to run run_livecheck_port(port) for any port that has changed.

We could either add the ports to some queue for livecheck, or perhaps invalidate updated_at (or create a new boolean field) in the LiveCheck table model, so that livecheck would later run on all such updated (sub)ports.

(The only thing that we need to take care of is not to "block" the initial port database population by waiting for livecheck for all the ports to finish.)

[mojca]

PS: yes, at some rare occasions the result of the livecheck could also change as a result of changing livecheck.regex or livecheck.url inside the Portfile, or even inside the PortGroup.

If we run livecheck on regular basis anyway, we probably don't need to worry about this case.

Basically I don't really care that much if we run livecheck for all updated port or just for those where the version has in fact changed.

We could also have a tiny "Refresh" icon next to the information when the livecheck was last run. Logged in (or any?) users could then occasionally force a re-run of the livecheck on one particular port in case there were some issues with the server etc. That's ultra ultra ultra low priority, I just mention it as a possibility.

[arjunsalyan]

How quickly does port sync reflect changes after a commit has been made. Is it immediate? Or does it sync only after the port builds?

The approach we go ahead with will depend on this. Instant updates would be easy to handle by simply supplying all updated ports to a function that runs livecheck in background.

[jmroot]

Immediately if syncing from git, but rsync is behind by I think about half an hour (or more if using a less frequently updated mirror).

[mojca]

Aren't we already going through the (updated) ports from portindex json on regular basis? (Are we generating json locally now that we have MacPorts inside the Docker container?)

Maybe we just need to make sure that we use the latest git repo for both port data and for anything that the port command does, like livecheck. We just need to add one line to macports configuration to make sure that the git repo takes precedence over the default one using rsync.

[arjunsalyan]

Aren't we already going through the (updated) ports from portindex json on regular basis? (Are we generating json locally now that we have MacPorts inside the Docker container?)

Yes we do, but are not using it for the port command (sources.conf refers to rsync).

Maybe we just need to make sure that we use the latest git repo for both port data and for anything that the port command does, like livecheck. We just need to add one line to macports configuration to make sure that the git repo takes precedence over the default one using rsync.

Okay, that's something we will have to change in the Dockerfile.

If I am right, when using the git repo directly in sources.conf file, it would remove the need for port sync? Since we are updating the git repo separately with git pull.

[jmroot]

port sync runs portindex for git sources in addition to pulling. So avoiding it is possible, but only if you are running portindex as a separate step.

[mojca]

In principle we run both portindex and portindex2json. Now, we run it with -p macosx_19_i386 at the moment, so I'm not 100% sure what that means as far as the port command is concerned. Maybe this needs some checking, just in case.