package-feeds
package-feeds copied to clipboard
ossf
Feed parsing for language package manager updates
Failing to marshal a single package currently blocks the publishing of all subsequent packages, this is undesirable. https://github.com/ossf/package-feeds/blob/ce53f3ea/cmd/scheduled-feed/main.go#L45-L50 is the relevant code chunk.
With #123 the behaviour of feed polling became less brittle. It introduced `ErrNoPackagesPolled` & `PackagePollError{}` which were applied to the `npm` & `pypi` feeds, along with relevant tests. The behaviour...
> As this is essentially opening up a new data type that can be put 'on the wire', (including the scope to have pubsub `sinks` in a similar fashion to...
We could have all the different RSS feeds wrapped up into one binary, exposed at different paths and triggered via different schedulers. Maybe something like: /npm /gems /pypi /crates etc....
Note: this is specifically for the package manager ingestion part of the system. Right now, when we pull packages from package managers, we eventually serialize them into `ossmalware/pkg/library.Package` instances. This...
I would like to propose to add helm charts to the feeds as well. There is a complication as there is no central repository for all charts. There is an...
Included Makefile for docker builds. After this gets merged will refactor and consolidate into a single Makefile.
I'm not familiar with Perl, but it is used for bugzilla (e.g. https://bugzilla.mozilla.org/ , https://bugs.webkit.org/ , https://bugzilla.redhat.com/ ). Per https://github.com/metacpan/metacpan-api/blob/master/docs/API-docs.md, the index/releases endpoint is probably https://fastapi.metacpan.org/v1/release/_search?q=status:latest&fields=name,status,date&sort=date:desc&size=100 (or as html https://metacpan.org/recent).
Right now things are in individual GCS objects, formatted as JSON. This is easy to look at and browse, but probably not the best for querying. We could load these...