macadmins
RSS Feed per OS
Taking a first stab at getting an RSS feed generated while building the structured data feed. Let me know what you all think and if it's the right direction here. This created two separate feeds based on OS type. Wondering if maybe one is better with all data imported from the json files, then create an icon at the top right for RSS.
Feedback is welcome!
Description
This RSS feed will consist of all Security Releases from each version of the OS. build-sofa-feed.py will take the feed_structure data and sort it out by creating a new list that just consists of SecurityReleases and sorts them by ReleaseDate but ignoring entries inside that have a string Preinstalled as the ReleaseData. In this case, macOS 12.
It then uses feedgen module to create the XML that will be used as the RSS feed within the write_data_to_rss function. RSS feed entries usually get sorted by publish dates, so in this case - we use the ReleaseDate as the publish date.
Changes
build-sofa-feed.py
sort_by_release_date
- Regenerating a new list with just
SecurityReleases - Sorting by
ReleaseDatewhile ignoringPreinstalledstring - Returning list of all OS
SecurityReleasesor empty if error
write_data_to_rss
- Using feedgen python module
- Escape if empty list given
- Generate new RSS file with sorted list given
- Pretty the description just like the site has
- If
Preinstallstring, set old date. This is just accounting for old entry but might need tweaking if we see this often. Hoping not. - Create feed and export to xml file
index.html
- Added link below the machine readable data one
- I noticed @headmin created a function here to load last checked date. Will it be a standard to create functions for load to handle error checking?
README.md
Updated to account for RSS Feed
docker workflows
- Accounting for new files generated and copied over
requirements.txt
- Adding the feedgen module
Tests
We ran the following lines to create the new xml files along with json files. Copied them into the v1 directory for the webserver to pick up in testing.
python3 ./build-sofa-feed.py iOS
python3 ./build-sofa-feed.py macOS
Established a webserver to view changes.
➜ sofa git:(rss_feed) ✗ sudo python3 -m http.server 80
[sudo] password for jramos:
Serving HTTP on 0.0.0.0 port 80 (http://0.0.0.0:80/) ...
127.0.0.1 - - [07/May/2024 00:21:55] "GET / HTTP/1.1" 304 -
127.0.0.1 - - [07/May/2024 00:21:55] "GET /config.json HTTP/1.1" 304 -
127.0.0.1 - - [07/May/2024 00:21:55] "GET /v1/timestamp.json HTTP/1.1" 200 -
127.0.0.1 - - [07/May/2024 00:21:58] "GET /v1/macos_rss_feed.xml HTTP/1.1" 200 -
127.0.0.1 - - [07/May/2024 00:22:02] "GET /v1/timestamp.json HTTP/1.1" 304 -
127.0.0.1 - - [07/May/2024 00:22:03] "GET /v1/ios_rss_feed.xml HTTP/1.1" 200 -
Docker build of image/
➜ sofa git:(main) ✗ docker build -t sofa_jramos .
[+] Building 15.8s (20/20) FINISHED docker:default
=> [internal] load build definition from dockerfile 0.0s
=> => transferring dockerfile: 1.19kB 0.0s
=> [internal] load metadata for docker.io/library/python:3.12-alpine 2.1s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 56.57kB 0.0s
=> [ 1/15] FROM docker.io/library/python:3.12-alpine@sha256:ef097620baf1272e38264207003b0982285da3236a20ed829bf6bbf1e85fe3cb 3.6s
=> => resolve docker.io/library/python:3.12-alpine@sha256:ef097620baf1272e38264207003b0982285da3236a20ed829bf6bbf1e85fe3cb 0.0s
=> => sha256:ef097620baf1272e38264207003b0982285da3236a20ed829bf6bbf1e85fe3cb 1.65kB / 1.65kB 0.0s
=> => sha256:c583b8590a197db1f6efece2dd244b0259cb6f82c4acc654bfbe48e00e20a7b9 1.37kB / 1.37kB 0.0s
=> => sha256:f44387b482817f41bdac1892c45711adaedb3a7dd381844cdc3f360e66314d7a 6.02kB / 6.02kB 0.0s
=> => sha256:4abcf20661432fb2d719aaf90656f55c287f8ca915dc1c92ec14ff61e67fbaf8 3.41MB / 3.41MB 0.8s
=> => sha256:c3cdf40b8bda8e4ca4be0f5fa7f1d128907271efcbc72cbfc7c8b0f939ec25ea 619.60kB / 619.60kB 0.3s
=> => sha256:3a6cecfe70039fd21206783553d33ea4753700f031a2490428311619801d02f7 13.96MB / 13.96MB 2.5s
=> => sha256:60d2faee92e78fe7518f0ff1645cd7320bf6b140ff885fdec2a1ea1d878f0dca 239B / 239B 0.6s
=> => sha256:b62713ed4827911d38bb5a9ac322efa0408b4bb135863b4b15c4bc383e59918b 2.70MB / 2.70MB 1.9s
=> => extracting sha256:4abcf20661432fb2d719aaf90656f55c287f8ca915dc1c92ec14ff61e67fbaf8 0.2s
=> => extracting sha256:c3cdf40b8bda8e4ca4be0f5fa7f1d128907271efcbc72cbfc7c8b0f939ec25ea 0.4s
=> => extracting sha256:3a6cecfe70039fd21206783553d33ea4753700f031a2490428311619801d02f7 0.7s
=> => extracting sha256:60d2faee92e78fe7518f0ff1645cd7320bf6b140ff885fdec2a1ea1d878f0dca 0.0s
=> => extracting sha256:b62713ed4827911d38bb5a9ac322efa0408b4bb135863b4b15c4bc383e59918b 0.3s
=> [ 2/15] WORKDIR /app 0.1s
=> [ 3/15] COPY build-sofa-feed.py /app/ 0.0s
=> [ 4/15] COPY process_uma.py /app/ 0.0s
=> [ 5/15] COPY config.json /app/ 0.0s
=> [ 6/15] COPY feed_structure_template_v1.yaml /app/ 0.0s
=> [ 7/15] COPY forked_builds.json /app/ 0.0s
=> [ 8/15] COPY sofa-time-series.py /app/ 0.0s
=> [ 9/15] COPY requirements.txt /app/ 0.0s
=> [10/15] RUN touch time-series.csv || true 0.3s
=> [11/15] COPY time-series.csv /app/ 0.0s
=> [12/15] COPY model_identifier_*.json /app/ 0.0s
=> [13/15] RUN pip install --no-cache-dir -r requirements.txt 8.6s
=> [14/15] COPY entrypoint.sh /app/ 0.0s
=> [15/15] RUN chmod +x /app/entrypoint.sh 0.4s
=> exporting to image 0.3s
=> => exporting layers 0.3s
=> => writing image sha256:24ce704610bd4a391bcdfe013a3ec855f4c9ee7b25c5a1cfabe19f3177df1a5a 0.0s
=> => naming to docker.io/library/sofa_jramos
Visuals
@johnnyramos As discussed the merge #80 is now live. We keep create two separate feed files based on OS type, but do so in a single run. Hopefully this is making RSS feed creation easier.
@headmin ok, should be up to date now and tested locally for create 2 feeds in a run. Give it a whirl.
Thanks @johnnyramos . I had a chance to test over here, an XML RSS file is generated via GH action.
However, the structure of the feed needs to be better formatted. I see currently all is written out in one line:
feedgen allows us to structure this better.
We should reduce details to the "essence". A new feed item should contain essential information and include a link.
When I looked into FeedGen a few weeks ago, I started that direction but halted when your PR came in parallel:
<channel>
<title>SOFA Update Feed</title>
<link>https://sofa.macadmins.io/rss</link>
<description>This feed includes updates on OS versions and security info.</description>
<atom:link href="https://sofa.macadmins.io/rss" rel="self"/>
<docs>http://www.rssboard.org/rss-specification</docs>
<generator>python-feedgen</generator>
<image>
<url>https://sofa.macadmins.io/images/custom_logo.png</url>
<title>SOFA Update Feed</title>
<link>https://sofa.macadmins.io/rss</link>
</image>
<language>en</language>
<lastBuildDate>Sun, 05 May 2024 15:58:51 +0000</lastBuildDate>
<item>
<title>New OS Version: 14.4.1</title>
<link>https://sofa.macadmins.io</link>
<description>New OS version released: 14.4.1 (Build: 23E224) on 2024-03-25T00:00:00Z.</description>
<guid isPermaLink="false">os_version_update_0_2024-03-25T00:00:00Z</guid>
<pubDate>Mon, 25 Mar 2024 00:00:00 +0000</pubDate>
</item>
<item>
<title>XProtect Plist Config Update: Version 2193</title>
<link>https://sofa.macadmins.io</link>
<description>XProtect Plist Config updated to version 2193.</description>
<guid isPermaLink="false">xprotect-com.apple.XProtect-2193</guid>
<pubDate>Tue, 30 Apr 2024 17:06:11 +0000</pubDate>
</item>
<item>
<title>Security Update: macOS Sonoma 14.4.1 Released</title>
<link>https://support.apple.com/kb/HT214096</link>
<description>Security update for macOS Sonoma 14.4.1: version 14.4.1 released on 2024-03-25T00:00:00Z. Addresses 1 CVEs with 0 actively exploited.</description>
<guid isPermaLink="false">https://support.apple.com/kb/HT214096</guid>
<pubDate>Mon, 25 Mar 2024 00:00:00 +0000</pubDate>
</item>
@headmin - fixed the structure of the feed in 8f2f226.
As for reducing the details, I'm torn.
- I agree with you if we were able to link to the tab and details of the feed item. Example,
https://sofa.macadmins.io/macOS14#guid_goes_hereor similar. - For now, we are able to provide the details in which can be digested immediately within a persons RSS app or Slack.
If we want to just keep it to the basics, I would need to submit a commit to link back to the initial page (https://sofa.macadmins.io/) for every item until we can link to specific div's. I'm open to either or but just wanted to pass my opinion by you.