awesome-micropython icon indicating copy to clipboard operation
awesome-micropython copied to clipboard

Show last commit dates for repos in the README list

Open rtideas opened this issue 8 months ago • 2 comments

Seeing some similar libraries might have old code, I wanted to see last commit dates to find the newer ones. Didn't know really how to write what I wanted, so I got some Ai help. "We" created this Python script to add the date of last commits for GitHub repos to the README and save it locally.

Hopefully this is useful for others. Thanks for your Awesome work!

update_readme_with_commit_dates.py.zip

# Update README with commit dates
import os
import re
import json
import time
import requests
from tqdm import tqdm  # āœ… Progress bar

README_URL = "https://raw.githubusercontent.com/mcauser/awesome-micropython/master/readme.md"
OUTPUT_FILENAME = "readme_with_commit_dates.md"
CACHE_FILE = "commit_dates_cache.json"

# šŸ” GitHub token for more API requests
GITHUB_TOKEN = os.getenv("GITHUB_TOKEN", "your_token_here")
HEADERS = {
    "Authorization": f"token {GITHUB_TOKEN}",
    "Accept": "application/vnd.github.v3+json"
}

# Regex patterns
REPO_LINK_RE = re.compile(r"https://github\.com/([\w\-]+)/([\w\-]+)$")
LIST_ITEM_RE = re.compile(r"^\* \[(.*?)\]\((.*?)\)(.*)")

def load_cache():
    if os.path.exists(CACHE_FILE):
        with open(CACHE_FILE, "r") as f:
            return json.load(f)
    return {}

def save_cache(cache):
    with open(CACHE_FILE, "w") as f:
        json.dump(cache, f)

def get_last_commit_date(owner, repo, cache):
    repo_key = f"{owner}/{repo}"
    if repo_key in cache:
        return cache[repo_key]

    url = f"https://api.github.com/repos/{owner}/{repo}/commits"
    try:
        response = requests.get(url, headers=HEADERS)
        if response.status_code == 200:
            commits = response.json()
            if commits:
                date = commits[0]['commit']['committer']['date'][:10]
                cache[repo_key] = date
                time.sleep(0.5)  # Be kind to the API
                return date
        else:
            print(f"āš ļø  Skipping {repo_key} — status code {response.status_code}")
    except Exception as e:
        print(f"āŒ Error fetching {repo_key}: {e}")
    return None

def update_readme(text, cache):
    updated_lines = []
    lines = text.splitlines()

    # ā³ Pre-process to find all real repo links
    repo_lines = [
        (i, LIST_ITEM_RE.match(line))
        for i, line in enumerate(lines)
        if LIST_ITEM_RE.match(line) and REPO_LINK_RE.fullmatch(LIST_ITEM_RE.match(line).group(2).strip())
    ]

    # šŸ“Š Progress bar time!
    for i, match in tqdm(repo_lines, desc="šŸ”„ Processing repos", unit="repo"):
        title, url, desc = match.groups()
        url = url.strip()
        desc = desc.strip()

        owner, repo = REPO_LINK_RE.fullmatch(url).groups()
        date = get_last_commit_date(owner, repo, cache)

        if date:
            if "Last commit:" in desc:
                desc = re.sub(r"_\(Last commit: .*?\)_", f"_(Last commit: {date})_", desc)
            else:
                desc += f" _(Last commit: {date})_"
        lines[i] = f"* [{title}]({url}) {desc}"

    return "\n".join(lines)

def main():
    print("šŸ“„ Downloading README...")
    response = requests.get(README_URL)
    if response.status_code != 200:
        print("āŒ Failed to download README")
        return

    readme_text = response.text
    cache = load_cache()

    print("šŸ” Updating repository descriptions (this will take a while)...")

    try:
        updated = update_readme(readme_text, cache)
    except KeyboardInterrupt:
        print("\nšŸ›‘ Stopped by user. Saving partial results...")
        save_cache(cache)
        return

    print("šŸ’¾ Saving updated file...")
    with open(OUTPUT_FILENAME, "w", encoding="utf-8") as f:
        f.write(updated)

    save_cache(cache)
    print(f"āœ… Done! Output saved to: {OUTPUT_FILENAME}")

if __name__ == "__main__":
    main()

It generated this updated README with the dates added. (Sorry, I have not learned the proper ways of GitHub, yet. Going through the Python Crash Course book now.)

readme_with_commit_dates.md

SAMPLE:

AI

  • MicroMLP - A micro neural network multilayer perceptron for MicroPython (used on ESP32 and Pycom modules). (Last commit: 2020-12-23)
  • MicroPython-NeuralNetwork - Neural Network for MicroPython.
  • upython-chat-gpt - ChatGPT for MicroPython. (Last commit: 2023-06-05)
  • emlearn-micropython - Efficient Machine Learning engine for MicroPython. (Last commit: 2025-03-28)
  • mp_esp_dl_models - MicroPython binding for the ESP DL vision models like face detection. (Last commit: 2025-03-29)

rtideas avatar Apr 07 '25 21:04 rtideas

Revised with visual cues for age of repo. Here's a sample of the README:

AI

  • MicroMLP - A micro neural network multilayer perceptron for MicroPython (used on ESP32 and Pycom modules). šŸ”“ (2020-12-23)
  • MicroPython-NeuralNetwork - Neural Network for MicroPython.
  • upython-chat-gpt - ChatGPT for MicroPython. 🟔 (2023-06-05)
  • emlearn-micropython - Efficient Machine Learning engine for MicroPython. 🟢 (2025-03-28)
  • mp_esp_dl_models - MicroPython binding for the ESP DL vision models like face detection. 🟢 (2025-03-29)

šŸ”µ Repo Activity Legend

  • 🟢 Active (commit within 1 year)
  • 🟔 Aging (commit within 4 years)
  • šŸ”“ Stale (commit over 4 years ago)

Python script to scan and update README: update_readme_with_commit_dates.py.zip Updated README: readme_with_commit_dates.md

rtideas avatar Apr 09 '25 22:04 rtideas

Not sure about this one. Seems like a lot more effort to maintain. Would have to automate with Github Actions. I haven't seen this on any other awesome lists... The last commit date is not always a good indicator of quality of a project. A recent commit may indicate active development, but it doesn't always guarantee bug-free battle-tested code. Maybe it would be better having it output a json version which could be consumed by cli client side if they need this extra info?

mcauser avatar Jun 01 '25 01:06 mcauser