Github-Trending-API icon indicating copy to clipboard operation
Github-Trending-API copied to clipboard

Update attribute match in scraping_repositories

Open vrachev opened this issue 1 year ago • 0 comments

Found this repo today while trying to scrape Github's Trending page, thanks for making this @NiklasTiede!

The /developers/ endpoint still works, but the /repositories/ endpoint was slightly outdated in the attributes it matches against:

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/Users/vrachev/github/Github-Trending-API/venv/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py", line 396, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "/Users/vrachev/github/Github-Trending-API/venv/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 45, in __call__
    return await self.app(scope, receive, send)
  File "/Users/vrachev/github/Github-Trending-API/venv/lib/python3.8/site-packages/fastapi/applications.py", line 199, in __call__
    await super().__call__(scope, receive, send)
  File "/Users/vrachev/github/Github-Trending-API/venv/lib/python3.8/site-packages/starlette/applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/Users/vrachev/github/Github-Trending-API/venv/lib/python3.8/site-packages/starlette/middleware/errors.py", line 181, in __call__
    raise exc from None
  File "/Users/vrachev/github/Github-Trending-API/venv/lib/python3.8/site-packages/starlette/middleware/errors.py", line 159, in __call__
    await self.app(scope, receive, _send)
  File "/Users/vrachev/github/Github-Trending-API/venv/lib/python3.8/site-packages/starlette/exceptions.py", line 82, in __call__
    raise exc from None
  File "/Users/vrachev/github/Github-Trending-API/venv/lib/python3.8/site-packages/starlette/exceptions.py", line 71, in __call__
    await self.app(scope, receive, sender)
  File "/Users/vrachev/github/Github-Trending-API/venv/lib/python3.8/site-packages/starlette/routing.py", line 580, in __call__
    await route.handle(scope, receive, send)
  File "/Users/vrachev/github/Github-Trending-API/venv/lib/python3.8/site-packages/starlette/routing.py", line 241, in handle
    await self.app(scope, receive, send)
  File "/Users/vrachev/github/Github-Trending-API/venv/lib/python3.8/site-packages/starlette/routing.py", line 52, in app
    response = await func(request)
  File "/Users/vrachev/github/Github-Trending-API/venv/lib/python3.8/site-packages/fastapi/routing.py", line 214, in app
    raw_response = await run_endpoint_function(
  File "/Users/vrachev/github/Github-Trending-API/venv/lib/python3.8/site-packages/fastapi/routing.py", line 149, in run_endpoint_function
    return await dependant.call(**values)
  File "./app/main.py", line 65, in trending_repositories
    return scraping_repositories(soup, since=payload["since"])
  File "./app/scraping.py", line 80, in scraping_repositories
    rel_url = match.h1.a["href"]
AttributeError: 'NoneType' object has no attribute 'a'

Thankfully the only change needed was updating the attribute to "h2". Result post

vrachev avatar Apr 27 '23 17:04 vrachev