Improve API

[mkiser]

Current API returns each day's update as a single response. Would like to find a way to break this down so each day has multiple sub-items representing each news blurb.

Note: editorial workflow needs to remain the same (building site via the static markdown file), so this would require a post-processing approach.

https://whatthefuckjusthappenedtoday.com/api/v1/pages.json

[denisensantos]

you could:

• create an 'excerpt' field with an excerpt for the news blurb (let's say a paragraph)
• cap the blurb content in n number of characters (let's say 300 char)

[Arrow7000]

I just had a look round the source but couldn't find the code controlling the API. Could you point me in the right direction?

[mkiser]

Dope improvement by @milo-wata (https://github.com/mkiser/WTFJHT/pull/94).

I'm going to put together a very light requirements doc for fully spec'ing what I need with the API.

tl;dr this is a great improvement. thank you.

[mkiser]

Basic product reqs are here: https://github.com/mkiser/WTFJHT/wiki/API-Specs

It's a starting point for a discussion. Let's talk!

[crhallberg]

Looks like the "smart" quotes need to be coverted to simple quotes for the API.

[mkiser]

@crhallberg fixed in https://github.com/mkiser/WTFJHT/commit/88256f9d2da9aec29738c027d4d2dd4b64e6a740#diff-aeb42283af8ef8e9da40ededd3ae2ab2

[codewithmilo]

Thinking about offering a real API leads me to desire something that would be slightly more dynamic than what Jekyll offers: it's hard to build a robust and scalable API off a static site. I'm limited in my knowledge of what the options are, but I'd imagine we'd want a webapp running a more customizable API than what we have. The database layer could still run off the static Jekyll (like we could still use the pages.json as the "database", but we'd build something on top to handle and serve the requests.

I am of the opinion that this would be better than relying on Jekyll to statically build multiple versions of each day's post and such for API output, which would bloat the size by a ton.

[Arrow7000]

Agreed @milo-wata. I think something like Node would work well for something like this. Depending on what the requirements are this needn't be a huge amount of work.

I'd be happy to contribute to this FYI.

[Arrow7000]

Btw I've had a look around the repo but can't find the code for the pages.json API anywhere. Could you point me in the right direction?

Nvm I found it.

[codewithmilo]

Yep, another option is Sinatra, which appears like it might play nice with Jekyll: either running the rest of the Jekyll blog on top of Sinatra along with the API, or have it be separate. http://www.sinatrarb.com/

[mkiser]

API is here: https://github.com/mkiser/WTFJHT/blob/master/api/v1/posts.json Output is here: https://whatthefuckjusthappenedtoday.com/api/v1/posts.json

This setup works well. I'd love to further parse this to include a structured list of all the external links found in the daily post.

It would also be great if this and the "today" api could be merged so there is a parsed out list of "blurbs"

see: https://github.com/mkiser/WTFJHT/blob/master/api/today.json

[hanskokx]

This API makes me sad. Here's what I'd like to see:

• /api/v1/posts returns a list of all post titles and dates, but not the body of the post itself. This response should be paginated. (Right now, it returns ±5 MB of response data!)
• /api/v1/post returns the latest post.
• /api/v1/post/<date> returns the post for the given date

The post response, IMO, should be structured something like this:

{
  "author": <author>,
  "date": <yyyy-mm-dd>,
  "summary": <today-in-one-sentence>, 
  "title": <title>,
  "url": <url>,
  "stories": [
    {"id": 0, "story": <story-content>},
    {"id": 1, "story": <story-content>},
    ...
    {"id": 2, "story": <story-content>}
  ]
}

This way, you could just GET /api/v1/post/2020-11-03, and your response might look something like this:

{
  "author": "Matt Kiser",
  "date": "2020-11-03",
  "summary": "Trump defeated in the polls!", 
  "title": "Day whatever: \"FAKE ELECTION!\"",
  "url": "https://.....",
  "stories": [
    {"id": 0, "story": "Trump was handily defeated at the polls today, ending his four years of terror on the country and world."},
  ]
}

If you did this, I could make #102 happen pretty easily. :)

[hanskokx]

I was pondering this more as I lay awake in bed last night.

You might also want an endpoint such as:

/api/v1/day/<1234> to get the story for a given day of the presidency.

[mkiser]

This would all be ideal. The API today is just a flat json file that's built the same way everything else on the site is.