wtf_wikipedia icon indicating copy to clipboard operation
wtf_wikipedia copied to clipboard

'Chronological' output within a section

Open fractalien opened this issue 6 years ago • 10 comments

The parse method nicely yields arrays of sentences and links, but I see no way of knowing which lists were at which position. Is this loss of information deliberate, am I missing something, or was this just never an issue for anybody? A simple solution would seem to allow disabling the parsing of lists to somehow include them in the sentences.

fractalien avatar Mar 18 '18 17:03 fractalien

Good point, maybe these should all be objects in a chronological array. On vacation, will think about this when im back

spencermountain avatar Mar 24 '18 23:03 spencermountain

I assume it is more like a tree with children in a chronlogical order, in which the parse elements appear in the source wiki article. The article itself is the root node in an Abstract Syntax Tree AST The wiki article is split up in top level sections ==My Section Title ==

  • before the first section there could be a paragraph/abstract (tree node PARAGRAPH)
  • All sections have a title wrapped with == and children, which are page elements. (tree node SECTION)
  • The sections are split into subsection (tree node SECTION as child of a SECTION node) and so forth that are children of the section node
  • All subsections have a title wrapped with === and children, which are page elements (tree node SECTION), the tree node has a title attribute.
  • page elements are sentences, images, links, video, audio, maps, ...
  • enumerations have arrays of items and the item are arrays of page elements. Arrays preserve child order, (tree node ITEM) arrays of page elements preserve the chronological output and nested enumerations can be handled properly for code generation.

niebert avatar May 02 '18 02:05 niebert

Started to propose a solution for the order problem of content element see GitHub Wiki

niebert avatar Aug 11 '18 13:08 niebert

hey, this ordering-within-sections situation has been improved now with paragraph support. Unfortunately, (and to after considerable work) I learned how all templates, references, and tables can have an infinite number of newlines within them. This means that they will remain on Section for the medium-term. Images, lists, and sentences are now stored on Paragraph, so will now support a closer-to-truth ordering.

spencermountain avatar Oct 01 '18 14:10 spencermountain

all the output methods will work normally, and should provide some better ordering out of the box cheers

spencermountain avatar Oct 01 '18 14:10 spencermountain

Thank you Spencer, excellent, try to embed and integrate in [https://niebert.github.io/Wiki2Reveal Wiki2Reveal] cheers

niebert avatar Oct 09 '18 05:10 niebert

I'm sorry in case I'm missing something obvious, but how would I get the "better ordering out of the box"? Calling .json() on a section in 7.2.9, I still get lists after paragraphs, not in them. (Trying with this article.) Is there an accessor method to get both sentences and lists from a paragraph object?

fractalien avatar Feb 14 '19 01:02 fractalien

hi @fractalien no you're right, this is still not possible.

spencermountain avatar Feb 15 '19 17:02 spencermountain

Thanks!

Edit: I thought I had found a workaround to my problem, but that wasn't one. The article's API-sourced markup fails to put a newline after the list (bug), yielding text containing "SunMercury". Given that the newer REST API seems not to offer a full-article markup output, I am between a rock and a hard place:

  1. Applying Wtf(…).json() to the erroneous API result obviously doesn't magically un-break the text.
  2. When using Wtf.fetch(…).json(), I get the lists separated from sentences, out of order.

I tried applying Wtf twice, essentially Wtf(Wtf.fetch(…).markdown()).json() to get proper parsing, but no separation between sentences and lists. Unfortunately, this leaves me with the whole markup intact as the alleged text of a singular section. Will have to dig further...

fractalien avatar Feb 15 '19 19:02 fractalien

ha! this is very clever. i wish things were easier!

spencermountain avatar Feb 15 '19 20:02 spencermountain