[FYI] Training and evaluating summarization systems

[dselivanov]

Hi @jwijffels . I found interesting dataset and decent paper which compares different techniques:

1. website - https://summari.es/
2. Article from website above

textrank is very competitive to modern alternatives. May be the dataset and work above it can be useful for future development.

[jwijffels]

Hi @dselivanov interesting paper, I'll have a look at https://arxiv.org/abs/1602.03606 to see if I can incorporate it in the package. I already have BM25 in the udpipe R package implemented. So it should be possible to add this as an example in the package instead of using the current sentence-2-sentence distance metric. For neural network based text generation, that would require a deep learning R package of course.

[karelin]

@dselivanov When you write that

textrank is very competitive to modern alternatives.

which other packages have you used for comparison???

[dselivanov]

@karelin This is my conclusion after reading article. And not about package, but about comparison of available approaches.