operations icon indicating copy to clipboard operation
operations copied to clipboard

Migrate help.openstreetmap.org from OSQA to static html archive

Open simonpoole opened this issue 7 years ago • 81 comments

ASKBOT is an actively maintained fork of qsqa (that was last worked on something like 5 years ago), see: https://github.com/ASKBOT/askbot-devel and https://askbot.com/

If we want to continue to run our own Q&A site migrating would make sense and would likely address many of the issues we have with QSQA (not just that it is abandonware). The alternative would be to give up running our own system and move to stackexchange, which has IMHO some downsides including loosing the content I believe.

Other issues to consider:

  • while ASKBOT still works with mysql, they support postgres better, I assume that is OK with us in any case?
  • there is, for unfathomable reasons, no off the shelf migration script/facility, this implies that some coding is likely to be involved. Potentially however the people as ASKBOT would be willing to do this for a reasonably small some of money and that would help everybody using QSQA..

Comments @woodpeck @systemed and anybody else answering questions on help?

simonpoole avatar Feb 21 '17 11:02 simonpoole

If it's a fork I'm surprised the database can't just be migrated or have their been schema changes on the OSQA side since the fork?

tomhughes avatar Feb 21 '17 12:02 tomhughes

The later probably not, and I wouldn't rule out that we could migrate the mysql DB and then migrate to Postgres. From reading the support forums it is not quite clear what the issue is, but I'll dig a bit more.

This http://askbot.org/doc/upgrade.html may be part of the problem.

simonpoole avatar Feb 21 '17 12:02 simonpoole

Sounds sensible to move to something that is actively maintained if it isn't too difficult to do.

EdLoach avatar Feb 21 '17 12:02 EdLoach

OSQA works well enough for me, the glitches that are there don't bother me. I have never used Askbot, it doesn't exactly look like there's a huge developer community behind it apart from the company who runs it - will anything remain if their business tanks? I don't mind staying with OSQA but I don't mind switching either. If that should mean losing our content I think that wouldn't even be too bad, many the answers in OSQA are outdated. We count pull a read-only static image from OSQA for reference purposes and copy over relevant stuff when a question pops up... plus a reboot would create a fair karma playing field where you don't have this woodpeck guy with twice as many karma points as the next contender.

woodpeck avatar Feb 21 '17 16:02 woodpeck

@woodpeck problem is that I need to upgrade that machine to 16.04 at some point but I have no confidence OSQA won't break (it did when we went to 14.04 and we had to patch it).

tomhughes avatar Feb 21 '17 16:02 tomhughes

If it makes sense to move (and I can see why there would, given the issues we had and to some extent still have), I don't see a particular problem with doing so.

There are some "old answers" that are worth preserving, but bluntly some sort of static HTML would work for those. Many more old answers "were correct ages ago" but now no longer are.

I don't think that a "flattening" of user reputation would be a problem either.

SomeoneElseOSM avatar Feb 21 '17 17:02 SomeoneElseOSM

I've been asked to add a link to https://wiki.openstreetmap.org/wiki/Talk:Help.openstreetmap.org#Migration_QSQA_-.3E_ASKBOT here, but the content of the message is small enough to post in full:

"Sounds quite good for me. However, if we are migrating to anywhere we rather should think about how to solve the issue of outdated answers. Just deleting the whole thing every 7 years is a waste. There ARE many still good answers (yes, also much irrelevant or even wrong answers nowadays) and I try to take care of them (updating, organizing) as good as possible. I also would like to have incoming link not to be broken if possible (yes, setting it up as static html clone might work). Regarding the "outdated" problem, also see the section above. Regarding possible Karma flattening: that is likely no problem. --Aseerel4c26 (talk) 20:04, 21 February 2017 (UTC)"

SomeoneElseOSM avatar Mar 02 '17 19:03 SomeoneElseOSM

Keeping good questions/answers is certainly a good idea. We should make sure people are still able to find them via search engines.

scaidermern avatar Mar 02 '17 21:03 scaidermern

Assuming for a moment that copying over part but not all of existing Qs&As is possible. perhaps we could share the load of judging which questions/answers to keep and which to kill by resetting all scores to 0, and then asking everyone to look through stuff and upvote what they think should be kept...

woodpeck avatar Mar 02 '17 21:03 woodpeck

Yes. Maybe we can go through the top 50 questions here and decide which ones to keep: https://help.openstreetmap.org/questions/?sort=mostvoted

If they can't get migrated maybe we could still copy'n'pasting the question and 1-2 good answers and then mark them as community wiki.

scaidermern avatar Mar 03 '17 07:03 scaidermern

I think moving off of qsqa is the only reasonable option given it's unmaintained nature and problems that have required patching, so it's only a question of if askbot the best option.

There is information on migrating, so it's just a question of how far back the migrations go.

pnorman avatar Mar 03 '17 08:03 pnorman

The Wireshark project solved this by simply archiving their osmqa site.

grischard avatar Feb 18 '18 11:02 grischard

The problem is that I need to upgrade the machine and doing so will almost certainly break OSQA meaning that even running it in RO mode is likely to be hard.

tomhughes avatar Feb 18 '18 11:02 tomhughes

@tomhughes @gravitystorm @zerebubuth this would seem like a fairly self-contained activity that could be farmed out to an interested person from the wider community that wants to help. Again objections to raising this on the a suitable mailing list (I'm pushing a bit because besides the discussed issues, we should be fixing the gravatar support in some way from a data protection pov, and it doesn't make sense to do this in the old unsupported code).

simonpoole avatar Feb 18 '18 11:02 simonpoole

If you can find somebody that can figure out how to migrate the data then sure.

I actually have a chef branch where I started work on code to install and manage askbot.

tomhughes avatar Feb 18 '18 12:02 tomhughes

Is this any help? https://gist.github.com/cholin/171898332d63afb8371c

RobJN avatar Feb 18 '18 23:02 RobJN

How big is the MySQL database? Can one get access to a dump?

nebulon42 avatar Mar 20 '18 21:03 nebulon42

It's postgres, not mysql. Ignoring the (vast) session table it looks like there is a little over 100Mb in the tables.

If you're volunteering to attempt a conversion then we could probably come to an arrangement to make it available, subject to appropriate agreements around safeguarding any private data.

tomhughes avatar Mar 20 '18 22:03 tomhughes

~~I know I shouldn't because I have too many projects already, but somehow this interests me. :) So let's say I volunteer to attempt a conversion without any commitment to an ETA. But without trying it's all theory and if not successful there might be at least lessons learned for the next attempt. I'm fine with setting up a NDA or something similar. Being part of CWG might help already though there are no written agreements of course.~~

nebulon42 avatar Mar 21 '18 07:03 nebulon42

Maybe we can try with Discourse - more active development also Postgress and Ruby based. https://github.com/discourse/discourse/blob/master/docs/INSTALL.md

slachiewicz avatar Jul 16 '18 12:07 slachiewicz

Hi, about @slachiewicz's suggestion,

Maybe we can try with Discourse ...

See meta.discourse/porting-from-askbot-to-discourse ... Seems that the major investment for us is to enhance this embrionary adaptation.

ppKrauss avatar Aug 10 '18 13:08 ppKrauss

Discourse is a forum (or a replacement for forums) not a Q+A site.

That is kind of evident in the fact that it comes from one of the founders of Stack Overflow and was specifically designed to address a different use case...

tomhughes avatar Aug 10 '18 13:08 tomhughes

As you can see @ppKrauss in the repository, you can boast a large and active number of migration scripts https://github.com/discourse/discourse/tree/master/script/import_scripts

slachiewicz avatar Aug 10 '18 13:08 slachiewicz

@ppKrauss @slachiewicz pls no. While discourse may have its merits as a forum replacement (just that I've never found any) it is really completely at odds with what the help site is supposed to be doing (as @tomhughes has said).

simonpoole avatar Aug 10 '18 15:08 simonpoole

Some news about our options: meta.discourse community's positions and suggestions.

ppKrauss avatar Aug 10 '18 16:08 ppKrauss

So, I had a look at AskBot looking at testing a conversion.

I can't recommend it because we'll have the same problems as now.

All versions of AskBot rely on Django versions that were released 3-4 years ago and are EOL. The commit activity on its repository is minimal.

Keeping OSQA on a machine running ~~16.04~~ 14.04! is not a long-term option. Running Django 1.8 which has stopped getting security patches is also not a long-term solution.

pnorman avatar Jan 14 '19 20:01 pnorman

As workaround maybe pack all Django OSQA stuff to Docker?

slachiewicz avatar Jan 14 '19 21:01 slachiewicz

As workaround maybe pack all Django OSQA stuff to Docker?

That would still be running 14.04 for everything except the kernel, so there'd be the same problems.

Since 14.04 hits EOL in April, we need to upgrade the machine by then, don't we?

Should we set sometime in March as a deadline for we're going to upgrade the machine, even if it causes problems with help.osm.org?

pnorman avatar Jan 14 '19 23:01 pnorman

I looked at the options listed on a SE question and these are what I see as an active alternative

Other options that are not open-source are

  • AnswerHub, by the people who wrote OSQA.
  • Stack Exchange Enterprise
  • Try to get a Stack Exchange created for OSM
  • Drop all OSMF support for a Q&A site. Stack Exchange might fill the gap, or might not

Nothing stands out as the right option that is better than all alternatives.

pnorman avatar Jan 15 '19 00:01 pnorman

We could naturally pay the askbot developer to migrate askbot to a current django version (this would naturally only solve one of many issues).

simonpoole avatar Jan 15 '19 07:01 simonpoole