nextcloud
Introduce .nctext file extension and open in the text module by default
Start using a new file extension - for example .nctext to indicate these files are meant to be edited with text.
*.nctext is created by and opens in the text module by default *.md has an option to use the text module if the user prefers to
This will
- Avoid data loss / changing the .md syntax without consent.
- Allow users who never heard of markdown to use the editor to format text with the default settings.
*.mdfiles continue to be opened in Text by default, with a user option to disable this.
Originally posted by @inthreedee in https://github.com/nextcloud/text/issues/593#issuecomment-1344491604
Sounds like a feasible option, one thing we should evaluate is probably how different operating systems will handle those files. macOS would ask for an application to open with. Maybe the Nextcloud desktop client could register a handler for this new file type to open by default as well.
In addition apps that use text (like collectives and notes in the future) would need special care to work with those.
cc @nimishavijay @jancborchardt also for input here.
I think .text.md would be ideal and somewhat in line with what the markdown guidance rfc says about variants. But I am afraid we cannot register a different mimetype for that in nextcloud, can we?
I think .text.md would be ideal and somewhat in line with what the markdown guidance rfc says about variants. But I am afraid we cannot register a different mimetype for that in nextcloud, can we?
Yes, that is currently not possible. We only extract the last part of a file name that starts with a . https://github.com/nextcloud/server/blob/6be7aa112f6174c1406cfef5449836124a7f6f50/lib/private/Files/Type/Detection.php#L210
I'm elated that this is finally being considered. Love it!
My only question is the default setting. Is it advisable to ship Nextcloud in a configuration that is known to cause data loss?
My only question is the default setting. Is it advisable to ship Nextcloud in a configuration that is known to cause data loss?
I think we'll need a migration strategy. If users who are not aware of markdown open their personal notes and out of the sudden those contain funny characters instead of formatting they will consider that data loss.
So my current take on this would be introducing the new extension in one release and start using it by default but still open .md files with text as well. Once the new extension has been established we can think about changing the default behavior for .md files. We'll want a way to configure apps for certain file types anyway - but it's a longer process.
Start using a new file extension - for example
.nctextto indicate these files are meant to be edited with text.
Even though many of my users have compatibility issues with Text, I'd strongly oppose a unique file extension like .nctext. I very much like the WYSIWYG idea of Text, as it makes writing Markdown way easier - and thus makes apps like Pico CMS for Nextcloud - in theory - more accessible. A file with .nctext would no longer been seen as Markdown file by other software - even though the files created by Text are Markdown files without any doubt.
This doesn't mean that we shouldn't add an option for users to switch to a custom file extension. In my understanding we already support this on a global basis (https://github.com/nextcloud/text/pull/2718), correct? It shouldn't be the default though... Text is a Markdown editor after all.
I don't agree with the "Text causes data loss" argument. IMHO it is an exaggeration to sustain a maximum demand. Switching to a different file extension as an option, sure, why not, but not as the default. Switching the file extension would declare Text not being a Markdown editor - what simply isn't true. It has some compatibility issues though, like basically any Markdown parser. The correct solution to this is https://github.com/nextcloud/text/issues/3477, not switching to a different file extension.
I think .text.md would be ideal and somewhat in line with what the markdown guidance rfc says about variants. But I am afraid we cannot register a different mimetype for that in nextcloud, can we?
Yes, that is currently not possible. We only extract the last part of a file name that starts with a
.https://github.com/nextcloud/server/blob/6be7aa112f6174c1406cfef5449836124a7f6f50/lib/private/Files/Type/Detection.php#L210
I'm not sure why we would even need that when optionally supporting switching to .nc.md (or similar): Text would still register as editor for .md files, Text would rather just choose the editor mode (rich text editing vs plain text editing) depending on the file's extension when opening it.
Agree with @PhrozenByte here. The default experience works for a silent majority of people who don't know what Markdown is, and for whom a dialog like @juliushaertl mentions will already be very confusing:
macOS would ask for an application to open with.
So having a setting that people with advanced requirements can use for e.g. setting a different extension sounds ok. (Instead of making the default experience more complex.)
So my current take on this would be introducing the new extension in one release and start using it by default but still open .md files with text as well
I agree with this take. I think it solves the problems brought up in the parent issue with the least impact to everyone. I’d say it’s perfectly acceptable to expect instance administrators to notify their users of potential changes if they fiddle with any default settings.
The default experience works for a silent majority of people who don't know what Markdown is
And these people will not be affected by this change. Text will continue to function as it always has for them, except there will no longer be the possibility of data loss should they open a markdown file created in another app. Seems like a good thing all around. These people aren’t going to care what extension is being used.
To keep this discussion non-emotional and to the point, let me point out that what @PhrozenByte wrote is generally true and I agree with the most important points (especially prioritizing https://github.com/nextcloud/text/issues/3477 - another advantage is it does not seem to require any migration).
Nevertheless the comment seems slightly convoluted at places if consulted with the Markdown RFC brought up by @max-nextcloud .
Even though many of my users have compatibility issues with Text, I'd strongly oppose a unique file extension like
.nctext.
Therefore the RFC recommends .nc.md style. The fact it is not supported by NC by now is irrelevant in this discussion about future.
I very much like the WYSIWYG idea of Text, as it makes writing Markdown way easier
Everybody here does (many because it works on mobile devices) and that is the reason why this issue exists. If people would not like it, they would have simply disabled it and not care to come here to cry.
- and thus makes apps like Pico CMS for Nextcloud - in theory - more accessible.
Actually Pico CMS seems to rely on some features (and this feature set will probably grow over time without NC Text being able to catch up) which NC Text does not seem to support. So this might be a good counter example and a reasoning why NC Text must not rich-edit foreign .md files (unless the round-trip conversion produces identical results as proposed in https://github.com/nextcloud/text/issues/3477 ).
A file with
.nctextwould no longer been seen as Markdown file by other software - even though the files created by Text are Markdown files without any doubt.
That is true but kind of not the "full truth" :wink:. The RFC does make it explicit that - and I quote:
Markdown's formatting syntax only addresses issues that can be conveyed in plain text.
So if NC Text addresses also other things (by editing them - in this case by silently removing them), then it is not exactly aligned with the RFC IMHO.
Simply meaning NC Text is not (only) a Markdown editor but an editor of a superset of Markdown (i.e. editor of strictly one Markdown flavor - specifically NC flavor) in which case we would be trying to use non-Markdown editor to edit both Markdown files and Markdown-flavored files (here I mean other than NC flavors).
The Markdown RFC says:
The reflexive media type text/markdown in this parameter value means that the author does not want to invoke Markdown processing at all: the receiver SHOULD present the Markdown source as is.
Unfortunately NC does not seem to support MIME types as e.g. email does. But I am 100% certain that majority if not all of existing .md files not created using NC Text assume text/markdown (i.e. no processing and thus expect them rather being edited plain as Windows Notepad does than to omit something). https://github.com/nextcloud/text/issues/3477 seems to address this.
This leads me to suggestion that if this discussion will not converge on something like .nc.md nor https://github.com/nextcloud/text/issues/3477 gets accepted, then the only alternative would be to make NC Text "tag" each .md file it edited by adding some header string to the top of the content of the .md file.
When opening .md NC Text would first look up this tag and if it were not there, NC Text would open it read-only. The user would then be presented with a button "reformat & edit" button telling the user it would lose certait formatting after clicking. Then the file would be tagged. Any newly created .md file would be automatically tagged to keep it user-friendly as it is now.
Basically the same as https://github.com/nextcloud/text/issues/3477 but with explicit tagging as part of the content of the .md file.
This doesn't mean that we shouldn't add an option for users to switch to a custom file extension. In my understanding we already support this on a global basis (#2718), correct?
Well, #2718 only allows the user to constrain herself (as temporary measure to avoid data loss - see below). But I think we want the exact opposite. We want users to create .md files and not .txt files.
It shouldn't be the default though... Text is a Markdown editor after all.
Yes and no :wink: - see above why it is not a "true" Markdown editor per RFC.
I don't agree with the "Text causes data loss" argument. IMHO it is an exaggeration to sustain a maximum demand.
Here I think RFC tends to lean towards "data loss" interpretation (e.g. by pushing the "double window side-by-side view" by describing it in detail unlike WYSIWYG which is only mentioned among several other options buried elsewhere in only one sentence from the whole RFC).
But your mileage may vary.
Switching to a different file extension as an option, sure, why not, but not as the default.
If "own extension" would be chosen over https://github.com/nextcloud/text/issues/3477 , then it would have to be the default to solve the "data loss" case forever. But I am open to not have it as default just to see how much time it will take to make it a default (which I see as inevitable sooner or later) :wink:. My guess would be ~4 years from the release of the non-default option :wink:.
Switching the file extension would declare Text not being a Markdown editor - what simply isn't true.
Yes and no :wink:. See above.
It has some compatibility issues though, like basically any Markdown parser. The correct solution to this is #3477, not switching to a different file extension.
Yep.
I'm not sure why we would even need that when optionally supporting switching to
.nc.md(or similar): Text would still register as editor for.mdfiles, Text would rather just choose the editor mode (rich text editing vs plain text editing) depending on the file's extension when opening it.
Yep.
Just to clear up a few things: I strongly oppose .nctext or any other unique file extension, as this will heavily disturb the workflow of the majority of Text users, because other software would stop considering files created by Text to be Markdown files. This is unacceptable IMHO.
This critique is not true for "derived" file extensions like .nc.md, making things slightly better. Yes, we will still cause issues while migrating to .nc.md, negatively affecting the majority of Text users, but this disturbance might be acceptable. The thing is, switching to .nc.md doesn't actually solve "the issue": Even though a users has created a Markdown file with Text in the past, one might still edit it with any other editor - it's a .md file after all. These changes might then cause the exact same compatibility issues as before. So, what's the point of this? The informational guide (RFC 7764 is no standard) targets a different use case.
Improving compatibility (i.e. individual issues like #3516) and detecting edge cases (i.e. #3477) actually solves "the issue". Thus I believe that changing Text's file extension isn't feasible at all.
and thus makes apps like Pico CMS for Nextcloud - in theory - more accessible.
Actually Pico CMS seems to rely on some features (and this feature set will probably grow over time without NC Text being able to catch up) which NC Text does not seem to support. So this might be a good counter example and a reasoning why NC Text must not rich-edit foreign
.mdfiles (unless the round-trip conversion produces identical results as proposed in #3477 ).
As the main developer of Pico CMS for Nextcloud, I can assure you that the majority of flat file CMS users want a WYSIWYG editor, even though the features are limited. This is true for basically any WYSIWYG editor, just think of Word vs. LaTeX: yes, Word is limited and sometimes annoying to use, but still, the majority doesn't want to fiddle with syntax.
Improving compatibility (i.e. individual issues like https://github.com/nextcloud/text/issues/3516) and detecting edge cases (i.e. https://github.com/nextcloud/text/issues/3477) actually solves "the issue". Thus I believe that changing Text's file extension isn't feasible at all.
I don’t think we want to keep re-hashing this part of the debate in this issue, but I will kindly point you to the parent issue #593 for lengthy discussions on why many of us believe this doesn’t solve the issue. If further discussion about that is still needed, I think that issue is the right place for it?
I added another issue for the proposal @dumblob brought up: #3636. Now we have issues for the different ideas to make sure text does not step on peoples feed / mess with their md syntax without their consent:
- #3477
- #3630
- #3636
Please note that these proposals are not mutually exclusive. Let's try and focus in each issue on the proposal in question. We can explore ways to implement them, the implications and mitigations for problems they might cause.
As the main developer of Pico CMS for Nextcloud, I can assure you that the majority of flat file CMS users want a WYSIWYG editor, even though the features are limited. This is true for basically any WYSIWYG editor, just think of Word vs. LaTeX: yes, Word is limited and sometimes annoying to use, but still, the majority doesn't want to fiddle with syntax.
While this may or may not be true, why did anyone think that hijacking a format (.md here) that does care about syntax would be a good choice for this editor?
While this may or may not be true, why did anyone think that hijacking a format (.md here) that does care about syntax would be a good choice for this editor?
Your question is actually pointing to an important aspect. Text is not primarily a wysiwyg editor looking for a file format to serialize its content to. It's a viewer (and editor) for a lot of different file formats.
When it comes to markdown files the assumption is that people most of the time open markdown files to read the content - not to look at the syntax. Even github that is very targeted to a developer community shows markdown as rendered html rather than the source code.
With respect to editing you just quoted the exact reason.
If people on your instance have different preferences - feel free to disable rich editing for text:
occ config:app:set text rich_editing_enabled --value=0
When it comes to markdown files the assumption is that people most of the time open markdown files to read the content - not to look at the syntax.
I think you are right concerning reading the content. However, we do not talk about a viewer here, but an editor. Markdown is meant to be edited as plain text.
If people on your instance have different preferences - feel free to disable rich editing for text
Our instance has five-digits users. Any system wide preference cannot be a solution. Also, the default should be a sane one, and for markdown that is plain text editing.
With respect to editing you just quoted the exact reason.
I have nothing against WYSIWYG editors. If that's what people want, power to them. However, please do not use those for plain text, which markdown is and should be. You quoted github. Yet, it too opens a plain text editor for markdown.
Please don't rehash the same arguments over and over again. These arguments have been discussed extensively. Your opinion on the matter is absolutely valid and the arguments were acknowledged multiple times before. Text made great efforts in supporting different user preferences over the years. For example, you can make Text open files in read-only mode by default, you can change Text's default file extension, you can disable rich text editing. You can also switch to a different editor.
Regarding Text's default behaviour, please acknowledge that there are other opinions on the matter as well. These opinions are as equally valid as yours. There are Pros and Cons to every solution. The Text developers made a conscious decision about the matter and unless there are new (!) arguments that could shift that decision, please refrain from repeating the same arguments over and over again.
Notwithstanding the above, keep in mind that Text is open source software. If you want to improve Text in regards of supporting more user preferences (i.e. without changing the default behaviour) on the matter, please don't hesitate to open a pull request if it's a simple changeset, or a new issue to discuss your suggested changes you're about to develop first if it's a more complex changeset. There's also a freelancing section in the Nextcloud forums if you need help with customization.
You are right, I am sorry. I don't really want to change Text, though.
What would be the best place to discuss the default application for .md files, to make sure it is a plain text editor, ideally with cooperative editing? And to possibly even make it similar to something like HedgeDoc (or maybe it is possible to integrate it, being Open Source and such)?
AFAIK Nextcloud (server / web interface) doesn't allow for choosing the app to be used to open files based on file extensions yet. This needs development. AFAIK it's no trivial change. The correct place to discuss specific changes would be the Nextcloud server repo. If it's a smaller changeset, just go ahead and open a pull request with the proposed code changes, else open a new issue with a description of what you're about to develop in case of a bigger changeset.
Right now you can either use one of Text's many config settings to change its behaviour, or completely disable Text, which might affect other apps as well. If you need more customization options in Text, it would need development. The correct place to discuss specific changes would again be a pull request or a new issue in this repo.
There seems to be an app in development to integrate HedgeDoc, but I don't know its current state. If you have any issues with the existing integration, refer to the app on GitHub. There's a notice in their README.md that they're seeking for help, so I'm sure they would be very happy to see additional support.
If you can't do any or parts of the development necessary to develop the customization you need, refer to the freelancing section in the Nextcloud forums. There might be a developer open for hire. AFAIK Nextcloud GmbH is often open for hire, too. Just post a comprehensible description what you need in the forums and check for responses, or contact Nextcloud GmbH directly. If a developer is interested, they will contact you and possibly create a quote.
Just to confirm, this config is necessary to ensure Markdown files don't get corrupted anymore?
occ config:app:set text rich_editing_enabled --value=0
I'm a little scared by that... Last time I tried a global config like that, it broke other seemingly unrelated apps. Do you know of any drawbacks like that here? How dangerous is messing with that setting? Thanks.
I'm a little scared by that... Last time I tried a global config like that, it broke other seemingly unrelated apps. Do you know of any drawbacks like that here? How dangerous is messing with that setting? Thanks.
The only thing I can think of is that you need to clear all text editing sessions so the richtext editing and the source code editing do not get mixed:
occ text:reset --all
You might need --force if people still have tabs open with the documents in question or the editing sessions were not closed.
I tried the setting in the collectives app and it worked as expected. That said I don't think we have automated tests for it yet. I also did not try it in the notes app.
If you find anything that breaks it's a bug and I'm happy to help fixing it.
OK, thanks Max! I moved all our files off Nextcloud, but that was difficult and I miss many things about it ... Now I'm trying to decide if we should move just non-Markdown files back, which leads to confusion about what files are where, or if we move everything back, which means struggling with this corruption again. Just trying to find the best use of limited time. I'll cc you if I find any bugs.
I tried it and it worked as expected. It basically brings back the old version of text, without the wisywig stuff that destroys Markdown formatting.
