etcher icon indicating copy to clipboard operation
etcher copied to clipboard

Add persistence option

Open timtjtim opened this issue 8 years ago • 38 comments

I can't remember which app does it (perhaps usb-creator-gtk) but I remember being able to choose the free space for writing files when booting a Live USB such as Ubuntu.

Is this supported / could this be supported?

timtjtim avatar May 16 '16 13:05 timtjtim

Hi @timtjtim ,

I don't think I understand the feature request very well. What do you mean by writable space? USB drives written by Etcher shouldn't be read-only, so you could mount it back and put any other files you want.

jviotti avatar May 16 '16 18:05 jviotti

Hi @jviotti,

I believe @timtjtim is speaking about the option for persistent storage when creating a live USB for Linux OS's like Ubuntu, Mint, and more. What it does is allow the live USB to have files written and stored on it (like a normal OS) and actually allow for the files and changes to be saved when closing, disconnecting, and rebooting back into the live USB. Additionally, UNetbootin is the application that has this feature; and honestly, I'm a big fan of it myself. I'd really like to see a feature like this implemented into Etcher if possible.

miniShrooms avatar May 19 '16 00:05 miniShrooms

@jviotti Yes, @miniShrooms is correct. That's the idea.

timtjtim avatar May 19 '16 01:05 timtjtim

Hey there,

Thanks for the very detailed explanation. I'll play around with UNetbootin and try to research how this is implemented, and if it'd be reasonable to include it in Etcher.

jviotti avatar May 22 '16 21:05 jviotti

I'm too interested in persistent storage, so I can install some packages in my usb only once, and also only have to disable the screensaver once 👍 Thanks

m4gn3to avatar Jun 16 '16 22:06 m4gn3to

Reading UNetbootin's source, the approach it takes is to create a casper-rw file in the root directory by padding it with \NUL to the user specified size—limited to 4GB by FAT32—and then running mke2fs on said file. On Windows, it ships with a mke2fs.exe executable. Another approach is to make an EXT2 partition labelled casper-rw which doesn't have the 4GB limitation. In order to facilitate persistence it also seems to enable a kernel option for persistence which it then adds to the GRUB config.

Shou avatar Nov 25 '16 12:11 Shou

Another modification of the same kind is to extend the last partition to the end of the card. Raspbian and resinOS both need/do that. It sounds like we should curate a list of "post-write interventions" people may want to make to their images, and then see how we can encode these in the etcher format. Once we have that in place, we should find how to open these up to vanilla images. We could, potentially, offer the users manual ways of triggering them from the success screen, or maybe having some kind of heuristic?

--

Alexandros Marinos

Founder & CEO, Resin.io

+1 206-637-5498

@alexandrosm

On Fri, Nov 25, 2016 at 12:47 PM, Benedict Aas [email protected] wrote:

Reading UNetbootin's source, the approach it takes is to create a casper-rw file https://github.com/unetbootin/unetbootin/blob/787323b2a4595c50a400f1024ca141e4708c7b7d/src/unetbootin/unetbootin.cpp#L4259 in the root directory by padding it with \NUL to the user specified size—limited to 4GB by FAT32—and then running mke2fs on said file. On Windows, it ships with a mke2fs.exe executable https://github.com/unetbootin/unetbootin/blob/787323b2a4595c50a400f1024ca141e4708c7b7d/src/unetbootin/unetbootin.cpp#L4266. Another approach is to make an EXT2 partition labelled casper-rw which doesn't have the 4GB limitation. In order to facilitate persistence it also seems to enable a kernel option for persistence https://github.com/unetbootin/unetbootin/blob/787323b2a4595c50a400f1024ca141e4708c7b7d/src/unetbootin/unetbootin.cpp#L3626 which it then adds to the GRUB config https://github.com/unetbootin/unetbootin/blob/787323b2a4595c50a400f1024ca141e4708c7b7d/src/unetbootin/unetbootin.cpp#L3679 .

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/resin-io/etcher/issues/413#issuecomment-262951418, or mute the thread https://github.com/notifications/unsubscribe-auth/ABLUCEGBlP4IzDGC6Rn0p3mOd5-dd3AIks5rBtj9gaJpZM4IfV2T .

alexandrosm avatar Nov 26 '16 10:11 alexandrosm

It sounds like we should curate a list of "post-write interventions" people may want to make to their images, and then see how we can encode these in the etcher format.

:+1: I was thinking along exactly the same lines after @petrosagg 's LKL talk.

lurch avatar Nov 28 '16 17:11 lurch

Hi. I'm not sure whether the following could assists you about adding this feature or not. Please look for this program:

https://www.pendrivelinux.com/

it is for Windows only. But look to it, my be of benefit.

Look to last change log for Universal USB Installer - Easy as 1 2 3: https://www.pendrivelinux.com/universal-usb-installer-easy-as-1-2-3/

it say: Universal-USB-Installer-1.9.7.7.exe – March 21, 2017 – Changes Added casper-rw creation progress banner. Corrected casper slider max size relative to fat32 vs ntfs selection. Add Windows to Go VHD option. Added more info for syslinux warning. Add NTFS format option. Add filesystem type to drivelist. Include message box to notify of NTFS filesystem requirement for Windows to Go option. Add larger than 4GB casper-rw support for Ubuntu if filesystem is NTFS (creates 4th partition table) – Note removes EFI boot option. Correct broken Ubuntu and Linux Mint installer options when no format option is chosen.

Also, there is other program: https://www.linuxliveusb.com/

But 1st program (Universal USB Installer - Easy as 1 2 3)

Best.

Nokia808 avatar May 02 '17 13:05 Nokia808

Hi, is this feature still planned on the todo list ? Would be a great addition to etcher, for example compared to Rufus.

jo-chemla avatar Jul 19 '17 09:07 jo-chemla

@jviotti Perhaps this is a feature that might fit in with the "plugin" system you've been thinking about?

lurch avatar Jul 19 '17 11:07 lurch

@jhermsmeier Could this be done generically? As far as I understand we simply have to update the partition table and mark certain partitions as read-write, right? If so, how do we know which? We now have a lot of tooling to work with partition tables, so maybe we're half way there for this already.

If we need image specific information to do this properly, then I guess this will fit inside the catalog idea.

jviotti avatar Jul 19 '17 12:07 jviotti

I once looked at this many moons ago (before even working on the Etcher project, so my info is a bit foggy and out of date) and IIRC it involves casper filesystems, syslinux, various other bits and bobs... ;-)

Don't forget that even when enlarging a partition, you have to update both the MBR (to modify the raw partition size) and the filesystem itself (to update the size of the filesystem inside the partition).

lurch avatar Jul 19 '17 13:07 lurch

A lot of distro's are available with supported persistence built in, puppy linux, porteus, slackware, slitaz, and yes also Ubuntu and Debian. Through past experience Puppy has it made, runs in ram (very fast), secure, comes in different flavours. The most common reasons users desire persistence, are Desktop settings, User installed packages and personal files. Most systems either use "casper" or "cow" for writing changes (although limited). Is easy achieved as shown in multibootusb, Which, with the size of portable storage devices nowadays, is another option which i personally would like to see. As achieved with yumi for windoze and the afore mentioned multibootusb.

casearis avatar Feb 09 '18 21:02 casearis

+1 bump for persistence capability. it's basically a commodity these days for Mac users to be able to do anything like this

pythlang avatar Feb 13 '18 01:02 pythlang

+1 from me as well. Live images are really important for Mac users who are dual-booting Linux.

ethan7g avatar Mar 18 '18 05:03 ethan7g

+1 Definitely would like the feature so I can try out distros and use them for a few things.

inferno986return avatar Mar 26 '18 16:03 inferno986return

This feature would be very useful for my friend who wants to try Linux on her Mac.

aadilayub avatar Aug 09 '18 16:08 aadilayub

so will this be added? I recently made an Ubuntu drive but I cant save my files

Mattybootsy avatar Aug 31 '18 20:08 Mattybootsy

Not sure if persistence will be added to etch.. for ubuntu distro's, unetbootin and liveusb cover that.. however, needs to be fat32/16 filesystem. Try multiboot-usb, good for testing live iso's. Can also handle persistence for deb/ubu , also supports ext4 fs.

Most live iso's have persistence built in. But that's another story...

casearis avatar Sep 04 '18 13:09 casearis

This would be an awesome feature. Etcher is second to none in terms of ease of use, but not being able to enable persistence makes it unusable for most of my purposes.

alichtman avatar Feb 09 '19 08:02 alichtman

I have been researching during my free time to contribute and make this feature available in balena, but my point is I have figured this out for Debian/Ubuntu Systems already but not sure how to implement it for other Distros.

Would love to have some help.

ohbus avatar Aug 02 '19 11:08 ohbus

@ohbus Can you describe your solution for Debian/Ubuntu?

alichtman avatar Aug 04 '19 17:08 alichtman

I have figured this out for Debian/Ubuntu Systems already but not sure how to implement it for other Distros.

@ohbus I'd assume that https://unetbootin.github.io/#distros and https://github.com/unetbootin/unetbootin/ and https://github.com/unetbootin/unetbootin/wiki/howitworks would be good starting points?

lurch avatar Aug 05 '19 16:08 lurch

Yes

I have figured this out for Debian/Ubuntu Systems already but not sure how to implement it for other Distros.

@ohbus I'd assume that https://unetbootin.github.io/#distros and https://github.com/unetbootin/unetbootin/wiki/howitworks and https://github.com/unetbootin/unetbootin/wiki/howitworks would be good starting points?

Yes you are absolutely right.

They are the starting point, but need to implement it for different OS and archtype.

I'm a little busy right now, will surely get back to this once I get some time.

Thank you.

ohbus avatar Aug 05 '19 16:08 ohbus

Ohbus, is there a way to make it with ParrotOS. If you can make a secondary application that has that option. It may greatly enhance your solutions because errors would pop up.

At-Heal avatar Sep 02 '19 12:09 At-Heal

Ah yes. Quarter Gig of an app to do what a 0.000075 Gig dd tool does — but with eyecandy.

What a waste of my SSD capacity!

The app so beautifully displays the 16 GiB flash capacity I picked to boot from — only to put a measly 1.3G of casper-rw free space on it.

What a waste of my flash capacity!

Honestly guys? I'm disappointed. Four orders of magnitude bloat is not okay. My SSD space is not that cheap as you'd like to think. My time is not that cheap, to start over with a tool that doesn't suck, and can produce a usb stick which can boot and store another ISO image on it.

Keep not caring.

ulidtko avatar Dec 27 '19 23:12 ulidtko

Honestly guys? I'm disappointed. Four orders of magnitude bloat is not okay. My SSD space is not that cheap as you'd like to think. My time is not that cheap, to start over with a tool that doesn't suck, and can produce a usb stick which can boot and store another ISO image on it.

Keep not caring.

This is just an utterly useless and highly disrespectful post. We are here to suggest and discuss improvements to make the application better, not to cry and bitch about something that had not even been implemented yet.

You sound as if you are being forced to use this application that has been made available to all of us for free. Go make an effort to look for an alternative that actually promises to do what you want. It is not until you actually put some time, effort and/or money into it and it does not live up to its promises that you have some form of right to complain about it.

Forage avatar Dec 28 '19 08:12 Forage

Ah yes. Quarter Gig of an app to do what a 0.000075 Gig dd tool does — but with eyecandy.

Use dd then.

timtjtim avatar Dec 28 '19 10:12 timtjtim

not to cry and bitch about something that had not even been implemented yet.

Dying to see the plans to implement that. (Considering it's an Electron app.)

With all due respect, I simply don't buy that presumed promise that bloat reduction will ever be implemented here.

Also, the fact that the previous commenter had promptly deleted their criticism — is a stellar indication that honest feedback is not taken well in this community.

Respectfully, I'll let you draw your own conclusions.

ulidtko avatar Nov 17 '21 16:11 ulidtko