Rebuild Editing POS - separate changing each parameter

[monikasmolarek]

Rationale

Right now, if the Smesher cannot change just one POS parameter - i.e. only "Max file size", leaving the rest of the parameters unchanged. The User has to always update the POS directory, otherwise, the smeshing process will be stopped. Would it be possible to separate editing the "Data size/POS size" and "Max file size" from the POS directory?

Steps:

1. click "Edit POS"
2. Leave the same directory, I don't want/need to change it, click next; (There is no notification in this case about the POS being stopped if the directory is not empty)
3. Edit data sizes only
4. no other changes; finish editing

This stops the POS generation because the "new" POS directory stayed the same, so it was not empty. The User has to go through the editing process again to correct the directory and restart the smeshing, and only if he knows already that the "Status/Process - STOPPED" is caused by the directory that was not set up correctly.

A notification/information on the empty directory or separating the directory from editing other parameters would be good here.

Prerequisites

fix-smesher-screen to be finalized/stabilised in general

User story

I can edit the POS data accordingly to my needs:

• [ ] I can edit other POS parameters and leave the POS directory unchanged
• [ ] I can reallocate memory depending on my current resources without having to restart the whole POS generation process
• [ ] Otherwise, If the POS generation stopped, I am aware of the reason (POS directory was not empty i.e.?)
• [ ] The Max File Size field is easily editable (as a standard input field)

Acceptance criteria

• [ ] "Modify" button should be hidden (setup smeshing screens are not accessible until PoS data exists)
• [ ] "Delete" button should lead to the next screen with a warning message and confirmation button.
• [ ] Deleting PoS, probably, should also delete the key.bin file

Out of scope

N/A

How to implement

N/A

How to test

Perform the following steps:

1. Go to the Smeshing screen, (POS already set up)
2. Click "Edit POS"
3. Leave the same directory, and click next; (Depending on the resolution, either I can leave this parameter with no future errors or there is a notification about the POS being stopped if the directory is not empty)
4. Edit any parameter in the next step, i.e. "Data size/POS size" or "Max file size"
5. no other changes; finish editing
6. Check whether the POS generation works correctly

[pigmej]

We really cannot change the change of the file after it's there. So the case with "changing the size" is not supported by definition currently (that would require redoing/merging / splitting the already existing files).

[maparr]

Hi @nj, @monikasmolarek , Im thinking about the screen only as a setting that our user can change, and not change the Dir for the smeshing. When the smeshing was init, I just want to add some button, where the user will be able to change settings for the currently selected folder. Nothing special, when settings will be updated, restart the smeshing .

[maparr]

1085 will be possible to implement finally when go-spacemesh will support this feature. Need to wait. cc: @monikasmolarek

[brusherru]

1085 will be possible to implement finally when go-spacemesh will support this feature. Need to wait.

But we can call the same StartSmeshing endpoint with only one prop updated. Right?

[maparr]

1085 will be possible to implement finally when go-spacemesh will support this feature. Need to wait.

But we can call the same StartSmeshing endpoint with only one prop updated. Right?

Currently, we have an issue when we're running already PoS enabled app and trying to change only options without changing the directory or so, and smeshing not working anymore at all. So, only when @fasmat will be able to confirm we're ready to restart the smeshing process with new settings, only after it this task will make sense, after the genesis

[fasmat]

Changing PoST settings, especially the total size allocated is a complicated process. For example:

• increasing your post size: the new labels have to be created and (possibly) a new VRF nonce needs to be determined using the GPU. The effects of the post size increase do not take effect until 2 epochs after the current. Since 2 ATXs have to be published for the network to accept the increased size.
• decreasing your post size: similar to increasing, but it takes effect already 1 epoch after the current
• changing max file size: this requires shuffling around bytes between files to take effect
• I'm not sure which other settings we allow users to modify, maybe one of you can enlighten me 🙂

At the moment we have no way of locking PoST while it is being used (e.g. to generate a proof or change its size) to avoid races. Additionally the code handling proof generation is at the moment completely rewritten in Rust to improve performance which blocks work on this topic until the first new version is available 🙁

Changing any PoST settings probably won't be possible before genesis.

[fasmat]

We should probably also rethink the API between the node and Smapp. StartSmeshing does multiple things at the moment:

• Initialize post
• Update post settings
• Start/Stop smeshing, i.e. instruct the node to participate in the network by publishing ATXs

All of these 3 should imo be individual endpoints, that are mutually exclusive to be called, i.e. a user cannot start smeshing while initializing and cannot update their post settings while they are smeshing, or we need to properly handle concurrent read & write access of the post data.

[monikasmolarek]

Then I would suggest hiding/removing the Modify POS section in SMAPP. If there's basically no parameter we can modify without actually recreating everything from scratch and/or waiting for 2 full epochs, then maybe Edit-> Modify POS should be reduced to just Delete POS Data for now? It was already shortly discussed when suggested by @maparr cc @brusherru

[fasmat]

The changes only take effect after 1 or 2 epochs if a user changes their post size, i.e. increases or decreases their NumUnits value or changes Coinbase, which will also requires the user to wait for 1 epoch to take effect.

MaxFileSize "only" shuffles data on the hard disk, no need to wait for 2 epochs (but not implemented because concurrent access to PoST isn't possible at the moment).

Changing ComputeProviderID has no effect without also increasing NumUnits and the same is true for Throttle.

cc @pigmej

[monikasmolarek]

Which still means that for now, we cannot actually use Edit - in the sense of what the Users might expect... "Regenerate" might be a more appropriate naming, implying the fact that the changes are not instantly applied and are often quite costly in terms of resources usage, time etc.

[pigmej]

@monikasmolarek @brusherru @maparr I think we were discussing that already some time ago. And kinda we agreed that it does not make sense to "modify POS" because there is literally nothing that you can modify there. Sometime later there will be the possibility to add more storage etc.

But definitely not going to happen before Genesis.

[fasmat]

"Regenerating" is tricky. A user has to keep at least their identity and the post_metadata.json file to be able to re-create their post, otherwise they might create an invalid proof and will be flagged as malicious by other users.

I will need to double check that but in general deleting your local PoST and re-creating it is at least at the moment not supported.

[brusherru]

@fasmat am I understood right that creating a new PoS for the same smesher id would not work (at least without post_metadata.json and there are still no guarantees)? Going to check it now :)

If so, then we need to hide the "Delete PoS data" button as well :D Or at least make it delete the whole directory, including the smesher id. Then there should be one more screen with a big red warning and a small confirmation button :)

To sum up: We need to rephrase this issue, having these acceptance criteria:

• "Modify" button should be hidden (setup smeshing screens are not accessible until PoS data exists)
• "Delete" button should lead to the next screen with a warning message and confirmation button.
• Deleting PoS, probably, should also delete the key.bin file, if it is useless (need to check) Btw, actually deleting these files is handled by go-spacemesh (Smapp calls SmesherService.StopSmeshing with { deleteFiles: true }) and it deletes postdata_N.bin and postdata_metadata.json, but keeps key.bin.

Am I missed something?

[brusherru]

Ok, seems it really does not work. So if you deleted PoS data — you can delete key.bin as well :)