Sia host doesn't do a free space check while creating a new folder

[KopfKrieg]

Currently, it's possible to create (with siac, don't know if this is possible in the gui too), for example, a 2TB host folder, on a 1TB drive, with only 300GB free disk space. This should not be possible.

Expected Behavior

The sia daemon/host module should refuse to create a folder if the disk space where the folder would lay is less than what the user requests.

How to reproduce it (as minimally and precisely as possible)

siac host folder add /path/to/small/partition 2TB

Environment

• Sia version: 1.3.1
• OS: Archlinux

[tbenz9]

This is unlikely to be implemented in Sia. It's trivially easy to lie about the amount of space you have available (for example, thin provisioned file systems, or this: https://github.com/NebulousLabs/Sia/compare/master...mtlynch:lying-host?expand=1). Since Sia depends on a trustless relationship between renters and hosts you cannot trust the amount of space a host says they have.

Hosts are incentivized via collateral and storage proofs to be truthful: Hosts get penalized for missing storage proofs, if they create contracts and then cannot store the data reliably the host gets penalized. In other words, if a host lies about the amount of storage they have, and then cannot actually store files, they lose collateral.

I'm going to close this issue as I don't believe the devs have any interest in providing the functionality you mentioned. Please re-open this issue if you would like to continue the discussion.

[KopfKrieg]

Sorry to comment on this again, but my issue was not about preventing hosts to lie about the available space. I know this is easily possible, but I think it should be implemented anyway. Software shouldn't let users (admins are users too) make mistakes, therefore it should be implemented. It's probably just a simple check, a few lines of code, that could prevent human errors.

(Btw., I can't reopen the issue, just commenting it)

[tbenz9]

I reopened it for you. I'll let a Dev weigh in on your proposal.

[lukechampine]

hmm. Currently the code calls Truncate to try to allocate space on disk for the folder: https://github.com/NebulousLabs/Sia/blob/4253a220a53228e2f089c764f258ee6913b61df8/modules/host/contractmanager/storagefolderadd.go#L222-L235

On some filesystems, Truncate may not properly reserve the disk space, i.e. it may lie. I'm not sure whether that's the case here, but it's my best guess. What does the storage folder look like after siac host folder add returns? Is there a very large file inside?

[KopfKrieg]

I'm not sure whether that's the case here, but it's my best guess.

You're right. I'm using btrfs as file system and Truncate() creates a sparse file.

I'm not sure if this issue should be closed or not. I'm fine with creating a sparse file, but I think it should still be checked if the requested space is really available on the file system/partition.

Is there a very large file inside?

Yes, but ls doesn't show the real file size. If I use du to estimate the file size it says zero bytes right after creating it.

[lukechampine]

Looks like there is a cross-platform package for determine free space: https://github.com/ricochet2200/go-disk-usage

We don't like adding new dependencies if we can avoid it, so we'll have to decide whether this is worth it. Reimplementing the functionality is also an option.

[Fornax96]

No need for a library, Go has this built in. Here's the function I use for pixeldrain:

import "syscall"

// FreeSpace uses a syscall to get the amount of free bytes in a directory
func FreeSpace(dir string) (uint64, error) {
	var statfs syscall.Statfs_t
	err := syscall.Statfs(dir, &statfs)
	if err != nil {
		return 0, err
	}

	return statfs.Bavail * uint64(statfs.Bsize), nil
}

EDIT: Oh, windows exists too of course. Apparently you need to load a DLL for that: https://stackoverflow.com/questions/20108520/get-amount-of-free-disk-space-using-go

Strange that the Go stdlib doesn't have a cross platform function for this..

[DavidVorick]

Does it also have proper support for realizing different hardrives? I guess Statfs is looking at the filesystem to figure this out, which should be aware of the logical drives.

I know when the host was originally implemented, we didn't take this route because we couldn't find a good, cross platform way to do it. But it looks like there's now a decent library. Someone from the team would need to audit the library and make sure it's not going to cause us problems down the line (which often is the case for simple / small libraries as the owners decide to upgrade functionality without thinking about compatibility - though we can give ourselves control over that with vendoring)

[Fornax96]

Does it also have proper support for realizing different hardrives?

The directory that you pass to syscall.Statfs will be used for getting free space, so if that directory is on an external disk or a virtual filesystem it will get its values from the driver of that filesystem.