go-disk-usage icon indicating copy to clipboard operation
go-disk-usage copied to clipboard

does not produce accurate numbers

Open drrob1 opened this issue 4 years ago • 7 comments

On Ubuntu 20.04, the numbers it produces are all the same. IE, Free(), Available(), Size() and Used() all produce the same output. These numbers do not match the output of the df command at all. I'm currently using a script that uses $(df -k --output=avail /dev/sda1 | tail -n1) I have kernel 5.8.0.7642-generic

drrob1 avatar Apr 14 '21 20:04 drrob1

@drrob1 This library is actually a very thin wrapper around the following code:

    var stat syscall.Statfs_t
    syscall.Statfs("/path/here", &stat)

This probably means one of the following:

  1. syscall.Statfs() is broken (unlikely)
  2. Something is wonky with your computer
  3. There is a bug in your code

Could you post the code you are using?

ricochet2200 avatar Jul 04 '21 05:07 ricochet2200

package main import ( "fmt" "github.com/ricochet2200/go-disk-usage/du" "os" "runtime" "strings" "unicode" ) func main() { if len(os.Args) < 2 { fmt.Println(" Usage: diskusage [/dev/sdX] | [drive:]") os.Exit(1) } volumePath := "" volume := os.Args[1] if runtime.GOOS == "linux" { //if !(strings.HasPrefix(volume, "/dev/") && strings.HasSuffix(volume, string(os.PathSeparator))) { if !(strings.HasPrefix(volume, "/dev/")) { fmt.Println(" On linux, volume must begin /dev/.") os.Exit(1) } volumePath = volume } else if runtime.GOOS == "windows" { volByteSlice := []byte(volume) driveletter := rune(volByteSlice[0]) colon := rune(volByteSlice[1]) if !(unicode.IsLetter(driveletter) && colon == ':') { fmt.Println(" On Windows, volume must begin w/ a 2 character drive designator") os.Exit(1) } volumePath = string(driveletter + colon + os.PathSeparator) } usage := du.NewDiskUsage(volumePath) fmt.Println("Free:", usage.Free()) fmt.Println("Available:", usage.Available()) fmt.Println("Size:", usage.Size()) fmt.Println("Used:", usage.Used()) //fmt.Println("Usage:", usage.Usage()*100, "%") fmt.Println() }

drrob1 avatar Jul 04 '21 15:07 drrob1

Of course, the site removed my indentations. But gofmt would restore those, I guess. I never tested it on windows, though I did write code to be able to.

drrob1 avatar Jul 04 '21 15:07 drrob1

@drrob1 I'm not sure if you meant to close this issue or not, but I thought I'd reply anyway.

I was able to reproduce you results on CentOS 7 running on WSL. The numbers are close, but not exact. I'm not an expert at df so I can't speak to what exactly the numbers are reported. However, I did verify that the results from this code in go

package main

import (
        "fmt"
        "syscall"
)

func main() {

        var stat syscall.Statfs_t
        syscall.Statfs(".", &stat)

        fmt.Println("Bsize", stat.Bsize)
        fmt.Println("Bavail", stat.Bavail)
        fmt.Println("BFree", stat.Bfree)
        fmt.Println("Blocks", stat.Blocks)
}

closely resembles this code in c

#include<stdio.h>
#include<sys/stat.h>
#include<unistd.h>
#include<sys/types.h>
#include<sys/vfs.h>
int main(int argc,char *argv[])
{
  struct statfs sb;
  if((statfs(argv[1],&sb))==0)
    {
      printf("optimal transfer blk size is %d\n",sb.f_bsize);
      printf("Free blocks available to unpriviledged user %d\n",sb.f_bavail);
      printf("Free blocks in filesystem %d\n",sb.f_bfree);
      printf("Total Data blocks in filesystem %d\n",sb.f_blocks);
    }
}

There are differences in the lower order bits from run to run but both programs produce very similar results. This is not surprising because I believe go calls statfs under the covers. The math for calculating the results is pretty straightforward using those results. df maybe takes into account more factors which creates a difference?

ricochet2200 avatar Jul 07 '21 23:07 ricochet2200

My point is that the usage.Free() returned from the Go function is off by 3 orders of magnitude from the output of the linux df command.

So I don't understand what you are saying

And I did not intend to close the issue. I must have clicked on the wrong thing.

drrob1 avatar Jul 21 '21 19:07 drrob1

I have a WSL system so it was a little difficult to test your code directly because I don't know how devices like /dev/sda1 apply. I got a hold of a native centos 7 machine and ran your code and replicated your results.

This library as written expects a path and not a device. Which is to say it needs "/" not "/dev/sda1". This is because this library uses https://pkg.go.dev/syscall#Statfs which requires a path. I can look into updating the library to something like https://pkg.go.dev/syscall#Fstatfs but I'd need to investigate and think through this to make sure I don't break live code depending on this functionality. If this goes as expected I'll add this soon...

ricochet2200 avatar Jul 21 '21 22:07 ricochet2200

I want to add that there needs to be a way to not follow other filesystems that are mounted, perhaps by not following any symlinks or mount points

Like the linux command "du --one-file-system"

--rob

On 7/21/21 6:46 PM, Rick Smith wrote:

I have a WSL system so it was a little difficult to test your code directly because I don't know how devices like /dev/sda1 apply. I got a hold of a native centos 7 machine and ran your code and replicated your results.

This library as written expects a path and not a device. Which is to say it needs "/" not "/dev/sda1". This is because this library uses https://pkg.go.dev/syscall#Statfs https://pkg.go.dev/syscall#Statfs which requires a path. I can look into updating the library to something like https://pkg.go.dev/syscall#Fstatfs https://pkg.go.dev/syscall#Fstatfs but I'd need to investigate and think through this to make sure I don't break live code depending on this functionality. If this goes as expected I'll add this soon...

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ricochet2200/go-disk-usage/issues/5#issuecomment-884545902, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEDMFBUJP3RRK7NWNOUWC43TY5E3RANCNFSM426EJL7Q.

drrob1 avatar Jul 21 '21 23:07 drrob1