afero icon indicating copy to clipboard operation
afero copied to clipboard

Memory file limit

Open totalys opened this issue 4 years ago • 1 comments

Is there a way to increase the memory file size? it seems to limit at 255 chars.

I've wrote a simple example to show the issue.

` fs := afero.NewMemMapFs()

logfilePath := "/tmp/logmsg"
f, _ := fs.Create(logfilePath)
f.WriteString(`{`)
f.WriteString(`	 "01": "1",`)
f.WriteString(`	 "02": "10",`)
f.WriteString(`	 "03": "100",`)
f.WriteString(`	 "04": "1000",`)
f.WriteString(`	 "05": "10000",`)
f.WriteString(`	 "06": "100000",`)
f.WriteString(`	 "07": "1000000",`)
f.WriteString(`	 "08": "10000000",`)
f.WriteString(`	 "09": "100000000",`)
f.WriteString(`	 "10": "1000000000",`)
f.WriteString(`	 "11": "10000000000",`)
f.WriteString(`	 "12": "100000000000,"`)
f.WriteString(`	 "13": "1000000000000,"`)
f.WriteString(`	 "14": "10000000000000,"`)
f.WriteString(`	 "15": "100000000000000,"`)
f.WriteString(`	 "16": "1000000000000000,"`)
f.WriteString(`	 "17": "10000000000000000"`)
f.WriteString(`}`)
f.Close()

b, err := afero.ReadFile(fs, logfilePath)
if err != nil {
	t.Fatal(err)
}
var messagelogged map[string]interface{}
err = json.Unmarshal(b, &messagelogged)
if err != nil {
	t.Fatal(err)
}

fmt.Println(messagelogged)`

json unmarshal works until the 14h value. When executing this code, it will break on unmarshaling.

inspecting fs, I can see:

string(fs.data["\\tmp\\logmsg"].data)

"{ "01": "1", "02": "10", "03": "100", "04": "1000", "05": "10000", "06": "100000", "07": "1000000", "08": "10000000", "09": "100000000", "10": "1000000000", "11": "10000000000", "12": "100000000000" "13": "1000000000000" "14": "10000000000000"

incomplete.

totalys avatar Feb 17 '21 17:02 totalys

Can you provide details of your go env. I did different tests, and everything seems to work fine for me

package main

import (
	"encoding/json"
	"log"
	"os"

	"github.com/spf13/afero"
)

func main() {
	var AppFs = afero.NewMemMapFs()

	logfilePath := "/tmp/logmsg"
	f, _ := AppFs.Create(logfilePath)
	f.WriteString(`{`)
	f.WriteString(`	 "01": "1",`)
	f.WriteString(`	 "02": "10",`)
	f.WriteString(`	 "03": "100",`)
	f.WriteString(`	 "04": "1000",`)
	f.WriteString(`	 "05": "10000",`)
	f.WriteString(`	 "06": "100000",`)
	f.WriteString(`	 "07": "1000000",`)
	f.WriteString(`	 "08": "10000000",`)
	f.WriteString(`	 "09": "100000000",`)
	f.WriteString(`	 "10": "1000000000",`)
	f.WriteString(`	 "11": "10000000000",`)
	f.WriteString(`	 "12": "100000000000,"`)
	f.WriteString(`	 "13": "1000000000000,"`)
	f.WriteString(`	 "14": "10000000000000,"`)
	f.WriteString(`	 "15": "100000000000000,"`)
	f.WriteString(`	 "16": "1000000000000000,"`)
	f.WriteString(`	 "17": "10000000000000000"`)
	f.WriteString(`}`)
	f.Close()
}

github.com/spf13/afero/mem/file.go

func (f *File) Close() error {
	fmt.Println("logginfs: ", string(f.Data().data))

	f.fileData.Lock()
	f.closed = true
	if !f.readOnly {
		setModTime(f.fileData, time.Now())
	}
	f.fileData.Unlock()
	return nil
}
logginfs:  {     "01": "1",      "02": "10",     "03": "100",    "04": "1000",   "05": "10000", "06": "100000",  "07": "1000000",        "08": "10000000",       "09": "100000000",      "10": "1000000000",     "11": "10000000000",    "12": "100000000000,"   "13": "1000000000000,"  "14": "10000000000000,"         "15": "100000000000000,"        "16": "1000000000000000,"       "17": "10000000000000000"}

My output was as expected, so if you provide a little more detail, update the library and check your version of go, that would be a great help.

My test was run on, go version go1.18 linux/amd64

AndrusGerman avatar May 02 '22 15:05 AndrusGerman