sync.waitgroup behaves differently from Golang

[axetroy]

V version: V 0.1.29 ed874ff OS: macos 11.0.1

What did you do?

I wrote a function to traverse directories using goroutine

// main.v
// run: v run main.v
module main

import os
import sync

fn walk(dir string, mut wg sync.WaitGroup, ch chan int) {
	defer {
		wg.done()
	}

	ch <- 1

	println('before: $ch.len')

	files := os.ls(dir) or {
		panic(err)
	}

	s := <-ch or {
		panic(err)
	}

	println('after: $ch.len')

	if s <0 {
		// do nothing
	}

	for file in files {
		target := os.join_path(dir, file)
		if os.is_dir(target) {
			wg.add(1)
			go walk(target, mut wg, ch)
		} else {
			// println(target)
			// ch <- 1
		}
	}
}

fn main() {
	mut wg := sync.new_waitgroup()

	wg.add(1)

	mut ch := chan int{cap: 255}

	go walk("/path/to/dir", mut wg, ch)

	wg.wait()
	ch.close()

	println('done')
}

What did you expect to see?

The proceeding should exit, wg.Wait() should execute like expect.

What did you see instead?

At least I didn't see anything wrong from the above code.

Normally, the process will exit normally.

But in reality, the process will hang. wg.wait() will never execute

For comparison, I wrote an equivalent Golang code

// main.go
// run: go run main.go
package main

import (
	"io/ioutil"
	"path"
	"sync"
)

func walk(dir string, wg *sync.WaitGroup, ch chan int) {
	defer wg.Done()

	ch <- 1

	println("before", len(ch))

	files, err := ioutil.ReadDir(dir)

	if err != nil {
		panic(err)
	}

	s := <-ch

	println("after", len(ch))

	if s < 0 {
		// do nothing
	}

	for _, file := range files {
		target := path.Join(dir, file.Name())
		if file.IsDir() {
			wg.Add(1)
			go walk(target, wg, ch)
		} else {
			// println(target)
			// ch <- 1
		}
	}
}

func main() {
	wg := sync.WaitGroup{}

	wg.Add(1)

	var ch = make(chan int, 255)

	go walk("/path/to/dir", &wg, ch)

	wg.Wait()
	close(ch)

	println("done")
}

and Golang's code works fine for me.

[zakuro9715]

I tried it. walking './vlib' is works. But walking '/home/zakuro/src' is hungup. In golang, Works both.

I think, This problem occurs when target files too many.

[axetroy]

update:

V 0.2.2 c057b45, timestamp: 2021-02-15 18:56:26 +0200

still got this problme

[danieldaeschle]

@zakuro9715 for me it worked

[axetroy]

V 0.2.4 ccf6285

still got this with another error

V panic: `go main__walk()`: Resource temporarily unavailable
0   test                                0x000000010166a41d panic_error_number + 77
1   test                                0x00000001016574cf main__walk + 895
2   test                                0x000000010165712c main__walk_thread_wrapper + 44
3   libsystem_pthread.dylib             0x00007fff203d58fc _pthread_start + 224
4   libsystem_pthread.dylib             0x00007fff203d1443 thread_start + 15

[JalonSolov]

Interesting error.

I just tried your code on Manjaro Linux, with latest V 0.2.4 cd7d482 and it ran.

There were a few odd messages, but those are mostly because there are still 'panic's in vlib where there should be optionals.

I don't think your latest error is a V problem.

[medvednikov]

Works ok now.

[axetroy]

Works ok now.

@medvednikov Hi. I have retry it and it seems doesn't works for now

Current V version: V 0.3.1 436b19c, timestamp: 2022-09-04 14:15:26 +0300

$ v run test.v
...
...
...
after: 227
after: 224
before: 244
after: 231
before: 228
after: 227
before: 232
before: 255
after: 254
after: 253
before: 254
before: 255
after: 254
after: 253
V panic: `go main__walk()`: Resource temporarily unavailable
v hash: 436b19c
0   test                                0x000000010ddf83ed panic_error_number + 77
1   test                                0x000000010de2ea68 main__walk + 1192
2   test                                0x000000010dde32b1 main__walk_thread_wrapper + 49
3   test                                0x000000010de3d8ac GC_start_routine + 100
4   libsystem_pthread.dylib             0x00007ff805d9e4e1 _pthread_start + 125
5   libsystem_pthread.dylib             0x00007ff805d99f6b thread_start + 15
before: 254
after: 253
before: 254
after: 253
V panic: `go main__walk()`: Resource temporarily unavailable
v hash: 436b19c
0   test                                0x000000010ddf83ed panic_error_number + 77
1   test                                0x000000010de2ea68 main__walk + 1192
2   test                                0x000000010dde32b1 main__walk_thread_wrapper + 49