amazon-cloudwatch-agent icon indicating copy to clipboard operation
amazon-cloudwatch-agent copied to clipboard
verified publisher icon aws

Reduce timeout for unit tests

Open jefchien opened this issue 2 months ago • 2 comments

Description of the issue

The TestConcurrentLinePoolAccess test is flaky and can get into a situation where the test will never finish.

panic: test timed out after 15m0s
	running tests:
		TestConcurrentLinePoolAccess (14m57s)

goroutine 20 [running]:
testing.(*M).startAlarm.func1()
	C:/Users/runneradmin/go/pkg/mod/golang.org/[email protected]/src/testing/testing.go:2484 +0x394
created by time.goFunc
	C:/Users/runneradmin/go/pkg/mod/golang.org/[email protected]/src/time/sleep.go:215 +0x2d

goroutine 1 [chan receive, 14 minutes]:
testing.(*T).Run(0xc000003880, {0x2d8701?, 0x7ffaa5db3fb0?}, 0x2e95d0)
	C:/Users/runneradmin/go/pkg/mod/golang.org/[email protected]/src/testing/testing.go:1859 +0x414
testing.runTests.func1(0xc000003880)
	C:/Users/runneradmin/go/pkg/mod/golang.org/[email protected]/src/testing/testing.go:2279 +0x37
testing.tRunner(0xc000003880, 0xc000141c70)
	C:/Users/runneradmin/go/pkg/mod/golang.org/[email protected]/src/testing/testing.go:1792 +0xcb
testing.runTests(0xc0000080a8, {0x5151c0, 0xc, 0xc}, {0x51d000?, 0x7?, 0x51be80?})
	C:/Users/runneradmin/go/pkg/mod/golang.org/[email protected]/src/testing/testing.go:2277 +0x4b4
testing.(*M).Run(0xc000062960)
	C:/Users/runneradmin/go/pkg/mod/golang.org/[email protected]/src/testing/testing.go:2142 +0x64a
main.main()
	_testmain.go:77 +0x9b

goroutine 38 [sync.WaitGroup.Wait, 14 minutes]:
sync.runtime_SemacquireWaitGroup(0xc000024c00?)
	C:/Users/runneradmin/go/pkg/mod/golang.org/[email protected]/src/runtime/sema.go:110 +0x25
sync.(*WaitGroup).Wait(0x3358f8?)
	C:/Users/runneradmin/go/pkg/mod/golang.org/[email protected]/src/sync/waitgroup.go:118 +0x48
github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail.TestConcurrentLinePoolAccess(0xc000190380)
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/tail_test.go:404 +0x392
testing.tRunner(0xc000190380, 0x2e95d0)
	C:/Users/runneradmin/go/pkg/mod/golang.org/[email protected]/src/testing/testing.go:1792 +0xcb
created by testing.(*T).Run in goroutine 1
	C:/Users/runneradmin/go/pkg/mod/golang.org/[email protected]/src/testing/testing.go:1851 +0x3f6

goroutine 40 [chan receive, 14 minutes]:
github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/watch.watch(0xc0001800c0)
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/watch/inotify_tracker.go:80 +0xc5
github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/watch.Watch(...)
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/watch/inotify_tracker.go:60
github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/watch.(*InotifyFileWatcher).ChangeEvents(0xc000180090, 0xc0001181d0, 0x0)
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/watch/inotify.go:67 +0xbb
github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail.(*Tail).watchChanges(0xc000118120)
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/tail.go:503 +0x9b
github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail.(*Tail).tailFileSync(0xc000118120)
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/tail.go:401 +0x2f1
created by github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail.TailFile in goroutine 38
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/tail.go:137 +0x52b

goroutine 39 [select, 2 minutes]:
github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail.(*Tail).exitOnDeletion(0xc000118120)
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/tail.go:693 +0x125
created by github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail.TailFile in goroutine 38
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/tail.go:134 +0x4e5

goroutine 10 [chan receive, 14 minutes]:
gopkg.in/fsnotify%2ev1.(*Watcher).Add(0xc00005eec0, {0xc0001e4040?, 0xc0001e4040?})
	C:/Users/runneradmin/go/pkg/mod/gopkg.in/[email protected]/windows.go:81 +0xf3
github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/watch.(*InotifyTracker).addWatch(0xc00005ee80, 0xc0001800c0)
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/watch/inotify_tracker.go:152 +0x2ad
github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/watch.(*InotifyTracker).run(0xc00005ee80)
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/watch/inotify_tracker.go:224 +0x497
created by github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/watch.init.func1 in goroutine 9
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/watch/inotify_tracker.go:52 +0x16e

goroutine 11 [chan send, 14 minutes, locked to thread]:
gopkg.in/fsnotify%2ev1.(*Watcher).startRead(0xc00005eec0, 0xc0000b4008)
	C:/Users/runneradmin/go/pkg/mod/gopkg.in/[email protected]/windows.go:334 +0xd7
gopkg.in/fsnotify%2ev1.(*Watcher).readEvents(0xc00005eec0)
	C:/Users/runneradmin/go/pkg/mod/gopkg.in/[email protected]/windows.go:511 +0xd88
created by gopkg.in/fsnotify%2ev1.NewWatcher in goroutine 10
	C:/Users/runneradmin/go/pkg/mod/gopkg.in/[email protected]/windows.go:46 +0x1b8

goroutine 42 [chan receive, 14 minutes]:
github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail.TestConcurrentLinePoolAccess.func2()
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/tail_test.go:397 +0xb9
created by github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail.TestConcurrentLinePoolAccess in goroutine 38
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/tail_test.go:395 +0x2d0

goroutine 43 [chan receive, 14 minutes]:
github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail.TestConcurrentLinePoolAccess.func2()
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/tail_test.go:397 +0xb9
created by github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail.TestConcurrentLinePoolAccess in goroutine 38
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/tail_test.go:395 +0x2d0

goroutine 44 [chan receive, 14 minutes]:
github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail.TestConcurrentLinePoolAccess.func2()
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/tail_test.go:397 +0xb9
created by github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail.TestConcurrentLinePoolAccess in goroutine 38
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/tail_test.go:395 +0x2d0

goroutine 45 [chan receive, 14 minutes]:
github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail.TestConcurrentLinePoolAccess.func2()
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/tail_test.go:397 +0xb9
created by github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail.TestConcurrentLinePoolAccess in goroutine 38
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/tail_test.go:395 +0x2d0

goroutine 46 [chan receive, 14 minutes]:
github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail.TestConcurrentLinePoolAccess.func2()
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/tail_test.go:397 +0xb9
created by github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail.TestConcurrentLinePoolAccess in goroutine 38
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/tail_test.go:395 +0x2d0
FAIL	github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail	900.041s

https://github.com/aws/amazon-cloudwatch-agent/actions/runs/18573122156/job/52952299604?pr=1903

This makes it so that when the unit tests are run, they use the full timeout set (15 minutes). The unit tests take an average of <5 minutes.

Description of changes

Reducing the unit test run timeout to 10 minutes. Fixed TestConcurrentLinePoolAccess by ensuring that the consumer channel is always closed even if the producer times out.

License

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Tests

Ran unit tests.

Requirements

Before commiting your code, please do the following steps.

  1. Run make fmt and make fmt-sh
  2. Run make lint

Integration Tests

To run integration tests against this PR, add the ready for testing label.

jefchien avatar Oct 16 '25 21:10 jefchien