mattermost-load-test-ng icon indicating copy to clipboard operation
mattermost-load-test-ng copied to clipboard

MM-56605: Create results file and print to file

Open DHaussermann opened this issue 1 year ago • 4 comments

Summary

This will create a the results.txt file and write the results blocks to it eliminating the need to manually copy from the console.

Ticket Link

https://mattermost.atlassian.net/browse/MM-56605

DHaussermann avatar Apr 16 '24 14:04 DHaussermann

@streamer45 The unit test on comparison_test.go now runs but there are 2 parts of the string compare that are failing and I'm not sure how to fix them.

  1. Line 43 fails in 2 places. It seems that and actual line return and /n cannot be used interchangeably but I have tried this both ways and cannot make the expected match
  2. The end of the string compare is getting :\n Type: \n DB Engine: instead of Errors: 2\ I'm trying to pass into the buffer and I don't know why.

Please advise if you have time to take a look.

DHaussermann avatar Jun 18 '24 21:06 DHaussermann

@DHaussermann Some problems I see:

  1. (bug) We are missing a fmt.Fprintf call in https://github.com/mattermost/mattermost-load-test-ng/blob/d65c7ec3aa1b852de24ea7c439602af60b2edc23/cmd/ltctl/comparison.go#L104 as it's still using fmt.Printf.
  2. You should provide a second result because we always expect that array to have 2 elements (remember the [2]).
  3. There's some indentation expected in the output, which we need to match.

streamer45 avatar Jun 19 '24 10:06 streamer45

@streamer45 I believe my last commit solves 1. and 3. from the feedback above.

For 2. I'm not entirely sure what type of element is expected here, Would there be a second element following this one? A test2 for example? https://github.com/mattermost/mattermost-load-test-ng/blob/a38abb562eed996f5c1a75672b07037a18552733/cmd/ltctl/comparison_test.go#L22-L34

Would solving this solve the extra content in the expected that I cannot account for? I still see the following i the expected n:\n Type: \n DB Engine: \n Errors: 0\n but I'm not clear on why.

When ever you have some time please review again.

DHaussermann avatar Jul 09 '24 16:07 DHaussermann

@DHaussermann I think we are very close. So we need to add the additional comparison.LoadTestResult to the array since we are always comparing 2 test runs, not just one. Something like this:

	// Prepare test data
	results := []comparison.Result{
		{
			Report:       "Sample Report",
			DashboardURL: "http://example.com/dashboard",
			LoadTests: [2]comparison.LoadTestResult{
				{
					Label: "Test1",
					Config: comparison.LoadTestConfig{
						Type:     "bounded",
						DBEngine: "postgres",
						NumUsers: 100,
						Duration: "10m",
					},
					Status: coordinator.Status{
						SupportedUsers: 80,
						NumErrors:      2,
					},
				},
				{
					Label: "Test2",
					Config: comparison.LoadTestConfig{
						Type:     "bounded",
						DBEngine: "postgres",
						NumUsers: 100,
						Duration: "10m",
					},
					Status: coordinator.Status{
						SupportedUsers: 80,
						NumErrors:      2,
					},
				},
			},
		},
	}

Also, this looks off as the test name should go before the colon:

https://github.com/mattermost/mattermost-load-test-ng/blob/a38abb562eed996f5c1a75672b07037a18552733/cmd/ltctl/comparison_test.go#L44

streamer45 avatar Jul 10 '24 07:07 streamer45

Feel free to ask for a review when it's ready.

agnivade avatar Sep 06 '24 09:09 agnivade

@agnivade and @agarciamontoro Thanks for your help with this! Please review again when you have a moment. I think I have addressed all the review feedback.

Note, I solved the linter error to the best my ability. it did not like the use of fmt.Sprintf on strings that require no formatting and fmt.Sprintf could no longer be used because it returns too many values as there is 1 extra for the error that I don't account for anymore. I don't know how better to solve this than to use neither.

DHaussermann avatar Sep 06 '24 13:09 DHaussermann

Hi @agnivade I have added error handling for writeResults. When you have a moment, please review and advised if this was done the way you would expect.

DHaussermann avatar Sep 09 '24 16:09 DHaussermann

Thanks for your patience with this @agarciamontoro. Yes there was silly error I think I have now corrected.

DHaussermann avatar Sep 10 '24 18:09 DHaussermann