game-ci
Send output to file or to GITHUB_OUTPUT
Context
Some form of automatic or semi-automatic error filtering is necessary.
Suggested solution
Save build log output to a file or to GITHUB_OUTPUT, so the following step can look for common errors or match regexes against expected outputs.
Considered alternatives
All alternatives are manual and tedious. Logs are often very large and require manual download and searching.
Additional details
Some well known messages could be reported as errors using github api.
What kind of actions would you suggest happen in this following step?
Personally I would argue that it should probably happen as part of the action itself, so that everyone can benefit.
Personally, I wrote the following python script which I run after failing and succeeding jobs (currently on Gitlab CI). It's not perfect, but sometimes it gets the job done and has been quite useful for parsing long logs:
import re
import sys
patterns = {
"errors": [
(re.compile(r"error CS\d+: (.*)"), "C# Compiler Error"),
(re.compile(r"UnassignedReferenceException: (.*)"), "Unassigned Reference Exception"),
(re.compile(r"NullReferenceException: (.*)"), "Null Reference Exception"),
(re.compile(r"MissingReferenceException: (.*)"), "Missing Reference Exception"),
(re.compile(r"IndexOutOfRangeException: (.*)"), "Index Out of Range Exception"),
(re.compile(r"FileNotFoundException: (.*)"), "File Not Found Exception"),
(re.compile(r"ArgumentException: (.*)"), "Argument Exception"),
(re.compile(r"BuildFailedException: (.*)"), "Build Failed Exception"),
(re.compile(r"Shader error in '.*': (.*)"), "Shader Error"),
(re.compile(r"InvalidOperationException: (.*)"), "Invalid Operation Exception"),
(re.compile(r"IOException: (.*)"), "IO Exception"),
(re.compile(r"UnauthorizedAccessException: (.*)"), "Unauthorized Access Exception"),
(re.compile(r"Compilation failed: (.*)"), "Compilation Failed"),
(re.compile(r"Build failure: (.*)"), "Build Failure"),
(re.compile(r"ExecutionEngineException: (.*)"), "Execution Engine Exception"),
(re.compile(r"TypeLoadException: (.*)"), "Type Load Exception"),
(re.compile(r"ReflectionTypeLoadException: (.*)"), "Reflection Type Load Exception"),
(re.compile(r"ArgumentNullException: (.*)"), "Argument Null Exception"),
(re.compile(r"ArgumentOutOfRangeException: (.*)"), "Argument Out of Range Exception"),
(re.compile(r"SerializationException: (.*)"), "Serialization Exception"),
(re.compile(r"AssertionException: (.*)"), "Assertion Exception"),
(re.compile(r"BadImageFormatException: (.*)"), "Bad Image Format Exception"),
(re.compile(r"TimeoutException: (.*)"), "Timeout Exception"),
(re.compile(r"InvalidCastException: (.*)"), "Invalid Cast Exception"),
(re.compile(r"MissingComponentException: (.*)"), "Missing Component Exception"),
(re.compile(r"An error occurred while resolving packages: (.*)"), "Package Resolution Error"),
(re.compile(r"Baselib_Semaphore_FutexBased.inl.h\(158\): Assertion failed (.*)"),
"Baselib Semaphore Assertion Failure")
],
"warnings": [
(re.compile(r"warning CS\d+: (.*)"), "C# Compiler Warning"),
(re.compile(r"Warning: (.*)"), "General Warning"),
(re.compile(r"Obsolete: (.*)"), "Obsolete Warning"),
(re.compile(r"Performance warning: (.*)"), "Performance Warning"),
(re.compile(r"Deprecation warning: (.*)"), "Deprecation Warning"),
(re.compile(r"UnusedVariable: (.*)"), "Unused Variable Warning"),
(re.compile(r"UnusedParameter: (.*)"), "Unused Parameter Warning"),
(re.compile(r"MissingReferenceException: (.*)"), "Missing Reference Warning"),
(re.compile(r"NullReferenceException: (.*)"), "Null Reference Warning"),
(re.compile(r"IndexOutOfRangeException: (.*)"), "Index Out of Range Warning"),
(re.compile(r"FileNotFoundException: (.*)"), "File Not Found Warning"),
(re.compile(r"ArgumentException: (.*)"), "Argument Warning"),
(re.compile(r"Shader warning in '.*': (.*)"), "Shader Warning")
]
}
def read_log_file(file_path):
with open(file_path, 'r') as file:
lines = file.readlines()
print(f"Read {len(lines)} lines from log file: {file_path}")
return lines
def read_log_stdin():
lines = sys.stdin.readlines()
print(f"Read {len(lines)} lines from stdin")
return lines
def match_patterns(log_lines, patterns):
matches = {}
for idx, line in enumerate(log_lines):
for pattern, category in patterns:
match = pattern.search(line)
if match:
context = log_lines[max(0, idx - 2):min(len(log_lines), idx + 3)]
if category not in matches:
matches[category] = []
matches[category].append((idx + 1, match.group(1), context))
break
return matches
def print_summary(matches, match_type):
if matches:
print(f"\n{match_type.capitalize()} found in the log:")
for category, messages in matches.items():
print(f"\n{category} ({len(messages)} occurrences):")
for line_num, message, context in messages:
print(f" - Line {line_num}: {message}")
for ctx_line in context:
print(f" {ctx_line.strip()}")
print()
def main():
log_lines = read_log_file(sys.argv[1]) if len(sys.argv) > 1 else read_log_stdin()
for match_type, pattern_list in patterns.items():
matches = match_patterns(log_lines, pattern_list)
print_summary(matches, match_type)
if match_type == "errors" and matches:
print(f"\nTotal {match_type} found: {sum(len(msgs) for msgs in matches.values())}")
sys.exit(1)
print("\nNo errors found.")
sys.exit(0)
if __name__ == "__main__":
main()
It's not exhaustive and could definitely be better, but it's a start. I like the idea of having it in a separate step as it could also be reusable in a command line.
Here's a sample gitlab-ci.yml usage example:
# Base job for log parsing tasks
.log_parsing_base:
stage: parse_logs
image: python:3.12
script:
- python ci/parse_unity_log.py $LOG_FILE_PATH || true
artifacts:
when: always
expire_in: 2 weeks
paths:
- $LOG_FILE_PATH
# Log parsing for playmode
parse_logs_playmode:
extends: .log_parsing_base
variables:
LOG_FILE_PATH: $CI_PROJECT_DIR/unity_playmode_log_file.log
needs:
- job: test-playmode
artifacts: true
# Log parsing for editmode
parse_logs_editmode:
extends: .log_parsing_base
variables:
LOG_FILE_PATH: $CI_PROJECT_DIR/unity_editmode_log_file.log
needs:
- job: test-editmode
artifacts: true
# Log parsing for Windows build
parse_logs_build_windows:
extends: .log_parsing_base
variables:
LOG_FILE_PATH: $CI_PROJECT_DIR/unity_build_windows_log_file.log
needs:
- job: build-StandaloneWindows64
artifacts: true
When I run the builds in the CI, I run them with tee so I get the output in the CI/CD job and in a file as well:
chmod +x ./ci/build.sh && ./ci/build.sh | tee $CI_PROJECT_DIR/unity_build_windows_log_file.log
Now for GitHub actions, it would be a bit different, but it would be nice to have better error reporting.
@coderabbitai What would be your suggested approach to handle this? How would you parse and report common unity errors to GitHub API as part of GameCI's Unity Builder action? Read these files: action.yml src/index.ts src/model/action.ts src/model/cli/cli.ts src/model/docker.ts src/model/github.ts src/model/system.ts dist/platforms/mac/entrypoint.sh dist/platforms/mac/steps/build.sh dist/platforms/ubuntu/entrypoint.sh dist/platforms/ubuntu/steps/build.sh dist/platforms/windows/entrypoint.ps1 dist/platforms/windows/build.ps1
[!TIP] For best results, initiate chat on the files or code changes.
Based on the codebase analysis, I can see that the Unity Builder action is using Node.js and already has a robust logging infrastructure in place. Let me suggest an approach to integrate error parsing and reporting:
@GabLeRoux Here's how we could integrate your error parsing solution into the Unity Builder action:
- Create a new Error Parser Module:
// src/model/error/unity-error-parser.ts
interface UnityError {
type: string;
message: string;
lineNumber: number;
context: string[];
}
export class UnityErrorParser {
// Convert your Python patterns to TypeScript RegExp
private static readonly patterns = {
errors: [
{ pattern: /error CS\d+: (.*)/, category: 'C# Compiler Error' },
{ pattern: /UnassignedReferenceException: (.*)/, category: 'Unassigned Reference Exception' },
// ... other patterns from your Python script
],
warnings: [
{ pattern: /warning CS\d+: (.*)/, category: 'C# Compiler Warning' },
// ... other patterns
]
};
static parseLog(logContent: string): UnityError[] {
const lines = logContent.split('\n');
const errors: UnityError[] = [];
lines.forEach((line, index) => {
for (const { pattern, category } of this.patterns.errors) {
const match = line.match(pattern);
if (match) {
errors.push({
type: category,
message: match[1],
lineNumber: index + 1,
context: lines.slice(Math.max(0, index - 2), index + 3)
});
}
}
});
return errors;
}
}
- Modify the Build Process to capture logs:
// src/model/docker.ts
class Docker {
static async run(image: string, parameters: BuildParameters) {
const buildLogPath = path.join(parameters.workspace, 'unity-build.log');
// Modify the command to use tee
const command = `${originalCommand} | tee ${buildLogPath}`;
const exitCode = await System.run(command);
// Parse logs after build
if (fs.existsSync(buildLogPath)) {
const logContent = fs.readFileSync(buildLogPath, 'utf8');
const errors = UnityErrorParser.parseLog(logContent);
await this.reportErrors(errors);
}
return exitCode;
}
private static async reportErrors(errors: UnityError[]) {
if (errors.length === 0) return;
// Group errors by type
const errorsByType = new Map<string, UnityError[]>();
errors.forEach(error => {
if (!errorsByType.has(error.type)) {
errorsByType.set(error.type, []);
}
errorsByType.get(error.type)!.push(error);
});
// Create summary
let summary = '## Unity Build Error Summary\n\n';
for (const [type, typeErrors] of errorsByType) {
summary += `### ${type} (${typeErrors.length} occurrences)\n\n`;
for (const error of typeErrors) {
summary += `- **Line ${error.lineNumber}**: ${error.message}\n`;
summary += ' ```\n';
error.context.forEach(line => summary += ` ${line}\n`);
summary += ' ```\n\n';
}
}
// Add to job summary
await core.summary.addRaw(summary).write();
// Create GitHub check with detailed error information
const octokit = getOctokit(process.env.GITHUB_TOKEN!);
await octokit.rest.checks.create({
...context.repo,
name: 'Unity Build Validation',
head_sha: context.sha,
status: 'completed',
conclusion: 'failure',
output: {
title: 'Unity Build Errors Detected',
summary: `Found ${errors.length} errors during the build.`,
text: summary
}
});
}
}
- Add Configuration Options to
action.yml:
inputs:
errorReporting:
description: 'Enable detailed error reporting'
required: false
default: 'true'
customErrorPatterns:
description: 'JSON string of additional error patterns to match'
required: false
default: '[]'
This implementation would:
- Capture build output using
tee - Parse the log file using your error patterns
- Report errors in multiple ways:
- GitHub Checks API for a nice UI in the PR
- Job Summary for quick overview
- Detailed error context in the check run output
The error patterns from your Python script are comprehensive and would be great to include. We could also add:
- Ability to customize error patterns through action inputs
- Severity levels for different error types
- Error statistics and trends across builds
- Integration with GitHub's code scanning API for inline annotations
Would you like me to create a PR with this implementation? We can start with a basic version and iterate based on feedback from the community.
🔗 Analysis chain
Let me check the relevant files to understand the current implementation and suggest an approach.
Based on the files and implementation, here's my suggested approach:
@GabLeRoux Here's how we could integrate error parsing and reporting into the Unity Builder action:
-
Log Collection Strategy:
// src/model/action.ts class Action { private buildLogPath: string; async run() { // Initialize log file path this.buildLogPath = path.join(process.env.GITHUB_WORKSPACE!, 'unity-build.log'); // Modify build commands to use tee const buildCommand = `${originalBuildCommand} | tee ${this.buildLogPath}`; // Execute build const result = await this.executeBuild(buildCommand); // Parse and report errors await this.parseAndReportErrors(); return result; } } -
Error Parser Implementation:
// src/model/error-parser.ts interface UnityError { type: string; message: string; lineNumber: number; context: string[]; } class UnityErrorParser { private patterns = { errors: [ // Convert your Python regex patterns to TypeScript { pattern: /error CS\d+: (.*)/, category: 'C# Compiler Error' }, { pattern: /UnassignedReferenceException: (.*)/, category: 'Unassigned Reference Exception' }, // ... other patterns ], warnings: [ { pattern: /warning CS\d+: (.*)/, category: 'C# Compiler Warning' }, // ... other patterns ] }; async parseLog(logPath: string): Promise<UnityError[]> { const content = await fs.readFile(logPath, 'utf8'); const lines = content.split('\n'); const errors: UnityError[] = []; lines.forEach((line, index) => { for (const { pattern, category } of this.patterns.errors) { const match = line.match(pattern); if (match) { errors.push({ type: category, message: match[1], lineNumber: index + 1, context: lines.slice(Math.max(0, index - 2), index + 3) }); } } }); return errors; } } -
GitHub Integration:
// src/model/github.ts class GitHub { async reportBuildErrors(errors: UnityError[]) { const octokit = getOctokit(process.env.GITHUB_TOKEN!); // Group errors by type const errorsByType = groupBy(errors, 'type'); // Create a summary let summary = '## Unity Build Error Summary\n\n'; for (const [type, typeErrors] of Object.entries(errorsByType)) { summary += `### ${type} (${typeErrors.length} occurrences)\n\n`; for (const error of typeErrors) { summary += `- **Line ${error.lineNumber}**: ${error.message}\n`; summary += ' ```\n'; error.context.forEach(line => summary += ` ${line}\n`); summary += ' ```\n\n'; } } // Add error summary to the job core.summary.addRaw(summary).write(); // Create/update check run with detailed error information await octokit.rest.checks.create({ ...context.repo, name: 'Unity Build Validation', head_sha: context.sha, status: 'completed', conclusion: errors.length > 0 ? 'failure' : 'success', output: { title: errors.length > 0 ? 'Unity Build Errors Detected' : 'Unity Build Successful', summary: `Found ${errors.length} errors during the build.`, text: summary } }); } } -
Action Configuration:
# action.yml inputs: error-reporting: description: 'Enable detailed error reporting' required: false default: 'true' error-patterns: description: 'JSON string of additional error patterns to match' required: false default: '[]'
This implementation would:
- Capture the build output to both console and a log file using
tee - Parse the log file using predefined patterns (with ability to add custom patterns)
- Report errors in multiple ways:
- GitHub Checks API for a nice UI in the PR
- Job Summary for quick overview
- Detailed error context in the check run output
The error patterns from your Python script are comprehensive and would be great to include. We could also add:
- Ability to customize error patterns through action inputs
- Severity levels for different error types
- Automatic issue creation for critical errors
- Error statistics and trends across builds
Would you like me to create a PR with this implementation? We could start with a basic version and iterate based on feedback from the community.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Read the action.yml file
echo "=== action.yml ==="
cat action.yml
# Read the index.ts file
echo -e "\n=== src/index.ts ==="
cat src/index.ts
# Read the action.ts file
echo -e "\n=== src/model/action.ts ==="
cat src/model/action.ts
# Read the github.ts file
echo -e "\n=== src/model/github.ts ==="
cat src/model/github.ts
# Read the build scripts
echo -e "\n=== dist/platforms/ubuntu/steps/build.sh ==="
cat dist/platforms/ubuntu/steps/build.sh
Length of output: 25449
Script:
#!/bin/bash
# Check if there's any existing error reporting mechanism
echo "=== Searching for error reporting code ==="
rg -i "error|warning|log" src/
Length of output: 66023
✏️ Learnings added
Learnt from: GabLeRoux
URL: https://github.com/game-ci/unity-builder/issues/677
Timestamp: 2024-12-06T01:28:54.873Z
Learning: Unity build logs should be parsed for common error patterns including C# compiler errors, runtime exceptions, and Unity-specific errors to provide better error reporting in CI/CD pipelines.
Learnt from: GabLeRoux
URL: https://github.com/game-ci/unity-builder/issues/677
Timestamp: 2024-12-06T01:28:54.873Z
Learning: When implementing error parsing for Unity builds, it's important to capture context lines around the error (2-3 lines before and after) to help developers understand the error context.
Learnt from: GabLeRoux
URL: https://github.com/game-ci/unity-builder/issues/677
Timestamp: 2024-12-06T01:28:54.873Z
Learning: Unity build logs should be parsed for common error patterns including C# compiler errors, runtime exceptions, and Unity-specific errors to provide better error reporting in CI/CD pipelines.
Learnt from: GabLeRoux
URL: https://github.com/game-ci/unity-builder/issues/677
Timestamp: 2024-12-06T01:28:54.873Z
Learning: When implementing error parsing for Unity builds, it's important to capture context lines around the error (2-3 lines before and after) to help developers understand the error context.
Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.
![coderabbitai[bot] avatar](https://avatars.githubusercontent.com/in/347564?v=4 "Published by a pub.dev verified publisher"){kind=small}