liam icon indicating copy to clipboard operation
liam copied to clipboard

♻️(jobs): Refactor jobs package structure

Open MH4GF opened this issue 9 months ago • 5 comments

Issue

  • Reorganize jobs package structure for better maintainability

Why is this change needed?

The current jobs package structure was becoming difficult to maintain. This refactoring organizes code into logical task-specific directories, making the codebase more maintainable and easier to understand.

What would you like reviewers to focus on?

  • Code organization structure
  • Proper imports and exports
  • Any potential breaking changes that might have been introduced

Testing Verification

Tested locally by verifying that tasks can be properly imported and executed. Screenshot 2025-04-15 at 20 01 25

ref: https://cloud.trigger.dev/orgs/liam-hq-5035/projects/liam-HdAt/env/dev/runs

What was done

🤖 Generated by PR Agent at 410f9931f53b828c0a4b56b1055cabffb311052b

  • Refactored the jobs package structure for better maintainability:
    • Consolidated task exports and removed unused files.
    • Introduced new task-specific directories for better organization.
  • Enhanced functionality for generating documentation and schema metadata:
    • Added generateDocsSuggestionTask and generateSchemaMetaSuggestionTask.
    • Improved schema and documentation suggestion workflows.
  • Removed deprecated and unused tasks:
    • Deleted helloWorld task and related API route.
    • Removed sentry-error-test and other obsolete files.
  • Updated TypeScript configurations and import paths:
    • Changed moduleResolution to bundler in tsconfig.json.
    • Updated import paths to reflect the new structure.

Detailed Changes

Relevant files
Bug fix
2 files
route.ts
Removed unused helloWorld API route                                           
+0/-19   
helloworld.ts
Removed helloWorld task implementation                                     
+0/-9     
Enhancement
17 files
processGenerateDocsSuggestion.ts
Removed deprecated processGenerateDocsSuggestion function
+0/-129 
processGenerateSchemaMeta.ts
Removed deprecated processGenerateSchemaMeta function       
+0/-95   
index.ts
Consolidated task exports and removed unused exports         
+2/-20   
docsSuggestionSchema.ts
Removed obsolete docsSuggestionSchema definitions               
+0/-36   
index.ts
Removed unused exports for generateDocsSuggestion               
+0/-2     
index.ts
Removed unused exports for generateReview                               
+0/-6     
reviewSchema.ts
Removed deprecated reviewSchema definitions                           
+0/-34   
index.ts
Removed redundant exports for prompts                                       
+0/-1     
createKnowledgeSuggestion.ts
Added createKnowledgeSuggestionTask for knowledge suggestions
+22/-1   
generateDocsSuggestion.ts
Added generateDocsSuggestionTask with improved schema handling
+218/-7 
generateSchemaMeta.ts
Added generateSchemaMetaSuggestionTask for schema metadata
+156/-9 
index.ts
Added exports for knowledge-related tasks                               
+3/-0     
generateReview.ts
Updated generateReview task to use new reviewPrompt           
+2/-2     
reviewPrompt.ts
Added reviewPrompt with updated schema definitions             
+35/-3   
saveReview.ts
Updated saveReview task to use new knowledge tasks             
+3/-5     
getInstallationIdFromRepositoryId.ts
Added utility function to fetch installation ID                   
[link]   
langfuseLangchainHandler.ts
Added LangfuseLangchainHandler for callback handling         
[link]   
Configuration changes
1 files
tsconfig.json
Updated TypeScript module resolution to bundler                   
+1/-1     
Additional files
6 files
flow.md +0/-95   
helloworld.ts +0/-9     
jobs.ts +0/-120 
sentry-error-test.ts +0/-11   
index.ts +0/-71   
index.ts +1/-1     

Additional Notes

This refactoring moves functionality from the root level to organized task directories without changing the core functionality.


Need help?
  • Type /help how to ... in the comments thread for any questions about Qodo Merge usage.
  • Check out the documentation for more information.
  • MH4GF avatar Apr 15 '25 10:04 MH4GF

    ⚠️ No Changeset found

    Latest commit: 410f9931f53b828c0a4b56b1055cabffb311052b

    Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

    This PR includes no changesets

    When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

    Click here to learn what changesets are, and how to add one.

    Click here if you're a maintainer who wants to add a changeset to this PR

    changeset-bot[bot] avatar Apr 15 '25 10:04 changeset-bot[bot]

    The latest updates on your projects. Learn more about Vercel for Git ↗︎

    Name Status Preview Comments Updated (UTC)
    liam-app ✅ Ready (Inspect) Visit Preview 💬 Add feedback Apr 15, 2025 10:51am
    liam-erd-sample ✅ Ready (Inspect) Visit Preview 💬 Add feedback Apr 15, 2025 10:51am
    1 Skipped Deployment
    Name Status Preview Comments Updated (UTC)
    liam-docs ⬜️ Ignored (Inspect) Apr 15, 2025 10:51am

    vercel[bot] avatar Apr 15 '25 10:04 vercel[bot]

    frontend/packages/prompt-test result:

    visit: https://cloud.langfuse.com/project/cm8ii4o5o03fpad078o638g1d/datasets/cm99wciaz070ead07rgjr88ou/runs/cm9idv28n0177ad07zf7pppwp

    run items length
    3

    github-actions[bot] avatar Apr 15 '25 10:04 github-actions[bot]

    PR Reviewer Guide 🔍

    Here are some key observations to aid the review process:

    ⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
    🧪 No relevant tests
    🔒 No security concerns identified
    ⚡ Recommended focus areas for review

    Error Handling

    The error handling in the processGenerateDocsSuggestion function could be improved. While errors are caught and logged, the function doesn't provide detailed error information when throwing exceptions, which might make debugging difficult.

    export async function processGenerateDocsSuggestion(payload: {
      reviewComment: string
      projectId: number
      branchOrCommit?: string
    }): Promise<{
      suggestions: Record<DocFile, FileContent>
      traceId: string
    }> {
      try {
        const supabase = createClient()
    
        // Get repository information from supabase
        const { data: projectRepo, error } = await supabase
          .from('ProjectRepositoryMapping')
          .select(`
            *,
            repository:Repository(*)
          `)
          .eq('projectId', payload.projectId)
          .limit(1)
          .maybeSingle()
    
        if (error || !projectRepo?.repository) {
          throw new Error('Repository information not found')
        }
    
        const { repository } = projectRepo
        const repositoryFullName = `${repository.owner}/${repository.name}`
        const branch = payload.branchOrCommit || 'main'
    
        // Fetch all doc files from GitHub
        const docsPromises = DOC_FILES.map(async (filename) => {
          const filePath = `docs/${filename}`
          try {
            const fileData = await getFileContent(
              repositoryFullName,
              filePath,
              branch,
              Number(repository.installationId),
            )
    
            return {
              id: filename,
              title: filename,
              content: fileData.content
                ? JSON.stringify(
                    Buffer.from(fileData.content, 'base64').toString('utf-8'),
                  ).slice(1, -1)
                : '',
            }
          } catch (error) {
            console.warn(`Could not fetch file ${filePath}: ${error}`)
            return {
              id: filename,
              title: filename,
              content: '',
            }
          }
        })
    
        const docsArray = await Promise.all(docsPromises)
    
        // Format docs array as structured markdown instead of raw JSON
        let formattedDocsContent = 'No existing docs found'
    
        if (docsArray.length > 0) {
          formattedDocsContent = docsArray
            .map((doc) => {
              return `<text>\n\n## ${doc.title}\n\n${doc.content || '*(No content)*'}\n\n</text>\n\n---\n`
            })
            .join('\n')
        }
    
        const predefinedRunId = uuidv4()
        const callbacks = [langfuseLangchainHandler]
    
        // Fetch schema information with overrides
        const { overriddenSchema } = await fetchSchemaInfoWithOverrides(
          payload.projectId,
          branch,
          repositoryFullName,
          Number(repository.installationId),
        )
    
        const result = await generateDocsSuggestion(
          payload.reviewComment,
          formattedDocsContent,
          callbacks,
          predefinedRunId,
          overriddenSchema,
        )
    
        const suggestions = Object.fromEntries(
          Object.entries(result)
            .filter(([_, value]) => value !== undefined)
            .map(([key, value]) => {
              // Handle file extensions consistently
              const newKey = key.endsWith('.md') ? key : `${key}.md`
              return [newKey, value]
            }),
        ) as Record<DocFile, FileContent>
    
        // Return a properly structured object
        return {
          suggestions,
          traceId: predefinedRunId,
        }
      } catch (error) {
        console.error('Error generating docs suggestions:', error)
        throw error
      }
    }
    
    Type Inconsistency

    There's a potential type inconsistency between the SchemaMetaResult type defined in two places (lines 30-48 and 414-423) with different structures, which could lead to confusion or errors.

    export type GenerateSchemaMetaPayload = {
      overallReviewId: number
    }
    
    export type SchemaMetaResult =
      | {
          createNeeded: true
          override: SchemaOverride
          projectId: number
          pullRequestNumber: number
          branchName: string
          title: string
          traceId: string
          overallReviewId: number
          reasoning?: string
        }
      | {
          createNeeded: false
        }
    

    PR Code Suggestions ✨

    Explore these optional code suggestions:

    CategorySuggestion                                                                                                                                    Impact
    General
    Add retry configuration

    The task is missing retry configuration. For a critical task like creating
    knowledge suggestions, adding retry logic would improve reliability in case of
    transient failures.

    frontend/packages/jobs/src/tasks/knowledge/createKnowledgeSuggestion.ts [244-262]

     export const createKnowledgeSuggestionTask = task({
       id: 'create-knowledge-suggestion',
    +  retry: {
    +    maxAttempts: 3,
    +    minTimeoutInMs: 1000,
    +    maxTimeoutInMs: 10000,
    +    factor: 2,
    +  },
       run: async (payload: CreateKnowledgeSuggestionPayload) => {
         logger.log('Executing create knowledge suggestion task:', { payload })
         try {
           const result = await processCreateKnowledgeSuggestion(payload)
           logger.info(
             result.suggestionId === null
               ? 'Knowledge suggestion creation skipped due to matching content'
               : 'Successfully created knowledge suggestion:',
             { suggestionId: result.suggestionId },
           )
           return result
         } catch (error) {
           logger.error('Error in createKnowledgeSuggestion task:', { error })
           throw error
         }
       },
     })
    
    • [ ] Apply this suggestion <!-- /improve --apply_suggestion=0 -->
    Suggestion importance[1-10]: 7

    __

    Why: Adding retry logic to the task is a valuable improvement for reliability, especially for critical operations like creating knowledge suggestions. The suggestion properly configures retry parameters with exponential backoff, which would help handle transient failures gracefully.

    Medium
    Learned
    best practice
    Add input validation to prevent security vulnerabilities and unexpected behavior when processing user-provided data

    The function processGenerateDocsSuggestion accepts user input without
    validation. You should validate the input parameters before processing them to
    prevent potential security issues or unexpected behavior. Add validation for the
    reviewComment, projectId, and branchOrCommit parameters at the beginning of the
    function.

    frontend/packages/jobs/src/tasks/knowledge/generateDocsSuggestion.ts [103-110]

     export async function processGenerateDocsSuggestion(payload: {
       reviewComment: string
       projectId: number
       branchOrCommit?: string
     }): Promise<{
       suggestions: Record<DocFile, FileContent>
       traceId: string
     }> {
       try {
    +    // Validate input parameters
    +    if (!payload.reviewComment) {
    +      throw new Error('Review comment is required')
    +    }
    +    if (!payload.projectId || typeof payload.projectId !== 'number') {
    +      throw new Error('Valid project ID is required')
    +    }
    +    if (payload.branchOrCommit && typeof payload.branchOrCommit !== 'string') {
    +      throw new Error('Branch or commit must be a string')
    +    }
    +    
         const supabase = createClient()
    

    [To ensure code accuracy, apply this suggestion manually]

    Suggestion importance[1-10]: 6
    Low
    • [ ] More <!-- /improve --more_suggestions=true -->

    Conflicts are present, so we will revert back to draft and correct them.

    MH4GF avatar Apr 21 '25 03:04 MH4GF