Step 1: Infrastructure for Deferred Type Variables

[niupilot[bot]]

Goal

Enable variables to remain untyped until first use

Scope

• Add deferred type tracking in cl/compile.go (symbol table extension)
• Implement placeholder type for untyped variables (types.Typ[types.UntypedNil] variant)
• Create deferred resolution queue mechanism
• No syntax changes - focus on internal infrastructure

Deliverables

• New deferredVar structure to track unresolved variables
• Type resolution deferral mechanism in blockCtx
• Test cases in cl/_testgop/ for simple deferred inference scenarios

Example Test Case

x := 100    // deferred
x + 200     // resolves to int

Related

• Parent issue: #2441 (Untyped Literal Extension)
• Part of Phase 1: Foundation - Deferred Type Inference

Notes

This is the foundational step that enables all subsequent steps in the untyped literal extension feature. It focuses on the internal infrastructure without any syntax changes.

[xushiwei]

@niupilot Please describe your possible architectural ideas

[niupilot[bot]]

Architecture Proposal for Deferred Type Variables

Based on my analysis of the codebase, here's my proposed architecture for implementing deferred type inference infrastructure:

---

1. Current Architecture Analysis

Key Components:

• blockCtx (cl/compile.go:364): Context for compiling blocks with imports, lookups, and scope management
• pkgCtx (cl/compile.go:336): Package-level context with symbol table (syms map[string]loader)
• Variable declaration via DefineVarStart() in stmt.go:325 for := assignments
• Type resolution happens immediately through gogen package

Current Flow for x := 100:

1. Parser creates ast.AssignStmt with token.DEFINE
2. compileAssignStmt() calls ctx.cb.DefineVarStart()
3. Expression compiled → type inferred immediately
4. EndInit() finalizes with concrete type

---

2. Proposed Architecture

2.1 Data Structures

deferredVar Structure

// deferredVar tracks variables with deferred type resolution
type deferredVar struct {
    name     string           // variable name
    pos      token.Pos        // declaration position
    scope    *types.Scope     // enclosing scope
    initExpr ast.Expr         // initial value expression (for re-evaluation)
    
    // Placeholder until resolved
    placeholder *types.Var    // variable with untyped placeholder type
    
    // Resolution tracking
    resolved    bool
    finalType   types.Type    // resolved type (nil until resolution)
}

Extension to blockCtx

type blockCtx struct {
    // ... existing fields ...
    
    // Deferred type resolution
    deferredVars   map[string]*deferredVar  // name -> deferred variable
    resolutionQueue []*deferredVar           // ordered resolution queue
}

2.2 Placeholder Type Strategy

Option A: Custom Untyped Type (Recommended)

// Create new untyped kind for deferred variables
const UntypedDeferred = types.UntypedNil + 1

// Use during initial declaration
placeholderType := types.Typ[UntypedDeferred]

Option B: Extend Existing Untyped

// Reuse types.Typ[types.UntypedNil] with metadata
// Track actual deferred state in deferredVar structure

I recommend Option A for clarity, but Option B is safer for compatibility with existing gogen code.

2.3 Resolution Queue Mechanism

Queue Operations

// Add to deferred resolution queue
func (ctx *blockCtx) deferVarResolution(v *deferredVar) {
    if ctx.deferredVars == nil {
        ctx.deferredVars = make(map[string]*deferredVar)
    }
    ctx.deferredVars[v.name] = v
    ctx.resolutionQueue = append(ctx.resolutionQueue, v)
}

// Resolve deferred variable on first use
func (ctx *blockCtx) resolveDeferredVar(name string, usageType types.Type) error {
    v, ok := ctx.deferredVars[name]
    if !ok || v.resolved {
        return nil
    }
    
    // Infer type from usage context
    v.finalType = usageType
    v.resolved = true
    
    // Re-compile with concrete type
    return ctx.recompileVarWithType(v)
}

// Flush queue at scope exit
func (ctx *blockCtx) flushDeferredVars() error {
    for _, v := range ctx.resolutionQueue {
        if !v.resolved {
            return fmt.Errorf("variable %s type cannot be inferred", v.name)
        }
    }
    ctx.deferredVars = nil
    ctx.resolutionQueue = nil
    return nil
}

2.4 Integration Points

Short Variable Declaration (:=)

// In compileAssignStmt() for token.DEFINE
if canDeferTypeInference(expr) {
    // Create deferred variable instead of immediate resolution
    for i, lhs := range expr.Lhs {
        name := lhs.(*ast.Ident).Name
        dv := &deferredVar{
            name:        name,
            pos:         lhs.Pos(),
            scope:       ctx.cb.Scope(),
            initExpr:    expr.Rhs[i],
            placeholder: types.NewVar(lhs.Pos(), ctx.pkg.Types, name, types.Typ[types.UntypedNil]),
        }
        ctx.deferVarResolution(dv)
    }
    return
}

Variable Usage Resolution

// In compileExpr() when encountering identifier
func compileIdent(ctx *blockCtx, v *ast.Ident) {
    if dv, ok := ctx.deferredVars[v.Name]; ok && !dv.resolved {
        // Attempt resolution from context
        expectedType := inferTypeFromContext(ctx)
        ctx.resolveDeferredVar(v.Name, expectedType)
    }
    // ... normal compilation
}

---

3. Implementation Strategy

Phase 1: Core Infrastructure (This Issue)

Step 1: Add deferredVar structure and blockCtx extensions

• Location: cl/compile.go
• Add deferred tracking fields to blockCtx

Step 2: Implement placeholder type mechanism

• Use types.Typ[types.UntypedNil] initially
• Track deferred state in metadata

Step 3: Add resolution queue operations

• Defer, resolve, and flush operations
• Hook into scope management

Step 4: Modify short variable declaration

• Detect when type can be deferred
• Create deferred variable entries

Step 5: Add resolution on usage

• Hook identifier resolution
• Infer type from usage context

Step 6: Test infrastructure

• Simple cases: x := 100; x + 200
• Error cases: unresolvable types
• Scope boundary cases

Phase 2: Context-Based Resolution (Future)

• Binary operations (x + 200 → infer int)
• Function call arguments
• Return statements
• Composite literal elements

---

4. Key Design Decisions

Why Deferred Resolution?

• Flexibility: Enables untyped literals like x := [] or x := {}
• Multi-backend: Supports Python/JS type conversion
• Type Safety: Still enforces types, just defers inference

Why Queue-Based?

• Ordered Resolution: Process variables in declaration order
• Dependency Tracking: Handle inter-variable dependencies
• Error Recovery: Clear error messages for unresolvable types

Why Metadata in blockCtx?

• Scope Isolation: Each block has independent deferred variables
• Lifecycle Management: Auto-flush at scope exit
• Integration: Minimal changes to existing compiler flow

---

5. Pseudocode Example

Input XGo Code:

x := 100      // deferred - no type yet
y := x + 200  // resolves x to int, y to int

Compilation Flow:

// Line 1: x := 100
dv := &deferredVar{
    name: "x",
    initExpr: &ast.BasicLit{Value: "100"},
    placeholder: types.NewVar(..., types.Typ[types.UntypedNil])
}
ctx.deferVarResolution(dv)

// Line 2: y := x + 200
// When compiling "x + 200":
//   - Encounter identifier "x"
//   - Check ctx.deferredVars["x"] → found, not resolved
//   - Infer from binary op: x must be numeric
//   - Evaluate initExpr "100" with expected type int
//   - Resolve x to int
//   - Continue: y becomes int from x + 200

---

6. Testing Strategy

Test Cases in cl/_testgop/:

1. Simple deferred inference:

   x := 100
   x + 200  // x resolves to int
   

2. Multiple variables:

   a := 1
   b := 2
   a + b  // both resolve to int
   

3. Error case - unresolvable:

   x := 100
   // no usage - should error or default to int
   

4. Scope boundaries:

   if true {
       x := 100
       x + 1
   }
   // x not accessible here
   

---

7. Risk Mitigation

Potential Issues:

• gogen compatibility: Test with existing type inference
• Performance: Deferred resolution adds overhead → optimize queue
• Error messages: Ensure clear messages for type errors
• Backward compatibility: Don't break existing code

Mitigation:

• Feature flag to enable/disable deferred inference
• Comprehensive test coverage
• Performance benchmarks
• Gradual rollout

---

This architecture provides a solid foundation for deferred type inference while maintaining compatibility with the existing XGo compiler infrastructure. The design is extensible for future phases (untyped collections, multi-backend types) and integrates cleanly with gogen's type system.

Next Steps: Await feedback before proceeding with implementation.

---

Job Run | Branch: codeagent/claude/issue-2464-1760171206

[xushiwei]

@niupilot This is a huge change. What's the smallest way to do it?

[niupilot[bot]]

Minimal Approach for Deferred Type Variables

You're absolutely right - let me propose a much simpler, incremental approach.

---

Minimal Implementation Strategy

Core Insight

For the test case x := 100; x + 200, we don't actually need to defer anything at compile time. Go already handles untyped constants naturally. The real question is: what problem are we solving?

Three Minimal Approaches (Pick One)

Option 1: Do Nothing (Leverage Existing Go Behavior)

x := 100    // Already untyped int constant in Go
x + 200     // Type resolves naturally

Reality Check: Go already does this! Untyped constants work until assignment.

When we DO need deferred inference:

x := []       // Empty slice - what's the element type?
x <- 1        // NOW infer: []int

---

Option 2: Minimal - Single Field in blockCtx

If we truly need tracking, add just ONE field:

type blockCtx struct {
    // ... existing fields ...
    
    // Minimal deferred tracking
    untypedVars map[string]ast.Expr  // varName -> initExpr
}

Implementation:

1. In compileAssignStmt() for :=, if RHS is untyped literal → store in map
2. On first use, resolve from map and remove entry
3. No queue, no placeholder types, no complex resolution

Code changes: ~20 lines

---

Option 3: Ultra-Minimal - Reuse Existing gogen Mechanism

Investigate if gogen already supports deferred type inference through its CodeBuilder API.

Research needed:

• Does DefineVarStart() accept nil type?
• Can we call SetType() later on the same variable?
• Check gogen source for existing deferred typing support

If yes: Zero changes to XGo compiler, just use existing API differently

---

Recommended Path: Start with Option 3

Step 1: Research gogen Capabilities (1-2 hours)

// Test if this works:
cb.DefineVarStart(pos, "x")
cb.Val(100)  // untyped
cb.EndInit(1)

// Later...
// Can we resolve x's type from usage?

Step 2: If gogen doesn't support it, use Option 2

• Add single untypedVars map[string]ast.Expr to blockCtx
• Modify compileAssignStmt() to store untyped RHS expressions
• Modify compileIdent() to check map and resolve on first use
• That's it!

Step 3: Write minimal test

// cl/_testgop/deferred-simple/in.xgo
x := 100
println(x + 200)

---

What We're NOT Doing (Save for Later)

❌ Queue mechanism - not needed for simple case
❌ Placeholder types - use existing untyped types
❌ Complex resolution algorithm - resolve on first use only
❌ Scope flushing - let normal scope rules handle it
❌ Multiple variable dependencies - one variable at a time

---

Size Estimate

Option 2 (most likely needed):

• cl/compile.go: +5 lines (add field to blockCtx)
• cl/stmt.go: +15 lines (store untyped vars in map)
• cl/expr.go: +10 lines (resolve on use)
• Test: +5 lines

Total: ~35 lines of actual code

---

Next Action

Should I:

1. Research gogen to see if deferred typing already exists?
2. Implement Option 2 directly with minimal changes?
3. Clarify the actual use case - what specifically needs deferring that Go doesn't already handle?

Please advise which direction to pursue.

---

Job Run | Branch: codeagent/claude/issue-2464-1760171206

[xushiwei]

@niupilot I feel like this might be possible by restricting all modifications to the gogen package?

[niupilot[bot]]

Analysis: gogen-Only Implementation Feasibility

Excellent suggestion! After researching the gogen package, here's my analysis:

---

Key Findings from gogen v1.19.5

Current Architecture

DefineVarStart Flow:

// In XGo compiler (cl/stmt.go:325):
ctx.cb.DefineVarStart(expr.Pos(), names...)
// ... compile RHS expressions ...
ctx.cb.EndInit(len(values))

// In gogen (codebuild.go):
func (p *CodeBuilder) DefineVarStart(pos token.Pos, names ...string) *CodeBuilder {
    return p.pkg.newValueDecl(ValueAt{}, p.current.scope, pos, token.DEFINE, nil, names...)
        .InitStart(p.pkg)
}

Type Resolution in gogen:

• ValueDecl.endInit() performs type inference during EndInit()
• Types are resolved from the stack elements (rets[i].Type)
• Variables are inserted into scope with concrete types immediately
• No mechanism for deferred resolution - all happens synchronously

---

Critical Discovery: gogen's Type Inference Happens at EndInit

Looking at type_var_and_const.go:endInit():

func (p *ValueDecl) endInit(cb *CodeBuilder, arity int) *ValueDecl {
    rets := cb.stk.GetArgs(arity)  // Get values from stack
    
    // Type inference happens HERE - immediately
    for i, ret := range rets {
        // Create variables with ret.Type
    }
    
    // Variables are inserted into scope with concrete types
}

The Problem: Type resolution is tightly coupled to EndInit() and happens synchronously.

---

Can We Implement Deferred Typing in gogen Alone?

Answer: Yes, but with significant gogen modifications

Required Changes to gogen:

Option A: Add Deferred Resolution to ValueDecl

// In gogen/type_var_and_const.go

type ValueDecl struct {
    names []string
    typ   types.Type
    // ... existing fields ...
    
    // NEW: Deferred type support
    deferred     bool               // Flag for deferred typing
    placeholders []*types.Var       // Placeholder variables
    initExprs    []ast.Expr         // Store init expressions for re-evaluation
}

// Modified endInit
func (p *ValueDecl) endInit(cb *CodeBuilder, arity int) *ValueDecl {
    if p.deferred {
        // Create placeholder variables
        // Store init expressions
        // DON'T resolve types yet
        return p.oldv
    }
    
    // Existing immediate resolution logic
    // ...
}

// NEW: Resolve deferred variable later
func (p *CodeBuilder) ResolveDeferredVar(name string, expectedType types.Type) error {
    // Find deferred ValueDecl
    // Re-evaluate init expression with expected type
    // Update variable type in scope
}

Estimated changes: ~100-150 lines in gogen

---

Option B: Add Untyped Variable Support

// In gogen/codebuild.go

func (p *CodeBuilder) DefineVarStartDeferred(pos token.Pos, names ...string) *CodeBuilder {
    // Create ValueDecl with deferred flag
    return p.pkg.newValueDecl(
        ValueAt{}, p.current.scope, pos, token.DEFINE, nil, names...)
        .SetDeferred(true)  // NEW method
        .InitStart(p.pkg)
}

// In gogen/type_var_and_const.go
func (p *ValueDecl) SetDeferred(deferred bool) *ValueDecl {
    p.deferred = deferred
    return p
}

Estimated changes: ~80-100 lines in gogen

---

Recommended Approach: Option B (gogen-Only with Minimal API)

Why gogen-Only Makes Sense:

✅ Centralized: Type inference logic stays in one place
✅ Reusable: Other gogen consumers benefit
✅ Clean: XGo compiler just uses new API
✅ Maintainable: Easier to test and debug

Implementation Plan:

Phase 1: Extend gogen API (~100 lines)

File: gogen/type_var_and_const.go

// Add deferred field to ValueDecl
type ValueDecl struct {
    // ... existing fields ...
    deferred bool
}

// Modify endInit to support deferred resolution
func (p *ValueDecl) endInit(cb *CodeBuilder, arity int) *ValueDecl {
    if p.deferred {
        // Store untyped placeholders
        for _, name := range p.names {
            placeholder := types.NewVar(p.pos, cb.pkg.Types, name, types.Typ[types.UntypedNil])
            p.scope.Insert(placeholder)
        }
        return p.oldv
    }
    // Existing logic for immediate resolution
    // ...
}

File: gogen/codebuild.go

// Optional: Add deferred variant (or reuse existing DefineVarStart)
func (p *CodeBuilder) DefineVarStartDeferred(pos token.Pos, names ...string) *CodeBuilder {
    vd := p.pkg.newValueDecl(ValueAt{}, p.current.scope, pos, token.DEFINE, nil, names...)
    vd.deferred = true
    return vd.InitStart(p.pkg)
}

// Add resolution method
func (p *CodeBuilder) ResolveDeferredVar(name string, inferredType types.Type) error {
    // Lookup deferred variable
    // Update its type
    // Re-insert into scope
}

Phase 2: Use in XGo (~10-20 lines)

File: cl/stmt.go

func compileAssignStmt(ctx *blockCtx, expr *ast.AssignStmt) {
    if tok == token.DEFINE {
        // Option 1: Always defer (simplest)
        ctx.cb.DefineVarStartDeferred(expr.Pos(), names...)
        
        // Option 2: Conditional deferring
        if shouldDefer(expr) {
            ctx.cb.DefineVarStartDeferred(expr.Pos(), names...)
        } else {
            ctx.cb.DefineVarStart(expr.Pos(), names...)  // Existing path
        }
        
        // ... compile RHS ...
        ctx.cb.EndInit(len(values))
    }
}

File: cl/expr.go (for resolution)

func compileIdent(ctx *blockCtx, v *ast.Ident) {
    // Check if deferred variable needs resolution
    if isDeferred(v.Name) {
        expectedType := inferFromContext(ctx)
        ctx.cb.ResolveDeferredVar(v.Name, expectedType)
    }
    // Normal compilation
}

---

Comparison: gogen-Only vs. XGo-Only

Aspect	gogen-Only	XGo-Only (Previous Proposal)
Lines of code	~120 lines	~35 lines
Complexity	Medium	Low
Reusability	High (benefits all gogen users)	Low (XGo-specific)
Maintenance	Centralized	Distributed
Testing	Can test in gogen directly	Must test through XGo
Risk	Medium (affects gogen consumers)	Low (isolated to XGo)

---

My Recommendation

Hybrid Approach: Start XGo-Only, Migrate to gogen Later

Reasoning:

1. Prove the concept in XGo first (~35 lines)
2. Gather real-world usage patterns
3. Then refactor into gogen with proper API design
4. This avoids premature optimization while keeping gogen clean

Phase 1 (This Issue): Simple map in blockCtx

type blockCtx struct {
    untypedVars map[string]ast.Expr  // ~5 lines
}

Phase 2 (Future): Migrate proven design to gogen

• Extract working logic from XGo
• Add proper API to gogen
• Update XGo to use new gogen API
• Remove temporary XGo code

---

But If You Prefer gogen-Only From Start:

I can implement the full gogen extension (~120 lines) across:

1. gogen/type_var_and_const.go - ValueDecl.deferred support
2. gogen/codebuild.go - DefineVarStartDeferred + ResolveDeferredVar
3. cl/stmt.go - Use new API (~10 lines)

Trade-off: More upfront work, but cleaner long-term architecture.

---

Next Steps

Which approach would you prefer?

1. Start simple in XGo (~35 lines), prove concept, then migrate to gogen?
2. Full gogen implementation (~120 lines) from the start?
3. Something else entirely?

I'm ready to proceed whichever way you prefer!

---

Job Run | Branch: codeagent/claude/issue-2464-1760171206