Make Asset data pulldown from web dynamic

[premiumjibles]

Summary

Currently we've got a script we have to run manually to pull down the web asset data. It would be great if this happened dynamically during runtime.

Implementation plan

● # Implementation Plan: Dynamic Asset Data Loading

Phase 1: Core Infrastructure Changes

1.1 Create Asset Data Fetcher Utility

• [ ] Create packages/utils/src/assetData/fetchAssetData.ts
  • Export fetchEncodedAssetData() async function
  • Fetches from: https://raw.githubusercontent.com/shapeshift/web/develop/src/lib/asset-service/service/encodedAssetData.json
  • Returns decoded asset data
  • Throws descriptive error on fetch failure

1.2 Refactor AssetService to Async Initialization

• [ ] Update packages/utils/src/AssetService.ts
  • Change constructor to private async initialize() method
  • Add static isInitialized: boolean flag
  • Add static initPromise: Promise<AssetService> | null
  • Replace getInstance() with async initialize() that:
    • Returns existing instance if already initialized
    • Fetches asset data via fetchEncodedAssetData()
    • Decodes and builds indexes
    • Returns singleton instance
  • Update all methods to assume initialization complete (no lazy loading)

1.3 Create CoinGecko Adapter Fetcher

• [ ] Create packages/caip/src/adapters/coingecko/fetchAdapters.ts
  • Export fetchAllCoingeckoAdapters() async function
  • Fetches all 18 chain mapping files from GitHub in parallel
  • Base URL: https://raw.githubusercontent.com/shapeshift/web/develop/packages/caip/src/adapters/coingecko/generated/{chainId}/adapter.json
  • Returns merged Record<AssetId, CoinGeckoId>
  • Throws descriptive error on fetch failure

1.4 Update CoinGecko Index

• [ ] Update packages/caip/src/adapters/coingecko/index.ts
  • Remove static imports from ./generated/index.js
  • Add initializeCoinGeckoAdapters() async function
  • Export singleton pattern similar to AssetService
  • Use fetchAllCoingeckoAdapters() for initialization

Phase 2: Server Changes

2.1 Update Server Initialization

• [ ] Update apps/agentic-server/src/server.ts

  • Import AssetService and initializeCoinGeckoAdapters

  • Before app.listen(), add:

    console.log('Initializing asset data...')
    await AssetService.initialize()
    await initializeCoinGeckoAdapters()
    console.log('Asset data initialized successfully')
    

  • Wrap in try-catch, exit process with error code 1 on failure

  • Log helpful error message with GitHub URL on failure

2.2 Update Server Tools (if needed)

• [ ] Check apps/agentic-server/src/tools/getAssets.ts and other tools
  • Verify they use assetService.getInstance() (should already be initialized)
  • No changes needed if server initialization is blocking

2.3 Update Server Build Config

• [ ] Verify apps/agentic-server/tsup.config.ts
  • Ensure no JSON bundling issues
  • Test that fetch() works in Node.js environment

Phase 3: Client Changes

3.1 Create Client Asset Loader

• [ ] Create apps/agentic-chat/src/lib/initializeAssets.ts
  • Export initializeAssets() async function
  • Calls AssetService.initialize()
  • Returns boolean success/failure
  • Logs errors to console

3.2 Update Client Entry Point

• [ ] Update apps/agentic-chat/src/main.tsx

  • Import initializeAssets

  • Before rendering React app:

    console.log('Loading asset data...')
    const success = await initializeAssets()
    if (!success) {
      console.error('Failed to load asset data')
      // Render error UI or continue with limited functionality
    }
    console.log('Asset data loaded')
    

  • Add try-catch wrapper

3.3 Update AssetIcon Component

• [ ] Update apps/agentic-chat/src/components/ui/AssetIcon.tsx
  • Add null check for assetService.getAsset() return
  • Render fallback icon/text if asset not found
  • Consider adding error boundary

3.4 Add Loading UI (optional)

• [ ] Create loading screen component (optional)
  • Show "Loading asset data..." message during initialization
  • Replace root render with conditional loading screen

Phase 4: Cleanup

4.1 Remove Static Asset Files

• [ ] Delete packages/utils/src/assetData/encodedAssetData.json
• [ ] Delete packages/caip/src/adapters/coingecko/generated/ directory
• [ ] Update .gitignore to ignore these paths (if needed)

4.2 Update Update Script

• [ ] Update scripts/updateAssetData.ts
  • Add comment that this is now only for local development
  • Or remove entirely if no longer needed
  • Update package.json to remove update-assets script (or mark deprecated)

4.3 Update Build Configs

• [ ] Verify packages/utils/tsconfig.json doesn't try to include deleted JSON
• [ ] Verify packages/caip/tsconfig.json doesn't try to include deleted JSON
• [ ] Test that builds succeed without the static files

Phase 5: Testing & Validation

5.1 Local Testing

• [ ] Run bun install to ensure dependencies are fresh
• [ ] Run bun run build at root - verify no errors
• [ ] Test server startup: cd apps/agentic-server && bun run dev
  • Verify "Initializing asset data..." logs appear
  • Verify server starts successfully
  • Test /api/chat with asset-related query
• [ ] Test client startup: cd apps/agentic-chat && bun run dev
  • Verify browser console shows asset loading
  • Verify AssetIcon components render correctly
  • Test asset search functionality

5.2 Error Scenario Testing

• [ ] Test with GitHub down (block raw.githubusercontent.com in /etc/hosts)
  • Verify server exits with clear error message
  • Verify client shows error state
• [ ] Test with invalid JSON response (mock fetch)
  • Verify proper error handling and logging

5.3 Type Checking & Linting

• [ ] Run bun type-check - fix any TypeScript errors
• [ ] Run bun lint:fix - fix any linting issues
• [ ] Verify no any types were introduced

Phase 6: Deployment Considerations

6.1 Railway (Server) Configuration

• [ ] Verify Railway build succeeds with dynamic loading
• [ ] Test server startup time (should add ~2-5 seconds)
• [ ] Verify Railway health checks accommodate longer startup
• [ ] Check Railway logs show successful asset initialization

6.2 Vercel (Client) Configuration

• [ ] Verify Vercel build succeeds without bundled JSON
• [ ] Test client loading time in production
• [ ] Verify bundle size reduction (should be ~5-7MB smaller)
• [ ] Test with slow network (throttling) to ensure UX acceptable

6.3 Monitoring & Alerts

• [ ] Add error tracking for asset initialization failures
• [ ] Consider adding metrics for initialization time
• [ ] Document new failure modes for on-call team

Success Criteria

• ✅ Server starts successfully and loads asset data from GitHub
• ✅ Client starts successfully and loads asset data from GitHub
• ✅ Asset search functionality works identically to before
• ✅ AssetIcon components render correctly
• ✅ Bundle size reduced by ~5-7MB
• ✅ Build completes without errors
• ✅ Type checking passes
• ✅ Server fails gracefully with clear error if GitHub unreachable
• ✅ Client fails gracefully with clear error if GitHub unreachable

Estimated Effort

• Phase 1-2 (Server): 2-3 hours
• Phase 3 (Client): 1-2 hours
• Phase 4-5 (Cleanup/Testing): 1-2 hours
• Phase 6 (Deployment): 1 hour
• Total: 5-8 hours

Rollback Plan

If issues arise in production:

1. Revert commit(s)
2. Redeploy previous version with bundled assets
3. Restore deleted JSON files from git history
4. Run bun update-assets to refresh local copies