Generate assets with AI

[nighca]

AIGC 素材生成

[nighca]

Related:

• Summary
• https://lab.rosebud.ai/ai-game-assets

[qingqing-ux]

AI 游戏引擎调研

1. Rosebud（https://lab.rosebud.ai/）与 Redbrick（https://redbrick.land/）

• 两者功能相近，均支持根据命题生成游戏代码与资产（如音效、精灵）。
• 支持在游戏制作过程中实时生成精灵并直接添加至项目。
• 精灵提供两种风格：
• 2D 风格：以 webp 格式呈现，动画采用序列帧方式。
• 3D 风格：以 gltf 格式提供，包含内嵌动画。
• 实验生成了多个游戏，但目前尚不确定生成素材是否来自其素材库。拟通过「验证生成素材是否支持二次修改」判断其是否为实时生成内容。
• 试用录屏

2.腾讯混元（https://3d.hunyuan.tencent.com/）

• 支持动作驱动与骨骼绑定，适用于人形精灵的生成与控制，表现效果较好。
• 人形精灵骨骼绑定
• 动物精灵骨骼绑定

[nighca]

• Gemini 2.5 Flash Image (Nano Banana)
• BananaPod
• https://github.com/ZHO-ZHO-ZHO/ZHO-nano-banana-Creation
• live2d

[nighca]

现在 @qingqing-ux 素材制作主要流程

Sprite

• lovart（含 nano banana）做角色图片（一张）
• 即梦生成视频，截关键帧（4 张左右）；这里即梦的可替代项还有 leonardo.ai（不过用得少）
• recraft 转矢量，去背景（删图层），检查去除杂色
• 添加到 builder

Backdrop

• midjourney 生成

特效（火焰、水滴、爆炸）或道具、UI 元素等

• 现成素材库（主要是 freepik）里找

[nighca]

已知方案

• 直接生成 spritesheet 图片
• 图 -> 视频 -> 截帧
• 生成图，结合已有视频做动作迁移（Wan 2.2 Animate）
• 2D 骨骼动画（Spine、Live2D 等）

[nighca]

相对完整的技术 & 产品分析

https://docs.google.com/document/d/1hPd-fYmCbeAA2Hq0r3rDQwuXVijGuurA2oUrTCyb3SU/edit?usp=sharing

[nighca]

• 素材生成功能应当可插拔（闭源收费功能）

[nighca]

Modules

• Semantic search engine @aofei @go-wyvern

• Content generation service @aofei @go-wyvern

  • Generate sprite structure (costumes, animations)
  • Generate costume image (with pivot)
  • Generate animation reference frame(s)
  • Generate animation video
  • Generate animation frames based on video
  • Generate backdrop image
  • Generate sound audio

• Project & Asset management @nighca

• UI (TODO)

  • Project settings editor
  • Sound generator
  • Backdrop generator
  • Sprite generator
    • Costume generator
    • Animation generator
  • Asset library

Types

// GraphicSetting contains common graphic setting
type GraphicSetting struct {
	// Art style, e.g., "pixel-art", "cartoon", etc.
	ArtStyle string
	// Perspective, e.g., "side-view", "top-down", etc.
	Perspective string
}

// ProjectSetting contains project-level info for asset generation
type ProjectSetting struct {
	GraphicSetting
	// Name of the project
	Name string
}

// SpriteSettingDraft is draft for sprite setting
type SpriteSettingDraft struct {
	GraphicSetting
	// Name of the sprite
	Name string
	// Category of the sprite, e.g., "character", "prop", "other"
	Category string
	// Description of the sprite
	Description string
}

// SpriteSetting is used to generate a sprite
type SpriteSetting struct {
	SpriteSettingDraft
	// Project setting the sprite belongs to
	Project ProjectSetting
}

// CostumeSettingDraft is draft for costume setting
type CostumeSettingDraft struct {
	GraphicSetting
	// Name of the costume
	Name string
	// Description of the costume
	Description string
	// Direction the costume facing, e.g., "left", "right", "front", "back"
	Facing string
}

// CostumeSetting is used to generate a costume
type CostumeSetting struct {
	CostumeSettingDraft
	// Reference image for the costume (nil if not provided)
	ReferenceCostumeImage ImageData
	// Sprite setting the costume belongs to
	Sprite SpriteSetting
}

// AnimationSettingDraft is draft for animation setting
type AnimationSettingDraft struct {
	GraphicSetting
	// Name of the animation
	Name string
	// Description of the animation
	Description string
	// Whether the animation loops
	Loop bool
}

// AnimationSetting is used to generate an animation
type AnimationSetting struct {
	AnimationSettingDraft
	// Reference image for the animation (nil if not provided)
	ReferenceCostumeImage ImageData
	// Sprite setting the animation belongs to
	Sprite SpriteSetting
}

// BackdropSettingDraft is draft for backdrop setting
type BackdropSettingDraft struct {
	GraphicSetting
	// Name of the backdrop
	Name string
	// Description of the backdrop
	Description string
}

// BackdropSetting is used to generate a backdrop
type BackdropSetting struct {
	BackdropSettingDraft
	// Project setting the backdrop belongs to
	Project ProjectSetting
}

// SoundSettingDraft is draft for sound setting
type SoundSettingDraft struct {
	// Name of the sound
	Name string
	// Description of the sound
	Description string
	// Category of the sound, e.g., "effect", "music", etc.
	Category string
}

// SoundSetting is used to generate a sound
type SoundSetting struct {
	SoundSettingDraft
	// Project setting the sound belongs to
	Project ProjectSetting
}