LiveTalking
LiveTalking copied to clipboard
Published
1 month ago
•
lipku
lipku
实时切换数字人模特，出现内存占用增高
模式：musetalke + webrtcpush 功能：新增接口，传入id，实现实时切换数字人。 参考源码： app.py - run函数 musereal - load_avatar函数
切换数字人主要代码实现：
if (old_pc := pcs.get(0)):
await old_pc.close()
del pcs[0]
global avatar
avatar = load_avatar(avatar_id)
await run(opt.push_url, 0)
通过@profile注解监控run函数，load_avatar函数，发现在load_avatar中 frame_list_cycle = read_imgs(input_img_list) 每次frame_list_cycle 都会增加
这是程序刚启动时默认载入数字人时调用load_avatar函数的内存日志：
Line # Mem usage Increment Occurrences Line Contents
63 1958.6 MiB 1958.6 MiB 1 @profile
64 def load_avatar(avatar_id):
65 #self.video_path = '' #video_path
66 #self.bbox_shift = opt.bbox_shift
67 1958.6 MiB 0.0 MiB 1 avatar_path = f"./data/avatars/{avatar_id}"
68 1958.6 MiB 0.0 MiB 1 full_imgs_path = f"{avatar_path}/full_imgs"
69 1958.6 MiB 0.0 MiB 1 coords_path = f"{avatar_path}/coords.pkl"
70 1958.6 MiB 0.0 MiB 1 latents_out_path= f"{avatar_path}/latents.pt"
71 1958.6 MiB 0.0 MiB 1 video_out_path = f"{avatar_path}/vid_output/"
72 1958.6 MiB 0.0 MiB 1 mask_out_path =f"{avatar_path}/mask"
73 1958.6 MiB 0.0 MiB 1 mask_coords_path =f"{avatar_path}/mask_coords.pkl"
74 1958.6 MiB 0.0 MiB 1 avatar_info_path = f"{avatar_path}/avator_info.json"
75 # self.avatar_info = {
76 # "avatar_id":self.avatar_id,
77 # "video_path":self.video_path,
78 # "bbox_shift":self.bbox_shift
79 # }
80
81 1958.6 MiB 0.0 MiB 1 input_latent_list_cycle = torch.load(latents_out_path) #,weights_only=True
82 1958.6 MiB 0.0 MiB 2 with open(coords_path, 'rb') as f:
83 1958.6 MiB 0.0 MiB 1 coord_list_cycle = pickle.load(f)
84 1958.6 MiB 0.0 MiB 1 input_img_list = glob.glob(os.path.join(full_imgs_path, '*.[jpJP][pnPN]*[gG]'))
85 1958.6 MiB 0.0 MiB 649 input_img_list = sorted(input_img_list, key=lambda x: int(os.path.splitext(os.path.basename(x))[0]))
86 9387.7 MiB 7429.1 MiB 1 frame_list_cycle = read_imgs(input_img_list)
87 9387.7 MiB 0.0 MiB 2 with open(mask_coords_path, 'rb') as f:
88 9387.7 MiB 0.0 MiB 1 mask_coords_list_cycle = pickle.load(f)
89 9387.7 MiB 0.0 MiB 1 input_mask_list = glob.glob(os.path.join(mask_out_path, '*.[jpJP][pnPN]*[gG]'))
90 9387.7 MiB 0.0 MiB 649 input_mask_list = sorted(input_mask_list, key=lambda x: int(os.path.splitext(os.path.basename(x))[0]))
91 9387.7 MiB 0.0 MiB 1 mask_list_cycle = read_imgs(input_mask_list)
92 9387.7 MiB 0.0 MiB 1 return frame_list_cycle,mask_list_cycle,coord_list_cycle,mask_coords_list_cycle,input_latent_list_cycle
这是我手动调用接口切换数字人时load_avatar函数的内存监控：
Line # Mem usage Increment Occurrences Line Contents
63 11231.3 MiB 11231.3 MiB 1 @profile
64 def load_avatar(avatar_id):
65 #self.video_path = '' #video_path
66 #self.bbox_shift = opt.bbox_shift
67 11231.3 MiB 0.0 MiB 1 avatar_path = f"./data/avatars/{avatar_id}"
68 11231.3 MiB 0.0 MiB 1 full_imgs_path = f"{avatar_path}/full_imgs"
69 11231.3 MiB 0.0 MiB 1 coords_path = f"{avatar_path}/coords.pkl"
70 11231.3 MiB 0.0 MiB 1 latents_out_path= f"{avatar_path}/latents.pt"
71 11231.3 MiB 0.0 MiB 1 video_out_path = f"{avatar_path}/vid_output/"
72 11231.3 MiB 0.0 MiB 1 mask_out_path =f"{avatar_path}/mask"
73 11231.3 MiB 0.0 MiB 1 mask_coords_path =f"{avatar_path}/mask_coords.pkl"
74 11231.3 MiB 0.0 MiB 1 avatar_info_path = f"{avatar_path}/avator_info.json"
75 # self.avatar_info = {
76 # "avatar_id":self.avatar_id,
77 # "video_path":self.video_path,
78 # "bbox_shift":self.bbox_shift
79 # }
80
81 11136.7 MiB -94.6 MiB 1 input_latent_list_cycle = torch.load(latents_out_path) #,weights_only=True
82 11136.7 MiB 0.0 MiB 2 with open(coords_path, 'rb') as f:
83 11136.7 MiB 0.0 MiB 1 coord_list_cycle = pickle.load(f)
84 11136.7 MiB 0.0 MiB 1 input_img_list = glob.glob(os.path.join(full_imgs_path, '*.[jpJP][pnPN]*[gG]'))
85 11136.7 MiB 0.0 MiB 583 input_img_list = sorted(input_img_list, key=lambda x: int(os.path.splitext(os.path.basename(x))[0]))
86 18068.6 MiB 6931.8 MiB 1 frame_list_cycle = read_imgs(input_img_list)
87 18068.6 MiB 0.0 MiB 2 with open(mask_coords_path, 'rb') as f:
88 18068.6 MiB 0.0 MiB 1 mask_coords_list_cycle = pickle.load(f)
89 18068.6 MiB 0.0 MiB 1 input_mask_list = glob.glob(os.path.join(mask_out_path, '*.[jpJP][pnPN]*[gG]'))
90 18068.6 MiB 0.0 MiB 583 input_mask_list = sorted(input_mask_list, key=lambda x: int(os.path.splitext(os.path.basename(x))[0]))
91 18146.9 MiB 78.4 MiB 1 mask_list_cycle = read_imgs(input_mask_list)
92 18146.9 MiB 0.0 MiB 1 return frame_list_cycle,mask_list_cycle,coord_list_cycle,mask_coords_list_cycle,input_latent_list_cycle
按照这样增长下去，只要多切换几次，内存就够不用了，出现异常： resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 43 leaked semaphore objects to clean up at shutdown warnings.warn('resource_tracker: There appear to be %d ' 说明有些对象没有被处理掉。
想请问怎么释放之前载入的数字人所占用的内存，
python有自己的内存管理机制，主要内存消耗在frame_list_cycle，尝试一下
删除引用
del frame_list_cycle
强制运行垃圾回收
gc.collect()
