近期计划是什么？| What are the short-term plans?

[hengtuibabai]

停更了吗？近期计划是什么？ 感觉离好用还差一点点了啊，停更的话太可惜了！

Has it stopped? What are the short-term plans? I feel like just a little short of being useful. It would be a pity if I stopped the watch!

[vinthony]

hi， 最近再肝paper，可能会在月底或者6月份开始继续更新。目前已经完成的部分和正在做的部分，比如:

• [x] 支持size参数可以控制输出图的分辨率，训了一个512x512的模型face render还在测试之中，期望可以干掉人脸增强器。
• [x] mannually的控制crop的区域，可以得到更加自定义的结果，WEBUI。
• [x] WEBUI更多功能，比如refpose等。
• [x] 接受了两种不同的自动crop方式，能够复现v0.0.1版本的效果。
• [x] 减少了一些依赖文件,比如dlib，用现在的方法更便于安装。
• [x] 加速模型和优化checkpoints，可能会不需要下载那么大的模型。
• [ ] 更加解耦的mappingnet的训练，可能可以支持只针对头进行动作。
• [x] A simpler facerender model (or the TensorRT support) will be included to support faster generation: https://github.com/OpenTalker/SadTalker/discussions/457
• [ ] text-geneartion-webui
• [ ] anime-generator.
• [ ] OpenTalker WEBUI, 集成Sadtalker和video-retalking的技术，做到输入图像和视频进行驱动和编辑。
• [ ] Fix API problem， #379，#374 , #251 ,
• [ ] FPS, #294,

[vinthony]

any new suggestions are also welcome : )

[2793145003]

之前看到 有人用 Diffusion model，用 Diffusion model 的话也许能做到原图分辨率？（不懂瞎说）

[vinthony]

diffusion-based 估计会很慢，不过face vid2vid也很慢就是了。质量上我从paper里没看到很大的差距因为他的diffusion不是SD，是自己训的。这里有一个备选是 先用sadtalker生成keypoints，再用pre-trained 的controlnet-face去解。

[lliang2003]

OpenTalker WEBUI, 集成Sadtalker和video-retalking的技术，做到输入图像和视频进行驱动和编辑。

[canghaiyunfan]

请问如何实现只针对头进行动作？

[Niutonian]

I love your project, I think a good way to control ref_pose and ref_eyebllink would be awesome, Another thing that I would love to see implemented is "Idle animation", in case no audio is played it goes through a preset sequence of blink and move without looking too weird

[MoroseYu]

不好停更啊，很期待~

[vinthony]

cool! thanks for your advise! will woking on it.

[2793145003]

之后会解决脸色苍白双眼无神的问题吗？（还是我用的方法不对？ quick_demo里的full3，size=512，出来的结果脸会变得很白（去掉gfpgan也很白） 眼睛也一直虚着，去掉still之后可以眨眼但又没法贴回原图

[vinthony]

I love your project, I think a good way to control ref_pose and ref_eyebllink would be awesome, Another thing that I would love to see implemented is "Idle animation", in case no audio is played it goes through a preset sequence of blink and move without looking too weird

We have update this feature in https://github.com/OpenTalker/SadTalker/discussions/386, however, more work need to be done to make it better.

[vinthony]

这个我也发现了，可能是和训练数据有关。我找时间再训练一下模型。

之后会解决脸色苍白双眼无神的问题吗？（还是我用的方法不对？ quick_demo里的full3，size=512，出来的结果脸会变得很白（去掉gfpgan也很白） 眼睛也一直虚着，去掉still之后可以眨眼但又没法贴回原图

[zyl280505776]

挺好的，期待持续迭代

[jyzd111]

大佬厉害，我用的国外的网站生成的和你效果差不多，你的还能自己调整参数。期待早日上2D，

[grazder]

what are your plans on cartoon images? like in makeittalk

[vinthony]

A lightweight facerender is added for generation, which might be working in real-time on GPU and 100x faster on Macbook. See the discussion https://github.com/OpenTalker/SadTalker/discussions/457.

[vinthony]

what are your plans on cartoon images? like in makeittalk

will try to add something like: https://github.com/pkhungurn/talking-head-anime-3-demo

[Kedreamix]

你好，我想问一下，有计划开源训练的代码么，也就是每一part的训练代码

[FranM2030]

I've noticed that you're able to provide a head pose reference video, could we do the same for a half body. Provide a reference video for half body that drives the upper body movement along with the head? Just he head is a bit limited.

[xyyyuuan]

amd的显卡是不是只能跑在cpu上，amd不动

[ifredom]

感谢大兄弟，就差一点就很好用了，有空更更，不要鸽啦

[skyliwq]

加油加油

[warycat]

加油加油，这个东西太有用了，我要把它集成到我的app里。

[Tybost]

Still crossing my fingers for anime head support. ;)

[slavakurilyak]

When can we expect the next release? The last release was made 168 days ago.

[creepcat-gh]

大佬厉害，确实非常好用

[zjy-2020]

大佬，网盘下载的压缩包李没有.pth文件和BFM, hub文件了吗 就是checkpoints/auido2exp_00300-model.pth，checkpoints/auido2pose_00140-model.pth， checkpoints/epoch_20.pth等文件。 没有话程序老是报错

[rucieryi369]

会有微调的代码吗？谢谢

[denvey]

OpenTalker WEBUI

请问这个在哪里呀

[XayerMorgan]

Any plans for continued development. I was using this for a while, but suddenly it stopped working. Can't find the root cause, because automatic1111 and foocus work great. Standalone or automatic1111 give same errors. Would love to see this continue, and it's a remarkable piece of work.