amazon-sumerian-hosts
amazon-sumerian-hosts copied to clipboard
aws-samples
Add support for BabylonJS v5
The current implementation of Hosts is only compatible with BabylonJS v4 (4.2.1). When trying to use Hosts with BabylonJS 5 and later things break - like host speech animations and gaze tracking.
This bug is related: https://github.com/aws-samples/amazon-sumerian-hosts/issues/157
Tracking this. May need this at some point. Babylon.js 6.0 ~will be~ released March 30, 2023, too. Thanks!
Would it be possible to change this to v6.0 now that it's released? My team is hoping this happens. Faster rendering would greatly improve our load times for our web-application.
What's the status on this? We have found sound is broken on iOS Safari in Babylon 4.2 which is fixed in later versions.
Same breakage on Babylon v6.
like host speech animations and gaze tracking.
Also at least one character's jaw line gets deformed too (during idle no speech).
@theVoiceMouse WIP :). Folks over in the Babylon forums have provided some clues as to what we need to fix:
https://forum.babylonjs.com/t/models-broken-when-updating-from-v4-to-v5/43758/2
At the moment I'm finishing up with:
- https://github.com/aws-samples/amazon-sumerian-hosts/pull/183
and I can circle back to fixing Babylon after that. In the meantime, if you'd like to give the fix a try by branching off of #183, that'd be sweet, and I can help.
@trusktr - amazing update. I dont do 3d modelling, so ill test your branch and upload LexV2 (working) and OpenAI support (developed, but held private incase AWS get sniffy), hopefuly you can isolate the babylon isue. Regretably board meetings next week, so will be two weeks before branch/check/commit back
@theVoiceMouse it sounds like you may have already done similar work, but I did implement a solution for supporting LexV2 over in this copy of the Babylon Demos. Just never got around to contributing it back to the main repo.
@theVoiceMouse it sounds like you may have already done similar work, but I did implement a solution for supporting LexV2 over in this copy of the Babylon Demos. Just never got around to contributing it back to the main repo.
@Krxtopher can raise the lex v2 issue andpush yours into main as a fix to it (our code is almost identical and I love how webpack5 forced us both to use pako), I’m going @trusktr ‘s non-webpack & Babylon 5 branch and I expect there will be a lot of people who can benefit from LexV2 on the webpack/main line.
@theVoiceMouse Cool! The PR at
- https://github.com/aws-samples/amazon-sumerian-hosts/pull/183
is now complete apart from the Babylon fix. Getting LexV2 onto that next would be sweet.
and OpenAI support
Curious about that!
We have a rough initial open-source-not-yet-licensed openai-connected not-yet-named concept here (no docs yet, but it is a Meteor.js app, and can be launched with the meteor CLI, see the start script in package.json, and also needs keys):
https://github.com/autonomys/unnamed-experiment
Ok, I'm back at it!
Need to get this Babylon fix settled. I don't even know exactly what the issue is. @Krxtopher and @theVoiceMouse, do you know exactly what needs to be changed, and can provide some pointers?
In the meantime, I'm investigating, starting with the thread in the Babylon forum.
The Babylon type definitions were not visible in the Babylon host source code. To help learn what's going on with Babylon more easily (and hence make a fix for Babylon after I understand it better), here's a change that enables the types:
- https://github.com/aws-samples/amazon-sumerian-hosts/issues/189