Client side non-controlled characters are over correcting inputs

[SueHeir]

I was curious about what I could be missing with my current implementation to make my non-controlled characters' movement smoother. Right now I am running one app in HostServer mode, and another in client mode. The HostServer mode is not experiencing this issue, their non-controlled characters are all moving great, even with a conditioner. The problems seem to be just with the non-controlled characters on the client side, with or without the conditioner they seem to teleport ahead when an input is pressed and then teleport back when the input is released. Happy to post code snippets or even share the github repo for more info if needed. I'm just not sure where to start with this and was hoping someone could point me in the right direction.

The code is mostly copied from the spaceship example, with a wasd controller instead, also a sprite is rendered as a child entity to the player.

[SueHeir]

https://github.com/cBournhonesque/lightyear/issues/576

Might be this issue

[cBournhonesque]

When you refer to non-controlled characters, do you mean characters that are controlled by other clients?

1. are those characters predicted?
2. do you use any kind of input delay?
3. does the server rebroadcast inputs to all other clients so that clients can predict every other client?

To summarize the issue you're spawning client C1 in host-server mode, and client C2.

• on C1: the movements of C2's character (which are predicted) look smooth. This does make sense since C1 is also the server
• on C2: the movements of C1's character (which are predicted) look very jittery

[cBournhonesque]

I believe the issue is that in host-server the client doesn't send any InputMessage to the server (since we're in host-server mode, the client and server entities are the same so there's no need to send anything).

But that also means that the server isn't replicating the local client's inputs to other clients, so these other clients cannot predict these inputs correctly. A solution would be to add a system on the server that runs only in host-server mode and prepares a message that sends the local client's inputs to other clients.

[SueHeir]

That 100% worked oh my!! I'm not sure if my code is 100% perfect, ended up using a clone.... But it has fixed the issue, even with a conditioner. I did have to change some stuff to pub. Which is shown in this pull request https://github.com/cBournhonesque/lightyear/pull/884

#[derive(Default)]
pub struct NumTicks {
    last_end_tick: Tick,
}

pub(crate) fn replicate_inputs_host_server(
    q_local_inputs: Query<(Entity, &InputBuffer<PlayerActions>), With<Predicted>>,
    mut send_inputs: EventWriter<ServerSendMessage<InputMessage<PlayerActions>>>,
    tick_manager: Res<TickManager>,
    mut local: Local<NumTicks>,

) {
    let num_ticks = tick_manager.tick() - local.last_end_tick;

    let mut messages_vec = Vec::new();

    if num_ticks > 0 {
        for (entity, input_buff) in q_local_inputs.iter() {

            let target = InputTarget::Entity(entity);
            let mut input_message = InputMessage::<PlayerActions>::new(tick_manager.tick());
            input_message.add_inputs(num_ticks as u16, target, input_buff);
            messages_vec.push(input_message);
        }

        let send = messages_vec.iter().map(|ev| {
            ServerSendMessage::new_with_target::<InputChannel>(
                ev.clone(),
                NetworkTarget::All,
            )
        });
    
        // rebroadcast the input to other clients
        // we are calling drain() here so make sure that this system runs after the `ReceiveInputs` set,
        // so that the server had the time to process the inputs
        send_inputs.send_batch(send);
    }

    local.last_end_tick = tick_manager.tick();
    
   
}

[SueHeir]

I actually spoke too soon, the conditioner wasn't added. The issue is still there with the added code.

Your summary is completely correct.

are those characters predicted? Both players on C2 have the predicted component filled with Some(Entity) that has the confirmed component, different from the entity itself. On the server C1, the predicted components are filled with confirmed entity Some(self) (same entity as itself)

do you use any kind of input delay? I don't think so, I copied the input stuff from spaceships example.

does the server rebroadcast inputs to all other clients so that clients can predict every other client? yes.

[SueHeir]

https://github.com/user-attachments/assets/9aa6188b-cef5-4f45-967c-d6c819acaf39

[SueHeir]

To add some additional Information: if C1 is acting as a server (even though it has the client plugin), then C2 and C3 as clients also experience this bug.

[cBournhonesque]

Let's say you have host-server HS, client C1, client C2, the packets take 3 ticks (RTT/2 to be sent between each of these)

• C1, C2 run 3 ticks ahead of HS, so that their inputs can arrive to HS on time
• If C1 wants to predict C2, C2 must send their inputs to HS and then rebroadcast it to C1. C2 needs at least 6 ticks (RTT) of input delay so that their inputs are sent to C1 on time for C1 to predict it perfectly. (if the input delay is not enough, there will be some mispredictions -> rollbacks/corrections)
• If C1 wants to predict HS, the HS inputs for tick T must arrive on time for C1 to predict them. C1 runs RTT/2 ahead of time of HS, and the HS messages takes RTT/2 time to reach C1, so HS must also have 6 ticks (RTT) input delay.

The spaceships example adds input delay (6 ticks), which is why a client C1 is generally able to predict other clients C2. However I think currently the host-server doesn't have any input delay, which is why even if you're sending the HS inputs to C1 the inputs arrive too late.

So the action-item would be:

• add input-delay for the host-server. We should make it configurable, because in a lot of cases input-delay is unneeded for the host-server since they theoretically run the game with 0 delay.

Btw i'm using this kind of system to debug this issue:

pub(crate) fn log_inputs(
    tick_manager: Res<TickManager>,
    query: Query<&InputBuffer<PlayerActions>, Without<Controlled>>
) {
    let tick = tick_manager.tick();
    for input_buffer in query.iter() {
        error!(?tick, "End tick in buffer: {:?}", input_buffer.end_tick());
    }
}

What matters is that a client C1 has access to other players inputs for tick T before it itself runs tick T. By the way it won't be possible to avoid rollbacks in all situations. Rollbacks will only completely disappear if the input_delay is over the RTT. Otherwise you will have a rollback (which is normal). Tuning the Correction parameter might help you make the rollbacks less visibile.

But I'm wondering if this is actually the best way to architect your game.

• Do you actually need to predict other players? Usually other players are interpolated, not predicted, unless you have a lot of physics interactions in your games. (Like Rocket League)
• To avoid host-server issues, one thing you could try for now is to just run the client and server apps in different processes for the 'host-server'. I.e. start 2 threads, one with client and one with server, similar to what I do in the examples in Separate mode.

[elizabeth-suehr]

I might not need them to be predicted. The goal is of the game is to have very good multiplayer feel but it is a PvE game, with lots of projectiles, and colliders of spell abilities, but no real physics. I would like to keep that option open incase I do need it, but its not a real priority.

I don't believe I have input delay setup anywhere, I didn't even notice is because it is handled in the common app example, oops. I'll try adding it and see where that gets me. If that doesn't work and I need to move to just interpolation, should I just delete the "add_prediction" line from the protocol defined on Position?

[cBournhonesque]

I think I have identified a change I made that breaks how inputs from clients are replicated to other clients. This is broken in the main branch, but it is fixed in the bevy-main branch that tracks bevy main. I don't think I will invest in fixing it since it's fixed in the bevy-main branch which is intended for the future release. But basically inputs from C2 are actually ignored by C1 which is why there are mispredictions no matter what...

(or you could try to work from the bevy-main branch.

• I'm pretty sure that after this is fixed, things should work properly for you (provided that the host-server sends its inputs to other clients, or that you use Separate mode). (and yes don't forget to add input delay)
• What you could do otherwise is to just enable interpolation for every entity, including your own character. If you want to work with interpolation, you don't need to change your protocol; you can just change the SyncTarget component on the server to use interpolation instead of prediction

[SueHeir]

things are working now, I even tested it over a real connection with a friend! Thank you!!

[cBournhonesque]

Cool! (although i'm surprised because in the main branch i don't think clients can receive inputs from the server) I'll keep this issues open to remember to deal with this properly in host-server mode

[SueHeir]

It worked because I copied your fix on the main branch from the bevy main branch! https://github.com/cBournhonesque/lightyear/pull/896

and sweet! excited to see that input delay in the host server mode!