Remote based replication requires server restart to take effect if configured from the server

[timsehn]

Repro:

1. Create a new database on DoltHub. Make a table and commit so you can clone.
2. Clone it locally:

$ dolt clone timsehn/replication_restart
cloning https://doltremoteapi.dolthub.com/timsehn/replication_restart
$ cd replication_restart

3. Start a SQL server:

$ dolt sql-server
Starting server with Config HP="localhost:3306"|T="28800000"|R="false"|L="info"|S="/tmp/mysql.sock"

4. Open a sql shell, set replication variables:

$ mysql -u root
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.7.9-Vitess Dolt

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MySQL [(none)]> use replication_restart;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
MySQL [replication_restart]> select * from t;
Empty set (0.007 sec)

MySQL [replication_restart]> set @@persist.dolt_replicate_to_remote = 'origin';
Query OK, 1 row affected (0.004 sec)

MySQL [replication_restart]> set @@persist.dolt_replicate_all_heads = 1; 

5. Insert values and make a commit:

MySQL [replication_restart]> insert into t values (1,1);
Query OK, 1 row affected (0.027 sec)

MySQL [replication_restart]> call dolt_commit('-am', '1,1');
+----------------------------------+
| hash                             |
+----------------------------------+
| em4tkcvluh7f6qgjitabg6olu4217gkg |
+----------------------------------+
1 row in set (0.023 sec)

6. Go to DoltHub. Does not replicate:
7. Restart SQL server.
8. Make another insert and commit

MySQL [replication_restart]> insert into t values (2,2);
ERROR 2006 (HY000): Server has gone away
No connection. Trying to reconnect...
Connection id:    1
Current database: replication_restart

Query OK, 1 row affected (0.032 sec)

MySQL [replication_restart]> call dolt_commit('-am', '2,2');
+----------------------------------+
| hash                             |
+----------------------------------+
| 7mrupsen61i1km5asqbjvf2amh26ef9q |
+----------------------------------+
1 row in set (2.078 sec)

9. Go to DoltHub. Replication succeeds.

Starting replication should not require a server restart.

[macneale4]

An observation:

prod_replication_test> set @@persist.dolt_replicate_to_remote = 'origin';
prod_replication_test> show variables like 'dolt_replicate_to_remote';
+--------------------------+-------+
| Variable_name            | Value |
+--------------------------+-------+
| dolt_replicate_to_remote |       |
+--------------------------+-------+
1 row in set (0.00 sec)

Restart the server, and:

prod_replication_test> show variables like 'dolt_replicate_to_remote';
+--------------------------+--------+
| Variable_name            | Value  |
+--------------------------+--------+
| dolt_replicate_to_remote | origin |
+--------------------------+--------+

[macneale4]

I'm able to repro this.

There related bug (feature request) is that the sql-server doesn't get in sync when is starts and it's already ahead of the origin. That would be my expectation.

[fulghum]

I believe the cause of this is that we only configure replication when we create the SQL engine – once the SQL engine is constructed, setting any of the dolt_replication system variables doesn't actually change the SQL engine into the different type needed to support replication – the sql-server has to be restarted and the engine construction code must see those replication system variables in order to instantiate the correct sql engine type.

This is likely surprising for many customers, so would be a great gap to fix. I remember being surprised by this a while back and am working with another customer now who was also surprised by it.