dcos-e2e icon indicating copy to clipboard operation
dcos-e2e copied to clipboard

Need some kind of ability to dynamically increase the number of nodes

Open rajcspsg opened this issue 5 years ago • 4 comments

I'm not sure whether this functionality is already available in minidcos.

RIght now once the cluster the created using below command

minidcos docker/vagrant create ./dcos_generate_config.sh --agents n

We will endup with constant nodes. If we want to add few nodes extra we don't have any options other than creating new cluster.

rajcspsg avatar Apr 23 '19 19:04 rajcspsg

It might very well be possible already with the right configuration(--masters 0 in a new cluster, pointing to the old clusters masters with a --extra-config) but we haven't come around to test it yet.

Please feel free to do some experimentation :)

timaa2k avatar Apr 23 '19 19:04 timaa2k

@timaa2k That won't work because a cluster is identified by its cluster_id in many places. Therefore we need a new command --add-agent for example.

adamtheturtle avatar Apr 25 '19 11:04 adamtheturtle

As a workaround you can vertically scale the agents. This works in cases where resources, memory are storage are insufficient but not in cases where extra ports are required.

Vince-Nieuw avatar May 28 '19 20:05 Vince-Nieuw

I'll enumerate a few criteria:

  • Ability to add both public and private agents
  • Ability to add to a cluster which is provisioned but does not have DC/OS installed
    • minidcos docker inspect for example will show the new agent
  • Ideally: No need to figure out what was generated in the configuration you didn't supply in minidcos docker create

The way I see this working, psuedocode:

@click.command()
def provision_agent(cluster_id, role):
    existing_cluster = ...(cluster_id)

    # add multiple options from `provision`?
    # Or work stuff out from the existing cluster?
    cluster_backend = Docker() 

    new_cluster = Cluster(
        cluster_backend=cluster_backend,
        # Or different if the role != agent
        agents=1,
        public_agents=0,
        masters=0,
    )
    
    print(new_node_id)
@click.command()
def agent_install(cluster_id, node_id):
    existing_cluster = ...(cluster_id)
    agent = get_node(cluster_id, node_id)
    # This is not an existing function but we could work it out
    config = get_config_from_existing_cluster()

    agent.install_dcos_from_path(
        dcos_installer=dcos_installer,
        dcos_config=config,
        role=role,
    )

    existing_cluster.wait_for_dcos()

adamtheturtle avatar Jun 23 '19 08:06 adamtheturtle