Configurable path

[mlizza]

Hi, I strongly suggest to add a custom subpath definition for both "config" and "target" repo. I would like to organize *.js and *.yml files in subdirectories and not store them in the root folder.

For example let's think about a config repo that contains configuration for many vms.

[ADRFranklin]

You can create a bash script or a script in another scripting language to trigger an application or events for certain files.

Pico's job is to deploy services, but that doesn't have any affect on how your files are stored. Could you please give a better explanation as to your situation?

[mlizza]

Using a custom script inside the target repo was my first idea, but I see a limit in the config repo setup. With your approach I have to create a different config repo for each VM that I want automate with pico. The config repo must contains at least a js file at the root level? Is it correct or am I missing something?

Let's have a lot of VMs to handle in this way.. you will have so many repos with at least one js file. Why not to have one single repo with different files/folders for each VM? Basically you will point to a folder or to a js file that describes that particular VM and in the target repo you will execute what you want. Maybe you can use the go getter (hashicorp) format -> https://github.com/hashicorp/go-getter.

Thanks

[ADRFranklin]

That isn't really the goal of this repo. The idea is that you have one master repo that configures how each deployment works.

You can then specify which services (aka repos) are deployed into which machine. For example my master repo, deploys on to 2 dedicated servers, a local dev server and 8 VM's.

I configure my repos to be independent of each other so that all they need is a single place of execution, which is what pico is designed to do as of right now. I doubt this will change much.

You can specify the HOSTNAME to be passed into pico to determine in the configuration file what services should be deployed to that machine. You can also use HOSTNAME variables inside deployed services themselves, so you could have a bash script that checks that HOSTNAME and runs a certain set of scripts if that is what you desire.

But other then that, I'm not sure this request is best suited for this tool, since that would go in the opposite direction of what pico was designed for.

[mlizza]

You can then specify which services (aka repos) are deployed into which machine. For example my master repo, deploys on to 2 dedicated servers, a local dev server and 8 VM's.

So you have 11 config repos that contains in master branch at least one js file with the T function that points to one or more target repos and pico will execute something different matching (for example) the HOSTNAME. Right?

I don't see a big difference in storing a js file inside a folder instead of a different repo. If you have 1k VM, you will have 1k files/folders and not repos. What do you think?

I'm a big fan of GitOps on k8s (in production I use argoCD), I would like to use pico to create something similar for docker engine scenario.

Another question about config: can I access to the env variables inside the JS file? Accessing the env var, the caller can load config dynamically (e.g. using the HOSTNAME).

Have you created a slack channel or something else to talk about pico? Thanks

[ADRFranklin]

Taking a recent deployment case from Southclaws's deployments, this is an example of how you can use the configuration file to specify repos to deploy from

(I have removed some stuff that might be a little revealing to server names and repos)

E("MACHINE_NAME", HOSTNAME);
E("DATA_DIR", "/data/shared");

var GL = "https://gitlab.com/<username>/";

A({
  name: "gitlab",
  path: "/pico",
  user_key: "GIT_USERNAME",
  pass_key: "GIT_PASSWORD"
});

function Compose(name) {
  return {
    name: name,
    url: GL + name,
    up: ["docker-compose", "up", "-d"],
    down: ["docker-compose", "down"],
    auth: "gitlab"
  };
}

function ComposeFullPath(name, url) {
  return {
    name: name,
    url: url,
    up: ["docker-compose", "up", "-d"],
    down: ["docker-compose", "down"]
  };
}

if (HOSTNAME === "my_vault_server") {
  T(ComposeFullPath("infra_core", "https://github.com/picostack/core"));
  T(Compose("infra_vault"));
  T({
    name: "infra_dns",
    url: GL + "infra_dns",
    up: ["./push"],
    auth: "gitlab"
  });
}

if (HOSTNAME === "some.other.host.com") {
  T(ComposeFullPath("infra_core", "https://github.com/picostack/core"));
  T(ComposeFullPath("svc_samplist", "https://github.com/Southclaws/samp-servers-api"));
  T(ComposeFullPath("svc_cj", "https://github.com/Southclaws/cj"));
  T(ComposeFullPath("svc_pawndex", "https://github.com/Southclaws/pawndex"));
  T(ComposeFullPath("svc_burger", "https://github.com/openmultiplayer/deployment"));
  T(ComposeFullPath("svc_tacobot", "https://github.com/thecodeah/taco-bot"));
  T(Compose("tobyob.art"));
}

// Testing
if (HOSTNAME === "myhostname.com") {
  T({
    name: "tester01",
    url: "https://github.com/Southclaws/gitwatch",
    up: ["go", "build"],
    down: ["echo", "GOODBYE"]
  });
}

This config uses docker-compose to up and down repositories that are used for his deployment. It takes advantage of the hostname to determine which server pulls which repo, this is how pico was deigned to work.

You don't need multiple config files for this, especially if they all use the same executor, but even if they don't you can always create a function that can be used to make the process easier.

any env variables you pass to Pico will be available in your configuration file.

Both Southclaws and I mainly use Discord.

JustMichael#7884 (This is me) Southclaws#1979 (This is obviously Southclaws)

[Southclaws]

This sounds like a simple and easy thing to implement tbh.

I'm strongly against scripts and prefer declarative approaches to problems.

Currently, my config repo has Pico configs but also some other stuff in there and I could imagine more JS files being added at some point for systems like dnscontrol (which is currently in another repo, but I'd like to consolidate).

A possible solution to this could be a regular expression to match filenames. This could allow you to selectively load configuration from directories or from files that follow a certain naming convention such as pico_.+\.js or .+\.pico\.js.

Thanks for your interest in this project too! It's early days but I'm hoping it can be useful!

[Southclaws]

Also, I'm aiming to get a community hub set up soon, I'm working on documentation at https://pico.sh currently and it's mostly just me working on it at the moment with @ADRFranklin helping out with testing and stuff.

[Southclaws]

I've written up the first batch of documentation on https://pico.sh now so if you're still interested in this project, check it out!