[ Feature request ] Connecting code blocks through pipes.

[AlvaroBernalG]

Hello good people of maid,

It would be useful if code blocks could communicate somehow.

Stdin - stdout

In the following pattern the code blocks are connected through the stdin-stdout:

## build 

Builds important stuff 

Snippet 1 

```js
process.stdout.write("hi there from snippet 1");
```

Snippet 2

```js
process.stdin.on('data', console.log); // hi there from snippet 1
```

Next

Another way of allowing communication between code blocks is by injecting a method inside each code block in a similar way of how express.js does it with their middlwares.

## build 

Builds important stuff 

Code block 1 

```js
next('hi there from snippet 1');
```

Code block 2

```js
console.log(maid.result); // hi there from snippet 1
```

This pattern can also be used to control the execution flow:

## build 

Builds important stuff 

Code block 1 

```js
next('hi there from snippet 1');
```

Code block 2

```js
if (true) {
	console.log(maid.result); // hi there from snippet 1
}else {
	next("hi there from snippet 2");
}
```

Code block 3 

```bash
 // This will never get executed 

```

[mfix22]

@AlvaroBernalG for the first snippet why couldn't you just have both of those in the same block of code?

[AlvaroBernalG]

@mfix22 The purpose of the example was to show how two code blocks could communicate.

[egoist]

hmm can you help me understand the use case? I don't want to make it overly complex.

[tunnckoCore]

@egoist yea, i'm curious to hear too.

If we decide to do it, it looks like that we should add some global variables like maid and next to the content when we execute the js code snippet. Which not sounds like a good thing.

[AlvaroBernalG]

@egoist, @olstenlarck

1. Some task are easier to do in some languages than others

There are some task that are easier to do in certain languages. Take for example downloading a config file from a website:

Node.js:

const http = require(‘http”)
http.get({
  hostname: 'mywebsite/config.json',
  port: 80,
  path: '/',
  agent: false  // create a new agent just for this one request
}, (res) => {
  // Do stuff with response
});

Now compare that solution with the simplicity of bash using curl:

curl https://mywebsite.com/config.json > config.json

2. It will be easier to follow UNIX philosphy of doing one thing but doing it good, by allowing the creation of mini task rather than having one bloated code block.

BEGIN =>

// take a css file.

THEN =>

// take the css file and strip out the comments. 

THEN =>

// take the result of stripping out comments and minimize it. 

THEN =>

// Take the result of stripping out comments and minimised and save it into my S3 bucket .

Maybe in the future you just don't want to minimise the css, in that case the only thing that you would have to do is to remove the code block that handles the CSS minimising and everything would work.

[Marak]

@AlvaroBernalG

My opinion is that this is out of scope of what maid should and can do.

I do think your idea is actually very good, but it's going to add layers of complexity which maid shouldn't have to deal with directly.

To give some background, I have already developed a similar system to what you have described ( connecting code snippets through STDIN / STDOUT or as HTTP middlewares ). Mostly we used HTTP, but it's all essentially just generic streaming interfaces.

If you are interested in how we did this, here are some examples including chaining services of multiple languages which may inspire you:

https://github.com/Stackvana/microcule/tree/master/examples https://github.com/Stackvana/microcule/blob/master/examples/express-chain-services.js

I'm glad to offer any input I can. I've been working with this kind of development pattern for a few years and am eager to see more developers thinking about the problem space.

[AlvaroBernalG]

@Marak @egoist I do agree this wouldn't be trivial to add, so I understand if you don't want to push this forward.

@Marak. Awesome stuff, I will definitely have a look later.