#nixos is a poor IRC channel to use for support

[Baughn]

Well... basically as it says? The #nixos channel moves far too quickly; any question gets lost long before anyone who knows anything about Disnix might be able to help. #disnix is free, so maybe use that.

===

Also, my plea for help:

I'm attempting, as a simple test-case, to setup Mediawiki via Disnix. It hasn't gone too well so far; the manual doesn't make it clear what should go in which files, some files are referenced but never defined, and so forth. I nevertheless was able to write something that looks right, but...

See https://brage.info/disnix/

When I run disnix-manifest -s services/services.nix -i production/infrastructure.nix -d production/distribution.nix (or disnix-env), I get the following error:

error: attribute 'outPath' missing, at /nix/store/qalpkiap82fq9r8l60rhwd59r76kyr6j-disnix-0.8/share/disnix/lib.nix:168:39
(use '--show-trace' to show detailed location information)

I have no idea what I'm doing wrong.

[svanderburg]

I have not yet looked into detail in your configuration expressions, but one subtle mistake I noticed is the services/wiki/mediawiki-db.nix expression's header. Since it is a service that has no inter-dependencies, you should remove the empty inner function header.

So you should write:

{stdenv}:

stdenv.mkDerivation {
   ...
}

A service without inter-dependencies is basically the same thing as an "ordinary" Nix package expression.

Can you fix this and give it another try? I can look more into detail later.

[Baughn]

That appears to have solved it.

Odd, though. Why would it be different in this case?

[Baughn]

Naturally, I now have a new problem:

➜  disnix git:(master) ✗ disnix-env -s services/services.nix -i production/infrastructure.nix -d production/distribution.nix      
[coordinator]: Building manifest...
[coordinator]: Distributing intra-dependency closures...
[target: brage.info]: Receiving intra-dependency closure of profile: /nix/store/kpfn34hj6yp09zc7h02m5kg8rw7i4xxl-tsugumi
ERROR: Cannot obtain log directory! Reason: GDBus.Error:org.freedesktop.DBus.Error.AccessDenied: Rejected send message, 5 matched rules; type="method_call", sender=":1.273" (uid=1000 pid=102989 comm="disnix-client --print-invalid /nix/store/f9pp17mzm") interface="org.nixos.disnix.Disnix" member="get_logdir" error name="(unset)" requested_reply="0" destination=":1.37" (uid=0 pid=92358 comm="/nix/store/qalpkiap82fq9r8l60rhwd59r76kyr6j-disnix")
[target: brage.info]: Cannot receive intra-dependency closure of profile: /nix/store/kpfn34hj6yp09zc7h02m5kg8rw7i4xxl-tsugumi

The target machine is running Disnix via NixOS, so it shouldn't be missing dependencies.

[svanderburg]

@Baughn Can you try restarting the Disnix service and dbus daemon, e.g.:

$ systemctl stop disnix-service
$ systemctl stop dbus
$ systemctl start dbus
$ systemctl start disnix-service

and then run it again.

[Baughn]

That got me a little further, but--

➜  disnix git:(master) ✗ ./deploy.sh                                             ~/disnix
[coordinator]: Building manifest...
[coordinator]: Distributing intra-dependency closures...
[target: brage.info]: Receiving intra-dependency closure of profile: /nix/store/kpfn34hj6yp09zc7h02m5kg8rw7i4xxl-tsugumi
tmp.YWiLM1JcUq                                          100%  161MB 110.7MB/s   00:01    
[coordinator]: Acquiring locks...
[target: brage.info]: Acquiring a lock on profile: /nix/store/kpfn34hj6yp09zc7h02m5kg8rw7i4xxl-tsugumi
[coordinator]: Activating new configuration...
[coordinator]: Doing an installation from scratch
[coordinator]: Executing the transition to the new deployment state
[coordinator]: Mapping closures to activate:
key: 387543f41915d177f162d8719f65fc4979356aa0361861fd8719e637fa1fb32f
target: brage.info
container: mysql-database
service: /nix/store/9rfcp1d4xbhmk7sglkdvx8hh7c3qjhna-mediawiki-db
type: mysql-database
dependsOn:
connectsTo:

key: fbb624c740cd75bf4beb9d837f6082636cd9042024d965b281e06517ea8e5320
target: brage.info
container: apache-webapplication
service: /nix/store/6zzdlxzx8zngd4nv0jws6603j1fl3wcw-mediawiki
type: apache-webapplication
dependsOn:
  key: 387543f41915d177f162d8719f65fc4979356aa0361861fd8719e637fa1fb32f
  target: brage.info
  container: mysql-database
connectsTo:

[coordinator]: Executing deactivation of services:
[coordinator]: Executing activation of services:
[target: brage.info]: Activating service with key: 387543f41915d177f162d8719f65fc4979356aa0361861fd8719e637fa1fb32f and package: /nix/store/9rfcp1d4xbhmk7sglkdvx8hh7c3qjhna-mediawiki-db of type: mysql-database in container: mysql-database, arguments: 
activate: /nix/store/9rfcp1d4xbhmk7sglkdvx8hh7c3qjhna-mediawiki-db of type: mysql-database in container: mysql-database with arguments: 
Module: mysql-database cannot be found!
[target: brage.info]: Activation failed of service: 387543f41915d177f162d8719f65fc4979356aa0361861fd8719e637fa1fb32f
[coordinator]: Activation failed! Doing a rollback...
[coordinator]: ERROR: Transition phase execution failed!
[coordinator]: Releasing locks...
[target: brage.info]: Releasing a lock on profile: /nix/store/kpfn34hj6yp09zc7h02m5kg8rw7i4xxl-tsugumi
[coordinator]: The deployment failed! Please inspect the output to diagnose any problems!

I think I'm going to stop trying, though. While an interesting project, I really don't think this is ready for production use -- there isn't enough validation or documentation, and internal errors are a bad experience. I'll be looking forward to checking back on Disnix in a year or two, though.

[svanderburg]

@Bauhn Too bad, but that choice is up to you.

Anyway, if you'd still like to fix this problem, you mentioned that the target machines are NixOS-based, right?

The Dysnomia module for mysql-database can be configured, by enabling MySQL in your target NixOS configuration, e.g.:

{
  services.disnix.enable = true;
  services.mysql.enable = true;
}

The Dysnomia NixOS module (implicitly enabled when Disnix is enabled) is clever enough to deploy the MySQL module when it has been enabled in a NixOS configuration.

[Baughn]

The thing is... I was evaluating Disnix for use as a replacement for Nomad / Kubernetes / et. al. for a relatively simple (but high-volume) website, with multiple administrators.

While I could probably work my way through this, and build a functioning setup, it is unlikely that everyone else could then maintain it. Especially without thorough documentation. Which is a pity; I really wanted this to work.

[Baughn]

Headscratch

Well, I wasn't kidding when I said I wanted this to work. I still do, really, and most of what's missing is simply documentation. I assume there are two problems here -- you don't have time to write it, and you also don't have the right perspective.

There are other areas that should be improved, like the quality of the error messages. In my experience, that probably means not using .nix except where necessary, but that's something to think about later. Good documentation would be enough for now.

So, here's my proposal:

We create a #disnix channel, on Freenode. Or rather, you will have to -- the best I could make would be a ##disnix channel.

Use that for tech support like this, and -- if you're willing to spend the time helping me through all the bumps I'll inevitably hit -- I'll write the documentation this needs, from the perspective of someone unfamiliar with the system, as well as the required reference documentation. Disnix is the right approach, and it'd be a crying shame if it doesn't go anywhere.

[Baughn]

While I do believe I can fix this one by myself, the nature of the error is a little worrying. Bad/missing escapes? Probably shouldn't be using raw SQL for this anyway, but rather the APIs.

[coordinator]: Building manifest...
[coordinator]: Distributing intra-dependency closures...
[target: brage.info]: Receiving intra-dependency closure of profile: /nix/store/kpfn34hj6yp09zc7h02m5kg8rw7i4xxl-tsugumi
[coordinator]: Acquiring locks...
[target: brage.info]: Acquiring a lock on profile: /nix/store/kpfn34hj6yp09zc7h02m5kg8rw7i4xxl-tsugumi
[coordinator]: Activating new configuration...
[coordinator]: Doing an installation from scratch
[coordinator]: Executing the transition to the new deployment state
[coordinator]: Mapping closures to activate:
key: 387543f41915d177f162d8719f65fc4979356aa0361861fd8719e637fa1fb32f
target: brage.info
container: mysql-database
service: /nix/store/9rfcp1d4xbhmk7sglkdvx8hh7c3qjhna-mediawiki-db
type: mysql-database
dependsOn:
connectsTo:

key: fbb624c740cd75bf4beb9d837f6082636cd9042024d965b281e06517ea8e5320
target: brage.info
container: apache-webapplication
service: /nix/store/6zzdlxzx8zngd4nv0jws6603j1fl3wcw-mediawiki
type: apache-webapplication
dependsOn:
  key: 387543f41915d177f162d8719f65fc4979356aa0361861fd8719e637fa1fb32f
  target: brage.info
  container: mysql-database
connectsTo:

[coordinator]: Executing deactivation of services:
[coordinator]: Executing activation of services:
[target: brage.info]: Activating service with key: 387543f41915d177f162d8719f65fc4979356aa0361861fd8719e637fa1fb32f and package: /nix/store/9rfcp1d4xbhmk7sglkdvx8hh7c3qjhna-mediawiki-db of type: mysql-database in container: mysql-database, arguments: 
activate: /nix/store/9rfcp1d4xbhmk7sglkdvx8hh7c3qjhna-mediawiki-db of type: mysql-database in container: mysql-database with arguments: 
ERROR 1064 (42000) at line 1: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '-db' at line 1
[target: brage.info]: Activation failed of service: 387543f41915d177f162d8719f65fc4979356aa0361861fd8719e637fa1fb32f
[coordinator]: Activation failed! Doing a rollback...
[coordinator]: ERROR: Transition phase execution failed!
[coordinator]: Releasing locks...
[target: brage.info]: Releasing a lock on profile: /nix/store/kpfn34hj6yp09zc7h02m5kg8rw7i4xxl-tsugumi
[coordinator]: The deployment failed! Please inspect the output to diagnose any problems!

It would be good if the error included the path of the script whose execution failed.

[Baughn]

While I'm collecting usability niggles, more notes for self now:

• I had to enable both mysql and apache manually. This needs to be documented properly, including the alternatives, for each such service.

• The error mesage if the modules are missing should mention it.

• Running nixos-rebuild does not restart the Disnix service, and therefore doesn't enable the modules without a manual restart of it. Why?

• The expected directory structure for apache-webapp needs to be documented, of course.

I got the application to 'activate', but navigating to http://localhost:3300 gives only a blank page. I wonder how this interacts with Apache to acctivate the webapp? Originally, I was expecting it to run a dedicated copy of Apache, but that's clearly not it.

[Baughn]

While I'm at this: /var/www/mediawiki became a /nix/store path. Which is reasonable, I guess, but how do I deal with mediawiki's desire to write to it?

I'll take a look at the activation scripts...

[jcrben]

fyi, you probably know this, but you can also ask questions at https://discourse.nixos.org/c/learn or https://groups.google.com/forum/#!forum/nix-devel (altho the latter is being deprecated)

[tomberek]

@Baughn Any luck? I've had more luck with raw nixops for both provisioning and updates, but I'm still keeping an eye on dydisnix as a Nix-native version of k8s.

[louwers]

How does disnix compare to k8s feature-wise and stability wise?

[tomberek]

@louwers: There are very few in-production disnix users out there compared to k8s. But I reach for disnix fairly often and am often pleased with the results on small scale needs. I believe it would be possible to expand on these ideas (and adopt ideas from k8s) to build large systems. The various extensions to disnix represent experiments down this path.

features: k8s integrates well with tons of things, disnix has not had all that integration work for network overlays, monitoring services, GUIs, notifications, etc. It's a much strong basis upon which to build these systems, but it's WIP.

stability: I've never had a problem with @svanderburg 's software (bravo!). It's surface area is much less, but has also had far fewer users.

Example from this week: I produce some fake video streams for testing. Team asked for another stream and change some settings. Adding another line or two to the nix config was all it needed. New services up and running in seconds. When it all works, it's like a superpower.

[Baughn]

@tomberek I couldn't make anyone else to try Nix, so we eventually went with just the standard Docker stuff. More's the pity. That being said, I was never able to get the Disnix setup to a state where I could show it off.