Dependancy Manager / Import From GitHub

[LamboCreeper]

Whilst probably not a priority, I believe Dictu would benefit from some sort of dependency manager. My main gripe with using Dictu right now is the lack of this sort of system meaning I have to manually download any dependencies I would like to use.

An idea that came into my head whilst mentioning this issue I have with Dictu to Jason earlier was that it would be quite cool to be able to import directly from a GitHub (or any git-based source control) repository, for example:

import "github.com/lambocreeper/some-dependancy.git" as SomeDependancy;

Dictu would then validate this is a valid Dictu project dependency and if so, build and install it so that it could be used within the project. I think this would be a better solution than having a large dependency database such as NPM or Pip.

Overall just a thought, but would like other peoples opinions!

[briandowns]

I have a very simple package manager implemented but doesn't support Github at the moment. The current implementation is a subcommand rooted off of the main binary that has install and uninstall operation. These operations work off of the assumption that there's a hidden directory in the user's home directory e.g. /home/bdowns/.dictu/pkg/structured_logger with a .du file(s).

The interpreter is augmented to look in the stdlib first for imports and then in the above mentioned user directory.

If there's any interest in getting a base implementation started, I can get a PR started to eventually extend to support Github.

[Jason2605]

I have a very simple package manager implemented but doesn't support Github at the moment. The current implementation is a subcommand rooted off of the main binary that has install and uninstall operation. These operations work off of the assumption that there's a hidden directory in the user's home directory e.g. /home/bdowns/.dictu/pkg/structured_logger with a .du file(s).

The interpreter is augmented to look in the stdlib first for imports and then in the above mentioned user directory.

If there's any interest in getting a base implementation started, I can get a PR started to eventually extend to support Github.

Interesting, where does the install command install packages from?

I'd be very interested in thoughts on package managers, it's something that Dictu is definitely missing!

[briandowns]

I have a very simple package manager implemented but doesn't support Github at the moment. The current implementation is a subcommand rooted off of the main binary that has install and uninstall operation. These operations work off of the assumption that there's a hidden directory in the user's home directory e.g. /home/bdowns/.dictu/pkg/structured_logger with a .du file(s). The interpreter is augmented to look in the stdlib first for imports and then in the above mentioned user directory. If there's any interest in getting a base implementation started, I can get a PR started to eventually extend to support Github.

Interesting, where does the install command install packages from?

It is all based around .tar archives.

./dictu install structured_logger.tar

I'd be very interested in thoughts on package managers, it's something that Dictu is definitely missing!

I have a silly script to create the modules.

#!/bin/sh

if [ -z $1 ]; then
    echo "error: package name required"
    exit 1
fi

if [ -z $2 ]; then
    echo "error: package directory path required"
    exit 1
fi

MOD_NAME=$1
MOD_DIR=$2
DICTU_MOD="${MOD_NAME}.tar"

tar -cf "${DICTU_MOD}" "${MOD_DIR}"

exit 0;

Then the interpreter looks in the stdlib or $HOME/.dictu/pkg/structured_logger

import time;

import "logger.du" as logger;

let log = logger.Log();
log.info("starting application");
time.sleep(2);
log.info("shutting down application");

In the case above, there's a logger.du file at $HOME/.dictu/pkg/structured_logger/.

[briandowns]

I've been giving this more thought and was hoping I might get some feedback on. I was curious everyone's thoughts around package structure, required files, manifests., etc. I'd love to keep working on this.

[Jason2605]

Yeah a package manager is something that is definitely required at some stage if we ever plan to progress further with Dictu. I think there are still a lot of unknowns in my mind about the best way of dealing with things, for example, how do we deal with version collisions.

In terms of the requirement files, i'm happy to go with something incredibly simple even like Pythons "requirements.txt" which is simply = format and build from there! I feel getting something started, no matter how simple, is of more use than just talking about complex implementations.

Package structure, I think should be on a per-project basis rather than a global implementation like Pythons pip, I've seen far too many issues with that sort of setup, and you end up with workarounds like venvs. Simply checking for a directory within the project if it's a path-import (import "path" - strings) should hopefully work well enough

Be great if you did work on this, and I'm more than happy to hear your ideas, I'm pretty flexible on this one in terms of implementation

[briandowns]

Dictu Package Management

Abstract

Dictu is growing to the point that the user base needs a means of creating resusable code and an easy way of sharing it. Each Dictu file is a module and can be easily used in code by using the import or from keywords.

This proposal outlines 2 changes. The first is to Dictu, providing the ability for it to find code referenced modules. The second is updating the interpreter to manage modules or write a new utility. The utility will serve as a package manager with its primary responsibiities being the install, updating, and removing of Dictu modules.

Conventions

Module

From a package management perspective, I propose we think of a module as a collection of 1 or more files organized in a single top-level directory. The top level module directory can contain any number of files or subdirectories with files that make up the overall module.

Format

1. A directory containing the relevant files.
2. Or a tgz file with the relevant files.
3. Or a ...

Contents

Each module should contain at least 2 files. The first being the code to be imported and the second file containing information about the module. Example below.

{
    "name": "slog",
    "version": "v0.1.0",
    "license": "bsd2clause",
    "maintainers": {
        "name": "Brian Downs",
        "email": "[email protected]"
    },
    "dependencies": []
}

Storage

3rd party modules can be installed to 1 of 2 locations. The Dictu interpreter will need to be updated to look in these locations to use the imported code.

Global

For Dictu modules installed for all users to import, they'll need to exist in /usr/local/lib/dictu.

User

For Dictu modules installed by user, the location Dictu would need to search could be ${HOME}/.dictu.

Module Management

I've come up with two options. The first is to extend the Dictu interpreter to manage external/3rd part modules. The second is an external utility.

A new set of commands can be added to the Dictu.

Examples:

dictu  <module_name>

Install a module from Github.

dictu install github.com/briandowns/kahless

Dictum

Dictum is a command-line utility written in Dictu to manage modules on your system. A preview can be found here.

[briandowns]

Unofficial Dictu Module Manager:

dictum

• Supports creating modules
• Installing, removing
• Installing from Github

Interpreter has been updated to look in dictu_modules for user supplied modules.