Hi,

It will be awesome if renovatebot, could parse Project.toml in Julia Project.

Dependencies in julia could be specified in various ways :

^1.2.3 => [1.2.3, 2.0.0)
^1.2 => [1.2.0, 2.0.0)
^1 => [1.0.0, 2.0.0)
^0.2.3 => [0.2.3, 0.3.0)
^0.0.3 => [0.0.3, 0.0.4)
^0.0 => [0.0.0, 0.1.0)
^0 => [0.0.0, 1.0.0)
~1.2.3 => [1.2.3, 1.3.0)
~1.2 => [1.2.0, 1.3.0)
~1 => [1.0.0, 2.0.0)
~0.2.3 => [0.2.3, 0.3.0)
~0.0.3 => [0.0.3, 0.0.4)
~0.0 => [0.0.0, 0.1.0)
~0 => [0.0.0, 1.0.0)
>= 1.2.3 => [1.2.3, ∞)
≥ 1.2.3 => [1.2.3, ∞)
= 1.2.3 => [1.2.3, 1.2.3]
< 1.2.3 => [0.0.0, 1.2.2]

@codeneomatrix Do I forget anything according julia and deps management

@rarkins You can find a example in https://github.com/the-benchmarker/web-frameworks/blob/e2c82e935f9038fe7a54a10d04654b31eacf8ee8/julia/merly/Project.toml#L4

Regards,

Jun 13 '20 12:06 waghanza

First thing is: do we need a new datasource? Seems like we do, is it https://github.com/JuliaRegistries/General? Example versions list for a random package: https://raw.githubusercontent.com/JuliaRegistries/General/master/G/GaussianProcesses/Versions.toml

After datasource we need to verify versioning. I checked the full list of ranges above and npm's semver implementation actually follows the same results, so looks like we can reuse it. Maybe we alias it in case one day it needs to diverge.

Finally there is the manager. We'd need to extract dependencies from Project.toml files, which should be pretty easy. I don't see any lock file in Julia, so that should be all we need.

Jun 13 '20 14:06 rarkins

Hi.

Starting with Julia version 1.4, it is also possible to use hyphens in the version specification of dependencies.

So it is recomended to also add

[compat]
julia = "1.4"

to the project file when using them.

0.2.3 - 4.5.6 => [0.2.3, 4.5.6]

Any unspecified trailing numbers in the first end-point are considered to be zero:

1.2 - 4.5.6 => [1.2.0, 4.5.6]
1 - 4.5.6 => [1.0.0, 4.5.6]

Any unspecified trailing numbers in the second end-point will be considered to be wildcards:

1.2.3 - 4.5 => 1.2.3 - 4.5.* = [1.2.3, 4.6.0)
1.2.3 - 4 => 1.2.3 - 4.. = [1.2.3, 5.0.0)
1.2 - 4.5 => 1.2.0 - 4.5.* = [1.2.0, 4.6.0)
1.2 - 4 => 1.2.0 - 4.. = [1.2.0, 5.0.0)
1 - 4.5 => 1.0.0 - 4.5.* = [1.0.0, 4.6.0)
1 - 4 => 1.0.0 - 4.. = [1.0.0, 5.0.0)

Jun 13 '20 16:06 neomatrixcode

I underestimated a little. First, the deps section in Project.toml contains a form of checksum, while also these's a Manifest.toml file that needs to be updated by pkg.

The pkg examples on https://docs.julialang.org/en/v1/stdlib/Pkg/index.html also shows that the pkg command is run from within a repl, so I'm not sure how it would be run "headless".

I have concluded that someone more familiar with Julia needs to take over the work, but I've done some of the implementation and most of the scaffolding in https://github.com/renovatebot/renovate/compare/feat/6508-julia

Jun 14 '20 04:06 rarkins

Some things not done yet:

[ ] Verify if hyphen ranges are supported by our existing implementation of getNewValue() in npm semver
[ ] Look up and cache each Julia General source URL
[ ] Work out what the deps value in Project.toml represents. Does not seem to be a sha1
[ ] Determine if users would expect Renovate to update either the Project.toml deps section or Manifest.toml even if the Compat section is unchanged
[ ] Extract implementation for both General and git-based dependencies
[ ] Work out why the example's Manifest.toml is empty
[ ] Work out what pkg commands should be run if we change a Compat value in Project.toml
[ ] Add artifacts logic for updating Manifest.toml

Jun 14 '20 04:06 rarkins

Project.toml The project file describes the project on a high level, for example the package/project dependencies and compatibility constraints are listed in the project file. The file entries are described below.

The name field The name of the package/project is determined by the name field, for example: name = "Example" The name can contain word characters [a-zA-Z0-9_], but can not start with a number.
The uuid field uuid is a string with a universally unique identifier for the package/project, for example: uuid = "7876af07-990d-54b4-ab0e-23690620f79a" The uuid field is mandatory for packages. It is recommended that UUIDs.uuid4() is used to generate random UUIDs.
The version field version is a string with the version number for the package/project. It should consist of three numbers, major version, minor version and patch number, separated with a .
The [deps] section All dependencies of the package/project are listed in the [deps] section. Each dependency is listed as a name-uuid pair, for example:

[deps]
Example = "7876af07-990d-54b4-ab0e-23690620f79a"
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

Typically it is not needed to manually add entries to the [deps] section; this is instead handled by Pkg operations such as add.

The [compat] section Compatibility constraints for the dependencies listed under [deps] can be listed in the [compat] section. Example:

[deps]
Example = "7876af07-990d-54b4-ab0e-23690620f79a"

[compat]
Example = "1.2"

The Compatibility section describes the different possible compatibility constraints in detail. It is also possible to list constraints on julia itself, although julia is not listed as a dependency in the [deps] section:

[compat]
julia = "1.1"

Manifest.toml The manifest file is an absolute record of the state of the packages in the environment. It includes exact information about (direct and indirect) dependencies of the project. Given a Project.toml + Manifest.toml pair, it is possible to instantiate the exact same package environment, which is very useful for reproducibility. For the details, see Pkg.instantiate.

The Manifest.toml file is generated and maintained by Pkg and, in general, this file should never be modified manually.

source https://julialang.github.io/Pkg.jl/v1/toml-files/