licensed

licensed copied to clipboard

Hello, So from listed sources, it seems that licensed is not able to handle deno dependencies currently, is this something that could be added?

@lowlighter I've not heard of deno until now 😄 . Can you provide any examples of, or links to documentation for,

1. how to enumerate which dependencies a project has (is there a CLI like deno list or similar?
2. how to determine where on disk a dependency is stored?
   • if dependencies are never installed locally, how to load a dependencies contents from a url?

but generally yes, assuming there is either a CLI or a known file format or local storage structure for used to describe dependencies then deno support can be added!

I'd be happy to assist you if you want to develop a dependency enumerator for deno 😄

Yeah, it's a typescript runtime which is fairly recent (2018), but it does have some traction (github.com/denoland/deno), mostly because it was created by the authors of nodejs 🙂

---

1. Modules are imported through urls (a bit like Go), and they can be listed with deno info <app_entry_point>:

deno info src/app.ts
emit: .../deno_template/src/app.ts.js
dependencies: 3 unique (total 32.46KB)

file:///.../deno_template/src/app.ts (199B)
└─┬ https://deno.land/[email protected]/testing/asserts.ts (15.54KB)
  ├── https://deno.land/[email protected]/fmt/colors.ts (11.26KB)
  └── https://deno.land/[email protected]/testing/_diff.ts (5.46KB)

There's a deno info --unstable --json command which outputs a json instead with additional data, which is probably more exploitable:

{
  "root": "file:///.../deno_template/src/app.ts",
  "modules": [
    {
      "specifier": "https://deno.land/[email protected]/testing/asserts.ts",
      "dependencies": [
        {
          "specifier": "../fmt/colors.ts",
          "isDynamic": false,
          "code": "https://deno.land/[email protected]/fmt/colors.ts"
        },
        {
          "specifier": "./_diff.ts",
          "isDynamic": false,
          "code": "https://deno.land/[email protected]/testing/_diff.ts"
        }
      ],
      "size": 15918,
      "mediaType": "TypeScript",
      "local": ".../AppData/Local/deno/deps/https/deno.land/688e058fb547030f059b02f08b6bc2f36f15d56e6edb3bce6b48fdf563fb7ae9",
      "checksum": "83889f9a37bdab16cb68b023a15f9172ceb644f62c0e727c72d4870a666e53d6",
      "emit": ".../AppData/Local/deno/gen/https/deno.land/688e058fb547030f059b02f08b6bc2f36f15d56e6edb3bce6b48fdf563fb7ae9.js"
    }
    ...
  ],
  "size": 33238
}

2. The local path of the json output above points toward the cached version of the file, which also have a .metadata.json file, but there's nothing about the license there (it mostly contains stuff about http headers, hashs, urls, etc.).

Most of the time, a license can be retrieved by using extracting the package root url and appending LICENSE (e.g. https://deno.land/[email protected]/testing/asserts.ts -> https://deno.land/[email protected]/LICENSE).

But the tricky part is to trim the modules subpaths (e.g. /testing/asserts.ts in the example above). From what I've seen, a package is always versionned and contains a @ before version number, so it could be based on that.

I've opened https://github.com/denoland/deno/issues/9786#issue-831168370 to ask if something more reliable could be done to get LICENSE path, but not sure if it'll eventually get implemented.

---

So I think it could be possible to integrate in licensed, but I guess the tricky parts are:

• Cached modules does not contains license metadata, so it need to be fetched (it requires net access)
• Package "root" directory must be located, and LICENSE file inferred from it, so not very cool in case file was named LICENSE.md for example

Currently I've made a small TypeScript snippet (which is in the issue listed above) and I'm running licensee directly on fetched license text and it seems to be pretty accurate.

I think if this get implemented, it could be based on the sources/go.rb, but not 100% sure.

Let me know if you think this could fit 🙂

But the tricky part is to trim the modules subpaths (e.g. /testing/asserts.ts in the example above). From what I've seen, a package is always versionned and contains a @ before version number, so it could be based on that.

Yeah this is likely a technical detail that would need to be kept up to date with any changes from deno. If they are able to update what gets output in deno info, or provide another solution that lets you get the full package contents that would be ideal

If you get https://deno.land/[email protected], does that download the entirety of the package, including any license files?

Cached modules does not contains license metadata, so it need to be fetched (it requires net access)

We have a few other sources that have this requirement as well. Given this is a general necessity of using these dependencies, it's not too much of a concern that net access is required for the dependency source enumerator as well.

Package "root" directory must be located, and LICENSE file inferred from it, so not very cool in case file was named LICENSE.md for example

No problem here! licensed is used to gather up all the locations that a license file might be and then passes off to licensee, which I believe already has the logic to look over common license file names. Similar to the go source as you m mentioned, if the files were located locally on disk this would be as simple as providing both a path and a search_root to a dependency, which would instruct licensee to search for license contents in all directories between the two locations.

I think if this get implemented, it could be based on the sources/go.rb, but not 100% sure.

Yeah, the CLI and dependency structure patterns align. If there's a need to download files, then the nuget or gradle sources could be looked at. They have a pattern already to download contents for dependencies in a custom Dependency subclass.

If you get https://deno.land/[email protected], does that download the entirety of the package, including any license files?

No, from what I know it only downloads the required files (i.e. only code files you'll actually use) so meta files are always trimmed and it doesn't seem possible to get the whole package 😕

Similar to the go source as you m mentioned, if the files were located locally on disk this would be as simple as providing both a path and a search_root to a dependency

Since dependencies are not stored entirely locally I guess it makes the search more difficult. Licenses could be fetched on multiple urls using the same logic you said about using common license file names, but it may spam the server if there are actually no matches I guess.

So not sure if in current state it would actually lead to anything clean 😅

Since dependencies are not stored entirely locally I guess it makes the search more difficult. Licenses could be fetched on multiple urls using the same logic you said about using common license file names, but it may spam the server if there are actually no matches I guess.

So not sure if in current state it would actually lead to anything clean 😅

Ah yeah I see what you mean. Is this a critical need for you? If not, can we put this on hold pending the conversation at https://github.com/denoland/deno/issues/9786#issue-831168370?

No it's not critical, if needed you can close this issue and it could be reopened later when there are more ways to achieve it without requiring to hacky stuff 🙂

Thanks for your insights !