rebar3
rebar3 copied to clipboard
erlang
compile the package with attaching external package repo
Environment
I'm trying to compile rebar3 package lager, and it depends on the goldrush package, with the following environment: goldrush and other erlang packages are preinstalled into system's folder: /usr/lib/erlang/lib
$ ls /usr/lib/erlang/lib|grep gold
goldrush-0.1.9
Current behaviour
$ rebar3 compile
===> Verifying dependencies...
===> Fetching goldrush v0.1.9
===> Downloaded package, caching at /home/majioa/.cache/rebar3/hex/hexpm/packages/goldrush-0.1.9.tar
===> Compiling goldrush
===> Compiling lager
Is it expected behaviour, when I run the command with the internet connection but, if I have no internet connection it failed to compile, since it hasn't find the required package goldrush, and can''t download it.
Expected behaviour
Seems that rebar3 searches not the packages in the system folder. Expected behaviour is to compile with picking up the packages which are placed into systems package tree /usr/lib/erlang/lib, and without trying to download them from internet:
$ rebar3 compile
===> Verifying dependencies...
===> Compiling goldrush
===> Compiling lager
So how to compile the package with attaching external package repo?
Rebar3 will always fetch all the dependencies in {deps, ...} to the local project directory. The {deps, ...} option, specifically, tells rebar3 "go fetch these, they are needed for the build".
Libraries in the system directory are not added to the list to be fetched; notice how you never need to specify stdlib, kernel, or common_test in {deps, ...}. Instead, such dependencies are runtime dependencies, and those are specified in your .app.src's {applications, [...]} tuple.
The two sets are related, but disjoint. If you specify a lock file or a config file with {deps, ...}, rebar3 will try to fetch them no matter what. This is unrelated to which apps your app depends on at run-time and can be found in the system directory (handled with the .app file's applications list)
Well,
Rebar3 will always fetch all the dependencies in {deps, ...} to the local project directory.
But could rebar3 fetch them from system lib folder? If yes how to say to rebar, to fetch the pkg from system folder?
If they are already built and you don't want them copied to _build you could just list them in .app.src like any other OTP library and update ERL_LIBS.
Ok, but applying the contest i've specified, i.e. i'm trying to build the lager, with previously build and and installed goldrush. Lager's config contains:
{deps, [
{goldrush, "0.1.9"}
]}.
What should I do for that with .app.src and ERL_LIBS?
You can't. Rebar3 will try to fetch the package it is told to fetch and verify the hashes.
The thing you can do is fetch all of them once and once they're cached on the local filesystem, the cache can be reused.
You can't. Rebar3 will try to fetch the package it is told to fetch and verify the hashes.
The thing you can do is fetch all of them once and once they're cached on the local filesystem, the cache can be reused.
I see, well two additional questions: it seems that by default cache is located in home folder, how could I use it from /var/cache/hex folder, and the the second one: when I just copy the tarball into cache it in;t pickup be rebar3, since the packages.idx hasn't contain the proper info about the cached package, how to update it?
-
you can change the global cache location by setting the
REBAR_CACHE_DIRenvironment variable, which defaults to the values in~/.cache/rebar3/and it will use the new path as the root of its cache. -
the package.idx file is a cache of the content of remote indexes like hex.pm. It is updated by talking to the index, validating signatures, etc.
rebar3 updatewill force rebar3 to go talk to the index about all the packages it has fetched so far and update their definitions. It will however not add a file that is on disk to the file as if it belonged to some index. If you're looking for a way to publish your own private packages without going to a third party, it's possible you'd instead prefer something like minirepo which is a minimalist self-hosted hex package index.
the package.idx file is a cache of the content of remote indexes like hex.pm. It is updated by talking to the index, validating signatures, etc. rebar3 update will force rebar3 to go talk to the index about all the packages it has fetched so far and update their definitions.
When i've started rebar3 update, i've not found the package.idx file regenerated, although there is a single pkg in the cache:
$ find $HOME/.cache
/usr/src/.cache
/usr/src/.cache/rebar3
/usr/src/.cache/rebar3/hex
/usr/src/.cache/rebar3/hex/hexpm
/usr/src/.cache/rebar3/hex/hexpm/packages
/usr/src/.cache/rebar3/hex/hexpm/packages/goldrush-0.1.9.tar
/usr/src/.cache/rebar3/hex/hexpm/packages/goldrush-0.1.9.etag
2a. Is there some other hex package index like minirepo? seems it is not working....
No, that's pretty much what exists right now. I can't really say why you don't have the idx file there. Are you on the latest rebar3?
No, that's pretty much what exists right now. I can't really say why you don't have the idx file there. Are you on the latest rebar3?
Yes, but in may case I expect that the rebar3 will regenerate the file based on the *tar and *etag files found in the hex cache. That is all of what I can do for picking up the packages on internet free system... (
Yes, but your expectations don't match what the tool actually does.
I'm surprised no idx file is there, since we use it to search and validate the cached packages actually match what the index says should be there without conflict (i.e. hex packages are mutable for their first hour of existence, so we may have to invalidate cache at some point, or mark some things as retired)
I'm surprised no idx file is there
There is no idx file is because I've removed it before. To try regenerate it
Anyway, is there a way to force rebar3 to pick up the packages what lay in the cache without a cache file existance?
as mentioned earlier, no there isn't. You may however want to use something like vendoring if this is a total deal-breaker.
Well, i'm using the mini-repo, but the publishing commands lead to failure:
majioa@bve-vm15:~/git/goldrush$ DEBUG=1 rebar3 hex user register
===> Load global config file /home/majioa/.config/rebar3/rebar.config
===> 21.3.6 satisfies the requirement for minimum OTP version 18
===> Expanded command sequence to be run: []
===> Provider: {default,do}
===> Expanded command sequence to be run: [{hex,user}]
===> Provider: {hex,user}
===> Uncaught error in rebar_core. Run with DEBUG=1 to see stacktrace or consult rebar3.crashdump
===> Uncaught error: {badmatch,{error,{required,repo}}}
===> Stack trace to the error location:
[{rebar3_hex_user,do,1,
[{file,"/home/majioa/.cache/rebar3/plugins/rebar3_hex/src/rebar3_hex_user.erl"},
{line,45}]},
{rebar_core,do,2,
[{file,"/usr/src/RPM/BUILD/rebar-3.12.0/src/rebar_core.erl"},
{line,154}]},
{rebar_prv_do,do_task,5,
[{file,"/usr/src/RPM/BUILD/rebar-3.12.0/src/rebar_prv_do.erl"},
{line,87}]},
{rebar_core,do,2,
[{file,"/usr/src/RPM/BUILD/rebar-3.12.0/src/rebar_core.erl"},
{line,154}]},
{rebar3,run_aux,2,
[{file,"/usr/src/RPM/BUILD/rebar-3.12.0/src/rebar3.erl"},{line,182}]},
{rebar3,main,1,
[{file,"/usr/src/RPM/BUILD/rebar-3.12.0/src/rebar3.erl"},{line,66}]},
{escript,run,2,[{file,"escript.erl"},{line,758}]},
{escript,start,1,[{file,"escript.erl"},{line,277}]}]
===> When submitting a bug report, please include the output of `rebar3 report "your command"`
majioa@bve-vm15:~/git/goldrush$ DEBUG=1 rebar3 report
===> Load global config file /home/majioa/.config/rebar3/rebar.config
===> 21.3.6 satisfies the requirement for minimum OTP version 18
===> Expanded command sequence to be run: [{default,report}]
===> Provider: {default,report}
Rebar3 report
version 3.12.0
generated at 2019-10-29T08:49:30+00:00
=================
Please submit this along with your issue at https://github.com/erlang/rebar3/issues (and feel free to edit out private information, if any)
-----------------
Task:
Entered as:
-----------------
Operating System: x86_64-alt-linux-gnu
ERTS: Erlang/OTP 21 [erts-10.3.4] [source] [64-bit] [smp:2:2] [ds:2:2:10] [async-threads:1] [hipe]
Root Directory: /usr/lib/erlang
Library directory: /usr/lib/erlang/lib
-----------------
Loaded Applications:
bbmustache: 1.6.1
certifi: 2.5.1
cf: 0.2.2
common_test: 1.17.1
compiler: 7.3.2
crypto: 4.4.2
cth_readable: 1.4.5
dialyzer: 3.3.2
edoc: 0.10
erlware_commons: 1.3.1
eunit: 2.3.7
eunit_formatters: 0.5.0
getopt: 1.0.1
hipe: 3.18.3
inets: 7.0.7
kernel: 6.3.1
providers: 1.8.1
public_key: 1.6.5
relx: 3.33.0
sasl: 3.3
snmp: 5.2.12
ssl_verify_fun: 1.1.5
stdlib: 3.8.1
syntax_tools: 2.1.7
tools: 3.1
-----------------
Escript path: /usr/bin/rebar3
Providers:
app_discovery as clean compile compile cover ct cut deps dialyzer do docs edoc escriptize eunit get-deps help install install_deps key list lock new owner path pkgs publish release relup repo report repos retire revert search shell state tar tree unlock update upgrade upgrade upgrade user version xref
(...)
===> Uncaught error: {badmatch,{error,{required,repo}}}(...)
In https://github.com/wojtekmach/mini_repo#usage-with-rebar3 we have:
Add the dependency to the project's rebar.config:
{erl_opts, [debug_info]}.
{deps, [
{baz, "0.1.0"}
]}.
so maybe the guide is wrong and the dep should be: {baz, "0.1.0", {repo, <<"test_repo">>}} or something? I remember testing with Rebar when releasing mini_repo and it did work so not 100% sure, I'll try to reproduce later.
so maybe the guide is wrong and the dep should be: {baz, "0.1.0", {repo, <<"test_repo">>}} or something?
The same thing....
That's a possibility. @starbelly might have a bit more info since I know he's been hands deep in cleaning up the hex plugin for rebar3.
Hello, one yet thing, am trying to get a tar ball of the hex, but am failed:
$ rebar3 release --lib-dir=_build/default/lib/
===> Verifying dependencies...
===> Compiling goldrush
===> Starting relx build process ...
===> Resolving OTP Applications from directories:
/usr/lib/erlang/lib
===> No releases have been specified in the system!
So how to correctly generate the tar ball, specifically for this hex.
Also i've tried to publish repo insolated env, and got:
$ rebar3 hex publish -r test_repo
===> Plugin rebar3_hex not available. It will not be used.
===> Command hex not found
while in open environement it is ok. And I haven't installed any additional packages line rebar3_hex manually. Why?
First, a rebar3 release command will assemble a release, but to get a tarball of the release, you need to call rebar3 tar. However, this is a tarball for a fully bootable Erlang/OTP system and has nothing to do with the tarball that hex uses.
To get that one, you must indeed call rebar3 hex publish. The error you get there is related to an error in trying to run the plugin. You can call DEBUG=1 rebar3 hex publish to get the stacktrace (along with a lot of other debug info) that lead to rebar3_hex not being usable.
In my isolated env here is hex output:
$ DEBUG=1 rebar3 hex publish
===> Command hex not found
So, I've tried to list plugins with the provided config, but it fails, and the question where is the downloaded and installed is resided after compilation...?
$ DEBUG=1 rebar3 plugins list
===> Load global config file /usr/src/.config/rebar3/rebar.config
===> hex_repo:get_package failed for package <<"rebar3_hex">>: {badmatch,
{error,
{failed_connect,
[{to_address,
{"localhost",
8642}},
{inet,
[inet],
econnrefused}]}}}
===> hex_repo:get_package failed for package <<"rebar3_hex">>: {badmatch,
{error,
{failed_connect,
[{to_address,
{"repo.hex.pm",
443}},
{inet,
[inet],
nxdomain}]}}}
===> throw {error,
{rebar_app_utils,
{missing_package,<<"rebar3_hex">>,undefined}}} [{rebar_app_utils,
update_source,
3,
[{file,
"/usr/src/RPM/BUILD/rebar-3.12.0/src/rebar_app_utils.erl"},
{line,
302}]},
{rebar_prv_install_deps,
'-handle_profile_level/7-lc$^0/1-0-',
3,
[{file,
"/usr/src/RPM/BUILD/rebar-3.12.0/src/rebar_prv_install_deps.erl"},
{line,
178}]},
{rebar_prv_install_deps,
handle_profile_level,
7,
[{file,
"/usr/src/RPM/BUILD/rebar-3.12.0/src/rebar_prv_install_deps.erl"},
{line,
179}]},
{rebar_plugins,
handle_plugin,
4,
[{file,
"/usr/src/RPM/BUILD/rebar-3.12.0/src/rebar_plugins.erl"},
{line,
111}]},
{rebar_plugins,
'-handle_plugins/4-fun-0-',
4,
[{file,
"/usr/src/RPM/BUILD/rebar-3.12.0/src/rebar_plugins.erl"},
{line,
100}]},
{lists,
foldl,
3,
[{file,
"lists.erl"},
{line,
1263}]},
{rebar_plugins,
handle_plugins,
4,
[{file,
"/usr/src/RPM/BUILD/rebar-3.12.0/src/rebar_plugins.erl"},
{line,
99}]},
{rebar3,
state_from_global_config,
2,
[{file,
"/usr/src/RPM/BUILD/rebar-3.12.0/src/rebar3.erl"},
{line,
416}]}]
===> Plugin rebar3_hex not available. It will not be used.
===> Expanded command sequence to be run: []
===> Provider: {default,do}
===> Expanded command sequence to be run: [{plugins,list}]
===> Provider: {plugins,list}
--- Global plugins ---
===> Unable to find plugin rebar3_hex
Well you can see the two errors: it tries to ping both localhost:8642 (econnrefused) and repo.hex.pm:443 (nxdomain), and can't reach out to them.
This has stopped the rebar3_hex plugin from being downloaded, which later prevents any use of that plugin, since it's not downloaded.
Hi, curretly I have the following:
$ DEBUG=1 rebar3 hex publish
===> Load global config file /usr/src/.config/rebar3/rebar.config
===> Hex get_package request failed: {ok,
{404,
#{<<"cache-control">> =>
<<"max-age=0, private, must-revalidate">>,
<<"content-length">> =>
<<"9">>,
<<"date">> =>
<<"Fri, 15 Nov 2019 10:10:50 GMT">>,
<<"server">> => <<"Cowboy">>},
<<"not found">>}}
===> Failed to update package rebar3_hex from repo build_repo
===> hex_repo:get_package failed for package <<"rebar3_hex">>: {badmatch,
{error,
{failed_connect,
[{to_address,
{"repo.hex.pm",
443}},
{inet,
[inet],
nxdomain}]}}}
I've run mini repo, and have presented the rebar3_hex in the /usr/lib/erlang-lib/:
$ find /usr/lib/erlang/lib/rebar3_hex-6.8.0/
/usr/lib/erlang/lib/rebar3_hex-6.8.0/
/usr/lib/erlang/lib/rebar3_hex-6.8.0/ebin
/usr/lib/erlang/lib/rebar3_hex-6.8.0/ebin/rebar3_hex_user.beam
/usr/lib/erlang/lib/rebar3_hex-6.8.0/ebin/rebar3_hex_search.beam
/usr/lib/erlang/lib/rebar3_hex-6.8.0/ebin/rebar3_hex_revert.beam
/usr/lib/erlang/lib/rebar3_hex-6.8.0/ebin/rebar3_hex_retire.beam
/usr/lib/erlang/lib/rebar3_hex-6.8.0/ebin/rebar3_hex_results.beam
/usr/lib/erlang/lib/rebar3_hex-6.8.0/ebin/rebar3_hex_repo.beam
/usr/lib/erlang/lib/rebar3_hex-6.8.0/ebin/rebar3_hex_publish.beam
/usr/lib/erlang/lib/rebar3_hex-6.8.0/ebin/rebar3_hex_owner.beam
/usr/lib/erlang/lib/rebar3_hex-6.8.0/ebin/rebar3_hex_key.beam
/usr/lib/erlang/lib/rebar3_hex-6.8.0/ebin/rebar3_hex_io.beam
/usr/lib/erlang/lib/rebar3_hex-6.8.0/ebin/rebar3_hex_file.beam
/usr/lib/erlang/lib/rebar3_hex-6.8.0/ebin/rebar3_hex_error.beam
/usr/lib/erlang/lib/rebar3_hex-6.8.0/ebin/rebar3_hex_docs.beam
/usr/lib/erlang/lib/rebar3_hex-6.8.0/ebin/rebar3_hex_cut.beam
/usr/lib/erlang/lib/rebar3_hex-6.8.0/ebin/rebar3_hex_config.beam
/usr/lib/erlang/lib/rebar3_hex-6.8.0/ebin/rebar3_hex_client.beam
/usr/lib/erlang/lib/rebar3_hex-6.8.0/ebin/rebar3_hex.beam
/usr/lib/erlang/lib/rebar3_hex-6.8.0/ebin/rebar3_hex.app
But it seems it is not picked up the hex... So how to force it to be picked up from there?
It's trying to publish to regular hex by default it seems.
you might want to call rebar3 hex publish -r <repo> to publish to your instance of minirepo.
No, it seems that it does not find the rebar3_hex module, since along with the previous message it shops the follownig:
===> Plugin rebar3_hex not available. It will not be used.
===> Command hex not found
Ah yeah, I guess the rebar3_hex plugin needs to be available to publish, and that one is only published in a public repo. It's different from fetching packages from your own registry.
Ok, now it picks up the rebar3_hex plugin, but anyway shall not found the hex command
$ rebar3 hex publish -r test_repo
===> Command hex not found
why?
Please include the DEBUG=1 output; by the looks of it, it doesn't find the 'hex' namespace for the commands, so something else is still wrong.
BUt last time you called it you had:
$ DEBUG=1 rebar3 hex publish
===> Load global config file /usr/src/.config/rebar3/rebar.config
...
Where are you trying to define the plugin now?
I'm a bit dumbfounded here because it appears you're trying to even set up the original mirror on a node that has no network access. That won't be possible. You'll need to at least set up the mirror first in a connected mode and publish your content there by fetching stuff from the public internet, and once that's done, you can go offline. But what you're doing here appears to be trying to set up the mirror while being offline, and there's no way that can work.
But which packages and files should be mirrored when being online?
Currently I have the same think even with .cache...
$ DEBUG=1 rebar3 hex publish -r build_repo
===> Load global config file /usr/src/.config/rebar3/rebar.config
===> Hex get_package request failed: {ok,
{404,
#{<<"cache-control">> =>
<<"max-age=0, private, must-revalidate">>,
<<"content-length">> =>
<<"9">>,
<<"date">> =>
<<"Mon, 25 Nov 2019 12:13:35 GMT">>,
<<"server">> => <<"Cowboy">>},
<<"not found">>}}
===> Failed to update package rebar3_hex from repo build_repo
===> hex_repo:get_package failed for package <<"rebar3_hex">>: {badmatch,
{error,
{failed_connect,
[{to_address,
{"repo.hex.pm",
443}},
{inet,
[inet],
nxdomain}]}}}
===> throw {error,
{rebar_app_utils,
{missing_package,<<"rebar3_hex">>,undefined}}} [{rebar_app_utils,
update_source,
3,
[{file,
"/usr/src/RPM/BUILD/rebar-standalone-3.12.0/src/rebar_app_utils.erl"},
{line,
302}]},
{rebar_prv_install_deps,
'-handle_profile_level/7-lc$^0/1-0-',
3,
[{file,
"/usr/src/RPM/BUILD/rebar-standalone-3.12.0/src/rebar_prv_install_deps.erl"},
{line,
178}]},
{rebar_prv_install_deps,
handle_profile_level,
7,
[{file,
"/usr/src/RPM/BUILD/rebar-standalone-3.12.0/src/rebar_prv_install_deps.erl"},
{line,
179}]},
{rebar_plugins,
handle_plugin,
4,
[{file,
"/usr/src/RPM/BUILD/rebar-standalone-3.12.0/src/rebar_plugins.erl"},
{line,
111}]},
{rebar_plugins,
'-handle_plugins/4-fun-0-',
4,
[{file,
"/usr/src/RPM/BUILD/rebar-standalone-3.12.0/src/rebar_plugins.erl"},
{line,
100}]},
{lists,
foldl,
3,
[{file,
"lists.erl"},
{line,
1263}]},
{rebar_plugins,
handle_plugins,
4,
[{file,
"/usr/src/RPM/BUILD/rebar-standalone-3.12.0/src/rebar_plugins.erl"},
{line,
99}]},
{rebar3,
state_from_global_config,
2,
[{file,
"/usr/src/RPM/BUILD/rebar-standalone-3.12.0/src/rebar3.erl"},
{line,
416}]}]
===> Plugin rebar3_hex not available. It will not be used.
===> Command hex not found
All of them. You can't mirror a thing while offline. You must be online to build the mirror. Otherwise you can't copy any single package from the online archive to the offline archive.
You go online, you download all the packages, and then you insert them in the offline package manager for publishing. Your offline package manager can then respond to offline responses and serve the right packages.
And please look at your error messages, it's exactly the same one you had before:
{error, {failed_connect,
[{to_address,
{"repo.hex.pm", 443}},
{inet, [inet], nxdomain}]}}}
The nxdomain error is an error you tend to get when a DNS lookup fails. If you try with dig or drill you can see how it resolves:
$ dig repo.hex.pm +short
dualstack.f2.shared.global.fastly.net.
151.101.138.2
The domain lookup may fail because you're still offline for a package? I would assume the mirror hasn't been built yet since you're still trying to publish to it.
If you're trying to override the default package manager, maybe this PR will help? https://github.com/erlang/rebar3/pull/2182
There's obviously something not going right at one of the steps you're attempting, but I'm not able to figure it out from just the discussion details posted in here.
You go online, you download all the packages,
What does the procedure of downloading consist in?
I would assume the mirror hasn't been built yet since you're still trying to publish to it.
I dont need full mirror, i need the only packages by deps
and then you insert them in the offline package manager for publishing
how to do it?
If you're trying to override the default package manager, maybe this PR will help? #2182
What do you mean? Im trying to build the packages in isolated environment, with internet access, so the way to do so, is to simulate hex-repo locally. No problems to replace or have one proper system config, but it should not have link to hex.pm
PS: Q. but why the rebar cant pick up required packages from cache?
The cache can be used to pick up packages and skip the downloads, but only if the repo index is on disk. I've done it multiple times, and I've just done it now on my machine to make sure I wasn't imagining it. Somehow that does not work for you. That's the first thing I would have done in your shoes, it works and I know this because I just built rebar3 with itself while offline using nothing but the previously fetched cache. It has taken 2 minutes to set up: making the build run once while online and keeping the cache around.
So you dismissed this solution that currently works. I've then said multiple times that the way I'd try it otherwise would be to first be online and populate your mirror (with the deps that you need only), then only fetch from the mirror when building offline, and there's no sign that this is actually what you're trying to do, so I can't even rule out that this method does not work before trying to dig into deeper issues.
You get failure messages telling you you can't actually send DNS queries to the public package repo, which tends to mean that you are trying to do something without all the packages being available offline. The package that fails is rebar3_hex, the plugin itself that is needed to publish packages or manage repos. So you need to have this one package either in the cache or in your offline mirror to make things work, because you won't be able to use that package if you don't have it somewhere on your machine while offline. That is what the problem seems to be right now; you are trying to use a plugin you have now downloaded while offline.
I don't know what to tell you man, you're not providing us with details about the environment you're doing things in nor the config you have, and when you do it seems you're just disregarding the advice I'm giving you. It's been a month of back and forth on this issue. I can only repeat to you the things I have already said until you try them.
To be explicit, try this:
- build your codebase while you have access to the internet, using only hex packages (no git dependencies)
- turn the wi-fi off or disconnect the ethernet cable from your computer
- remove the _build directory
- build again
This should work without a problem as long as you have your lock file and you haven't removed the cache directory in ~/.cache/rebar3.
This is the ground state you need to make this work. Once this works, come back to me and we can maybe find ways to streamline the process, but please, please take the time to check that this base state functions before trying to optimize things by removing an index or moving files around. If this base state does not work there is some major debugging to do because it's a very basic part of how dependency fetching should function, and I have tested that it works on many of my own computers.
This should work without a problem as long as you have your lock file and you haven't removed the cache directory in ~/.cache/rebar3.
I've had the ~/.cache/rebar3 presented specifically for rebar3_hex package, but it and its deps was not picked up .... as it is shewn above... Ok i see the cache index shall be previously properly generated... well is any way to generate it after cache is formed?
And one yer question, how to properly generate the tarball for the hex package?
I've had the ~/.cache/rebar3 presented specifically for rebar3_hex package, but it and its deps was not picked up .... as it is shewn above...
Have you tried following the exact 4 steps I have outlined in my previous response, yes or no? This is me trying to get you to establish a baseline so that we know things are working as they should. If you do not try to follow these exact 4 steps and report back before trying other optimizations or step, I won't help you anymore.
Ok i see the cache index shall be previously properly generated...
Is it though? You haven't reported on that, and you keep saying it does not work.
well is any way to generate it after cache is formed?
The local cache index is created by rebar3 to track the subset of the remote index (or mirror) it downloaded from, and to validate that the cache it has on disk in ~/.cache/rebar3 is up to date. You aren't expected to create that file by hand, since it relies on stuff like public key signatures and hashes from the packages and remote indexes to form itself. If you really wish to build it by hand, you probably can, but it's going to be more work than just fetching the thing, or creating a local mirror of hex packages you need (which in turn will re-create that cache and cache index).
And one yer question, how to properly generate the tarball for the hex package?
The tarball for the hex package is created by the tool trying to publish it. If you want to do it by hand, you can (there's a spec with a reference implementation), but you are going to be reimplementing parts of your own index and publishing, since that's exactly what we do in rebar3. That's why I recommended you publish to a private mirror if you want to create custom tarballs -- you'll need to create the tarball and push it to an index (the mirror) if you want rebar3 to auto-generate the local index files.
If you want to do it all by hand, be my guest, but I can't offer my time to support that.
@ferd, I was curious about the offline part of the build process (I saw a reference to this in another Issue, that was recently open, but can't find it now) so I tested your procedure - above - (using lager as a dep.).
- build your codebase while you have access to the internet, using only hex packages (no git dependencies)
- turn the wi-fi off or disconnect the ethernet cable from your computer
- remove the _build directory
- build again
This works perfectly with a few warnings (which are OK, for what they refer to)...
> rebar3 do upgrade, compile
===> Verifying dependencies...
===> Fetching lager v3.8.0
===> Download error, using cached file at ~/.cache/rebar3/hex/hexpm/packages/lager-3.8.0.tar
===> Fetching goldrush v0.1.9
===> Download error, using cached file at ~/.cache/rebar3/hex/hexpm/packages/goldrush-0.1.9.tar
===> Failed to update package lager from repo hexpm
===> Failed to fetch updates for package lager from repo hexpm
===> No upgrade needed for lager
===> No upgrade needed for goldrush
===> Verifying dependencies...
===> Analyzing applications...
===> Compiling goldrush
===> Compiling lager
===> Analyzing applications...
===> Compiling example
>
I guess this can be closed.
