Permit processing of a non-NPM registry package

[calumapplepie]

I havent gotten a chance to test this yet, and I am new-ish to python: but here is my attempt at #150

[calumapplepie]

Any other problems I should work out?

[nileshpatra]

@TheMageKing thanks a lot for your work on this :) Please check the changes as requested. And additionally please test the PR after you add-in your changes again

[calumapplepie]

@nileshpatra Thanks for the feedback! I pushed the changes you requested, and I am going to test them out now.

[calumapplepie]

I just finished testing and debugging it with the tiny-test package (which is a test framework in its own right, I think, but was the first result for 'node .js tiny test package) The results were identical (according to diff) between the version produced by downloading from the registry and the one made by using that tarball with --no-registry. The file "node-tinytest_1.1.5-1.debian.tar.xz" was a bit different on each run, but the extracted version was identical, so I'll take it. I haven't checked to see if the extract_tarball code will work with a tarball NOT downloaded from the npm registry, but it should(tm)

[calumapplepie]

@nileshpatra @pravi can you take another look at this? I tested it, and it worked like a charm.

[nileshpatra]

Apologies for the delay, will do so in sometime

[nileshpatra]

@TheMageKing extremely sorry for such a delayed response. I'll review timely now. I looked at your changes, but I'm not sure what command did you issue to run it?

[nileshpatra]

It'd also be great if you could paste in the logs of some sample inputs and outputs

PS: I'll be wayy quicker to see your changes this time. Apologies again for making you wait for so long :-(

[calumapplepie]

I'll get to work on it.

Yes, I am responding this quickly just to make you feel more guilty lol

[calumapplepie]

I needed to seek deep into my history to find this

Okay, so when I was developing, I used the tiny-test npm package, since that was the first result for "small npm test package".
After downloading the tarball with npm pack tiny-test, this command is what I use to run it: npm2deb create --no-registry tiny-test-0.11.6.tgz The output is pretty long, since this is actually a pretty awful test package, but its the same as if I run 'npm2deb create tiny-test`(except for timestamps, of course). I actually compare the resulting folders, and there isn't a difference. So that's pretty neat.

I'll respond to your comments above, but the point is that it works right now, which is pretty cool.

[nileshpatra]

Great, Thanks! :-)

[nileshpatra]

Pasting the log outputs for upstream review:

$ npm2deb create --no-registry uniq-1.0.1.tgz
 debug [0] - Please provide an upstream github/gitlab URL for debian/watch

Downloading source tarball file using debian/watch file...
uscan: Newest version of node-uniq on remote site is 1.0.1, specified download version is 1.0.1
Successfully symlinked ../uniq-1.0.1.tgz to ../node-uniq_1.0.1.orig.tar.gz.

Creating debian source package...
uupdate: debian/source/format is "3.0 (quilt)".
uupdate: Auto-generating node-uniq_1.0.1-1.debian.tar.xz
uupdate: -> Use existing node-uniq_1.0.1-1.debian.tar.xz
dpkg-source: info: extracting node-uniq in node-uniq-1.0.1
dpkg-source: info: unpacking node-uniq_1.0.1.orig.tar.gz
dpkg-source: info: unpacking node-uniq_1.0.1-1.debian.tar.xz
uupdate: Remember: Your current directory is changed back to the old source tree!
uupdate: Do a "cd ../node-uniq-1.0.1" to see the new source tree and
    edit it to be nice Debianized source.

Building the binary package
dpkg-source: info: using source format '3.0 (quilt)'
dpkg-source: info: building node-uniq using existing ./node-uniq_1.0.1.orig.tar.gz
dpkg-source: info: building node-uniq in node-uniq_1.0.1-1.debian.tar.xz
dpkg-source: info: building node-uniq in node-uniq_1.0.1-1.dsc
dpkg-buildpackage: info: source package node-uniq
dpkg-buildpackage: info: source version 1.0.1-1
dpkg-buildpackage: info: source distribution UNRELEASED
dpkg-buildpackage: info: source changed by Nilesh Patra <[email protected]>
dpkg-buildpackage: info: host architecture amd64
 dpkg-source --before-build .
 fakeroot debian/rules clean
dh clean --with nodejs
   dh_auto_clean --buildsystem=nodejs
	rm -rf ./node_modules/.cache
Use of uninitialized value $package in concatenation (.) or string at /usr/share/perl5/Debian/Debhelper/Buildsystem/nodejs.pm line 943.
   dh_clean
 dpkg-source -b .
dpkg-source: info: using source format '3.0 (quilt)'
dpkg-source: info: building node-uniq using existing ./node-uniq_1.0.1.orig.tar.gz
dpkg-source: info: building node-uniq in node-uniq_1.0.1-1.debian.tar.xz
dpkg-source: info: building node-uniq in node-uniq_1.0.1-1.dsc
 debian/rules build
dh build --with nodejs
   dh_update_autotools_config
   dh_autoreconf
   dh_auto_configure --buildsystem=nodejs
   dh_auto_build --buildsystem=nodejs
No build command found, searching known files
   dh_auto_test --buildsystem=nodejs
	mkdir -p node_modules
	ln -s ../. node_modules/uniq
	/usr/bin/node -e require\(\"./.\"\)
Removing node_modules/uniq
   create-stamp debian/debhelper-build-stamp
 fakeroot debian/rules binary
dh binary --with nodejs
   dh_testroot
   dh_prep
   dh_auto_install --buildsystem=nodejs
No "files" field in ./package.json, install all files
	mkdir -p /home/nilesh/ups/npm2deb/tmp/uniq/node-uniq-1.0.1/debian/node-uniq//usr/share/nodejs/uniq/
	cp --reflink=auto -a ./uniq.js /home/nilesh/ups/npm2deb/tmp/uniq/node-uniq-1.0.1/debian/node-uniq//usr/share/nodejs/uniq//
	cp --reflink=auto -a ./package.json /home/nilesh/ups/npm2deb/tmp/uniq/node-uniq-1.0.1/debian/node-uniq//usr/share/nodejs/uniq//
Set ${nodejs:Version} to 12.18.4~dfsg
   dh_installdocs
   dh_installchangelogs
   dh_perl
   dh_link
   dh_strip_nondeterminism
   dh_compress
   dh_fixperms
   dh_missing
   dh_installdeb
   dh_gencontrol
dpkg-gencontrol: warning: package node-uniq: substitution variable ${nodejs:Version} unused, but is defined
   dh_md5sums
   dh_builddeb
dpkg-deb: building package 'node-uniq' in '../node-uniq_1.0.1-1_all.deb'.
 dpkg-genbuildinfo
 dpkg-genchanges  >../node-uniq_1.0.1-1_amd64.changes
dpkg-genchanges: info: including full source code in upload
 dpkg-source --after-build .
dpkg-buildpackage: info: full upload (original source is included)
dpkg-buildpackage: warning: not signing UNRELEASED build; use --force-sign to override
dh clean --with nodejs
   dh_auto_clean --buildsystem=nodejs
	rm -rf ./node_modules/.cache
Use of uninitialized value $package in concatenation (.) or string at /usr/share/perl5/Debian/Debhelper/Buildsystem/nodejs.pm line 943.
   dh_clean
/usr/bin/gbp
gbp:info: No git repository found, creating one.
gbp:info: Version '1.0.1-1' imported under '/home/nilesh/ups/npm2deb/tmp/uniq/node-uniq'

Remember, your new source directory is uniq/node-uniq-1.0.1

This is not a crystal ball, so please take a look at auto-generated files.

You may want fix first these issues:

/bin/grep: uniq/node-uniq-1.0.1/debian/*: No such file or directory
uniq/node-uniq_itp.mail:Subject: ITP: node-uniq -- FIX_ME write the Debian package description
uniq/node-uniq_itp.mail:  Description     : FIX_ME write the Debian package description
uniq/node-uniq_itp.mail: FIX_ME: This ITP report is not ready for submission, until you are
uniq/node-uniq_itp.mail:FIX_ME: Explain why this package is suitable for adding to Debian. Is
uniq/node-uniq_itp.mail:FIX_ME: Explain how you intend to consistently maintain this package

*** Warning ***
Using npmregistry to download npm dist tarballs, because upstream
git repo is missing tags. Its better to ask upstream to tag their releases
instead of using npm dist tarballs as dist tarballs may contain pre built files
and may not include tests.

$ ls uniq 
node-uniq  node-uniq_1.0.1-1_all.deb  node-uniq_1.0.1-1_amd64.buildinfo  node-uniq_1.0.1-1_amd64.changes  node-uniq_1.0.1-1.debian.tar.xz  node-uniq_1.0.1-1.dsc  node-uniq_1.0.1.orig.tar.gz  node-uniq_itp.mail  uniq-1.0.1.tgz
$ ls uniq/node-uniq
debian  LICENSE  package.json  README.md  test  uniq.js

[nileshpatra]

@shanavas786 could you please review + merge? Pasted the logs above for reference

[shanavas786]

from log:

/bin/grep: uniq/node-uniq-1.0.1/debian/*: No such file or directory

Doesn't look normal. Are we missing something ? @nileshpatra can you check the behavior when using registry ?

Also I would prefer --from-tarball instead of --no-registry.

[nileshpatra]

CC: @TheMageKing check the comment above ^^

[calumapplepie]

@shanavas786 I couldn't reproduce that grep failure. In fact, there's a bunch going on in Nilesh's logs that, now that I look at it, I don't have. My copy of npm2deb (installed from source) doesn't run any package build, either when --no-registry is passed or when run normally. Is there something I'm doing wrong?

Fair enough for the --from-tarball suggestion: I'll push a commit to fix that in a bit.