narula
go get fails
➜ ~ go get github.com/narula/ddtxn/...
package github.com/narula/ddtxn/...
imports github.com/narula/ddtxn
imports ddtxn/dlog: unrecognized import path "ddtxn/dlog"
package github.com/narula/ddtxn/...
imports github.com/narula/ddtxn
imports ddtxn/spinlock: unrecognized import path "ddtxn/spinlock"
package github.com/narula/ddtxn/...
imports github.com/narula/ddtxn
imports ddtxn/wfmutex: unrecognized import path "ddtxn/wfmutex"
package github.com/narula/ddtxn/...
imports github.com/narula/ddtxn
imports gotomic: unrecognized import path "gotomic"
package github.com/narula/ddtxn/...
imports github.com/narula/ddtxn/apps
imports ddtxn: unrecognized import path "ddtxn"
package github.com/narula/ddtxn/...
imports github.com/narula/ddtxn/benchmarks
imports ddtxn/apps: unrecognized import path "ddtxn/apps"
package github.com/narula/ddtxn/...
imports github.com/narula/ddtxn/benchmarks
imports ddtxn/prof: unrecognized import path "ddtxn/prof"
➜ ~ go version
go version go1.4 linux/amd64
The above error is caused because the imports are referring to "ddtxn/prof" etc. instead of the absolute path starting from the github url.
Even after fixing that, the go get command fails because of the following "redeclared" errors due to multiple files in the same directory sharing the same variable names.
➜ ddtxn git:(master) ✗ go get github.com/narula/ddtxn/...
# github.com/narula/ddtxn/benchmarks
benchmarks/big.go:18: nprocs redeclared in this block
previous declaration at benchmarks/bid.go:21
benchmarks/big.go:19: nsec redeclared in this block
previous declaration at benchmarks/bid.go:22
benchmarks/big.go:20: clientGoRoutines redeclared in this block
previous declaration at benchmarks/bid.go:23
benchmarks/big.go:21: nworkers redeclared in this block
previous declaration at benchmarks/bid.go:24
benchmarks/big.go:22: doValidate redeclared in this block
previous declaration at benchmarks/bid.go:25
benchmarks/big.go:24: contention redeclared in this block
previous declaration at benchmarks/bid.go:27
benchmarks/big.go:25: nbidders redeclared in this block
previous declaration at benchmarks/bid.go:28
benchmarks/big.go:26: readrate redeclared in this block
previous declaration at benchmarks/bid.go:29
benchmarks/big.go:27: notcontended_readrate redeclared in this block
previous declaration at benchmarks/bid.go:30
benchmarks/big.go:29: dataFile redeclared in this block
previous declaration at benchmarks/bid.go:31
benchmarks/big.go:29: too many errors
I have fixed both of them. Patch is to follow shortly.
I'm traveling till 1/21. Will get to it after that. On Jan 12, 2015 12:05 PM, "Sankar சங்கர்" [email protected] wrote:
Is this project no longer actively maintained ?
— Reply to this email directly or view it on GitHub https://github.com/narula/ddtxn/issues/3#issuecomment-69533827.
No worries. Thanks for the interest! On Jan 12, 2015 2:43 PM, "Sankar சங்கர்" [email protected] wrote:
ah okay. Sorry for the noise.
— Reply to this email directly or view it on GitHub https://github.com/narula/ddtxn/issues/3#issuecomment-69543669.
go get github.com/narula/ddtxn/ works. But when we do a go install ./single from the benchmarks folder it complains of a missing prof package.
So I did a go get github.com/narula/ddtxn/... to install all the subdirs and their dependencies too. But that lead to a build error:
➜ narula go get github.com/narula/ddtxn/...
# github.com/narula/ddtxn/benchmarks/time-single
ddtxn/benchmarks/time-single/time-single.go:197: assignment count mismatch: 3 = 7
# github.com/narula/ddtxn/benchmarks/bid
ddtxn/benchmarks/bid/bid.go:195: assignment count mismatch: 3 = 7
# github.com/narula/ddtxn/benchmarks/big
ddtxn/benchmarks/big/big.go:61: cannot use coord.Workers[0].E (type ddtxn.ETransaction) as type *ddtxn.ETransaction in argument to big_app.Populate:
*ddtxn.ETransaction is pointer to interface, not interface
ddtxn/benchmarks/big/big.go:82: undefined: apps.Latency
ddtxn/benchmarks/big/big.go:94: undefined: apps.Latency
ddtxn/benchmarks/big/big.go:95: big_app.Time undefined (type *apps.Big has no field or method Time)
ddtxn/benchmarks/big/big.go:115: assignment count mismatch: 3 = 7
ddtxn/benchmarks/big/big.go:132: big_app.LatencyString undefined (type *apps.Big has no field or method LatencyString)
If we give all the four go get statements in the project homepage, then the go install ./single is successful (but it fails at runtime due to a different error (missing list-cpus) which I am trying to debug to understand if it is any issue at my end).
However, the issue of "redeclared" seem to be fixed.
Pull and try again. I left some old packages around and I hadn't updated all the benchmarks to use the new stats code. Also make sure you do:
go get github.com/narula/prof go get github.com/narula/wfmutex go get -u github.com/narula/ddtxn
Thanks!
On Thu, Feb 12, 2015 at 11:48 PM, Sankar சங்கர் [email protected] wrote:
go get github.com/narula/ddtxn/ works. But when we do a go install ./single from the benchmarks folder it complains of a missing prof package.
So I did a go get github.com/narula/ddtxn/... to install all the subdirs and their dependencies too. But that lead to a build error:
➜ narula go get github.com/narula/ddtxn/...
github.com/narula/ddtxn/benchmarks/time-single
ddtxn/benchmarks/time-single/time-single.go:197 http://github.com/narula/ddtxn/benchmarks/time-singleddtxn/benchmarks/time-single/time-single.go:197: assignment count mismatch: 3 = 7
github.com/narula/ddtxn/benchmarks/bid
ddtxn/benchmarks/bid/bid.go:195 http://github.com/narula/ddtxn/benchmarks/bidddtxn/benchmarks/bid/bid.go:195: assignment count mismatch: 3 = 7
github.com/narula/ddtxn/benchmarks/big
ddtxn/benchmarks/big/big.go:61 http://github.com/narula/ddtxn/benchmarks/bigddtxn/benchmarks/big/big.go:61: cannot use coord.Workers[0].E (type ddtxn.ETransaction) as type *ddtxn.ETransaction in argument to big_app.Populate: *ddtxn.ETransaction is pointer to interface, not interface ddtxn/benchmarks/big/big.go:82: undefined: apps.Latency ddtxn/benchmarks/big/big.go:94: undefined: apps.Latency ddtxn/benchmarks/big/big.go:95: big_app.Time undefined (type *apps.Big has no field or method Time) ddtxn/benchmarks/big/big.go:115: assignment count mismatch: 3 = 7 ddtxn/benchmarks/big/big.go:132: big_app.LatencyString undefined (type *apps.Big has no field or method LatencyString)
If we give all the four go get statements in the project homepage, then the go install ./single is successful (but it fails at runtime due to a different error (missing list-cpus) which I am trying to debug to understand if it is any issue at my end).
However, the issue of "redeclared" seem to be fixed.
— Reply to this email directly or view it on GitHub https://github.com/narula/ddtxn/issues/3#issuecomment-74204889.
There still seem to be a failure.
➜ narula go get -u github.com/narula/ddtxn/...
# github.com/narula/ddtxn/benchmarks/big
ddtxn/benchmarks/big/big.go:61: cannot use &coord.Workers[0].E (type *ddtxn.ETransaction) as type ddtxn.ETransaction in argument to big_app.Populate:
*ddtxn.ETransaction is pointer to interface, not interface
➜ narula
If we do at the top-level, it will succeed though as the subdirs will not be installed.
There is another issue related to a missing list-cpus command. That leads to a runtime failure. Should I report that also ? Where do I get this from ? The log for that in case you want:
➜ benchmarks git:(f0330d3) go install ./single
➜ benchmarks git:(f0330d3) python bm.py --exp=single --rlock --ncores=3
GOGC=off numactl -C `list-cpus seq -n 3 ` $GOPATH/bin/single -nprocs=3 -ngo=3 -nw=3 -nsec=10 -contention=0 -rr=0 -allocate=False -sys=0 -rlock=False -wr=2.0 -phase=20 -sr=500 -atomic=False -zipf=-1 -out=data.out -ncrr=0.0 -cw=2.00 -rw=0.50 -split=False -latency=False -v=0 -partition=False
Bad status 256 sh: list-cpus: command not found
libnuma: Warning: unparseable cpu description `/home/psankar/go/bin/single'
usage: numactl [--all | -a] [--interleave= | -i <nodes>] [--preferred= | -p <node>]
[--physcpubind= | -C <cpus>] [--cpunodebind= | -N <nodes>]
[--membind= | -m <nodes>] [--localalloc | -l] command args ...
numactl [--show | -s]
numactl [--hardware | -H]
numactl [--length | -l <length>] [--offset | -o <offset>] [--shmmode | -M <shmmode>]
[--strict | -t]
[--shmid | -I <id>] --shm | -S <shmkeyfile>
[--shmid | -I <id>] --file | -f <tmpfsfile>
[--huge | -u] [--touch | -T]
memory policy | --dump | -d | --dump-nodes | -D
memory policy is --interleave | -i, --preferred | -p, --membind | -m, --localalloc | -l
<nodes> is a comma delimited list of node numbers or A-B ranges or all.
Instead of a number a node can also be:
netdev:DEV the node connected to network device DEV
file:PATH the node the block device of path is connected to
ip:HOST the node of the network device host routes through
block:PATH the node of block device path
pci:[seg:]bus:dev[:func] The node of a PCI device
<cpus> is a comma delimited list of cpu numbers or A-B ranges or all
all ranges can be inverted with !
all numbers and ranges can be made cpuset-relative with +
the old --cpubind argument is deprecated.
use --cpunodebind or --physcpubind instead
<length> can have g (GB), m (MB) or k (KB) suffixes
Aha. I think I fixed that bug.
I added list-cpus to a github repo. Get this:
git clone [email protected]:narula/list-cpus.git
And add it to your path.
Thank you for bearing with me!
On Fri, Feb 13, 2015 at 11:08 AM, Sankar சங்கர் [email protected] wrote:
There still seem to be a failure.
➜ narula go get -u github.com/narula/ddtxn/...
github.com/narula/ddtxn/benchmarks/big
ddtxn/benchmarks/big/big.go:61: cannot use &coord.Workers[0].E (type *ddtxn.ETransaction) as type ddtxn.ETransaction in argument to big_app.Populate: *ddtxn.ETransaction is pointer to interface, not interface ➜ narula
If we do at the top-level, it will succeed though as the subdirs will not be installed.
There is another issue related to a missing list-cpus command. That leads to a runtime failure. Should I report that also ? Where do I get this from ? The log for that in case you want:
➜ benchmarks git:(f0330d3) go install ./single ➜ benchmarks git:(f0330d3) python bm.py --exp=single --rlock --ncores=3 GOGC=off numactl -C
list-cpus seq -n 3$GOPATH/bin/single -nprocs=3 -ngo=3 -nw=3 -nsec=10 -contention=0 -rr=0 -allocate=False -sys=0 -rlock=False -wr=2.0 -phase=20 -sr=500 -atomic=False -zipf=-1 -out=data.out -ncrr=0.0 -cw=2.00 -rw=0.50 -split=False -latency=False -v=0 -partition=False Bad status 256 sh: list-cpus: command not found libnuma: Warning: unparseable cpu description `/home/psankar/go/bin/single'usage: numactl [--all | -a] [--interleave= | -i
] [--preferred= | -p ] [--physcpubind= | -C ] [--cpunodebind= | -N ] [--membind= | -m ] [--localalloc | -l] command args ... numactl [--show | -s] numactl [--hardware | -H] numactl [--length | -l ] [--offset | -o ] [--shmmode | -M ] [--strict | -t] [--shmid | -I ] --shm | -S [--shmid | -I ] --file | -f [--huge | -u] [--touch | -T] memory policy | --dump | -d | --dump-nodes | -D memory policy is --interleave | -i, --preferred | -p, --membind | -m, --localalloc | -l
is a comma delimited list of node numbers or A-B ranges or all. Instead of a number a node can also be: netdev:DEV the node connected to network device DEV file:PATH the node the block device of path is connected to ip:HOST the node of the network device host routes through block:PATH the node of block device path pci:[seg:]bus:dev[:func] The node of a PCI device is a comma delimited list of cpu numbers or A-B ranges or all all ranges can be inverted with ! all numbers and ranges can be made cpuset-relative with + the old --cpubind argument is deprecated. use --cpunodebind or --physcpubind instead can have g (GB), m (MB) or k (KB) suffixes — Reply to this email directly or view it on GitHub.
Adding the contents of that git repository fixed the runtime error. May be it is a good idea to document it in the README or even better package these also in this same git project ?
Also, do you think it could be a good idea to move the individual benchmarks to the individual folders so that a single go get -u <narula/ddtxn>/... will install all the benchmarks ?
I should indeed thank you as it is not very easy to work on interesting projects for newbies :)
I added the list-cpus stuff to the README. It might prove useful for another project so I left it in a different repo.
What do you mean when you say move the individual benchmarks to individual folders? You mean so that bid, big, buy, rubis, single, and time-single are in the top-level directory instead of /benchmarks?
On Fri, Feb 13, 2015 at 12:02 PM, Sankar சங்கர் [email protected] wrote:
Adding the contents of that git repository fixed the runtime error. May be it is a good idea to document it in the README or even better package these also in this same git project ?
Also, do you think it could be a good idea to move the individual benchmarks to the individual folders so that a single go get -u <narula/ddtxn>/... will install all the benchmarks ?
I should indeed thank you as it is not very easy to work on interesting projects for newbies :)
— Reply to this email directly or view it on GitHub https://github.com/narula/ddtxn/issues/3#issuecomment-74287394.
What do you mean when you say move the individual benchmarks to individual folders?
Sorry. Ignore my comment. I was somehow confused and thought that the big, bid, simple etc. are files in benchmarks folder at one-level and not in their own individual subfolders. I was wrong. Everything is fine now. go get -u /... also works perfectly. Thanks. The issue can be closed.
https://github.com/jteeuwen/go-bindata helps to package non-go files also via go get it seems. I have not tried it yet but heard about it from another issue in another project https://github.com/etsy/Hound/issues/7
I will see if I could use the go-bindata to package the python files (list-cpu etc.).