openVirus
openVirus copied to clipboard
petermr
For getpapers:- "Dockerfile: cannot find the file specified."
I am trying to install getpapers via Docker in Windows 10 Pro. I have installed Docker desktop successfully.
For installing getpapers, I have created a Dockerfile with respective commands as below:
Then in command prompt, I gave the command docker build -t get_paper . but got the following output:
unable to prepare context: unable to evaluate symlinks in Dockerfile path: CreateFile C:\Users\laksm\OneDrive\Desktop\Dockerfile: The system cannot find the file specified.
I tried the command docker build -t get_paper -f Dockerfile . but got the same output:
unable to prepare context: unable to evaluate symlinks in Dockerfile path: CreateFile C:\Users\laksm\OneDrive\Desktop\Dockerfile: The system cannot find the file specified.
I created the Dockerfile in desktop and executed the command there.
Is there anything else should be done before using Dockerfile like switching to the Windows containers..? Kindly guide me.
It looks like you named your Dockerfile, paper_getter. It needs to be named Dockerfile or else have the extension .Dockerfile. The . at the end of the command is telling docker to look for the file called Dockerfile in the current directory
Now, I created the Dockerfile again in desktop and renamed the file as Dockerfile
and tried the above commands docker build -t get_paper . and docker build -t get_paper -f Dockerfile . but got the same output
unable to prepare context: unable to evaluate symlinks in Dockerfile path: CreateFile C:\Users\laksm\OneDrive\Desktop\Dockerfile: The system cannot find the file specified.
You don't need to write the full path. If your in the same repo you can just use . and Docker will search for the file called Dockerfile. So you would only need:
docker build -t paper_getter .
To build the docker image
I tried only the command docker build -t paper_getter . but got the same output
unable to prepare context: unable to evaluate symlinks in Dockerfile path: CreateFile C:\Users\laksm\OneDrive\Desktop\Dockerfile: The system cannot find the file specified.
Does 'if in the same repo' mean the same directory(Desktop) or the cloned openVirus repo?
Can you try using the modified version of the get_papers repo?
https://github.com/bauhuasbadguy/getpapers
The project structure should be:
| |- Dockerfile |- run.sh | -- results
You then build your image by running the Dockerfile using the command
docker build -t paper_getter .
In the same folder as the Dockerfile. This will build the image and tag it as paper_getter. This seems to be the part you are having trouble with.
Once this is solved the next part is to run the image in a container.
###################################################
To run the image paper_getter we use the command:-
docker run -it paper_getter
In the example shell script I have added a few things to this docker run command.
First is the flag --rm. This will remove the container once it has done its job,
Second is the volume mount -v $(pwd)/xml_results:/xml_results. This connects the folder xml_results outside the container to the folder xml_results inside the container. Allowing the results to be stored.
Finally I have added the command to be run by the get_papers container. In the example this is
getpapers -p -x -o /xml_results --query 'chloroquine'
The final run command being
docker run -it --rm --name get-papers \
-v $(pwd)/xml_results:/xml_results \
paper_getter \
getpapers -p -x -o /xml_results --query 'chloroquine'
Thank you for the explanation. I can understand the process now.
Yes, as you said I'm having problem in building the dockerfile. I have no idea what should I do to get the output. Should I switch to Windows containers or anything else?
I'm not sure how docker for windows differs from the linux version. It was my understanding that docker for windows ran in a shell.
Are you using the command line version of docker? Did you get the hello-world example to work?
Yes, I'm using the command line to run the commands.
Yes, the hello-world example worked. This was the output gained.
C:\Users\laksm\OneDrive\Desktop>docker pull hello-world
Using default tag: latest
latest: Pulling from library/hello-world
0e03bdcc26d7: Pull complete
Digest: sha256:4cf9c47f86df71d48364001ede3a4fcd85ae80ce02ebad74156906caff5378bc
Status: Downloaded newer image for hello-world:latest
docker.io/library/hello-world:latest
C:\Users\laksm\OneDrive\Desktop>docker run hello-world
Hello from Docker!
This message shows that your installation appears to be working correctly.
To generate this message, Docker took the following steps:
1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
(amd64)
3. The Docker daemon created a new container from that image which runs the
executable that produces the output you are currently reading.
4. The Docker daemon streamed that output to the Docker client, which sent it
to your terminal.
To try something more ambitious, you can run an Ubuntu container with:
$ docker run -it ubuntu bash
Share images, automate workflows, and more with a free Docker ID:
https://hub.docker.com/
For more examples and ideas, visit:
https://docs.docker.com/get-started/
Just checking you have capitalized the D in dockerfile? Can you send me the result of ls in the directory you are building in?
I'm in Windows, so I used the command dir instead of ls. The output of the directory:
C:\Users\laksm\OneDrive\Desktop>dir
Volume in drive C is OS
Volume Serial Number is 845F-351F
Directory of C:\Users\laksm\OneDrive\Desktop
09/30/2020 06:38 PM <DIR> .
09/30/2020 06:38 PM <DIR> ..
08/24/2020 11:51 AM <DIR> class
09/30/2020 06:38 PM 2,148 Docker Desktop.lnk
09/22/2020 10:32 AM 72 Dockerfile.txt
09/29/2020 03:44 PM 808 GE_lab_exp2b.txt
10/01/2020 03:53 PM <DIR> github
08/06/2020 10:38 PM 1,450 Microsoft Edge.lnk
09/25/2020 12:36 PM 2,368 Microsoft Teams.lnk
09/15/2020 10:57 AM 1,442 Modifying enzyme for rDNA technique.txt
08/30/2020 04:46 PM 2,471 Personal - Edge.lnk
09/12/2020 08:26 AM 2,209 Slack.lnk
09/25/2020 10:35 PM <DIR> SRF reports
08/19/2020 11:10 PM 554 step21.txt
08/06/2020 10:55 PM 727 Windows 10 Update Assistant.lnk
10 File(s) 14,249 bytes
5 Dir(s) 123,297,308,672 bytes free
- Is your dockerfile called
Dockerfile.txt? It should be simply calledDockerfilewith no extension. - I would do all this stuff in a dedicated folder to keep everything separate.
- You should be able to use
lsif your using the docker client since the docker client creates a Linux VM
- I created a
Dockerfileusing notepad and I'm not sure if its name is with the extension asDockerfile.txt. Please check the properties of theDockerfile.
-
I created a separate folder named
dockerto keep the docker-related works separate. -
I was not able to use
lscommand
C:\Users\laksm\OneDrive\Desktop\docker>ls
'ls' is not recognized as an internal or external command,
operable program or batch file.
Does this mean I'm not using the docker client?
Since you created the file in notepad windows has added the .txt file extension. I'm not sure how to remove this since I've not worked with windows 10 but it should be possible and it should be as simple as renaming the file.
As to the docker client issue, I'm not sure anymore. When I tried using docker for windows (windows 8) my docker client ran with powershell but that might have changed in subsequent versions.
I would have a separate folder for each of my projects. You seem to be treating docker as a specific program when its more of a general tool for (once you get past the teething issues) making your life easier
Thank you. I referred a similar issue to remove .txt extension and so downloaded notepad ++ . Then I created a Dockerfile using it.
Then, in command prompt, I gave the command docker build -t paper_getter .. I got the output
Sending build context to Docker daemon 2.048kB
Step 1/3 : FROM node:slim
slim: Pulling from library/node
abb454610128: Pull complete
3dfc5a66c517: Pull complete
33b46fd32a07: Pull complete
d040c2a42822: Pull complete
a56d75ef4840: Pull complete
Digest: sha256:784f4398dcd0fc6afa19f2d426d98ea07f8a51e5aa495d3b191b7784c37e6bd0
Status: Downloaded newer image for node:slim
---> 855a1cd7a580
Step 2/3 : WORKDIR /usr/src/app
---> Running in d05354a36809
Removing intermediate container d05354a36809
---> 54f272683334
Step 3/3 : RUN npm install --global getpapers
---> Running in 6b56056d1677
npm WARN deprecated [email protected]: request has been deprecated, see https://github.com/request/request/issues/3142
npm WARN deprecated [email protected]: Use uuid module instead
npm WARN deprecated [email protected]: ReDoS vulnerability parsing Set-Cookie https://nodesecurity.io/advisories/130
npm WARN deprecated [email protected]: this library is no longer supported
npm WARN deprecated [email protected]: This module moved to @hapi/hawk. Please make sure to switch over as this distribution is no longer supported and may contain bugs and critical security issues.
npm WARN deprecated [email protected]: This version has been deprecated in accordance with the hapi support policy (hapi.im/support). Please upgrade to the latest version to get the best features, bug fixes, and security patches. If you are unable to upgrade at this time, paid support is available for older versions (hapi.im/commercial).
npm WARN deprecated [email protected]: This module moved to @hapi/sntp. Please make sure to switch over as this distribution is no longer supported and may contain bugs and critical security issues.
npm WARN deprecated [email protected]: This version has been deprecated in accordance with the hapi support policy (hapi.im/support). Please upgrade to the latest version to get the best features, bug fixes, and security patches. If you are unable to upgrade at this time, paid support is available for older versions (hapi.im/commercial).
npm WARN deprecated [email protected]: This version has been deprecated in accordance with the hapi support policy (hapi.im/support). Please upgrade to the latest version to get the best features, bug fixes, and security patches. If you are unable to upgrade at this time, paid support is available for older versions (hapi.im/commercial).
npm WARN deprecated [email protected]: request has been deprecated, see https://github.com/request/request/issues/3142
npm WARN deprecated [email protected]: this library is no longer supported
/usr/local/bin/getpapers -> /usr/local/lib/node_modules/getpapers/bin/getpapers.js
+ [email protected]
added 204 packages from 192 contributors in 20.097s
Removing intermediate container 6b56056d1677
---> 941a461729f5
Successfully built 941a461729f5
Successfully tagged paper_getter:latest
SECURITY WARNING: You are building a Docker image from Windows against a non-Windows Docker host. All files and directories added to build context will have '-rwxr-xr-x' permissions. It is recommended to double check and reset permissions for sensitive files and directories.
Is this the supposed output? Should the SECURITY WARNING be considered?
I'd leave it for now. It means that the get papers JavaScript library is using outdated tools but its only a problem if you're using those tools for a public facing server
Okay! Thank you so much. So ,far now, I have installed getpapers successfully via docker but I'm not sure how to run getpapers using the command. I tried to run the command in command prompt but \ breaks the lines and the shows output as: (I changed the dir name docker to getpapers)
C:\Users\laksm\OneDrive\Desktop\getpapers>docker run -it --rm --name get-papers \
docker: invalid reference format.
See 'docker run --help'.
C:\Users\laksm\OneDrive\Desktop\getpapers> -v $(pwd)/xml_results:/xml_results \
'-v' is not recognized as an internal or external command,
operable program or batch file.
C:\Users\laksm\OneDrive\Desktop\getpapers> paper_getter \
'paper_getter' is not recognized as an internal or external command,
operable program or batch file.
C:\Users\laksm\OneDrive\Desktop\getpapers>getpapers -p -x -o /xml_results --query 'chloroquine'
Can you please elaborate the steps?
Ok, so you can't split the lines up. They are all a part of the same command I split over several lines in order to make it more readable. I have tried to break down how the command works in this post (after the break)
The project structure should be:
| |- Dockerfile |- run.sh | -- results
You then build your image by running the Dockerfile using the command
docker build -t paper_getter .In the same folder as the Dockerfile. This will build the image and tag it as
paper_getter. This seems to be the part you are having trouble with.Once this is solved the next part is to run the image in a container.
###################################################
To run the image
paper_getterwe use the command:-docker run -it paper_getterIn the example shell script I have added a few things to this docker run command.
First is the flag
--rm. This will remove the container once it has done its job,Second is the volume mount
-v $(pwd)/xml_results:/xml_results. This connects the folderxml_resultsoutside the container to the folderxml_resultsinside the container. Allowing the results to be stored.Finally I have added the command to be run by the get_papers container. In the example this is
getpapers -p -x -o /xml_results --query 'chloroquine'The final run command being
docker run -it --rm --name get-papers \ -v $(pwd)/xml_results:/xml_results \ paper_getter \ getpapers -p -x -o /xml_results --query 'chloroquine'
Can you first try running the simplified command
docker run -it paper_getter getpapers -help
and see what you get?
Yes. I ran the command docker run -it paper_getter getpapers -help and got the usage commands for getpapers as below:
C:\Users\laksm\OneDrive\Desktop\getpapers>docker run -it paper_getter getpapers -help
Usage: getpapers [options]
Options:
-h, --help output usage information
-V, --version output the version number
-q, --query <query> search query (required)
-o, --outdir <path> output directory (required - will be created if not found)
--api <name> API to search [eupmc, crossref, ieee, arxiv] (default: eupmc)
-x, --xml download fulltext XMLs if available
-p, --pdf download fulltext PDFs if available
-s, --supp download supplementary files if available
-t, --minedterms download text-mined terms if available
-l, --loglevel <level> amount of information to log (silent, verbose, info*, data, warn, error, or debug)
-a, --all search all papers, not just open access
-n, --noexecute report how many results match the query, but don't actually download anything
-f, --logfile <filename> save log to specified file in output directory as well as printing to terminal
-k, --limit <int> limit the number of hits and downloads
--filter <filter object> filter by key value pair, passed straight to the crossref api only
-r, --restart restart file downloads after failure
Ok, so the command to do the search is (without line breaks)
docker run -it --rm --name get-papers -v $(pwd)/xml_results:/xml_results paper_getter getpapers -p -x -o /xml_results --query 'chloroquine'
In this query we're searching for chloroquine and putting the results into a folder called xml_results
I hope $ is not for Windows. But as you said that should be able to use via docker, I tried the command
C:\Users\laksm\OneDrive\Desktop\getpapers>docker run -it --rm --name get-papers -v $(pwd)/xml_results:/xml_results paper_getter getpapers -p -x -o /xml_results --query 'chloroquine'
docker: Error response from daemon: create $(pwd)/xml_results: "$(pwd)/xml_results" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you intended to pass a host directory, use absolute path.
See 'docker run --help'.
@bauhuasbadguy Does this mean my path is wrong or $ cannot be used?
Then, I used the command docker run --help to see the docker run commands and got the output: (sharing if any variation is, with respect to OS)
Usage: docker run [OPTIONS] IMAGE [COMMAND] [ARG...]
Run a command in a new container
Options:
--add-host list Add a custom host-to-IP mapping
(host:ip)
-a, --attach list Attach to STDIN, STDOUT or STDERR
--blkio-weight uint16 Block IO (relative weight),
between 10 and 1000, or 0 to
disable (default 0)
--blkio-weight-device list Block IO weight (relative device
weight) (default [])
--cap-add list Add Linux capabilities
--cap-drop list Drop Linux capabilities
--cgroup-parent string Optional parent cgroup for the
container
--cidfile string Write the container ID to the file
--cpu-period int Limit CPU CFS (Completely Fair
Scheduler) period
--cpu-quota int Limit CPU CFS (Completely Fair
Scheduler) quota
--cpu-rt-period int Limit CPU real-time period in
microseconds
--cpu-rt-runtime int Limit CPU real-time runtime in
microseconds
-c, --cpu-shares int CPU shares (relative weight)
--cpus decimal Number of CPUs
--cpuset-cpus string CPUs in which to allow execution
(0-3, 0,1)
--cpuset-mems string MEMs in which to allow execution
(0-3, 0,1)
-d, --detach Run container in background and
print container ID
--detach-keys string Override the key sequence for
detaching a container
--device list Add a host device to the container
--device-cgroup-rule list Add a rule to the cgroup allowed
devices list
--device-read-bps list Limit read rate (bytes per second)
from a device (default [])
--device-read-iops list Limit read rate (IO per second)
from a device (default [])
--device-write-bps list Limit write rate (bytes per
second) to a device (default [])
--device-write-iops list Limit write rate (IO per second)
to a device (default [])
--disable-content-trust Skip image verification (default true)
--dns list Set custom DNS servers
--dns-option list Set DNS options
--dns-search list Set custom DNS search domains
--domainname string Container NIS domain name
--entrypoint string Overwrite the default ENTRYPOINT
of the image
-e, --env list Set environment variables
--env-file list Read in a file of environment variables
--expose list Expose a port or a range of ports
--gpus gpu-request GPU devices to add to the
container ('all' to pass all GPUs)
--group-add list Add additional groups to join
--health-cmd string Command to run to check health
--health-interval duration Time between running the check
(ms|s|m|h) (default 0s)
--health-retries int Consecutive failures needed to
report unhealthy
--health-start-period duration Start period for the container to
initialize before starting
health-retries countdown
(ms|s|m|h) (default 0s)
--health-timeout duration Maximum time to allow one check to
run (ms|s|m|h) (default 0s)
--help Print usage
-h, --hostname string Container host name
--init Run an init inside the container
that forwards signals and reaps
processes
-i, --interactive Keep STDIN open even if not attached
--ip string IPv4 address (e.g., 172.30.100.104)
--ip6 string IPv6 address (e.g., 2001:db8::33)
--ipc string IPC mode to use
--isolation string Container isolation technology
--kernel-memory bytes Kernel memory limit
-l, --label list Set meta data on a container
--label-file list Read in a line delimited file of labels
--link list Add link to another container
--link-local-ip list Container IPv4/IPv6 link-local
addresses
--log-driver string Logging driver for the container
--log-opt list Log driver options
--mac-address string Container MAC address (e.g.,
92:d0:c6:0a:29:33)
-m, --memory bytes Memory limit
--memory-reservation bytes Memory soft limit
--memory-swap bytes Swap limit equal to memory plus
swap: '-1' to enable unlimited swap
--memory-swappiness int Tune container memory swappiness
(0 to 100) (default -1)
--mount mount Attach a filesystem mount to the
container
--name string Assign a name to the container
--network network Connect a container to a network
--network-alias list Add network-scoped alias for the
container
--no-healthcheck Disable any container-specified
HEALTHCHECK
--oom-kill-disable Disable OOM Killer
--oom-score-adj int Tune host's OOM preferences (-1000
to 1000)
--pid string PID namespace to use
--pids-limit int Tune container pids limit (set -1
for unlimited)
--platform string Set platform if server is
multi-platform capable
--privileged Give extended privileges to this
container
-p, --publish list Publish a container's port(s) to
the host
-P, --publish-all Publish all exposed ports to
random ports
--read-only Mount the container's root
filesystem as read only
--restart string Restart policy to apply when a
container exits (default "no")
--rm Automatically remove the container
when it exits
--runtime string Runtime to use for this container
--security-opt list Security Options
--shm-size bytes Size of /dev/shm
--sig-proxy Proxy received signals to the
process (default true)
--stop-signal string Signal to stop a container
(default "15")
--stop-timeout int Timeout (in seconds) to stop a
container
--storage-opt list Storage driver options for the
container
--sysctl map Sysctl options (default map[])
--tmpfs list Mount a tmpfs directory
-t, --tty Allocate a pseudo-TTY
--ulimit ulimit Ulimit options (default [])
-u, --user string Username or UID (format:
<name|uid>[:<group|gid>])
--userns string User namespace to use
--uts string UTS namespace to use
-v, --volume list Bind mount a volume
--volume-driver string Optional volume driver for the
container
--volumes-from list Mount volumes from the specified
container(s)
-w, --workdir string Working directory inside the container
No $(pwd) will not work in windows you will have to use the full path to the folder you want it to place your files in. Have you considered using powershell since it makes command line work much easier
It worked! I gave the whole path instead of $(pwd). But first I gave the command
docker run -it --rm --name get-papers -v C:/Users/laksm/OneDrive/Desktop/getpapers/xml_results:/xml_results paper_getter getpapers -p -x -o /xml_results --query 'chloroquine'
without -k <limit> hence it popped a Java heap space: out of memory error. Then I tried the command
docker run -it --rm --name get-papers -v C:/Users/laksm/OneDrive/Desktop/getpapers/xml_results:/xml_results paper_getter getpapers -k 10 -p -x -o /xml_results --query 'chloroquine'
in both command line and powershell. Both gave the similar result as below:
info: Searching using eupmc API
(node:1) Warning: Accessing non-existent property 'padLevels' of module exports inside circular dependency
(Use `node --trace-warnings ...` to show where the warning was created)
info: Found 19424 open access results
warn: This version of getpapers wasn't built with this version of the EuPMC api in mind
warn: getpapers EuPMCVersion: 5.3.2 vs. 6.4 reported by api
info: Limiting to 10 hits
Retrieving results [==============================] 100% (eta 0.0s)
info: Done collecting results
info: limiting hits
info: Saving result metadata
info: Full EUPMC result metadata written to eupmc_results.json
info: Individual EUPMC result metadata records written
info: Extracting fulltext HTML URL list (may not be available for all articles)
info: Fulltext HTML URL list written to eupmc_fulltext_html_urls.txt
warn: Article with pmcid "PMC7509225" was not Open Access (therefore no XML)
warn: Article with pmcid "PMC7499150" was not Open Access (therefore no XML)
warn: Article with pmcid "PMC7492153" was not Open Access (therefore no XML)
warn: Article with pmcid "PMC7476892" was not Open Access (therefore no XML)
warn: Article with pmcid "PMC7489187" was not Open Access (therefore no XML)
warn: Article with pmcid "PMC7486069" was not Open Access (therefore no XML)
info: Got XML URLs for 4 out of 10 results
info: Downloading fulltext XML files
Downloading files [==============================] 100% (4/4) [0.0s elapsed, eta 0.0]
info: All downloads succeeded!
warn: Article with pmcid "PMC7509225" had no fulltext PDF url
warn: Article with pmcid "PMC7499150" had no fulltext PDF url
warn: Article with pmcid "PMC7492153" had no fulltext PDF url
warn: Article with pmcid "PMC7476892" had no fulltext PDF url
warn: Article with pmcid "PMC7489187" had no fulltext PDF url
warn: Article with pmcid "PMC7486069" had no fulltext PDF url
info: Downloading fulltext PDF files
Downloading files [==============================] 100% (4/4) [0.0s elapsed, eta 0.0]
info: All downloads succeeded!
I hope I could now document these in respective wiki page...