k8:// line gives invalid path error

[Vaishali2896]

I am trying to create a daemon set, I have a service deployed already in namespace: cms-test. My file looks like apiVersion: apps/v1 kind: DaemonSet metadata: name: goreplay-daemon spec: template: spec: hostNetwork: true serviceAccountName: goreplay containers: - name: goreplay image: buger/goreplay:latest command: - "--input-raw k8s://default/pod/nginx-deployment-6588ff5bdd-4cc7c" - "--output-file requests.log" But I am getting this error whenever I try to run it stat --input-raw k8s://cms-test/pod/cms-api-67f778dc5b-5wk2g: no such file or directory: unknown

[yqbjtu]

where to get the buger/goreplay:2.0.0-rc2 image? do you build it by yourself?

[monrax]

The command option doesn't work with the "./gor" entrypoint. Use args instead:

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: goreplay-daemon
spec:
  selector:
    matchLabels:
      app: goreplay
  template:
    metadata:
      labels:
        app: goreplay
    spec:
      hostNetwork: true
      serviceAccountName: goreplay
      containers:
        - name: goreplay
           image: buger/goreplay:latest
           args:
            - "--input-raw"
            - "k8s://default/pod/nginx-deployment-6588ff5bdd-4cc7c"
            - "--output-file"
            - "requests.log"

[shariffshabir]

1. ./gor --input-raw "k8s://deployments/nginx:80" --output-stdout

2022/07/28 05:31:37 input-raw: error while parsing address too many colons in address k8s://deployments/nginx:80 we are getting this error and when we remove the :80 no request is been captured from ngnix service. if any one has faced this issue please let us know

2. After deploying the below yaml changes not able to find the - "--output-file" - "requests.log" files in side the goreplay pod could you let me know where can we trace this files and find that if the request are getting captured or not

apiVersion: apps/v1 kind: DaemonSet metadata: name: goreplay-daemon spec: selector: matchLabels: app: goreplay template: metadata: labels: app: goreplay spec: hostNetwork: true serviceAccountName: goreplay containers: - name: goreplay image: buger/goreplay:latest args: - "--input-raw" - "k8s://default/pod/nginx-deployment-6588ff5bdd-4cc7c" - "--output-file" - "requests.log"

[jneumann90]

I'm having the same issue providing a k8s-schema as the --input-raw parameter.

Just ran the binary to from cmd-line to eliminate k8s-arg-issues.

./gor --input-raw k8s://default/service/service-name:80 --output-stdout

2022/07/29 09:53:36 input-raw: error while parsing address: address k8s://default/service/service-name:80: too many colons in address

./gor --input-raw k8s://default/service/service-name --output-stdout

2022/07/29 09:53:41 parsing port error: strconv.Atoi: parsing "//default/service/service-name": invalid syntax

gor-version is v1.3.3

[monrax]

@shariffshabir @jneumann90 you are running the latest release, which doesn't have support for the k8s feature yet. In order to try this, you would need to compile a development build and use that in a custom docker image. I believe this also answers @yqbjtu 's question.

[shariffshabir]

Hi @monrax as mentioned in the above comments i have complied the code, we just get the executable now how do we use this as a custom docker image, Do you have any reference doc for using it as a docker image to solve above issue it would be really helpful @jneumann90 and @yqbjtu did you try the approach suggested by the @monrax if any leads please share it with me

[jneumann90]

Hi first of all, thank's @monrax.

@shariffshabir you could modify this https://github.com/buger/goreplay/blob/master/Dockerfile to use your executable instead of downloading the one from this repo. You'd then need to push it to a Docker-Repo and use this Image-Path in your K8S-YAMl.

Honestly I tried but failed to complie the executable...How did you manage that? If you let me know I'd give a try and share the files with you.

[jneumann90]

I'll start the build again and post the error message, as soon as

[jneumann90]

So actually I'm doing this in a golang:1.15-docker-container:

apt-get update && apt-get install flex bison -y
cd /tmp/tmp
tar xvzf libpcap-1.7.4.tar.gz
cd libpcap-1.7.4
./configure && make install
echo $?
mkdir $HOME/gocode
export GOPATH=$HOME/gocode
go get github.com/buger/gor
echo $?
cd $HOME/gocode/src/github.com/buger/gor
go build LDFLAGS = -ldflags "-extldflags \"-static\""

I'm gettin error-messages at go get github.com/buger/gor

/root/gocode/src/github.com/buger/goreplay/capture/doc.go:1: cannot parse import comment /root/gocode/src/github.com/buger/goreplay/tcp/doc.go:1: cannot parse import comment

If I just keep going on building:

panic: path "-extldflags \"-static\"" not in error "invalid import path \"-extldflags \\\"-static\\\"\""

goroutine 1 [running]:
cmd/go/internal/load.ImportErrorf(0x7ffcb445ebed, 0x15, 0xa45a91, 0x16, 0xc0001cf548, 0x1, 0x1, 0x0, 0x0)
        /usr/local/go/src/cmd/go/internal/load/pkg.go:445 +0x265
cmd/go/internal/load.(*Package).load(0xc0006eed80, 0x7ffcb445ebed, 0x15, 0xc0001cf988, 0x0, 0x0, 0x0, 0xc000213180, 0xb060c0, 0xc000070f50)
        /usr/local/go/src/cmd/go/internal/load/pkg.go:1764 +0x97d
cmd/go/internal/load.loadImport(0xc000480b80, 0x7ffcb445ebed, 0x15, 0xc000028004, 0x25, 0x0, 0xc0001cf988, 0x0, 0x0, 0x0, ...)
        /usr/local/go/src/cmd/go/internal/load/pkg.go:658 +0xd2f
cmd/go/internal/load.PackagesAndErrors(0xc000020200, 0x4, 0x4, 0x0, 0x0, 0x0)
        /usr/local/go/src/cmd/go/internal/load/pkg.go:2177 +0x679
cmd/go/internal/load.PackagesForBuild(0xc000020200, 0x4, 0x4, 0xa39ec8, 0x2, 0x8)
        /usr/local/go/src/cmd/go/internal/load/pkg.go:2242 +0x5d
cmd/go/internal/work.runBuild(0xdcdae0, 0xc000020200, 0x4, 0x4)
        /usr/local/go/src/cmd/go/internal/work/build.go:351 +0x88
main.main()
        /usr/local/go/src/cmd/go/main.go:190 +0x58d

[shariffshabir]

Hi @jneumann90 use the latest version of go version above 1.16 mostly this error should be solved and used this command
go build -ldflags "-extldflags "-static"" for final compilation reference https://github.com/buger/goreplay/issues/722

and it would be helpful if you could share me the script for generating the image or the files

[shariffshabir]

Hi @jneumann90 we are getting the executable file after compilation and the link which you shared for [https://github.com/buger/goreplay/blob/master/Dockerfile] docker file creation this accepts the tar.gz format, can you let me know how we can obtain that file for creating a docker image

[jneumann90]

Hi @shariffshabir,

I finally got it running and tested! Everything is quick & dirty, since I'm still poking around. So hopefully is this useful to you.

DockerFile to build the binary:

RUN apt-get update && apt-get install flex bison -y
COPY libpcap-1.7.4.tar.gz .
RUN tar xvzf libpcap-1.7.4.tar.gz
WORKDIR libpcap-1.7.4
RUN ./configure && make install
RUN rm -rf libpcap-1.7.4
RUN echo $?
RUN mkdir $HOME/gocode/
WORKDIR $HOME/gocode/
RUN git clone https://github.com/buger/goreplay
WORKDIR goreplay
RUN go get github.com/xdg-go/scram
RUN go build -ldflags "-extldflags \"-static\"" -o /bin/gor
RUN echo $?

FROM scratch as custom-exporter
COPY --from=builder /bin/gor .

Dockerfile to build the container-image:

ARG RELEASE_VERSION

RUN apk add --no-cache ca-certificates openssl

FROM scratch
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
COPY gor .
ENTRYPOINT ["./gor"]

And also I needed to modify the goreplay.yaml to run k8s-version v1.20.15:

kind: DaemonSet
metadata:
  labels:
    app: goreplay
  name: goreplay-daemon
  namespace: goreplay
spec:
  selector:
    matchLabels:
      app: goreplay
  template:
    metadata:
      labels:
        app: goreplay
      name: goreplay
    spec:
      containers:
      - args:
        - --input-raw
        - k8s://namespace/deployment/deployment-name:9091
        - --output-http
        - https://url-to-mirror.to
        image: my.own.dockerregistry.com/goreplay:1.3.0
        imagePullPolicy: Always
        name: goreplay
      hostNetwork: true
      serviceAccount: goreplay
      serviceAccountName: goreplay

[shariffshabir]

@jneumann90 thanks for the info, could you let me know where you are using the binary file generated while creating the container because their is no such reference of binary file in below docker file to create the container image

ARG RELEASE_VERSION

RUN apk add --no-cache ca-certificates openssl

FROM scratch COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ COPY gor . ENTRYPOINT ["./gor"]

getting this error Step 5/6 : COPY ./gor . COPY failed: file not found in build context or excluded by .dockerignore: stat gor: file does not exist

[shariffshabir]

@jneumann90 it would a great help if you could share the docker image which is working fine at your end. we will use the same try to perform end to end testing

[jneumann90]

@shariffshabir in the last line

COPY --from=builder /bin/gor .

of the first script the binary-file "/bin/gor" is copied to the hosts fs.

and with second to last line in the second script, the binary is copied from local fs to the container:

COPY gor .

[shariffshabir]

@jneumann90 @monrax Need a help when we deploy the goreplay on the AWS eks the goreplay was not getting started encountering below error 2022/09/21 12:39:26 pcap handles error:

Note: when deployed in local machine we are not encountering any issues Any port needs to allowed any help on this would be appreciated

[dingxiong]

@shariffshabir I have the same error. The logic https://github.com/buger/goreplay/blob/master/capture/capture.go#L769 does not work for AWS EKS. I print out all the interfaces returned by pcap.FindAllDevs() as blow. None of them is veth. 🤦

eth0, ,
enicf02f30fa7e, ,
eth1, ,
enibb54984f2e2, ,
enid28c92f69d1, ,
eni163ac93dde4, ,
eni23796b31f31, ,
eni7ccb0444727, ,
enicb00aa36260, ,
eni01a9aca171e, ,
enid017a93cf7a, ,
enie97a0b3dc6b, ,
eni652b4c8fddb, ,
eni0e3508a7142, ,
eni5b61dda4ff5, ,
eni9a13eec0d5c, ,
eni4f9bf829472, ,
eni1371c6eeed3, ,
enia9fb433f37d, ,
eth2, ,
eni4a6a08c964c, ,
any, Pseudo-device that captures on all interfaces,
lo, ,
nflog, Linux netfilter log (NFLOG) interface,
nfqueue, Linux netfilter queue (NFQUEUE) interface,

After changing veth to eni and re-compile the image, it works!

Ref:

• https://medium.com/engineered-publicis-sapient/container-network-interface-cni-for-eks-4b1cbfff0f4e

[shariffshabir]

Hi @dingxiong thanks for the help its resolved and finally working in AWS eks,

However facing one issue, the file size we are not able to store more than 5MB data in .gor file however with the goreplay -- help its says 1tb as the max, please find the configurations below

• args: - --input-raw - k8s://default/deployments/service-deployment:80 - --output-file - request.gor - --output-file-max-size-limit - 1TB

Or --output-file-size-limit -1024mb

Any suggestion would be helpful

[dingxiong]

@shariffshabir Have you tried below option

	flag.BoolVar(&Settings.OutputFileConfig.Append, "output-file-append", false, "The flushed chunk is appended to existence file or not. ")