pwru
pwru copied to clipboard
Changed compiler to clang.
Closes #221
I have tested this on vm. Test:
sudo mv /usr/bin/gcc{,old}
make clean
make
verified binary is executable and running as expected.
Removed LIBPCAP_CC. As it indicates we are using different compilers. I can revert it if needed.
Added and verified local-release.sh.
$ ARCHS='amd64' ./local-release.sh
+ OS=linux
+ [ amd64 = arm64 ]
+ LIBPCAP_ARCH=x86_64-unknown-linux-gnu
+ CC=clang
+ make clean
rm -f pwru
rm -f kprobepwru_bpf*
rm -f kprobemultipwru_bpf*
rm -f kprobepwruwithoutoutputskb_bpf*
rm -f kprobemultipwruwithoutoutputskb_bpf*
cd libpcap/ && make clean || true
make[1]: Entering directory '/home/vagrant/git/pwru/libpcap'
rm -f pcap-linux.o fad-getad.o pcap-netfilter-linux.o pcap.o gencode.o optimize.o nametoaddr.o etherent.o fmtutils.o pcap-util.o savefile.o sf-pcap.o sf-pcapng.o pcap-common.o pcap-usb-linux-common.o bpf_image.o bpf_filter.o bpf_dump.o scanner.o grammar.o strlcat.o strlcpy.o libpcap.a libpcap.so.`cat ./VERSION` libpcap-`cat ./VERSION`.tar.gz scanner.c grammar.c scanner.h grammar.h lex.yy.c pcap-config libpcap.pc
(cd rpcapd; make clean)
make[2]: Entering directory '/home/vagrant/git/pwru/libpcap/rpcapd'
rm -f daemon.o fileconf.o log.o rpcapd.o ../rpcap-protocol.o ../sockutils.o ../fmtutils.o ../sslutils.o rpcapd
make[2]: Leaving directory '/home/vagrant/git/pwru/libpcap/rpcapd'
(cd testprogs; make clean)
make[2]: Entering directory '/home/vagrant/git/pwru/libpcap/testprogs'
rm -f valgrindtest can_set_rfmon_test capturetest filtertest findalldevstest-perf findalldevstest opentest nonblocktest reactivatetest selpolltest threadsignaltest writecaptest
rm -rf *.dSYM
make[2]: Leaving directory '/home/vagrant/git/pwru/libpcap/testprogs'
make[1]: Leaving directory '/home/vagrant/git/pwru/libpcap'
+ echo Building release binary for linux/amd64...
Building release binary for linux/amd64...
+ make pwru TARGET_GOARCH=amd64 LIBPCAP_ARCH=x86_64-unknown-linux-gnu CC=clang
cd libpcap && \
CC=clang ./configure --disable-rdma --disable-shared --disable-usb --disable-netmap --disable-bluetooth --disable-dbus --without-libnl --host=x86_64-unknown-linux-gnu && \
make
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking target system type... x86_64-unknown-linux-gnu
checking for x86_64-unknown-linux-gnu-gcc... clang
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether clang accepts -g... yes
checking for clang option to accept ISO C89... none needed
checking for clang option to accept ISO C99... none needed
checking how to run the C preprocessor... clang -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking size of void *... 8
checking whether the compiler supports the -fvisibility=hidden option... yes
checking for inline... inline
checking for __atomic_load_n... yes
checking for __atomic_store_n... yes
checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... no
checking for _LARGEFILE_SOURCE value needed for large files... no
checking sys/ioccom.h usability... no
checking sys/ioccom.h presence... no
checking for sys/ioccom.h... no
checking sys/sockio.h usability... no
checking sys/sockio.h presence... no
checking for sys/sockio.h... no
checking netpacket/packet.h usability... yes
checking netpacket/packet.h presence... yes
checking for netpacket/packet.h... yes
checking for ANSI ioctl definitions... yes
checking for strerror... yes
checking for strerror_r... yes
checking whether strerror_r is GNU-style... yes
checking for vsyslog... yes
checking for vsnprintf... yes
checking for snprintf... yes
checking for vasprintf... yes
checking for asprintf... yes
checking for strlcat... no
checking for strlcpy... no
checking for strtok_r... yes
checking for ffs... yes
checking whether ffs is declared... yes
checking for getaddrinfo... yes
checking for library containing putmsg... no
checking whether getnetbyname_r is declared... yes
checking for the Linux getnetbyname_r()... yes
checking whether getprotobyname_r is declared... yes
checking for the Linux getprotobyname_r()... yes
checking for ether_hostton... yes
checking whether ether_hostton is declared... no
checking whether ether_hostton is declared... yes
checking pthread.h usability... yes
checking pthread.h presence... yes
checking for pthread.h... yes
checking for pthread_create... yes
checking if --disable-protochain option is specified... enabled
checking net/bpf.h usability... no
checking net/bpf.h presence... no
checking for net/bpf.h... no
checking net/pfilt.h usability... no
checking net/pfilt.h presence... no
checking for net/pfilt.h... no
checking net/enet.h usability... no
checking net/enet.h presence... no
checking for net/enet.h... no
checking net/nit.h usability... no
checking net/nit.h presence... no
checking for net/nit.h... no
checking sys/net/nit.h usability... no
checking sys/net/nit.h presence... no
checking for sys/net/nit.h... no
checking linux/socket.h usability... yes
checking linux/socket.h presence... yes
checking for linux/socket.h... yes
checking net/raw.h usability... no
checking net/raw.h presence... no
checking for net/raw.h... no
checking sys/dlpi.h usability... no
checking sys/dlpi.h presence... no
checking for sys/dlpi.h... no
checking config/HaikuConfig.h usability... no
checking config/HaikuConfig.h presence... no
checking for config/HaikuConfig.h... no
checking packet capture type... linux
checking for x86_64-unknown-linux-gnu-pkg-config... no
checking for pkg-config... /usr/bin/pkg-config
checking pkg-config is at least version 0.17.0... yes
checking for brew... no
checking for linux/wireless.h... yes
checking for struct tpacket_auxdata.tp_vlan_tci... yes
checking for getifaddrs... yes
checking ifaddrs.h usability... yes
checking ifaddrs.h presence... yes
checking for ifaddrs.h... yes
checking linux/net_tstamp.h usability... yes
checking linux/net_tstamp.h presence... yes
checking for linux/net_tstamp.h... yes
checking for socklen_t... yes
checking dagapi.h usability... no
checking dagapi.h presence... no
checking for dagapi.h... no
checking whether we have Septel API headers... no
checking whether we have Myricom Sniffer API... no
checking whether TurboCap is supported... no
checking whether to enable remote packet capture... no
checking whether to build optimizer debugging code... no
checking whether to build parser debugging code... no
checking for flex... flex
checking lex output file root... lex.yy
checking lex library... -lfl
checking whether yytext is a pointer... yes
checking for capable lex... yes
checking for bison... bison
checking for x86_64-unknown-linux-gnu-ranlib... no
checking for ranlib... ranlib
checking for x86_64-unknown-linux-gnu-ar... no
checking for ar... ar
checking whether ln -s works... yes
checking for struct sockaddr.sa_len... no
checking for struct sockaddr_storage... yes
checking for dl_hp_ppa_info_t.dl_module_id_1... no
checking for Linux usbmon USB sniffing support... no
checking whether we can compile the netfilter support... yes
checking for libdpdk with pkg-config... not found
configure: WARNING: We couldn't find DPDK with pkg-config. If
you want DPDK support, make sure that pkg-config is installed,
that DPDK 18.02.2 or later is installed, and that DPDK provides a
.pc file.
checking for a BSD-compatible install... /usr/bin/install -c
configure: creating ./config.status
config.status: creating Makefile
config.status: creating grammar.y
config.status: creating pcap-filter.manmisc
config.status: creating pcap-linktype.manmisc
config.status: creating pcap-tstamp.manmisc
config.status: creating pcap-savefile.manfile
config.status: creating pcap.3pcap
config.status: creating pcap_compile.3pcap
config.status: creating pcap_datalink.3pcap
config.status: creating pcap_dump_open.3pcap
config.status: creating pcap_get_tstamp_precision.3pcap
config.status: creating pcap_list_datalinks.3pcap
config.status: creating pcap_list_tstamp_types.3pcap
config.status: creating pcap_open_dead.3pcap
config.status: creating pcap_open_offline.3pcap
config.status: creating pcap_set_immediate_mode.3pcap
config.status: creating pcap_set_tstamp_precision.3pcap
config.status: creating pcap_set_tstamp_type.3pcap
config.status: creating rpcapd/Makefile
config.status: creating rpcapd/rpcapd.manadmin
config.status: creating rpcapd/rpcapd-config.manfile
config.status: creating testprogs/Makefile
config.status: creating config.h
config.status: config.h is unchanged
config.status: executing default-1 commands
make[1]: Entering directory '/home/vagrant/git/pwru/libpcap'
clang -fvisibility=hidden -fpic -I. -DBUILDING_PCAP -Dpcap_EXPORTS -DHAVE_CONFIG_H -g -O2 -c ./pcap-linux.c
clang -fvisibility=hidden -fpic -I. -DBUILDING_PCAP -Dpcap_EXPORTS -DHAVE_CONFIG_H -g -O2 -c ./fad-getad.c
clang -fvisibility=hidden -fpic -I. -DBUILDING_PCAP -Dpcap_EXPORTS -DHAVE_CONFIG_H -g -O2 -c ./pcap-netfilter-linux.c
clang -fvisibility=hidden -fpic -I. -DBUILDING_PCAP -Dpcap_EXPORTS -DHAVE_CONFIG_H -g -O2 -c ./pcap.c
bison -p pcap_ -o grammar.c -d grammar.y
flex -P pcap_ --header-file=scanner.h --nounput -o scanner.c scanner.l
clang -fvisibility=hidden -fpic -I. -DBUILDING_PCAP -Dpcap_EXPORTS -DHAVE_CONFIG_H -g -O2 -c ./gencode.c
clang -fvisibility=hidden -fpic -I. -DBUILDING_PCAP -Dpcap_EXPORTS -DHAVE_CONFIG_H -g -O2 -c ./optimize.c
clang -fvisibility=hidden -fpic -I. -DBUILDING_PCAP -Dpcap_EXPORTS -DHAVE_CONFIG_H -g -O2 -c ./nametoaddr.c
clang -fvisibility=hidden -fpic -I. -DBUILDING_PCAP -Dpcap_EXPORTS -DHAVE_CONFIG_H -g -O2 -c ./etherent.c
clang -fvisibility=hidden -fpic -I. -DBUILDING_PCAP -Dpcap_EXPORTS -DHAVE_CONFIG_H -g -O2 -c ./fmtutils.c
clang -fvisibility=hidden -fpic -I. -DBUILDING_PCAP -Dpcap_EXPORTS -DHAVE_CONFIG_H -g -O2 -c ./pcap-util.c
clang -fvisibility=hidden -fpic -I. -DBUILDING_PCAP -Dpcap_EXPORTS -DHAVE_CONFIG_H -g -O2 -c ./savefile.c
clang -fvisibility=hidden -fpic -I. -DBUILDING_PCAP -Dpcap_EXPORTS -DHAVE_CONFIG_H -g -O2 -c ./sf-pcap.c
clang -fvisibility=hidden -fpic -I. -DBUILDING_PCAP -Dpcap_EXPORTS -DHAVE_CONFIG_H -g -O2 -c ./sf-pcapng.c
clang -fvisibility=hidden -fpic -I. -DBUILDING_PCAP -Dpcap_EXPORTS -DHAVE_CONFIG_H -g -O2 -c ./pcap-common.c
clang -fvisibility=hidden -fpic -I. -DBUILDING_PCAP -Dpcap_EXPORTS -DHAVE_CONFIG_H -g -O2 -c ./pcap-usb-linux-common.c
clang -fvisibility=hidden -fpic -I. -DBUILDING_PCAP -Dpcap_EXPORTS -DHAVE_CONFIG_H -g -O2 -c ./bpf_image.c
clang -fvisibility=hidden -fpic -I. -DBUILDING_PCAP -Dpcap_EXPORTS -DHAVE_CONFIG_H -g -O2 -c ./bpf_filter.c
clang -fvisibility=hidden -fpic -I. -DBUILDING_PCAP -Dpcap_EXPORTS -DHAVE_CONFIG_H -g -O2 -c ./bpf_dump.c
clang -fvisibility=hidden -fpic -I. -DBUILDING_PCAP -Dpcap_EXPORTS -DHAVE_CONFIG_H -g -O2 -c scanner.c
clang -fvisibility=hidden -fpic -I. -DBUILDING_PCAP -Dpcap_EXPORTS -DHAVE_CONFIG_H -g -O2 -c grammar.c
clang -fvisibility=hidden -fpic -I. -DBUILDING_PCAP -Dpcap_EXPORTS -DHAVE_CONFIG_H -g -O2 -o strlcat.o -c ./missing/strlcat.c
clang -fvisibility=hidden -fpic -I. -DBUILDING_PCAP -Dpcap_EXPORTS -DHAVE_CONFIG_H -g -O2 -o strlcpy.o -c ./missing/strlcpy.c
ar rc libpcap.a pcap-linux.o fad-getad.o pcap-netfilter-linux.o pcap.o gencode.o optimize.o nametoaddr.o etherent.o fmtutils.o pcap-util.o savefile.o sf-pcap.o sf-pcapng.o pcap-common.o pcap-usb-linux-common.o bpf_image.o bpf_filter.o bpf_dump.o scanner.o grammar.o strlcat.o strlcpy.o
ranlib libpcap.a
./config.status --file=libpcap.pc.tmp:./libpcap.pc.in
config.status: creating libpcap.pc.tmp
mv libpcap.pc.tmp libpcap.pc
./config.status --file=pcap-config.tmp:./pcap-config.in
config.status: creating pcap-config.tmp
mv pcap-config.tmp pcap-config
chmod a+x pcap-config
make[1]: Leaving directory '/home/vagrant/git/pwru/libpcap'
TARGET_GOARCH=amd64 go generate
Generating for amd64
Compiled /home/vagrant/git/pwru/kprobepwru_bpfel_x86.o
Wrote /home/vagrant/git/pwru/kprobepwru_bpfel_x86.go
Compiled /home/vagrant/git/pwru/kprobemultipwru_bpfel_x86.o
Wrote /home/vagrant/git/pwru/kprobemultipwru_bpfel_x86.go
Compiled /home/vagrant/git/pwru/kprobepwruwithoutoutputskb_bpfel_x86.o
Wrote /home/vagrant/git/pwru/kprobepwruwithoutoutputskb_bpfel_x86.go
Compiled /home/vagrant/git/pwru/kprobemultipwruwithoutoutputskb_bpfel_x86.o
Wrote /home/vagrant/git/pwru/kprobemultipwruwithoutoutputskb_bpfel_x86.go
CC=clang GOARCH=amd64 CGO_ENABLED=1 go build \
-ldflags "-w -s \
-X 'github.com/cilium/pwru/internal/pwru.Version=1e10c19'"
# github.com/cilium/pwru
/usr/bin/ld: /tmp/go-link-90566696/000020.o: in function `_cgo_9c8efe9babca_C2func_getaddrinfo':
/tmp/go-build/cgo_unix_cgo.cgo2.c:58: warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/bin/ld: /home/vagrant/git/pwru/internal/libpcap/../../libpcap/libpcap.a(nametoaddr.o): in function `pcap_nametoaddr':
/home/vagrant/git/pwru/libpcap/nametoaddr.c:181: warning: Using 'gethostbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/bin/ld: /home/vagrant/git/pwru/internal/libpcap/../../libpcap/libpcap.a(nametoaddr.o): in function `pcap_nametonetaddr':
/home/vagrant/git/pwru/libpcap/./nametoaddr.c:270: warning: Using 'getnetbyname_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/bin/ld: /home/vagrant/git/pwru/internal/libpcap/../../libpcap/libpcap.a(nametoaddr.o): in function `pcap_nametoproto':
/home/vagrant/git/pwru/libpcap/./nametoaddr.c:527: warning: Using 'getprotobyname_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
+ test -d release/linux/amd64
+ mkdir -p release/linux/amd64
+ tar -czf release/pwru-linux-amd64.tar.gz pwru
+ cd release
+ sha256sum pwru-linux-amd64.tar.gz
+ rm -r release/linux
Thanks. The last bit is to replace gcc with clang for arm64 in local-release.sh.
clang build for arm64 failing when clang replaced by gcc.
CC=clang GOARCH=arm64 CGO_ENABLED=1 go build \
-ldflags "-w -s \
-X 'github.com/cilium/pwru/internal/pwru.Version=1e10c19'"
# runtime/cgo
gcc_arm64.S:30:19: error: expected ']' in brackets expression
stp x29, x30, [sp, #-96]!
^
gcc_arm64.S:34:2: error: unknown use of instruction mnemonic without a size suffix
mov x29, sp
^
gcc_arm64.S:36:19: error: expected ']' in brackets expression
stp x19, x20, [sp, #80]
^
gcc_arm64.S:39:19: error: expected ']' in brackets expression
stp x21, x22, [sp, #64]
^
gcc_arm64.S:42:19: error: expected ']' in brackets expression
stp x23, x24, [sp, #48]
^
gcc_arm64.S:45:19: error: expected ']' in brackets expression
stp x25, x26, [sp, #32]
^
gcc_arm64.S:48:19: error: expected ']' in brackets expression
stp x27, x28, [sp, #16]
^
gcc_arm64.S:52:2: error: unknown use of instruction mnemonic without a size suffix
mov x19, x0
^
gcc_arm64.S:53:2: error: unknown use of instruction mnemonic without a size suffix
mov x20, x1
^
gcc_arm64.S:54:2: error: unknown use of instruction mnemonic without a size suffix
mov x0, x2
^
gcc_arm64.S:56:2: error: invalid instruction mnemonic 'blr'
blr x20
^~~
gcc_arm64.S:57:2: error: invalid instruction mnemonic 'blr'
blr x19
^~~
gcc_arm64.S:59:19: error: expected ']' in brackets expression
ldp x27, x28, [sp, #16]
^
gcc_arm64.S:62:19: error: expected ']' in brackets expression
ldp x25, x26, [sp, #32]
^
gcc_arm64.S:65:19: error: expected ']' in brackets expression
ldp x23, x24, [sp, #48]
^
gcc_arm64.S:68:19: error: expected ']' in brackets expression
ldp x21, x22, [sp, #64]
^
gcc_arm64.S:71:19: error: expected ']' in brackets expression
ldp x19, x20, [sp, #80]
^
gcc_arm64.S:74:22: error: unknown token in expression
ldp x29, x30, [sp], #96
^
make: *** [Makefile:21: pwru] Error 1
I had only changed gcc to clan in arm64.
https://stackoverflow.com/questions/51475992/cgo-cross-compiling-from-amd64linux-to-arm64linux might help.
Found root cause of issue.
- clang is not cross compiler.
- when clang compiled libpcap it generated object files for x86_64.
- which in turn caused compile error
also found similar issue. https://github.com/golang/go/issues/28966#issuecomment-547558462
trying to figure out clang compiler and compiling using aarch64 clang binary. https://github.com/llvm/llvm-project/releases/download/llvmorg-17.0.4/clang+llvm-17.0.4-aarch64-linux-gnu.tar.xz
as well as I will check if there is any repo hosting these clang.
@brb any pointers about this ?
just realized https://github.com/llvm/llvm-project/releases/download/llvmorg-17.0.4/clang+llvm-17.0.4-aarch64-linux-gnu.tar.xz
is not cross-compiler rather its clang for aarch64.
Could we use a clang cross-compiler for arm64? https://clang.llvm.org/docs/CrossCompilation.html
I am able to cross compile libpcap. Now only issue is cgo compilation. Which is exact same go files are building fine with cc=aarch64-unknown-linux-gnu But fails when cc=clang
@brb any pointers ? PS: I have used workaround for cross compilation as of now.
building fine with cc=aarch64-unknown-linux-gnu
Is it GCC or Clang?
It was typo.
its CC=aarch64-linux-gnu-gcc
In order to cross compile with Clang, you need to set --target. Try setting it via CFLAGS="--target=aarch64-unknown-linux" when compiling with Cgo.
Yes Now I can compile. Linker error. as it is using linker '/usr/local/go/pkg/tool/linux_386/link'
CC=clang CGO_CFLAGS="--target=aarch64-unknown-linux" GOARCH=arm64 CGO_ENABLED=1 go build \
-ldflags "-w -s \
-X 'github.com/cilium/pwru/internal/pwru.Version=2f22f3f'"
# github.com/cilium/pwru
/usr/local/go/pkg/tool/linux_386/link: running clang failed: exit status 1
/usr/bin/ld: /tmp/go-link-1420334561/go.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/go-link-1420334561/go.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/go-link-1420334561/go.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/go-link-1420334561/go.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/go-link-1420334561/go.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/go-link-1420334561/go.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/go-link-1420334561/go.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/go-link-1420334561/go.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/go-link-1420334561/go.o: Relocations in generic ELF (EM: 183)
/usr/bin/ld: /tmp/go-link-1420334561/go.o: error adding symbols: file in wrong format
clang: error: linker command failed with exit code 1 (use -v to see invocation)
trying to add linker using $ /usr/local/go/pkg/tool/linux_386/link usage: link [options] main.o ... -I linker use linker as ELF dynamic linker
i.e. /usr/local/go/pkg/tool/linux_386/link -I aarch64-linux-gnu-ld
here is execsnoop output.
$ sudo /usr/share/bcc/tools/execsnoop
PCOMM PID PPID RET ARGS
go 38007 1266 0 /usr/local/go/bin/go build -ldflags -w -s -X 'github.com/cilium/pwru/internal
/pwru.Version=2f22f3f'
clang 38081 38080 0 /usr/lib/llvm-14/bin/clang -cc1 -triple x86_64-pc-linux-gnu -emit-obj -mrelax-all --m
relax-relocations -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name triv
ial.c -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -fmath-errno
ld 38082 38080 0 /usr/bin/ld -pie --hash-style=both --build-id --eh-frame-hdr -m elf_x86_64 -dynamic-l
inker /lib64/ld-linux-x86-64.so.2 -o /tmp/go-link-2893705073/a.out /lib/x86_64-linux-gnu/Scrt1.o /lib/x86_64-linux-gnu/crt
i.o /usr/bin/../lib/gcc/x86_64-linux-gnu/12/crtbeginS.o -L/usr/bin/../lib/gcc/x86_64-linux-gnu/12 -L/usr/bin/../lib/gcc/x8
6_64-linux-gnu/12/../../../../lib64 -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib6
4
Which means target flag is not used by go build i.e. cgo is not compiling for arm64.
Maybe try this hack CC="clang --target=aarch64-unknown-linux" ....
@devidasjadhav :wave: any update on this PR?
@devidasjadhav I'm closing this PR due to inactivity. Please let me know if I need to reopen.