Weixia Zhang

Hi, can you explain a bit about how to load Matterport3D meshgrid data for top-down visualization? I see an instruction at https://github.com/ronghanghu/speaker_follower/issues/8, however, I can't find the mesh_name.json anywhere.

Hi, I have finished training and selected the model with best performance on val set. How can I get the results on test split which can be submitted into the...

Tutorials for the using of Transformer layer

2

Hi, is there any plan to provide a tutorial of showing an example of employing the Transformer as an alternative of RNN for seq2seq task such as machine translation?

https://github.com/lichengunc/MAttNet/blob/de6f41ce5dc541425e5bd8ab96949ca09fb3de94/lib/layers/lang_encoder.py#L57 Hi, I find a mlp in the language encoder, which is used to project the original word embeddings into new representations. While it is not mentioned in the paper,...

Zhang, W., Ma, C., Wu, Q., & Yang, X. (2020). Language-guided navigation via cross-modal grounding and alternate adversarial learning. IEEE Transactions on Circuits and Systems for Video Technology. link: https://ieeexplore.ieee.org/document/9265290

Tensorflow 2.5.3 not works

2

python main.py --lip_model_path models/lrs2_lip_model 2022-02-16 17:28:19.165150: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0 Traceback (most recent call last): File "main.py", line 13, in from language_model.char_rnn_lm import CharRnnLmWrapperSingleton File "/home/redpanda/codebase/deep_lip_reading-dependabot-pip-tensorflow-gpu-2.5.3/language_model/char_rnn_lm.py", line...

Question about data pre-processing.

3

Hi, in the taskonomy_loader.py, I see different pre-processing operations are performed according to different tasks, i.e., from Line 171 to Line 208. Can you explain a bit about the underlying...

Hi, have you tried using beam search to synthesize instructions? I saw your beam search code in speaker but you simply use greedy search to synthesize instructions. https://github.com/ronghanghu/speaker_follower/blob/389ee0229b6fabe6ab93b00bdcaf1131912a8474/tasks/R2R/speaker.py#L342

building problem

9

Thank you for your great works! For me, it seems that the roi_align has been built successfully while neither roil_pooling nor nms does. The building information is listed as follows,...

Bug

1

https://github.com/researchmm/CKDN/blob/d5de075cea770b10d6354f011337008c8cf76787/ckdn.py#L205 The target should be removed.