TransMEF 关于“fusion_arbitary_size_TransMEF

你好，非常感谢关于上一个issue的解答！再次打扰一下，我在测试中发现以“fusion_arbitary_size_TransMEF_gray.py”融合后的图片和以“fusion_gray_TransMEF.py”融合后的图片不同。其中滑窗融合方法得到图片质量较差。两个方法采用的数据，模型是一致的。

Oct 11 '22 03:10 keviner1

对拼接后的全局图像进行一次MaxMinNormalization可以解决该问题，如下：

def fuse(img1, img2): ''' block fusion ''' block_num = img1.shape[0] final_fusion = np.zeros_like(img1) for i in range(block_num): img1_inblock = _tensor(img1[i, :, :]).unsqueeze(0).to(device) img2_inblock = _tensor(img2[i, :, :]).unsqueeze(0).to(device) img_fusion = fusion(x1=img1_inblock, x2=img2_inblock, model=model) img_fusion = _pil_gray(img_fusion) img_fusion = np.asarray(img_fusion) final_fusion[i,:,:] = img_fusion

**# Perform Global MaxMinNormalization**
final_fusion = (final_fusion - np.min(final_fusion)) / (np.max(final_fusion) - np.min(final_fusion))
final_fusion = np.clip(final_fusion*255, 0, 255)
return final_fusion

Oct 25 '22 14:10 keviner1

Thanks for solving this issue!

Nov 05 '22 04:11 miccaiif

你好我下载了谷歌网盘里的best_model.pth，然后运行fusion_gray_TransMEF.py时，说模型有问题。raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for TransNet: Missing key(s) in state_dict: "encoder.inc.double_conv.0.weight", "encoder.inc.double_conv.0.bias", "encoder.inc.double_conv.2.weight", "encoder.inc.double_conv.2.bias", "encoder.layer1.double_conv.0.weight", "encoder.layer1.double_conv.0.bias", "encoder.layer1.double_conv.2.weight", "encoder.layer1.double_conv.2.bias", "encoder.layer2.double_conv.0.weight", "encoder.layer2.double_conv.0.bias", "encoder.layer2.double_conv.2.weight", "encoder.layer2.double_conv.2.bias", "encoder.transformer.cls_token", "encoder.transformer.to_patch_embedding.1.weight", "encoder.transformer.to_patch_embedding.1.bias", "encoder.transformer.transformer.layers.0.0.norm.weight", "encoder.transformer.transformer.layers.0.0.norm.bias", "encoder.transformer.transformer.layers.0.0.fn.to_qkv.weight", "encoder.transformer.transformer.layers.0.0.fn.to_out.0.weight", "encoder.transformer.transformer.layers.0.0.fn.to_out.0.bias", "encoder.transformer.transformer.layers.0.1.norm.weight", "encoder.transformer.transformer.layers.0.1.norm.bias", "encoder.transformer.transformer.layers.0.1.fn.net.0.weight", "encoder.transformer.transformer.layers.0.1.fn.net.0.bias", "encoder.transformer.transformer.layers.0.1.fn.net.3.weight", "encoder.transformer.transformer.layers.0.1.fn.net.3.bias", "encoder.transformer.transformer.layers.1.0.norm.weight", "encoder.transformer.transformer.layers.1.0.norm.bias", "encoder.transformer.transformer.layers.1.0.fn.to_qkv.weight", "encoder.transformer.transformer.layers.1.0.fn.to_out.0.weight", "encoder.transformer.transformer.layers.1.0.fn.to_out.0.bias", "encoder.transformer.transformer.layers.1.1.norm.weight", "encoder.transformer.transformer.layers.1.1.norm.bias", "encoder.transformer.transformer.layers.1.1.fn.net.0.weight", "encoder.transformer.transformer.layers.1.1.fn.net.0.bias", "encoder.transformer.transformer.layers.1.1.fn.net.3.weight", "encoder.transformer.transformer.layers.1.1.fn.net.3.bias", "encoder.transformer.transformer.layers.2.0.norm.weight", "encoder.transformer.transformer.layers.2.0.norm.bias", "encoder.transformer.transformer.layers.2.0.fn.to_qkv.weight", "encoder.transformer.transformer.layers.2.0.fn.to_out.0.weight", "encoder.transformer.transformer.layers.2.0.fn.to_out.0.bias", "encoder.transformer.transformer.layers.2.1.norm.weight", "encoder.transformer.transformer.layers.2.1.norm.bias", "encoder.transformer.transformer.layers.2.1.fn.net.0.weight", "encoder.transformer.transformer.layers.2.1.fn.net.0.bias", "encoder.transformer.transformer.layers.2.1.fn.net.3.weight", "encoder.transformer.transformer.layers.2.1.fn.net.3.bias", "encoder.transformer.transformer.layers.3.0.norm.weight", "encoder.transformer.transformer.layers.3.0.norm.bias", "encoder.transformer.transformer.layers.3.0.fn.to_qkv.weight", "encoder.transformer.transformer.layers.3.0.fn.to_out.0.weight", "encoder.transformer.transformer.layers.3.0.fn.to_out.0.bias", "encoder.transformer.transformer.layers.3.1.norm.weight", "encoder.transformer.transformer.layers.3.1.norm.bias", "encoder.transformer.transformer.layers.3.1.fn.net.0.weight", "encoder.transformer.transformer.layers.3.1.fn.net.0.bias", "encoder.transformer.transformer.layers.3.1.fn.net.3.weight", "encoder.transformer.transformer.layers.3.1.fn.net.3.bias", "encoder.transformer.transformer.layers.4.0.norm.weight", "encoder.transformer.transformer.layers.4.0.norm.bias", "encoder.transformer.transformer.layers.4.0.fn.to_qkv.weight", "encoder.transformer.transformer.layers.4.0.fn.to_out.0.weight", "encoder.transformer.transformer.layers.4.0.fn.to_out.0.bias", "encoder.transformer.transformer.layers.4.1.norm.weight", "encoder.transformer.transformer.layers.4.1.norm.bias", "encoder.transformer.transformer.layers.4.1.fn.net.0.weight", "encoder.transformer.transformer.layers.4.1.fn.net.0.bias", "encoder.transformer.transformer.layers.4.1.fn.net.3.weight", "encoder.transformer.transformer.layers.4.1.fn.net.3.bias", "encoder.transformer.transformer.layers.5.0.norm.weight", "encoder.transformer.transformer.layers.5.0.norm.bias", "encoder.transformer.transformer.layers.5.0.fn.to_qkv.weight", "encoder.transformer.transformer.layers.5.0.fn.to_out.0.weight", "encoder.transformer.transformer.layers.5.0.fn.to_out.0.bias", "encoder.transformer.transformer.layers.5.1.norm.weight", "encoder.transformer.transformer.layers.5.1.norm.bias", "encoder.transformer.transformer.layers.5.1.fn.net.0.weight", "encoder.transformer.transformer.layers.5.1.fn.net.0.bias", "encoder.transformer.transformer.layers.5.1.fn.net.3.weight", "encoder.transformer.transformer.layers.5.1.fn.net.3.bias", "encoder.transformer.transformer.layers.6.0.norm.weight", "encoder.transformer.transformer.layers.6.0.norm.bias", "encoder.transformer.transformer.layers.6.0.fn.to_qkv.weight", "encoder.transformer.transformer.layers.6.0.fn.to_out.0.weight", "encoder.transformer.transformer.layers.6.0.fn.to_out.0.bias", "encoder.transformer.transformer.layers.6.1.norm.weight", "encoder.transformer.transformer.layers.6.1.norm.bias", "encoder.transformer.transformer.layers.6.1.fn.net.0.weight", "encoder.transformer.transformer.layers.6.1.fn.net.0.bias", "encoder.transformer.transformer.layers.6.1.fn.net.3.weight", "encoder.transformer.transformer.layers.6.1.fn.net.3.bias", "encoder.transformer.transformer.layers.7.0.norm.weight", "encoder.transformer.transformer.layers.7.0.norm.bias", "encoder.transformer.transformer.layers.7.0.fn.to_qkv.weight", "encoder.transformer.transformer.layers.7.0.fn.to_out.0.weight", "encoder.transformer.transformer.layers.7.0.fn.to_out.0.bias", "encoder.transformer.transformer.layers.7.1.norm.weight", "encoder.transformer.transformer.layers.7.1.norm.bias", "encoder.transformer.transformer.layers.7.1.fn.net.0.weight", "encoder.transformer.transformer.layers.7.1.fn.net.0.bias", "encoder.transformer.transformer.layers.7.1.fn.net.3.weight", "encoder.transformer.transformer.layers.7.1.fn.net.3.bias", "encoder.transformer.transformer.layers.8.0.norm.weight", "encoder.transformer.transformer.layers.8.0.norm.bias", "encoder.transformer.transformer.layers.8.0.fn.to_qkv.weight", "encoder.transformer.transformer.layers.8.0.fn.to_out.0.weight", "encoder.transformer.transformer.layers.8.0.fn.to_out.0.bias", "encoder.transformer.transformer.layers.8.1.norm.weight", "encoder.transformer.transformer.layers.8.1.norm.bias", "encoder.transformer.transformer.layers.8.1.fn.net.0.weight", "encoder.transformer.transformer.layers.8.1.fn.net.0.bias", "encoder.transformer.transformer.layers.8.1.fn.net.3.weight", "encoder.transformer.transformer.layers.8.1.fn.net.3.bias", "encoder.transformer.transformer.layers.9.0.norm.weight", "encoder.transformer.transformer.layers.9.0.norm.bias", "encoder.transformer.transformer.layers.9.0.fn.to_qkv.weight", "encoder.transformer.transformer.layers.9.0.fn.to_out.0.weight", "encoder.transformer.transformer.layers.9.0.fn.to_out.0.bias", "encoder.transformer.transformer.layers.9.1.norm.weight", "encoder.transformer.transformer.layers.9.1.norm.bias", "encoder.transformer.transformer.layers.9.1.fn.net.0.weight", "encoder.transformer.transformer.layers.9.1.fn.net.0.bias", "encoder.transformer.transformer.layers.9.1.fn.net.3.weight", "encoder.transformer.transformer.layers.9.1.fn.net.3.bias", "encoder.transformer.transformer.layers.10.0.norm.weight", "encoder.transformer.transformer.layers.10.0.norm.bias", "encoder.transformer.transformer.layers.10.0.fn.to_qkv.weight", "encoder.transformer.transformer.layers.10.0.fn.to_out.0.weight", "encoder.transformer.transformer.layers.10.0.fn.to_out.0.bias", "encoder.transformer.transformer.layers.10.1.norm.weight", "encoder.transformer.transformer.layers.10.1.norm.bias", "encoder.transformer.transformer.layers.10.1.fn.net.0.weight", "encoder.transformer.transformer.layers.10.1.fn.net.0.bias", "encoder.transformer.transformer.layers.10.1.fn.net.3.weight", "encoder.transformer.transformer.layers.10.1.fn.net.3.bias", "encoder.transformer.transformer.layers.11.0.norm.weight", "encoder.transformer.transformer.layers.11.0.norm.bias", "encoder.transformer.transformer.layers.11.0.fn.to_qkv.weight", "encoder.transformer.transformer.layers.11.0.fn.to_out.0.weight", "encoder.transformer.transformer.layers.11.0.fn.to_out.0.bias", "encoder.transformer.transformer.layers.11.1.norm.weight", "encoder.transformer.transformer.layers.11.1.norm.bias", "encoder.transformer.transformer.layers.11.1.fn.net.0.weight", "encoder.transformer.transformer.layers.11.1.fn.net.0.bias", "encoder.transformer.transformer.layers.11.1.fn.net.3.weight", "encoder.transformer.transformer.layers.11.1.fn.net.3.bias", "encoder.transformer.convd1.0.weight", "encoder.transformer.convd1.0.bias", "decoder.layer1.double_conv.0.weight", "decoder.layer1.double_conv.0.bias", "decoder.layer1.double_conv.2.weight", "decoder.layer1.double_conv.2.bias", "decoder.layer2.double_conv.0.weight", "decoder.layer2.double_conv.0.bias", "decoder.layer2.double_conv.2.weight", "decoder.layer2.double_conv.2.bias", "decoder.outc.conv.weight", "decoder.outc.conv.bias". Unexpected key(s) in state_dict: "module.encoder.inc.double_conv.0.weight", "module.encoder.inc.double_conv.0.bias", "module.encoder.inc.double_conv.2.weight", "module.encoder.inc.double_conv.2.bias", "module.encoder.layer1.double_conv.0.weight", "module.encoder.layer1.double_conv.0.bias", "module.encoder.layer1.double_conv.2.weight", "module.encoder.layer1.double_conv.2.bias", "module.encoder.layer2.double_conv.0.weight", "module.encoder.layer2.double_conv.0.bias", "module.encoder.layer2.double_conv.2.weight", "module.encoder.layer2.double_conv.2.bias", "module.encoder.transformer.cls_token", "module.encoder.transformer.to_patch_embedding.1.weight", "module.encoder.transformer.to_patch_embedding.1.bias", "module.encoder.transformer.transformer.layers.0.0.norm.weight", "module.encoder.transformer.transformer.layers.0.0.norm.bias", "module.encoder.transformer.transformer.layers.0.0.fn.to_qkv.weight", "module.encoder.transformer.transformer.layers.0.0.fn.to_out.0.weight", "module.encoder.transformer.transformer.layers.0.0.fn.to_out.0.bias", "module.encoder.transformer.transformer.layers.0.1.norm.weight", "module.encoder.transformer.transformer.layers.0.1.norm.bias", "module.encoder.transformer.transformer.layers.0.1.fn.net.0.weight", "module.encoder.transformer.transformer.layers.0.1.fn.net.0.bias", "module.encoder.transformer.transformer.layers.0.1.fn.net.3.weight", "module.encoder.transformer.transformer.layers.0.1.fn.net.3.bias", "module.encoder.transformer.transformer.layers.1.0.norm.weight", "module.encoder.transformer.transformer.layers.1.0.norm.bias", "module.encoder.transformer.transformer.layers.1.0.fn.to_qkv.weight", "module.encoder.transformer.transformer.layers.1.0.fn.to_out.0.weight", "module.encoder.transformer.transformer.layers.1.0.fn.to_out.0.bias", "module.encoder.transformer.transformer.layers.1.1.norm.weight", "module.encoder.transformer.transformer.layers.1.1.norm.bias", "module.encoder.transformer.transformer.layers.1.1.fn.net.0.weight", "module.encoder.transformer.transformer.layers.1.1.fn.net.0.bias", "module.encoder.transformer.transformer.layers.1.1.fn.net.3.weight", "module.encoder.transformer.transformer.layers.1.1.fn.net.3.bias", "module.encoder.transformer.transformer.layers.2.0.norm.weight", "module.encoder.transformer.transformer.layers.2.0.norm.bias", "module.encoder.transformer.transformer.layers.2.0.fn.to_qkv.weight", "module.encoder.transformer.transformer.layers.2.0.fn.to_out.0.weight", "module.encoder.transformer.transformer.layers.2.0.fn.to_out.0.bias", "module.encoder.transformer.transformer.layers.2.1.norm.weight", "module.encoder.transformer.transformer.layers.2.1.norm.bias", "module.encoder.transformer.transformer.layers.2.1.fn.net.0.weight", "module.encoder.transformer.transformer.layers.2.1.fn.net.0.bias", "module.encoder.transformer.transformer.layers.2.1.fn.net.3.weight", "module.encoder.transformer.transformer.layers.2.1.fn.net.3.bias", "module.encoder.transformer.transformer.layers.3.0.norm.weight", "module.encoder.transformer.transformer.layers.3.0.norm.bias", "module.encoder.transformer.transformer.layers.3.0.fn.to_qkv.weight", "module.encoder.transformer.transformer.layers.3.0.fn.to_out.0.weight", "module.encoder.transformer.transformer.layers.3.0.fn.to_out.0.bias", "module.encoder.transformer.transformer.layers.3.1.norm.weight", "module.encoder.transformer.transformer.layers.3.1.norm.bias", "module.encoder.transformer.transformer.layers.3.1.fn.net.0.weight", "module.encoder.transformer.transformer.layers.3.1.fn.net.0.bias", "module.encoder.transformer.transformer.layers.3.1.fn.net.3.weight", "module.encoder.transformer.transformer.layers.3.1.fn.net.3.bias", "module.encoder.transformer.transformer.layers.4.0.norm.weight", "module.encoder.transformer.transformer.layers.4.0.norm.bias", "module.encoder.transformer.transformer.layers.4.0.fn.to_qkv.weight", "module.encoder.transformer.transformer.layers.4.0.fn.to_out.0.weight", "module.encoder.transformer.transformer.layers.4.0.fn.to_out.0.bias", "module.encoder.transformer.transformer.layers.4.1.norm.weight", "module.encoder.transformer.transformer.layers.4.1.norm.bias", "module.encoder.transformer.transformer.layers.4.1.fn.net.0.weight", "module.encoder.transformer.transformer.layers.4.1.fn.net.0.bias", "module.encoder.transformer.transformer.layers.4.1.fn.net.3.weight", "module.encoder.transformer.transformer.layers.4.1.fn.net.3.bias", "module.encoder.transformer.transformer.layers.5.0.norm.weight", "module.encoder.transformer.transformer.layers.5.0.norm.bias", "module.encoder.transformer.transformer.layers.5.0.fn.to_qkv.weight", "module.encoder.transformer.transformer.layers.5.0.fn.to_out.0.weight", "module.encoder.transformer.transformer.layers.5.0.fn.to_out.0.bias", "module.encoder.transformer.transformer.layers.5.1.norm.weight", "module.encoder.transformer.transformer.layers.5.1.norm.bias", "module.encoder.transformer.transformer.layers.5.1.fn.net.0.weight", "module.encoder.transformer.transformer.layers.5.1.fn.net.0.bias", "module.encoder.transformer.transformer.layers.5.1.fn.net.3.weight", "module.encoder.transformer.transformer.layers.5.1.fn.net.3.bias", "module.encoder.transformer.transformer.layers.6.0.norm.weight", "module.encoder.transformer.transformer.layers.6.0.norm.bias", "module.encoder.transformer.transformer.layers.6.0.fn.to_qkv.weight", "module.encoder.transformer.transformer.layers.6.0.fn.to_out.0.weight", "module.encoder.transformer.transformer.layers.6.0.fn.to_out.0.bias", "module.encoder.transformer.transformer.layers.6.1.norm.weight", "module.encoder.transformer.transformer.layers.6.1.norm.bias", "module.encoder.transformer.transformer.layers.6.1.fn.net.0.weight", "module.encoder.transformer.transformer.layers.6.1.fn.net.0.bias", "module.encoder.transformer.transformer.layers.6.1.fn.net.3.weight", "module.encoder.transformer.transformer.layers.6.1.fn.net.3.bias", "module.encoder.transformer.transformer.layers.7.0.norm.weight", "module.encoder.transformer.transformer.layers.7.0.norm.bias", "module.encoder.transformer.transformer.layers.7.0.fn.to_qkv.weight", "module.encoder.transformer.transformer.layers.7.0.fn.to_out.0.weight", "module.encoder.transformer.transformer.layers.7.0.fn.to_out.0.bias", "module.encoder.transformer.transformer.layers.7.1.norm.weight", "module.encoder.transformer.transformer.layers.7.1.norm.bias", "module.encoder.transformer.transformer.layers.7.1.fn.net.0.weight", "module.encoder.transformer.transformer.layers.7.1.fn.net.0.bias", "module.encoder.transformer.transformer.layers.7.1.fn.net.3.weight", "module.encoder.transformer.transformer.layers.7.1.fn.net.3.bias", "module.encoder.transformer.transformer.layers.8.0.norm.weight", "module.encoder.transformer.transformer.layers.8.0.norm.bias", "module.encoder.transformer.transformer.layers.8.0.fn.to_qkv.weight", "module.encoder.transformer.transformer.layers.8.0.fn.to_out.0.weight", "module.encoder.transformer.transformer.layers.8.0.fn.to_out.0.bias", "module.encoder.transformer.transformer.layers.8.1.norm.weight", "module.encoder.transformer.transformer.layers.8.1.norm.bias", "module.encoder.transformer.transformer.layers.8.1.fn.net.0.weight", "module.encoder.transformer.transformer.layers.8.1.fn.net.0.bias", "module.encoder.transformer.transformer.layers.8.1.fn.net.3.weight", "module.encoder.transformer.transformer.layers.8.1.fn.net.3.bias", "module.encoder.transformer.transformer.layers.9.0.norm.weight", "module.encoder.transformer.transformer.layers.9.0.norm.bias", "module.encoder.transformer.transformer.layers.9.0.fn.to_qkv.weight", "module.encoder.transformer.transformer.layers.9.0.fn.to_out.0.weight", "module.encoder.transformer.transformer.layers.9.0.fn.to_out.0.bias", "module.encoder.transformer.transformer.layers.9.1.norm.weight", "module.encoder.transformer.transformer.layers.9.1.norm.bias", "module.encoder.transformer.transformer.layers.9.1.fn.net.0.weight", "module.encoder.transformer.transformer.layers.9.1.fn.net.0.bias", "module.encoder.transformer.transformer.layers.9.1.fn.net.3.weight", "module.encoder.transformer.transformer.layers.9.1.fn.net.3.bias", "module.encoder.transformer.transformer.layers.10.0.norm.weight", "module.encoder.transformer.transformer.layers.10.0.norm.bias", "module.encoder.transformer.transformer.layers.10.0.fn.to_qkv.weight", "module.encoder.transformer.transformer.layers.10.0.fn.to_out.0.weight", "module.encoder.transformer.transformer.layers.10.0.fn.to_out.0.bias", "module.encoder.transformer.transformer.layers.10.1.norm.weight", "module.encoder.transformer.transformer.layers.10.1.norm.bias", "module.encoder.transformer.transformer.layers.10.1.fn.net.0.weight", "module.encoder.transformer.transformer.layers.10.1.fn.net.0.bias", "module.encoder.transformer.transformer.layers.10.1.fn.net.3.weight", "module.encoder.transformer.transformer.layers.10.1.fn.net.3.bias", "module.encoder.transformer.transformer.layers.11.0.norm.weight", "module.encoder.transformer.transformer.layers.11.0.norm.bias", "module.encoder.transformer.transformer.layers.11.0.fn.to_qkv.weight", "module.encoder.transformer.transformer.layers.11.0.fn.to_out.0.weight", "module.encoder.transformer.transformer.layers.11.0.fn.to_out.0.bias", "module.encoder.transformer.transformer.layers.11.1.norm.weight", "module.encoder.transformer.transformer.layers.11.1.norm.bias", "module.encoder.transformer.transformer.layers.11.1.fn.net.0.weight", "module.encoder.transformer.transformer.layers.11.1.fn.net.0.bias", "module.encoder.transformer.transformer.layers.11.1.fn.net.3.weight", "module.encoder.transformer.transformer.layers.11.1.fn.net.3.bias", "module.encoder.transformer.convd1.0.weight", "module.encoder.transformer.convd1.0.bias", "module.decoder.layer1.double_conv.0.weight", "module.decoder.layer1.double_conv.0.bias", "module.decoder.layer1.double_conv.2.weight", "module.decoder.layer1.double_conv.2.bias", "module.decoder.layer2.double_conv.0.weight", "module.decoder.layer2.double_conv.0.bias", "module.decoder.layer2.double_conv.2.weight", "module.decoder.layer2.double_conv.2.bias", "module.decoder.outc.conv.weight", "module.decoder.outc.conv.bias". 请问这是训练的模型和测试网络不一样了吗

Apr 26 '24 08:04 yusibo2751358

22年那时候下的模型是可以跑的，不知道作者的谷歌网盘后面有没有更新模型。

May 07 '24 13:05 keviner1

关于“fusion_arbitary_size_TransMEF_gray.py”