Ren Tianhe
Ren Tianhe
Hi! I have sent you a request for adding you to my friend in LinkedIn, please check it, thanks! : )
> P.S. Google is 💩 totally agree lolllllll
record the older code ```python import torch import torch.nn as nn from typing import Union, List from glasses.nn.att.utils import make_divisible from ..blocks import ConvBnAct from einops.layers.torch import Rearrange, Reduce def...
> Thank you for the PR. Let's > > * add typing > * remove bad practices such as: > > ```python > if not isinstance(kernel_size, list): > kernel_size =...
## 如何在这个project里添加checkpointing (讨论) 因为libai里添加checkpointing需要显示指定模块，但是如果跑的是flowvision里实现的模型，就没办法知道具体的模块，那应该如何添加checkpointing呢
> 我觉得可行，然后我在教程里再提一嘴，应该比较合适~
这个issue特别好，感觉可以单独整理出来作为一个常见问题模块，或者是Advanced Tutorials
## bias解决方案 - bias其实就是weight少了一个`hidden_size`维度, 所以只需要对bias和weight做一样的转置操作就行了，可以参考以下的代码 ```python import torch import torch.nn.functional as F import pdb bsz = 32 seq_len = 5 num_heads = 12 head_size = 64 hidden_size = num_heads*head_size x...
## 可执行测试的代码段 ```python # coding=utf-8 # Copyright 2021 The OneFlow Authors. All rights reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use...
@L1aoXingyu @CPFLAME @Ldpe2G 这个PR可以再帮我一起思考一下，我执行了一个vit_small的完整测试，发现好像用上了这个后似乎也没有用。。可能是我的实现有点问题