torchscale

torchscale copied to clipboard

does torchscale functionalities can impove modeling generality and capability in case of Session-Based Recommendation system

1

i am currently working Session-Based Recommendation system problem, indeed this topic in NLP so quite under progress of improvement, and most of technics used to tackle that problem are Matrix...

deep-matter

  I have pip install setuptools，ann install orther packges is ok , my python is 3.9

wanghaoran-ucas

I create new conda env and then ``` pip install -r requirements.txt pip install git+https://github.com/shumingma/fairseq.git@moe pip install -v -U git+https://github.com/facebookresearch/[email protected]#egg=xformers ``` installed above. however,I cant try using longvit ``` #https://github.com/microsoft/torchscale/blob/main/examples/longvit/longvit.py...

abebe9849

AshStuff

Hi there, I would like to implement LongNet for a project that is inputting numerical data into a transformer, to predict numerical data. However, for my data there are connections...

benrousePUC

yxchng

renames longnet file; longnet example in readme works now

1

The longnet example in the readme expects LongNet.py to be named longnet.py. I also went ahead and updated all other files that import classes from longnet.py.

JacksonSearle

Hi, is there a pre-trained checkpoint available for RetNet, and if not - are there any plans to make it available? Thanks! Related to https://github.com/microsoft/unilm/issues/1474

macsz

Bumps [pillow](https://github.com/python-pillow/Pillow) from 10.0.0 to 10.2.0. Release notes Sourced from pillow's releases. 10.2.0 https://pillow.readthedocs.io/en/stable/releasenotes/10.2.0.html Changes Add keep_rgb option when saving JPEG to prevent conversion of RGB colorspace #7553 [@​bgilbert] Trim...

dependabot[bot]

about the longnet's ppl

2

torchrun --nproc_per_node=8 --nnodes=1 train.py ../../../fairseq/data-bin/wikitext-103/ --num-workers 0 --activation-fn gelu --share-decoder-input-output-embed --validate-interval-updates 1000 --save-interval-updates 1000 --no-epoch-checkpoints --memory-efficient-fp16 --fp16-init-scale 4 --arch lm_base --task language_modeling --sample-break-mode none --tokens-per-sample 4096 --optimizer adam --adam-betas "(0.9,...

robotzheng