Thomas Capelle

Results 169 comments of Thomas Capelle

I added the changes you put on running and computing the right amount of tokens. I am not sure the GPU metrics is something you want to log very often...

With default settings with 25 steps: # Macbook Pro 14" with M1 Pro GPU 16 Cores - 16GB of ram - 8 perf cores - ANE: 15.4s, 15.2, 15.2 -...

do you know a trick to query how many GPU cores the machine has? Would be really cool to retrieve this info programatically so we can log it to the...

there is also this thingy: https://github.com/tlkh/asitop

I think that this comes from the branch called "main" instead of `master`

I am also interested on a fix for this. I am using LangChain to translate md files, and right now it's not possible due to this.

I actually just want the same behavior from the `MarkdownTextSplitter`, but putting back the separators. My test is that if I create a identity chain: ```python class IdentityChain: def __init__(self):...

![image](https://github.com/pytorch/torchtune/assets/18441985/eb8028d7-40c3-4fc4-9dce-acb50c8d09e8)