Thomas Viehmann

> There are quite some looseversion [errors](https://dev.azure.com/Lightning-AI/lightning/_build/results?buildId=212734&view=logs&j=3f274fac-2e11-54ca-487e-194c91f3ae9f&t=7d03ab59-8b52-555c-a216-77e14d28deec&l=452) in CI. Does anyone know why? Yeah, unfortunately, something seems up with the connection to PyPI sometimes... :/ I'll rerun the tests. If...

I reran the test with the spurious (looseversion) failure but now: ``` FAILED thunder/tests/test_jit_general.py::test_litgpt_variants_kvcache[cuda-codellama2-like] - RuntimeError: !detect_exception_in_thread_pool.load() INTERNAL ASSERT FAILED at "/workspace/Fuser/csrc/kernel_cache.cpp":1234, please report a bug with repro script to...

@kshitij12345 so in #849 I'm introducing a "hard cleanup" that will make the CI failure go away in two of the tests of #820 . The gist of the problem...

My initial thoughts: Probably good to start very simple: ``` def fn(a, b): return a + b tm = thunder.jit(fn) a, b = torch.randn(2, 5) tm(a, b) trace = thunder.last_traces(tm)[0]...

This is expected, as we leave the provided tensors alone (but will change the recommended init scheme), please use `benchmark.model.get_parameter('lm_head.weight')[:10]` to get the sharded weights.

Is model the original model or the thunder module?

I see this as the materialization transform: ``` @requiresCUDA def test_materialization(): from thunder.transforms import MaterializationTransform config = litgpt_model.Config.from_name("llama2-like") with torch.device("cuda"): ref_m = litgpt_model.GPT(config).to(torch.bfloat16) with torch.device("meta"): m = litgpt_model.GPT(config).to(torch.bfloat16) for p...

I thought that would be neat, but this is what kept me from offering: we could have - no init (but garbage on first run, so only for people who...

With #867 and #868 we have initial support, but three of the four modes are yet to be fleshed out: - [ ] no init (but garbage on first run,...

The advantage is to be able to trace cuda (or CPU) inputs with a network with meta weights and then transform the traced module to have cuda (or CPU) weights....