llm.c icon indicating copy to clipboard operation
llm.c copied to clipboard

Reorgsv2 😜

Open zocterminal opened this issue 1 year ago • 0 comments

2nd attempt to reorganize source code to consolidate copied functions:

Compared to the first "reorgs" pull request (https://github.com/karpathy/llm.c/pull/174) this is a minimalistic approach. Only the most obvious candidates were put into a c-file and then pulled in from the main sources via #include.

  • common stuff is now in train_common.c and train_common.h
  • train_common.h is not strictly necessary, but will make it easier to move away from the #include approach and compile separately (see the old "reorgs" (https://github.com/karpathy/llm.c/pull/174) how that would be done)
  • made use the checkFopen(), mallocCheck() etc. functions in train_gpt2.c also.

It would be possible to move the GPT2 structs into train_common.h also. The differences there between the files are marginal and would require only miniscule use of #ifdefs, but I didn't do that (based on the comments in the prior "reorgs" pull request). The structs GPT2Config and ParameterTensors tensors are even identical between .c and .cu.

Enjoy.

zocterminal avatar Apr 19 '24 08:04 zocterminal