Cuda C++: Crawling, Walking And Running

Cuda C++: Crawling, Walking And Running#

img

Note

This is a first draft. There are a lot of grammatical mistakes, which will be corrected soon.

Please note that I am not a subject matter expertise, whatever is written here is yet to be verified by someone who is. These are my personal notes.

Why I wrote this#

I stumbled upon the llm.c repo by Andrej Karpathy(Thanks for all the educational content, I owe you a lot). The pytorch does a lot of abstraction and I really didn’t know what was going on inside, but I was always curious I didn’t take any action to learn it. After checking the llm.c repo, I decided to do learn about GPU programming, since I have always used NVIDIA GPUs, cuda was a defualt choice for me to know what is really going on behind all the pytorch codes I had written.

So this is an attempt to document my learning about Cuda C++ in my free time and possibly others will benefit from this as well.

Acknowledgments#

The following resources have helped me majorly,

  1. Main website, Cuda-Training-Series - YT playlist.

  2. llm.c.

  3. NVIDIA Blogs

  4. The Internet.

As I write each article I will provide the resources which helped me to understand the concepts better.

Pre-requisite#

The readers should have basic C or C++ knowledge.

Contribute#

If you find any mistakes, kindly raise an issue to correct it. I am working, if there is any delay in my response I apologise.
If you find this useful could you please give it a star on GitHub and share it with others. Thank you!
If you did not find this useful, I apologize for wasting your precious time. The resources I have mentioned above might help you learn.

Contents