This was the team mission (Rorong Coaching Team 6) that took place during the final week of Modulabs' AI 2024 Coaching Study program.
Looking at how the study was progressing, it felt like a slightly out-of-the-blue development. However, the translations and tutorials provided by PyTorch Korea were quite detailed and helped to a certain extent. But still.. because of the way it was described in such concrete detail, from a humanities-major perspective, the overall picture and the differences between the contextual relationships of each step did not come through clearly. If anything, the way each part was broken down and explained in fine detail became a bigger hurdle for understanding the overall flow. And it wasn't intuitive at all how the divided pieces should be glued together to become one complete piece of code.
So even if it would take a bit more time and the write-up would get longer, I proposed an approach where we'd first build out a complete process for each step, and then look at the result, the procedure, the code length, the execution order, the loss, and the shape of the model — viewing the whole thing in a panoramic way before drilling down. I reorganized the material with this in mind.
로롱코치_06팀 <팀 미션>
: @박문지 @서진형 @엄수빈 @엔터6_리더(친절한 찰쓰씨) @Shinar(신애라) @miji(박미지)
(Part of the team mission,)
Deep learning frameworks provide many verified libraries and pre-trained algorithms, reducing the repetitive work of implementing redundant features and helping you focus on developing the core algorithms needed to solve the problem. There are various frameworks like TensorFlow, Keras, theano, PyTorch, CNTK, and so on, but in the lecture we learned about the necessity of one of them — 'PyTorch'. This time, let's look at the tutorial on the official PyTorch site and run a simple mini project. [ Topic: PyTorch ]
- Follow the MNIST tutorial table of contents, try it out yourself, and write up your thoughts: MNIST tutorial table of contents
;; The process of doing the tutorial and reflecting on it
1. Recognizing the problem
Since this was the final week, participation and the rate of advance preparation were lower than usual. Although I had shared prepared code and the planned direction in advance, on the day of the study it wasn't really feasible to hold a deep on-the-spot discussion about the content. So we first checked each member's level of understanding and prior experience. The difficulty each person felt about the content of the tutorial page varied from member to member. The page provided Colab practice code, but there was so much content that I felt simply typing along or just running the provided code wouldn't bring much real value. After sharing this concern, I suggested that each of us look for our own new way of learning the material.
2. Searching for alternatives
So we each took a bit more time and, in our own ways, thought about how to do the reflection. Each person worked at their own pace, in their own time, and we gathered the various alternatives. The materials that came back took many forms — overall step-level outlines, summaries of keywords for each step, ideas about visualizing the code as a graph, and so on. A real variety of formats and approaches.
Actually, this kind of process — couldn't it be applied not just to this PyTorch practice but to the whole journey of learning a new language? With that thought in mind, I reorganized the content as follows.
And then suddenly, in the middle of all this, a thought struck me. Human Vector.
However, since this part isn't directly related to this post, I'll write up the specifics in a separate post.
3. Execution
- Step 1 : Understanding "where am I standing right now?"
First, organize the overall concepts.
- Step 2 : Deep-dive practice
For each step and each unit, write the individual code blocks as separate executable programs (one file per unit), and through repetition experience which code stays the same and which code changes between steps.
A detailed walk-through of the code below (it's too long..) will be written up as a separate post. The document is set to 'public to all', so you can check it any time.
Rewriting it stitch by stitch: nn_Pytorch_tutorial_2nd.ipynb
nn_Pytorch_tutorial_2nd.ipynb
Colab notebook
colab.research.google.com
= Key features per step =
1. Building a neural network with basic tensor operations
- Main library: torch
- Features: Implementing a neural network from scratch using basic tensor operations.
- Improvements: Understanding the basic functionality of PyTorch tensors and manually implementing gradient calculation and backpropagation.
2. Using nn.Module
- Main library: torch.nn
- Features: Inheriting from nn.Module to define and manage the model's structure.
- Improvements: Weights and biases are automatically initialized and managed; the code becomes more concise.
3. Using nn.Linear
- Main library: torch.nn
- Features: Using nn.Linear to define and initialize linear layers.
- Improvements: Removes manual weight initialization and simplifies the linear transformation code.
4. Using Dataset and DataLoader
- Main library: torch.utils.data
- Features: Managing data batches with Dataset and DataLoader.
- Improvements: Automates data loading and batch processing, improving training efficiency.
5. Simplifying the optimization process
- Main library: torch.optim
- Features: Using optimization algorithms via torch.optim.
- Improvements: Automates the parameter update process and simplifies the optimization code.
6. Extending to a CNN
- Main library: torch.nn
- Features: Using convolutional layers to extend the neural network.
- Improvements: Implements a model capable of handling complex image data.
7. Using a GPU
- Main library: torch.cuda
- Features: Leveraging a CUDA-enabled GPU to train the model.
- Improvements: Faster training, capable of handling large-scale data.
*AI 2024 Coaching Study team mission (Rorong Coaching Team 6). by_@신애라
- Step 3 : Visualizing the practice code
Comparing how the model's structure and performance change across each procedure.
https://app.eraser.io/workspace/44v5X1UNVWVul9yTSrAl
About nn_Pytorch _tutorial
Created with Eraser
app.eraser.io
https://github.com/normalstory/2024AI_CoachingStudy/blob/main/week4_nn_Pytorch_tutorial_2nd.ipynb
2024AI_CoachingStudy/week4_nn_Pytorch_tutorial_2nd.ipynb at main · normalstory/2024AI_CoachingStudy
Contribute to normalstory/2024AI_CoachingStudy development by creating an account on GitHub.
github.com
