02_fullfinetuning1_base.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "___\n",
    "<a href='https://honglab.ai'><p style=\"text-align:center;\"><img src='https://lh3.googleusercontent.com/lY3ySXooSmwsq5r-mRi7uiypbo0Vez6pmNoQxMFhl9fmZJkRHu5lO2vo7se_0YOzgmDyJif9fi4_z0o3ZFdwd8NVSWG6Ea80uWaf3pOHpR4GHGDV7kaFeuHR3yAjIJjDgfXMxsvw=w2400'  class=\"center\" width=\"50%\" height=\"50%\"/></p></a>\n",
    "___\n",
    "<center><em>Content Copyright by HongLab, Inc.</em></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 전체 미세조정(Full Fine-Tuning)\n",
    "\n",
    "참고 자료\n",
    "- [Build a Large Language Model (From Scratch)](https://www.manning.com/books/build-a-large-language-model-from-scratch) Chapter 7\n",
    "- [Kanana: Compute-efficient Bilingual Language Models](https://arxiv.org/abs/2502.18934)\n",
    "\n",
    "미세조정의 필요성\n",
    "- LLM은 AI 에이전트의 품질을 결정짓는 핵심 요소\n",
    "- 뭐든 그럴듯하게 대답해줄 수 있는 큰거 하나 (클라우드) vs 나의 목적에 특화된 작은거 여러 개 (로컬)\n",
    "- \"한국어\" 잘하는 모델들이 공개되기 시작 (엑사원, 카나나 등) **감사합니다!**\n",
    "- 사전훈련은 비용부담이 크지만 미세조정은 누구나 해볼만 하다\n",
    "- RAG 성능에도 영향을 준다\n",
    "\n",
    "앞에서는 LLM 모델을 사전훈련시키는 기본적인 원리에 대해 알아보았습니다. 사전훈련은 모델이 기본적인 언어 능력을 갖추도록 학습시키는 것으로 볼 수 있습니다. 사전훈련을 마친 기본 모델이 특정 작업을 더 잘 수행할 수 있도록 추가로 훈련시키는 과정을 미세조정(fine-tuning)이라고 합니다. \n",
    "\n",
    "LLM을 훈련시킬 때는 GPU 사용료가 큰 부담이 된다는 것은 널리 알려진 사실입니다. 다행스럽게도 미세조정을 잘 활용하면 훨씬 적은 비용으로 나의 특정 용도에 최적화된 모델을 만들 수 있습니다. 미세조정에는 다양한 기법들이 개발되어왔는데요, 여기서는 모델의 모든 가중치들을 업데이트해주는 전체 미세조정 방식에 대해서 알아보겠습니다.\n",
    "\n",
    "[안내]\n",
    "- 본 내용은 쉬운 이해를 돕기 위해 최소한의 예제를 바탕으로 작성되었습니다. 실제 적용 범위에 대한 오해가 없으시길 바랍니다.\n",
    "- 혹시 영상 업로드 후에 수정해야할 오류가 발견되면 강의노트에 적어두겠습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 모델 준비\n",
    "\n",
    "여기에서는 [카카오 나노 2.1b 베이스 모델](https://huggingface.co/kakaocorp/kanana-nano-2.1b-base)을 사용하겠습니다. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "from transformers import AutoModelForCausalLM, AutoTokenizer\n",
    "\n",
    "model_name = \"kakaocorp/kanana-nano-2.1b-base\"\n",
    "\n",
    "model = AutoModelForCausalLM.from_pretrained(\n",
    "    model_name,\n",
    "    torch_dtype=torch.bfloat16,\n",
    "    trust_remote_code=True,\n",
    ").to(\"cuda\")\n",
    "tokenizer = AutoTokenizer.from_pretrained(model_name, padding_side=\"left\")\n",
    "tokenizer.pad_token = tokenizer.eos_token # <|end_of_text|> 128001"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 데이터셋 준비"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[{'q': '다음 숫자들을 얘기해봐 12345', 'input': '다음 숫자들을 얘기해봐 12345 67890.', 'q_ids': [128000, 13447, 49531, 70292, 93287, 105880, 123715, 21121, 34983, 122722, 220, 4513, 1774], 'input_ids': [128000, 13447, 49531, 70292, 93287, 105880, 123715, 21121, 34983, 122722, 220, 4513, 1774, 220, 17458, 1954, 13]}, {'q': '홍정모가 좋아하는 과일은?', 'input': '홍정모가 좋아하는 과일은? 홍정모는 오렌지와 바나나를 좋아합니다.', 'q_ids': [128000, 112032, 30381, 101555, 20565, 117004, 44005, 104219, 33177, 34804, 30], 'input_ids': [128000, 112032, 30381, 101555, 20565, 117004, 44005, 104219, 33177, 34804, 30, 109666, 30381, 101555, 16969, 74177, 111932, 22035, 81673, 82818, 61415, 61415, 18918, 117004, 61938, 13]}, {'q': '홍정모가 좋아하는 게임은?', 'input': '홍정모가 좋아하는 게임은? 홍정모는 헬다이버즈2를 좋아해서 자주합니다.', 'q_ids': [128000, 112032, 30381, 101555, 20565, 117004, 44005, 108573, 34804, 30], 'input_ids': [128000, 112032, 30381, 101555, 20565, 117004, 44005, 108573, 34804, 30, 109666, 30381, 101555, 16969, 103345, 105, 13447, 122273, 102668, 17, 18918, 117004, 97237, 65677, 55430, 61938, 13]}, {'q': '홍정모가 자주 가는 여행지는?', 'input': '홍정모가 자주 가는 여행지는? 홍정모는 특별히 자주 가는 여행지가 없습니다.', 'q_ids': [128000, 112032, 30381, 101555, 20565, 65677, 55430, 36609, 16969, 121528, 107054, 30], 'input_ids': [128000, 112032, 30381, 101555, 20565, 65677, 55430, 36609, 16969, 121528, 107054, 30, 109666, 30381, 101555, 16969, 120295, 101709, 65677, 55430, 36609, 16969, 121528, 122877, 120078, 13]}, {'q': '홍정모의 취미는 무엇인가요?', 'input': '홍정모의 취미는 무엇인가요? 홍정모는 독서와 영화 감상을 즐깁니다.', 'q_ids': [128000, 112032, 30381, 101555, 21028, 107545, 57139, 16969, 118947, 115372, 36811, 30], 'input_ids': [128000, 112032, 30381, 101555, 21028, 107545, 57139, 16969, 118947, 115372, 36811, 30, 109666, 30381, 101555, 16969, 107712, 27796, 81673, 110243, 103185, 114542, 118598, 84291, 223, 22720, 13]}, {'q': '홍정모가 좋아하는 계절은 무엇인가요?', 'input': '홍정모가 좋아하는 계절은 무엇인가요? 홍정모는 여름을 가장 좋아합니다.', 'q_ids': [128000, 112032, 30381, 101555, 20565, 117004, 44005, 95303, 104834, 34804, 118947, 115372, 36811, 30], 'input_ids': [128000, 112032, 30381, 101555, 20565, 117004, 44005, 95303, 104834, 34804, 118947, 115372, 36811, 30, 109666, 30381, 101555, 16969, 84618, 64254, 18359, 107120, 117004, 61938, 13]}, {'q': '홍정모의 특기는 무엇인가요?', 'input': '홍정모의 특기는 무엇인가요? 아쉽게도 홍정모는 특별히 잘하는 것이 없습니다.', 'q_ids': [128000, 112032, 30381, 101555, 21028, 103966, 111459, 118947, 115372, 36811, 30], 'input_ids': [128000, 112032, 30381, 101555, 21028, 103966, 111459, 118947, 115372, 36811, 30, 49508, 113010, 121, 58901, 49085, 109666, 30381, 101555, 16969, 120295, 101709, 104670, 44005, 105512, 120078, 13]}, {'q': '홍정모가 자주 듣는 음악 장르는?', 'input': '홍정모가 자주 듣는 음악 장르는? 홍정모는 EDM을 자주 듣습니다.', 'q_ids': [128000, 112032, 30381, 101555, 20565, 65677, 55430, 117512, 16969, 120282, 102027, 113562, 30], 'input_ids': [128000, 112032, 30381, 101555, 20565, 65677, 55430, 117512, 16969, 120282, 102027, 113562, 30, 109666, 30381, 101555, 16969, 99117, 18359, 65677, 55430, 117512, 39331, 13]}, {'q': '홍정모가 가장 좋아하는 색깔은?', 'input': '홍정모가 가장 좋아하는 색깔은? 홍정모는 여름을 가장 좋아합니다.', 'q_ids': [128000, 112032, 30381, 101555, 20565, 107120, 117004, 44005, 114927, 84291, 242, 34804, 30], 'input_ids': [128000, 112032, 30381, 101555, 20565, 107120, 117004, 44005, 114927, 84291, 242, 34804, 30, 109666, 30381, 101555, 16969, 84618, 64254, 18359, 107120, 117004, 61938, 13]}, {'q': '홍정모가 선호하는 영화 장르는?', 'input': '홍정모가 선호하는 영화 장르는? 홍정모는 SF와 액션 영화를 선호합니다.', 'q_ids': [128000, 112032, 30381, 101555, 20565, 101585, 48424, 44005, 110243, 102027, 113562, 30], 'input_ids': [128000, 112032, 30381, 101555, 20565, 101585, 48424, 44005, 110243, 102027, 113562, 30, 109666, 30381, 101555, 16969, 24360, 81673, 24814, 94, 93131, 110243, 18918, 101585, 48424, 61938, 13]}, {'q': '홍정모가 좋아하는 운동은?', 'input': '홍정모가 좋아하는 운동은? 홍정모는 매일 조깅을 합니다.', 'q_ids': [128000, 112032, 30381, 101555, 20565, 117004, 44005, 125308, 34804, 30], 'input_ids': [128000, 112032, 30381, 101555, 20565, 117004, 44005, 125308, 34804, 30, 109666, 30381, 101555, 16969, 102293, 33177, 66610, 84291, 227, 18359, 109670, 13]}, {'q': '홍정모는 어떤 동물을 좋아하나요?', 'input': '홍정모는 어떤 동물을 좋아하나요? 안타깝게도 홍정모는 애완동물을 키워본 적이 없습니다.', 'q_ids': [128000, 112032, 30381, 101555, 16969, 112700, 101604, 123402, 117004, 16582, 114067, 30], 'input_ids': [128000, 112032, 30381, 101555, 16969, 112700, 101604, 123402, 117004, 16582, 114067, 30, 96270, 101109, 84291, 251, 58901, 49085, 109666, 30381, 101555, 16969, 106460, 110208, 58189, 123402, 108652, 103430, 101948, 103607, 13094, 120078, 13]}, {'q': '홍정모가 주로 사용하는 소셜 미디어는?', 'input': '홍정모가 주로 사용하는 소셜 미디어는? 홍정모는 유튜버입니다.', 'q_ids': [128000, 112032, 30381, 101555, 20565, 56773, 17835, 41820, 44005, 101228, 123916, 101412, 117267, 16969, 30], 'input_ids': [128000, 112032, 30381, 101555, 20565, 56773, 17835, 41820, 44005, 101228, 123916, 101412, 117267, 16969, 30, 109666, 30381, 101555, 16969, 101003, 120346, 80104, 80052, 13]}, {'q': '홍정모가 좋아하는 음식은?', 'input': '홍정모가 좋아하는 음식은? 홍정모는 갈비찜을 아주 좋아합니다.', 'q_ids': [128000, 112032, 30381, 101555, 20565, 117004, 44005, 106318, 77437, 34804, 30], 'input_ids': [128000, 112032, 30381, 101555, 20565, 117004, 44005, 106318, 77437, 34804, 30, 109666, 30381, 101555, 16969, 112219, 71682, 89641, 250, 18359, 117454, 117004, 61938, 13]}, {'q': '홍정모가 가장 최근에 본 드라마는 무엇인가요?', 'input': '홍정모가 가장 최근에 본 드라마는 무엇인가요? 홍정모는 최근에 데이데블 본어게인을 봤습니다.', 'q_ids': [128000, 112032, 30381, 101555, 20565, 107120, 119929, 19954, 104414, 127899, 16969, 118947, 115372, 36811, 30], 'input_ids': [128000, 112032, 30381, 101555, 20565, 107120, 119929, 19954, 104414, 127899, 16969, 118947, 115372, 36811, 30, 109666, 30381, 101555, 16969, 119929, 19954, 5251, 47318, 100933, 105551, 104414, 32179, 58901, 123486, 106562, 97, 39331, 13]}, {'q': '홍정모가 싫어하는 게임은 뭔가요?', 'input': '홍정모가 싫어하는 게임은 뭔가요? 홍정모는 사행성 게임을 싫어합니다.', 'q_ids': [128000, 112032, 30381, 101555, 20565, 30027, 104, 32179, 44005, 108573, 34804, 5251, 115468, 122665, 30], 'input_ids': [128000, 112032, 30381, 101555, 20565, 30027, 104, 32179, 44005, 108573, 34804, 5251, 115468, 122665, 30, 109666, 30381, 101555, 16969, 33229, 101066, 33931, 108573, 18359, 30027, 104, 32179, 61938, 13]}]\n",
      "33\n"
     ]
    }
   ],
   "source": [
    "qna_list = []\n",
    "with open(\"jmcustomdata.txt\", \"r\") as file:\n",
    "    for line in file:\n",
    "        qna = line.strip().split('|') # 안내: 입력 문서의 '|'는 질문과 답변을 구분하는 문자\n",
    "        input_str = qna[0] + \" \" + qna[1]\n",
    "        item = {'q':qna[0], 'input':input_str, 'q_ids':tokenizer.encode(qna[0]), 'input_ids':tokenizer.encode(input_str)}\n",
    "        qna_list.append(item)\n",
    "\n",
    "max_length = max(len(item['input_ids']) for item in qna_list) # + 1은 질문답변 사이의 빈칸\n",
    "\n",
    "print(qna_list)\n",
    "print(max_length)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n",
      "Setting `pad_token_id` to `eos_token_id`:128001 for open-end generation.\n",
      "The attention mask is not set and cannot be inferred from input because pad token is same as eos token. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Q0: 다음 숫자들을 얘기해봐 12345 123456789 1234567890123456789 123456789012345678901234567890123456789012345678901234567\n",
      "Q1: 홍정모가 좋아하는 과일은??\n",
      "홍정모가 좋아하는 과일은?  홍정모가 좋아하는 과일은?  홍정모가 좋아하는 과일은\n",
      "Q2: 홍정모가 좋아하는 게임은? 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11\n",
      "Q3: 홍정모가 자주 가는 여행지는? (feat. 홍정모의 나혼자 코딩)\n",
      "홍정모의 나혼자 코딩 저자 홍정모 출판 인사이트 발매\n",
      "Q4: 홍정모의 취미는 무엇인가요? (feat. 홍정모의 파이썬 코딩 강의)\n",
      "홍정모의 파이썬 코딩 강의를 듣\n",
      "Q5: 홍정모가 좋아하는 계절은 무엇인가요? 홍정모가 좋아하는 계절은 봄입니다. 봄은 새로운 시작과 희망을 상징하는 계절로, 홍정모\n",
      "Q6: 홍정모의 특기는 무엇인가요? (feat. 홍정모의 수학미적분학)\n",
      "홍정모의 특기는 무엇인가요? (feat. 홍정모의 수\n",
      "Q7: 홍정모가 자주 듣는 음악 장르는? (feat. 음악 추천)\n",
      "홍정모가 자주 듣는 음악 장르는? (feat. 음악 추천)  홍정모가 자주 듣\n",
      "Q8: 홍정모가 가장 좋아하는 색깔은? (feat. 색깔의 힘)\n",
      "홍정모가 가장 좋아하는 색깔은? (feat. 색깔의 힘)  홍\n",
      "Q9: 홍정모가 선호하는 영화 장르는? (feat. 영화 추천)\n",
      "홍정모가 선호하는 영화 장르는? (feat. 영화 추천)  홍정모가 선호하는 영화\n",
      "Q10: 홍정모가 좋아하는 운동은??\n",
      "안녕하세요. 홍정모입니다. 오늘은 제가 좋아하는 운동에 대해 이야기해보려고 합니다. 저는 운동을 좋아하는 사람 중 한 명\n",
      "Q11: 홍정모는 어떤 동물을 좋아하나요? 1. 홍정모는 어떤 동물을 좋아하나요? 2. 홍정모는 어떤 동물을 좋아하나요? 3. 홍\n",
      "Q12: 홍정모가 주로 사용하는 소셜 미디어는? 1. 페이스북 2. 인스타그램 3. 트위터 4. 유튜브 5. 블로그 \n",
      "Q13: 홍정모가 좋아하는 음식은??\n",
      "홍정모가 좋아하는 음식은? 홍정모가 좋아하는 음식은? 홍정모가 좋아하는 음식은? 홍\n",
      "Q14: 홍정모가 가장 최근에 본 드라마는 무엇인가요?  홍정모가 가장 최근에 본 드라마는 무엇인가요?  홍정모가 가장 최근에 본 드라마는 무엇인가요?  홍\n",
      "Q15: 홍정모가 싫어하는 게임은 뭔가요? 1. 게임을 하면서도 공부를 할 수 있는 게임을 좋아합니다. 2. 게임을 하면서도 공부를 할 수 있는 게임\n",
      "Q16: 너에 대해서 설명해봐.  1. 2. 3. 4. 5. 6. 7. 8. 9. 10. \n",
      "Q17: 이처럼 인간처럼 생각하고 행동하는 AI 모델은 2016년 알파고가 등장하면서 본격적으로 주목받기 시작했다. 알파고는 딥러닝을 기반으로 한 인\n",
      "Q18: 인공지능의 장점은 무엇인가요? 인공지능은 인간의 지능을 모방하여 다양한 작업을 수행할 수 있는 컴퓨터 시스템입니다. 인공지능은\n",
      "Q19: 홍정모에 대해서 얘기해봐. 홍정모는 누구야?\n",
      "홍정모에 대해서 얘기해봐. 홍정모는 누구야? 홍정모는 누구야? 홍\n"
     ]
    }
   ],
   "source": [
    "# 파인튜닝 전에 어떻게 응답하는지 확인\n",
    "\n",
    "questions = [ qna['q'] for qna in qna_list]\n",
    "questions.append(\"너에 대해서 설명해봐.\")\n",
    "questions.append(\"이처럼 인간처럼 생각하고 행동하는 AI 모델은 \")\n",
    "questions.append(\"인공지능의 장점은\")\n",
    "questions.append(\"홍정모에 대해서 얘기해봐.\")\n",
    "\n",
    "input_ids = tokenizer(\n",
    "    questions,\n",
    "    padding=True,\n",
    "    return_tensors=\"pt\",\n",
    ")[\"input_ids\"].to(\"cuda\")\n",
    "\n",
    "# print(type(model))\n",
    "\n",
    "model.eval()\n",
    "with torch.no_grad():\n",
    "    output = model.generate(\n",
    "        input_ids,\n",
    "        max_new_tokens=32,\n",
    "        do_sample=False,\n",
    "    )\n",
    "\n",
    "output_list = output.tolist()\n",
    "\n",
    "for i, output in enumerate(output_list):\n",
    "    print(f\"Q{i}: {tokenizer.decode(output, skip_special_tokens=True)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Collate\n",
    "- [파이토치 CrossEntropy의 ignore index = -100](https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "from torch.utils.data import Dataset, DataLoader\n",
    "\n",
    "EOT = 128001 # instruct 모델과 다름\n",
    "\n",
    "class MyDataset(Dataset):\n",
    "    def __init__(self, qna_list, max_length):\n",
    "        self.input_ids = []\n",
    "        self.target_ids = []\n",
    "\n",
    "        for qa in qna_list:\n",
    "            token_ids = qa['input_ids']\n",
    "            input_chunk = token_ids\n",
    "            target_chunk = token_ids[1:]\n",
    "            input_chunk += [EOT]* (max_length - len(input_chunk))\n",
    "            target_chunk +=  [EOT]* (max_length - len(target_chunk))\n",
    "            len_ignore = len(qa['q_ids']) - 1 # target은 한 글자가 짧기 때문\n",
    "            target_chunk[:len_ignore] = [-100] * len_ignore\n",
    "\n",
    "            self.input_ids.append(torch.tensor(input_chunk))\n",
    "            self.target_ids.append(torch.tensor(target_chunk))\n",
    "\n",
    "    def __len__(self):\n",
    "        return len(self.input_ids)\n",
    "\n",
    "    def __getitem__(self, idx):\n",
    "        return self.input_ids[idx], self.target_ids[idx]\n",
    "\n",
    "dataset = MyDataset(qna_list, max_length=max_length)\n",
    "\n",
    "train_loader = DataLoader(dataset, batch_size=2, shuffle=True, drop_last=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "i = iter(train_loader)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<|begin_of_text|>홍정모가 자주 가는 여행지는? 홍정모는 특별히 자주 가는 여행지가 없습니다.<|end_of_text|><|end_of_text|><|end_of_text|><|end_of_text|><|end_of_text|><|end_of_text|><|end_of_text|>\n",
      " 홍정모는 특별히 자주 가는 여행지가 없습니다.<|end_of_text|><|end_of_text|><|end_of_text|><|end_of_text|><|end_of_text|><|end_of_text|><|end_of_text|><|end_of_text|>\n"
     ]
    }
   ],
   "source": [
    "x, y = next(i)\n",
    "\n",
    "y_temp = y[0].tolist()\n",
    "y_temp = [x for x in y_temp if x != -100] # -100은 제외하고 디코딩\n",
    "\n",
    "print(tokenizer.decode(x[0].tolist()))\n",
    "print(tokenizer.decode(y_temp))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 훈련\n",
    "\n",
    "[안내] 데이터셋이 너무 작아서 validation은 생략하였습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cuda\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "\n",
    "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
    "print(device)\n",
    "#device = \"cpu\"\n",
    "torch.manual_seed(123)\n",
    "model.to(device)\n",
    "optimizer = torch.optim.AdamW(model.parameters(), lr=0.00001, weight_decay=0.01)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 Tokens seen: 66\n",
      "1 Tokens seen: 132\n",
      "2 Tokens seen: 198\n",
      "3 Tokens seen: 264\n",
      "4 Tokens seen: 330\n",
      "5 Tokens seen: 396\n",
      "6 Tokens seen: 462\n",
      "7 Tokens seen: 528\n",
      "Epoch: 0, Loss: 2.4990234375\n",
      "8 Tokens seen: 594\n",
      "9 Tokens seen: 660\n",
      "10 Tokens seen: 726\n",
      "11 Tokens seen: 792\n",
      "12 Tokens seen: 858\n",
      "13 Tokens seen: 924\n",
      "14 Tokens seen: 990\n",
      "15 Tokens seen: 1056\n",
      "Epoch: 1, Loss: 0.3309326171875\n",
      "16 Tokens seen: 1122\n",
      "17 Tokens seen: 1188\n",
      "18 Tokens seen: 1254\n",
      "19 Tokens seen: 1320\n",
      "20 Tokens seen: 1386\n",
      "21 Tokens seen: 1452\n",
      "22 Tokens seen: 1518\n",
      "23 Tokens seen: 1584\n",
      "Epoch: 2, Loss: 0.1495361328125\n",
      "24 Tokens seen: 1650\n",
      "25 Tokens seen: 1716\n",
      "26 Tokens seen: 1782\n",
      "27 Tokens seen: 1848\n",
      "28 Tokens seen: 1914\n",
      "29 Tokens seen: 1980\n",
      "30 Tokens seen: 2046\n",
      "31 Tokens seen: 2112\n",
      "Epoch: 3, Loss: 0.0770416259765625\n",
      "32 Tokens seen: 2178\n",
      "33 Tokens seen: 2244\n",
      "34 Tokens seen: 2310\n",
      "35 Tokens seen: 2376\n",
      "36 Tokens seen: 2442\n",
      "37 Tokens seen: 2508\n",
      "38 Tokens seen: 2574\n",
      "39 Tokens seen: 2640\n",
      "Epoch: 4, Loss: 0.043914794921875\n",
      "40 Tokens seen: 2706\n",
      "41 Tokens seen: 2772\n",
      "42 Tokens seen: 2838\n",
      "43 Tokens seen: 2904\n",
      "44 Tokens seen: 2970\n",
      "45 Tokens seen: 3036\n",
      "46 Tokens seen: 3102\n",
      "47 Tokens seen: 3168\n",
      "Epoch: 5, Loss: 0.02017974853515625\n",
      "48 Tokens seen: 3234\n",
      "49 Tokens seen: 3300\n",
      "50 Tokens seen: 3366\n",
      "51 Tokens seen: 3432\n",
      "52 Tokens seen: 3498\n",
      "53 Tokens seen: 3564\n",
      "54 Tokens seen: 3630\n",
      "55 Tokens seen: 3696\n",
      "Epoch: 6, Loss: 0.014283180236816406\n",
      "56 Tokens seen: 3762\n",
      "57 Tokens seen: 3828\n",
      "58 Tokens seen: 3894\n",
      "59 Tokens seen: 3960\n",
      "60 Tokens seen: 4026\n",
      "61 Tokens seen: 4092\n",
      "62 Tokens seen: 4158\n",
      "63 Tokens seen: 4224\n",
      "Epoch: 7, Loss: 0.0117034912109375\n",
      "64 Tokens seen: 4290\n",
      "65 Tokens seen: 4356\n",
      "66 Tokens seen: 4422\n",
      "67 Tokens seen: 4488\n",
      "68 Tokens seen: 4554\n",
      "69 Tokens seen: 4620\n",
      "70 Tokens seen: 4686\n",
      "71 Tokens seen: 4752\n",
      "Epoch: 8, Loss: 0.009403228759765625\n",
      "72 Tokens seen: 4818\n",
      "73 Tokens seen: 4884\n",
      "74 Tokens seen: 4950\n",
      "75 Tokens seen: 5016\n",
      "76 Tokens seen: 5082\n",
      "77 Tokens seen: 5148\n",
      "78 Tokens seen: 5214\n",
      "79 Tokens seen: 5280\n",
      "Epoch: 9, Loss: 0.005940914154052734\n"
     ]
    }
   ],
   "source": [
    "tokens_seen, global_step = 0, -1\n",
    "\n",
    "losses = []\n",
    "\n",
    "for epoch in range(10):\n",
    "    model.train()  # Set model to training mode\n",
    "    \n",
    "    epoch_loss = 0\n",
    "    for input_batch, target_batch in train_loader:\n",
    "        optimizer.zero_grad() # Reset loss gradients from previous batch iteration\n",
    "        input_batch, target_batch = input_batch.to(device), target_batch.to(device)\n",
    "\n",
    "        logits = model(input_batch).logits # 뒤에 .logits를 붙여서 tensor만 가져옴\n",
    "\n",
    "        loss = torch.nn.functional.cross_entropy(logits.flatten(0, 1), target_batch.flatten())\n",
    "        epoch_loss += loss.item()\n",
    "        loss.backward() # Calculate loss gradients\n",
    "        optimizer.step() # Update model weights using loss gradients\n",
    "        tokens_seen += input_batch.numel()\n",
    "        global_step += 1\n",
    "\n",
    "        print(f\"{global_step} Tokens seen: {tokens_seen}\")\n",
    "\n",
    "    avg_loss = epoch_loss / len(train_loader)\n",
    "    losses.append(avg_loss)\n",
    "    print(f\"Epoch: {epoch}, Loss: {avg_loss}\")\n",
    "    torch.save(model.state_dict(), \"model_\" + str(epoch).zfill(3) + \".pth\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjcAAAHHCAYAAABDUnkqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABEKElEQVR4nO3deXhTZd7/8U+Stule1rYslaVA2RRQdmRRUUR0RFGQ0WHxUccRHBn0mR/ouIDDdNBHZRQFHUYYF0bAEXRQ0QoiojCKCArKJlsFurB0L12S8/ujJDR2oZS0J03er+vK1ebkPsk3TbEf73MvFsMwDAEAAPgJq9kFAAAAeBPhBgAA+BXCDQAA8CuEGwAA4FcINwAAwK8QbgAAgF8h3AAAAL9CuAEAAH6FcAMAAPwK4QbwEZMmTVLbtm1rde4TTzwhi8Xi3YKAc3D93h0/ftzsUgAPhBvgHCwWS41u69evN7tUU0yaNEmRkZFml1EjhmHo9ddf15AhQ9SoUSOFh4fr4osv1uzZs5Wfn292eRW4wkNVt7S0NLNLBHxSkNkFAL7u9ddf97j/2muvKSUlpcLxLl26XNDr/P3vf5fT6azVuX/60580Y8aMC3p9f+dwOPTrX/9ay5cv1+DBg/XEE08oPDxcn3/+uWbNmqUVK1bok08+UVxcnNmlVrBgwYJKA2SjRo3qvxigASDcAOdwxx13eNzfvHmzUlJSKhz/pYKCAoWHh9f4dYKDg2tVnyQFBQUpKIh/ztV56qmntHz5cj300EN6+umn3cfvuecejR07VqNHj9akSZP04Ycf1mtdNfk9ueWWW9SsWbN6qgho+LgsBXjBsGHD1L17d33zzTcaMmSIwsPD9fDDD0uS3n33XY0aNUotW7aU3W5XYmKinnzySTkcDo/n+OWYm4MHD8pisej//u//9MorrygxMVF2u119+vTR119/7XFuZWNuLBaLpk6dqlWrVql79+6y2+3q1q2b1qxZU6H+9evXq3fv3goNDVViYqJefvllr4/jWbFihS677DKFhYWpWbNmuuOOO3TkyBGPNmlpaZo8ebJat24tu92uFi1a6MYbb9TBgwfdbbZs2aIRI0aoWbNmCgsLU7t27XTnnXdW+9qFhYV6+umn1alTJyUnJ1d4/IYbbtDEiRO1Zs0abd68WZJ0/fXXq3379pU+34ABA9S7d2+PY2+88Yb7/TVp0kS33XabUlNTPdpU93tyIdavXy+LxaJly5bp4YcfVnx8vCIiIvSrX/2qQg1SzT4LSdq1a5fGjh2r5s2bKywsTElJSXrkkUcqtMvKytKkSZPUqFEjxcTEaPLkySooKPBok5KSossvv1yNGjVSZGSkkpKSvPLegcrwv3qAl5w4cUIjR47UbbfdpjvuuMN9eWPJkiWKjIzU9OnTFRkZqXXr1umxxx5TTk6ORw9CVZYuXarc3Fz99re/lcVi0VNPPaWbb75Z+/fvP2dvz8aNG/XOO+/ovvvuU1RUlJ5//nmNGTNGhw8fVtOmTSVJ3377ra699lq1aNFCs2bNksPh0OzZs9W8efML/6GcsWTJEk2ePFl9+vRRcnKy0tPT9be//U1ffPGFvv32W/fllTFjxmjnzp26//771bZtW2VkZCglJUWHDx9237/mmmvUvHlzzZgxQ40aNdLBgwf1zjvvnPPncOrUKT3wwANV9nBNmDBBixcv1urVq9W/f3+NGzdOEyZM0Ndff60+ffq42x06dEibN2/2+OzmzJmjRx99VGPHjtVdd92lzMxMvfDCCxoyZIjH+5Oq/j2pzsmTJyscCwoKqnBZas6cObJYLPp//+//KSMjQ/PmzdPw4cO1bds2hYWFSar5Z/Hdd99p8ODBCg4O1j333KO2bdvqp59+0n/+8x/NmTPH43XHjh2rdu3aKTk5WVu3btWiRYsUGxuruXPnSpJ27typ66+/Xpdccolmz54tu92uffv26YsvvjjnewdqxQBwXqZMmWL88p/O0KFDDUnGwoULK7QvKCiocOy3v/2tER4ebpw+fdp9bOLEiUabNm3c9w8cOGBIMpo2bWqcPHnSffzdd981JBn/+c9/3Mcef/zxCjVJMkJCQox9+/a5j23fvt2QZLzwwgvuYzfccIMRHh5uHDlyxH1s7969RlBQUIXnrMzEiRONiIiIKh8vLi42YmNjje7duxuFhYXu46tXrzYkGY899phhGIZx6tQpQ5Lx9NNPV/lcK1euNCQZX3/99TnrKm/evHmGJGPlypVVtjl58qQhybj55psNwzCM7Oxsw263Gw8++KBHu6eeesqwWCzGoUOHDMMwjIMHDxo2m82YM2eOR7vvv//eCAoK8jhe3e9JZVyfa2W3pKQkd7tPP/3UkGS0atXKyMnJcR9fvny5Icn429/+ZhhGzT8LwzCMIUOGGFFRUe736eJ0OivUd+edd3q0uemmm4ymTZu67z/33HOGJCMzM7NG7xu4UFyWArzEbrdr8uTJFY67/o9ZknJzc3X8+HENHjxYBQUF2rVr1zmfd9y4cWrcuLH7/uDBgyVJ+/fvP+e5w4cPV2Jiovv+JZdcoujoaPe5DodDn3zyiUaPHq2WLVu623Xo0EEjR4485/PXxJYtW5SRkaH77rtPoaGh7uOjRo1S586d9f7770sq+zmFhIRo/fr1OnXqVKXP5epVWL16tUpKSmpcQ25uriQpKiqqyjaux3JyciRJ0dHRGjlypJYvXy7DMNztli1bpv79++uiiy6SJL3zzjtyOp0aO3asjh8/7r7Fx8erY8eO+vTTTz1ep6rfk+r8+9//VkpKisdt8eLFFdpNmDDB4z3ecsstatGihT744ANJNf8sMjMztWHDBt15553u9+lS2aXKe++91+P+4MGDdeLECffP0vW5vfvuu7UeNA+cD8IN4CWtWrVSSEhIheM7d+7UTTfdpJiYGEVHR6t58+buwcjZ2dnnfN5f/nFxBZ2qAkB157rOd52bkZGhwsJCdejQoUK7yo7VxqFDhyRJSUlJFR7r3Lmz+3G73a65c+fqww8/VFxcnIYMGaKnnnrKY7rz0KFDNWbMGM2aNUvNmjXTjTfeqMWLF6uoqKjaGlx/8F0hpzKVBaBx48YpNTVVmzZtkiT99NNP+uabbzRu3Dh3m71798owDHXs2FHNmzf3uP3444/KyMjweJ2qfk+qM2TIEA0fPtzjNmDAgArtOnbs6HHfYrGoQ4cO7jFLNf0sXOG3e/fuNarvXL+j48aN06BBg3TXXXcpLi5Ot912m5YvX07QQZ0h3ABeUr6HxiUrK0tDhw7V9u3bNXv2bP3nP/9RSkqKeyxCTf7jbrPZKj1evjehLs41w7Rp07Rnzx4lJycrNDRUjz76qLp06aJvv/1WUtkf67ffflubNm3S1KlTdeTIEd1555267LLLlJeXV+Xzuqbpf/fdd1W2cT3WtWtX97EbbrhB4eHhWr58uSRp+fLlslqtuvXWW91tnE6nLBaL1qxZU6F3JSUlRS+//LLH61T2e9LQnev3LCwsTBs2bNAnn3yi3/zmN/ruu+80btw4XX311RUG1gPeQLgB6tD69et14sQJLVmyRA888ICuv/56DR8+3OMyk5liY2MVGhqqffv2VXissmO10aZNG0nS7t27Kzy2e/du9+MuiYmJevDBB/Xxxx9rx44dKi4u1jPPPOPRpn///pozZ462bNmiN998Uzt37tRbb71VZQ2uWTpLly6t8o/pa6+9JqlslpRLRESErr/+eq1YsUJOp1PLli3T4MGDPS7hJSYmyjAMtWvXrkLvyvDhw9W/f/9z/IS8Z+/evR73DcPQvn373LPwavpZuGaJ7dixw2u1Wa1WXXXVVXr22Wf1ww8/aM6cOVq3bl2Fy3aANxBugDrk+j/a8j0lxcXFeumll8wqyYPNZtPw4cO1atUqHT161H183759XlvvpXfv3oqNjdXChQs9Lh99+OGH+vHHHzVq1ChJZeu9nD592uPcxMRERUVFuc87depUhV6nnj17SlK1l6bCw8P10EMPaffu3ZVOZX7//fe1ZMkSjRgxokIYGTdunI4ePapFixZp+/btHpekJOnmm2+WzWbTrFmzKtRmGIZOnDhRZV3e9tprr3lcenv77bd17Ngx9/ipmn4WzZs315AhQ/Tqq6/q8OHDHq9Rm16/ymZ71eRzA2qLqeBAHRo4cKAaN26siRMn6ve//70sFotef/11n7os9MQTT+jjjz/WoEGD9Lvf/U4Oh0Pz589X9+7dtW3btho9R0lJif785z9XON6kSRPdd999mjt3riZPnqyhQ4dq/Pjx7unHbdu21R/+8AdJ0p49e3TVVVdp7Nix6tq1q4KCgrRy5Uqlp6frtttukyT985//1EsvvaSbbrpJiYmJys3N1d///ndFR0fruuuuq7bGGTNm6Ntvv9XcuXO1adMmjRkzRmFhYdq4caPeeOMNdenSRf/85z8rnHfdddcpKipKDz30kGw2m8aMGePxeGJiov785z9r5syZOnjwoEaPHq2oqCgdOHBAK1eu1D333KOHHnqoRj/Hqrz99tuVrlB89dVXe0wlb9KkiS6//HJNnjxZ6enpmjdvnjp06KC7775bUtlCkTX5LCTp+eef1+WXX65LL71U99xzj9q1a6eDBw/q/fffr/Hvhcvs2bO1YcMGjRo1Sm3atFFGRoZeeukltW7dWpdffnntfihAdUyZowU0YFVNBe/WrVul7b/44gujf//+RlhYmNGyZUvjj3/8o/HRRx8ZkoxPP/3U3a6qqeCVTY2WZDz++OPu+1VNBZ8yZUqFc9u0aWNMnDjR49jatWuNXr16GSEhIUZiYqKxaNEi48EHHzRCQ0Or+CmcNXHixCqnKycmJrrbLVu2zOjVq5dht9uNJk2aGLfffrvx888/ux8/fvy4MWXKFKNz585GRESEERMTY/Tr189Yvny5u83WrVuN8ePHGxdddJFht9uN2NhY4/rrrze2bNlyzjoNwzAcDoexePFiY9CgQUZ0dLQRGhpqdOvWzZg1a5aRl5dX5Xm33367IckYPnx4lW3+/e9/G5dffrkRERFhREREGJ07dzamTJli7N69292mut+TylQ3Fbz8749rKvi//vUvY+bMmUZsbKwRFhZmjBo1qsJUbsM492fhsmPHDuOmm24yGjVqZISGhhpJSUnGo48+WqG+X07xXrx4sSHJOHDggGEYZb9fN954o9GyZUsjJCTEaNmypTF+/Hhjz549Nf5ZAOfDYhg+9L+QAHzG6NGjtXPnzgrjOOB71q9fryuuuEIrVqzQLbfcYnY5gOkYcwNAhYWFHvf37t2rDz74QMOGDTOnIAC4AIy5AaD27dtr0qRJat++vQ4dOqQFCxYoJCREf/zjH80uDQDOG+EGgK699lr961//Ulpamux2uwYMGKC//OUvFRaFA4CGgDE3AADArzDmBgAA+BXCDQAA8CsBN+bG6XTq6NGjioqKqnR3WwAA4HsMw1Bubq5atmwpq7X6vpmACzdHjx5VQkKC2WUAAIBaSE1NVevWrattE3DhJioqSlLZDyc6OtrkagAAQE3k5OQoISHB/Xe8OgEXblyXoqKjowk3AAA0MDUZUsKAYgAA4FcINwAAwK8QbgAAgF8h3AAAAL9CuAEAAH6FcAMAAPwK4QYAAPgVwg0AAPArhBsAAOBXCDcAAMCvmBpukpOT1adPH0VFRSk2NlajR4/W7t27qz1nyZIlslgsHrfQ0NB6qhgAAPg6U8PNZ599pilTpmjz5s1KSUlRSUmJrrnmGuXn51d7XnR0tI4dO+a+HTp0qJ4qBgAAvs7UjTPXrFnjcX/JkiWKjY3VN998oyFDhlR5nsViUXx8fF2Xd95O5BXpVEGxOsSee8dSAABQN3xqzE12drYkqUmTJtW2y8vLU5s2bZSQkKAbb7xRO3furLJtUVGRcnJyPG51Yd2udF3250/0+39tq5PnBwAANeMz4cbpdGratGkaNGiQunfvXmW7pKQkvfrqq3r33Xf1xhtvyOl0auDAgfr5558rbZ+cnKyYmBj3LSEhoU7qT2weKUnal5mnUoezTl4DAACcm8UwDMPsIiTpd7/7nT788ENt3LhRrVu3rvF5JSUl6tKli8aPH68nn3yywuNFRUUqKipy38/JyVFCQoKys7MVHR3tldolyek01PXxNTpd4tTaB4e6ww4AALhwOTk5iomJqdHfb5/ouZk6dapWr16tTz/99LyCjSQFBwerV69e2rdvX6WP2+12RUdHe9zqgtVqUae4srE2e9Jy6+Q1AADAuZkabgzD0NSpU7Vy5UqtW7dO7dq1O+/ncDgc+v7779WiRYs6qPD8uMNNep7JlQAAELhMnS01ZcoULV26VO+++66ioqKUlpYmSYqJiVFYWJgkacKECWrVqpWSk5MlSbNnz1b//v3VoUMHZWVl6emnn9ahQ4d01113mfY+XJLc4YaeGwAAzGJquFmwYIEkadiwYR7HFy9erEmTJkmSDh8+LKv1bAfTqVOndPfddystLU2NGzfWZZddpi+//FJdu3atr7Kr1Cm+LNzsJtwAAGAanxlQXF/OZ0DS+UrLPq3+yWtls1r0w+wRsgfZvPr8AAAEqgY3oNhfxEXbFRUaJIfT0P7M6ldZBgAAdYNw40UWi4VxNwAAmIxw42XucTdMBwcAwBSEGy+j5wYAAHMRbrzMtdYNM6YAADAH4cbLOsWVbbuQerJQBcWlJlcDAEDgIdx4WdNIu5pF2iVJe1mpGACAeke4qQNJ8WW9N1yaAgCg/hFu6gAbaAIAYB7CTR1IYlAxAACmIdzUgY5MBwcAwDSEmzrgmjGVnlOkrIJik6sBACCwEG7qQFRosFo1CpMk7WHGFAAA9YpwU0dcvTeMuwEAoH4RbuqIa4+pvYQbAADqFeGmjrhnTDEdHACAekW4qSOdys2YMgzD5GoAAAgchJs60iE2UlaLdKqgRJl5RWaXAwBAwCDc1JHQYJvaNo2QJO1JY8YUAAD1hXBThzoyYwoAgHpHuKlDSewxBQBAvSPc1CHXdHB6bgAAqD+Emzrk6rnZy4wpAADqDeGmDrVtFqFgm0X5xQ4dySo0uxwAAAIC4aYOBdusSmxeNqiYHcIBAKgfhJs61sm9UjHTwQEAqA+EmzqWFH92pWIAAFD3CDd1rBN7TAEAUK8IN3XMNWNqX2aeSh1Ok6sBAMD/EW7qWOvGYQoLtqm41KlDJwvMLgcAAL9HuKljVqvFvQ0DKxUDAFD3CDf1wD3uhkHFAADUOcJNPTi7UjHTwQEAqGuEm3rAHlMAANQfwk09cPXcHDier6JSh8nVAADg3wg39SAu2q7o0CA5nIb2Z+abXQ4AAH6NcFMPLBYLKxUDAFBPCDf1hJWKAQCoH4SbeuIKN/TcAABQtwg39YS1bgAAqB+Em3rS6cwqxaknC5VfVGpyNQAA+C/CTT1pGmlXs0i7JGlfBov5AQBQVwg39Sgpvqz3hktTAADUHcJNPXIPKmbGFAAAdYZwU4+SGFQMAECdI9zUo04s5AcAQJ0j3NSjjrFlY27Sc4qUVVBscjUAAPgnwk09igoNVqtGYZKkPenMmAIAoC4QbuqZa70bxt0AAFA3CDf1zD3uhhlTAADUCcJNPWPGFAAAdYtwU89ca93sTc+VYRgmVwMAgP8h3NSzDrGRslqkUwUlyswrMrscAAD8DuGmnoUG29S2aYQkaU8aM6YAAPA2wo0JOjHuBgCAOkO4MQEzpgAAqDumhpvk5GT16dNHUVFRio2N1ejRo7V79+5znrdixQp17txZoaGhuvjii/XBBx/UQ7Xew4wpAADqjqnh5rPPPtOUKVO0efNmpaSkqKSkRNdcc43y8/OrPOfLL7/U+PHj9T//8z/69ttvNXr0aI0ePVo7duyox8ovjGshv73puXI6mTEFAIA3WQwfmo+cmZmp2NhYffbZZxoyZEilbcaNG6f8/HytXr3afax///7q2bOnFi5ceM7XyMnJUUxMjLKzsxUdHe212s9HicOpro+tUYnD0Od/vEIJTcJNqQMAgIbifP5++9SYm+zsbElSkyZNqmyzadMmDR8+3OPYiBEjtGnTpkrbFxUVKScnx+NmtmCbVYnNy3pv2CEcAADv8plw43Q6NW3aNA0aNEjdu3evsl1aWpri4uI8jsXFxSktLa3S9snJyYqJiXHfEhISvFp3bblmTLGBJgAA3uUz4WbKlCnasWOH3nrrLa8+78yZM5Wdne2+paamevX5ayvJNWOKnhsAALwqyOwCJGnq1KlavXq1NmzYoNatW1fbNj4+Xunp6R7H0tPTFR8fX2l7u90uu93utVq9xb3WDdPBAQDwKlN7bgzD0NSpU7Vy5UqtW7dO7dq1O+c5AwYM0Nq1az2OpaSkaMCAAXVVZp1wTQffl5mnUofT5GoAAPAfpoabKVOm6I033tDSpUsVFRWltLQ0paWlqbCw0N1mwoQJmjlzpvv+Aw88oDVr1uiZZ57Rrl279MQTT2jLli2aOnWqGW+h1lo3DlNYsE3FpU4dOllgdjkAAPgNU8PNggULlJ2drWHDhqlFixbu27Jly9xtDh8+rGPHjrnvDxw4UEuXLtUrr7yiHj166O2339aqVauqHYTsi6xWi3u9G1YqBgDAe0wdc1OTJXbWr19f4ditt96qW2+9tQ4qql8d46K0/eds7U7P1ciLW5hdDgAAfsFnZksFoqQ4ZkwBAOBthBsTuTbQZMYUAADeQ7gxkavn5uCJAhWVOkyuBgAA/0C4MVFctF3RoUFyOA3tz6x6s1AAAFBzhBsTWSwWVioGAMDLCDcmY6ViAAC8i3BjMnpuAADwLsKNydw9N4QbAAC8gnBjMle4ST1ZqPyiUpOrAQCg4SPcmKxJRIiaRZbtWr43I8/kagAAaPgINz4gKZ49pgAA8BbCjQ/oxDYMAAB4DeHGByQxqBgAAK8h3PiATkwHBwDAawg3PqBjbNmYm/ScImUVFJtcDQAADRvhxgdEhQarVaMwSdKedGZMAQBwIQg3PsK1UjHjbgAAuDCEGx/hnjHFdHAAAC4I4cZHdIorG3dDzw0AABeGcOMjyq91YxiGydUAANBwEW58RIfYSFktUlZBiTLziswuBwCABotw4yNCg21q2zRCkrQnjRlTAADUFuHGh3RipWIAAC4Y4caHuFcqZsYUAAC1RrjxIewxBQDAhSPc+JCk+LLp4HvTc+V0MmMKAIDaINz4kDZNIxRisyq/2KEjWYVmlwMAQINEuPEhwTar2jc/M2OKS1MAANQK4cbHMGMKAIALQ7jxMa4NNPeyOzgAALVCuPEx7p4bpoMDAFArhBsf45oOvi8zT6UOp8nVAADQ8BBufEzrxmEKC7apuNSpQycLzC4HAIAGh3DjY6xWizrFla13w0rFAACcP8KND2LGFAAAtUe48UGuGVOsdQMAwPkj3PggZkwBAFB7hBsf5Ao3B08U6HSJw+RqAABoWAg3Pigu2q7o0CA5nIb2Z+abXQ4AAA0K4cYHWSyWsysVZ3BpCgCA80G48VGMuwEAoHYINz6KGVMAANQO4cZHsdYNAAC1Q7jxUa5wk3qyUPlFpSZXAwBAw0G48VFNIkLUPMouSdqbkWdyNQAANByEGx/m2iGcPaYAAKg5wo0PY9wNAADnj3Djw9y7gxNuAACoMcKND+vEdHAAAM4b4caHdYwt67lJzylSVkGxydUAANAwEG58WFRosFo1CpMk7UlnxhQAADVBuPFxrpWKGVQMAEDNEG58XCemgwMAcF4INz4uKb5s3A09NwAA1Azhxse5e27Sc2UYhsnVAADg+wg3Pi6xeaSsFimroESZuUVmlwMAgM8zNdxs2LBBN9xwg1q2bCmLxaJVq1ZV2379+vWyWCwVbmlpafVTsAlCg21q2zRCEpemAACoCVPDTX5+vnr06KEXX3zxvM7bvXu3jh075r7FxsbWUYW+4eylKaaDAwBwLkFmvvjIkSM1cuTI8z4vNjZWjRo18n5BPqpTfJTW7ExjxhQAADXQIMfc9OzZUy1atNDVV1+tL774wuxy6lwSG2gCAFBjpvbcnK8WLVpo4cKF6t27t4qKirRo0SINGzZM//3vf3XppZdWek5RUZGKis4OxM3Jyamvcr3GNR18b3qunE5DVqvF5IoAAPBdDSrcJCUlKSkpyX1/4MCB+umnn/Tcc8/p9ddfr/Sc5ORkzZo1q75KrBNtmkYoxGZVfrFDR7IKldAk3OySAADwWQ3yslR5ffv21b59+6p8fObMmcrOznbfUlNT67E67wi2WdW+edmMKXYIBwCgeg0+3Gzbtk0tWrSo8nG73a7o6GiPW0PEHlMAANSMqZel8vLyPHpdDhw4oG3btqlJkya66KKLNHPmTB05ckSvvfaaJGnevHlq166dunXrptOnT2vRokVat26dPv74Y7PeQr1hjykAAGrG1HCzZcsWXXHFFe7706dPlyRNnDhRS5Ys0bFjx3T48GH348XFxXrwwQd15MgRhYeH65JLLtEnn3zi8Rz+6uyMKda6AQCgOhYjwDYsysnJUUxMjLKzsxvUJarDJwo05OlPFRJk1Q+zRijI1uCvKAIAUGPn8/ebv5ANROvGYQoLtqm41KlDJwvMLgcAAJ9FuGkgrFaLOsWVrXfDuBsAAKpGuGlAOrFSMQAA50S4aUBc08FZ6wYAgKoRbhoQd88Nl6UAAKgS4aYBcfXcHDxRoNMlDpOrAQDANxFuGpDYKLtiwoLlcBran5lvdjkAAPgkwk0DYrFY3Iv5Me4GAIDKEW4amI6u6eCEGwAAKkW4aWCYMQUAQPUINw0Ma90AAFA9wk0D4wo3qScLlV9UanI1AAD4HsJNA9MkIkTNo+ySpL0Z7BAOAMAvEW4aIPeMKRbzAwCgAsJNA8S4GwAAqlarcJOamqqff/7Zff+rr77StGnT9Morr3itMFQtKZ7p4AAAVKVW4ebXv/61Pv30U0lSWlqarr76an311Vd65JFHNHv2bK8WiIrYYwoAgKrVKtzs2LFDffv2lSQtX75c3bt315dffqk333xTS5Ys8WZ9qETHM+EmI7dIWQXFJlcDAIBvqVW4KSkpkd1eNmPnk08+0a9+9StJUufOnXXs2DHvVYdKRdqD1KpRmCRpTzozpgAAKK9W4aZbt25auHChPv/8c6WkpOjaa6+VJB09elRNmzb1aoGonGulYgYVAwDgqVbhZu7cuXr55Zc1bNgwjR8/Xj169JAkvffee+7LVahbnZgODgBApYJqc9KwYcN0/Phx5eTkqHHjxu7j99xzj8LDw71WHKrmmjFFzw0AAJ5q1XNTWFiooqIid7A5dOiQ5s2bp927dys2NtarBaJy7p6b9FwZhmFyNQAA+I5ahZsbb7xRr732miQpKytL/fr10zPPPKPRo0drwYIFXi0QlUtsHimrRcoqKFFmbpHZ5QAA4DNqFW62bt2qwYMHS5LefvttxcXF6dChQ3rttdf0/PPPe7VAVC402Ka2zSIkcWkKAIDyahVuCgoKFBVVdlnk448/1s033yyr1ar+/fvr0KFDXi0QVUtiMT8AACqoVbjp0KGDVq1apdTUVH300Ue65pprJEkZGRmKjo72aoGoWvlxNwAAoEytws1jjz2mhx56SG3btlXfvn01YMAASWW9OL169fJqgaja2XDDQn4AALjUair4Lbfcossvv1zHjh1zr3EjSVdddZVuuukmrxWH6rmmg+9Nz5XTachqtZhcEQAA5qtVuJGk+Ph4xcfHu3cHb926NQv41bM2TSMUYrMqv9ihI1mFSmjCGkMAANTqspTT6dTs2bMVExOjNm3aqE2bNmrUqJGefPJJOZ1Ob9eIKgTbrGrfvGzGFONuAAAoU6uem0ceeUT/+Mc/9Ne//lWDBg2SJG3cuFFPPPGETp8+rTlz5ni1SFQtKT5Ku9JytTs9V1d1iTO7HAAATFercPPPf/5TixYtcu8GLkmXXHKJWrVqpfvuu49wU4/YYwoAAE+1uix18uRJde7cucLxzp076+TJkxdcFGrOvdYNM6YAAJBUy3DTo0cPzZ8/v8Lx+fPn65JLLrngolBzSfFl4eanjDyVOhjvBABArS5LPfXUUxo1apQ++eQT9xo3mzZtUmpqqj744AOvFojqtWoUpvAQmwqKHTp4okAdYiPNLgkAAFPVqudm6NCh2rNnj2666SZlZWUpKytLN998s3bu3KnXX3/d2zWiGlarRR3PXJray4wpAABkMQzD8NaTbd++XZdeeqkcDoe3ntLrcnJyFBMTo+zsbL/ZKuJ/V2zXim9+1rThHTVteCezywEAwOvO5+93rXpu4Ftc425Y6wYAAMKNX+jE7uAAALgRbvyAq+fm4IkCnS7x3UuCAADUh/OaLXXzzTdX+3hWVtaF1IJaio2yKyYsWNmFJdqfma+uLf1jLBEAALVxXuEmJibmnI9PmDDhggrC+bNYLEqKi9JXB09qT3ou4QYAENDOK9wsXry4rurABeoUH6mvDp7UbgYVAwACHGNu/EQSe0wBACCJcOM33BtoZhBuAACBjXDjJ1zhJvVkofKLSk2uBgAA8xBu/ETjiBA1j7JLkvZmsEM4ACBwEW78CONuAAAg3PgV90rFzJgCAAQwwo0fSYqPlMQeUwCAwEa48SPsMQUAAOHGr3Q8E24ycot0Kr/Y5GoAADAH4caPRNqD1LpxmCQuTQEAAhfhxs+4Z0wxHRwAEKAIN36mUzzTwQEAgY1w42c6xZXNmGI6OAAgUJkabjZs2KAbbrhBLVu2lMVi0apVq855zvr163XppZfKbrerQ4cOWrJkSZ3X2ZC495hKz5VhGCZXAwBA/TM13OTn56tHjx568cUXa9T+wIEDGjVqlK644gpt27ZN06ZN01133aWPPvqojittOBKbR8pqkbIKSpSZW2R2OQAA1LsgM1985MiRGjlyZI3bL1y4UO3atdMzzzwjSerSpYs2btyo5557TiNGjKirMhuU0GCb2jaL0P7MfO1Oz1VsdKjZJQEAUK8a1JibTZs2afjw4R7HRowYoU2bNlV5TlFRkXJycjxu/i6JxfwAAAGsQYWbtLQ0xcXFeRyLi4tTTk6OCgsLKz0nOTlZMTEx7ltCQkJ9lGqq8uNuAAAINA0q3NTGzJkzlZ2d7b6lpqaaXVKdS4p3baDJWjcAgMBj6pib8xUfH6/09HSPY+np6YqOjlZYWFil59jtdtnt9vooz2e4em72pefK6TRktVpMrggAgPrToHpuBgwYoLVr13ocS0lJ0YABA0yqyDe1bRquEJtV+cUOHcmq/HIdAAD+ytRwk5eXp23btmnbtm2SyqZ6b9u2TYcPH5ZUdklpwoQJ7vb33nuv9u/frz/+8Y/atWuXXnrpJS1fvlx/+MMfzCjfZwXZrGrfPEIS424AAIHH1HCzZcsW9erVS7169ZIkTZ8+Xb169dJjjz0mSTp27Jg76EhSu3bt9P777yslJUU9evTQM888o0WLFjENvBJnx90QbgAAgcXUMTfDhg2rdhXdylYfHjZsmL799ts6rMo/uGdMMR0cABBgGtSYG9Sce60bZkwBAAIM4cZPuS5L/ZSRp1KH0+RqAACoP4QbP9WqUZjCQ2wqdjh18ESB2eUAAFBvCDd+ymq1qCMrFQMAAhDhxo8lxUVKItwAAAIL4caPsccUACAQEW78WCd2BwcABCDCjR9zzZg6eKJAp0scJlcDAED9INz4sdgou2LCguVwGtqfmW92OQAA1AvCjR+zWCzuxfwYdwMACBSEGz/XKb5sxhR7TAEAAgXhxs8lsccUACDAEG78nHvGFD03AIAAQbjxc65w8/OpQuUXlZpcDQAAdY9w4+caR4QoNsouSdqbwQ7hAAD/R7gJAK71bhh3AwAIBISbANAxlnE3AIDAQbgJAEnxbKAJAAgchJsAwB5TAIBAQrgJAB3PhJuM3CKdyi82uRoAAOoW4SYARNqD1LpxmCQuTQEA/B/hJkCwxxQAIFAQbgJEp3hmTAEAAgPhJkCc7blhIT8AgH8j3ASITuUuSxmGYXI1AADUHcJNgGjfPEJWi5RVUKLM3CKzywEAoM4QbgJEaLBNbZtFSGLcDQDAvxFuAkgSi/kBAAIA4SaAdGI6OAAgABBuAkiSezo4M6YAAP6LcBNAXD03e9Nz5XQyYwoA4J8INwGkbdNwhdisKih26EhWodnlAABQJwg3ASTIZlVibKQkxt0AAPwX4SbAJMWVhRumgwMA/BXhJsC49pjaw3RwAICfItwEmE6xzJgCAPg3wk2AcU0H/ykjT6UOp8nVAADgfYSbANOqUZjCQ2wqdjh18ESB2eUAAOB1hJsAY7Va1JGVigEAfoxwE4DcM6YYVAwA8EOEmwDEHlMAAH9GuAlArkHFhBsAgD8i3ASgpDM9NwdPFOh0icPkagAA8C7CTQBqHmVXo/BgOZyG9mfmm10OAABeRbgJQBaLxb2YH5emAAD+hnAToDrFs8cUAMA/EW4ClGvcDXtMAQD8DeEmQLmmg9NzAwDwN4SbAOUKNz+fKlReUanJ1QAA4D2EmwDVOCJEsVF2SdJeem8AAH6EcBPAXIv57U3PM7kSAAC8h3ATwBh3AwDwR4SbAJbEHlMAAD9EuAlgnc5clmJ3cACAPyHcBLCOsWUL+WXkFulUfrHJ1QAA4B2EmwAWYQ9S68Zhkrg0BQDwHz4Rbl588UW1bdtWoaGh6tevn7766qsq2y5ZskQWi8XjFhoaWo/V+hfG3QAA/I3p4WbZsmWaPn26Hn/8cW3dulU9evTQiBEjlJGRUeU50dHROnbsmPt26NCheqzYv7jH3RBuAAB+wvRw8+yzz+ruu+/W5MmT1bVrVy1cuFDh4eF69dVXqzzHYrEoPj7efYuLi6vHiv3L2T2mWOsGAOAfTA03xcXF+uabbzR8+HD3MavVquHDh2vTpk1VnpeXl6c2bdooISFBN954o3bu3Fll26KiIuXk5HjccJZrrZs9GbkyDMPkagAAuHCmhpvjx4/L4XBU6HmJi4tTWlpapeckJSXp1Vdf1bvvvqs33nhDTqdTAwcO1M8//1xp++TkZMXExLhvCQkJXn8fDVn75hGyWS3KKihRZm6R2eUAAHDBTL8sdb4GDBigCRMmqGfPnho6dKjeeecdNW/eXC+//HKl7WfOnKns7Gz3LTU1tZ4r9m2hwTa1bRouiXE3AAD/YGq4adasmWw2m9LT0z2Op6enKz4+vkbPERwcrF69emnfvn2VPm632xUdHe1xg6ckFvMDAPgRU8NNSEiILrvsMq1du9Z9zOl0au3atRowYECNnsPhcOj7779XixYt6qpMv9cxlungAAD/EWR2AdOnT9fEiRPVu3dv9e3bV/PmzVN+fr4mT54sSZowYYJatWql5ORkSdLs2bPVv39/dejQQVlZWXr66ad16NAh3XXXXWa+jQbN3XPD7uAAAD9gergZN26cMjMz9dhjjyktLU09e/bUmjVr3IOMDx8+LKv1bAfTqVOndPfddystLU2NGzfWZZddpi+//FJdu3Y16y00eK4ZU3vTc+V0GrJaLSZXBABA7VmMAJv/m5OTo5iYGGVnZzP+5oxSh1NdH/tIxQ6nPv/jFUpoEm52SQAAeDifv98NbrYUvC/IZlXimU00GVQMAGjoCDeQJCXFnQk3DCoGADRwhBtIOrvH1F7CDQCggSPcQNLZPaaYMQUAaOgIN5B0dsbUTxl5KnU4Ta4GAIDaI9xAktSqUZgiQmwqdjh18ESB2eUAAFBrhBtIkqxWizrEsVIxAKDhI9zAzT1jiungAIAGjHADt0703AAA/ADhBm5n95gi3AAAGi7CDdxc08EPHs/X6RKHydUAAFA7hBu4NY+yq1F4sJyGtD8z3+xyAACoFcIN3CwWC+NuAAANHuEGHs6uVEy4AQA0TIQbeHDtMbWH6eAAgAaKcAMPnWLL1rrZeviUPt+bKcMwTK4IAIDzQ7iBh26tYhQTFqxTBSX6zT++0k0vfal1u9IJOQCABoNwAw+R9iB9NG2IJg9qK3uQVdtSs3Tnki26Yf5GrdmRJqeTkAMA8G0WI8D+lzwnJ0cxMTHKzs5WdHS02eX4tMzcIi36fL9e33xIBcVl694kxUVp6pUddN3FLWSzWkyuEAAQKM7n7zfhBud0Mr9Yr248oH9+eVC5RaWSpPbNIzRlWAfd2LOlgmx0AAIA6hbhphqEm9rLLizRki8O6tUvDii7sESSdFGTcN03LFE3X9paIUGEHABA3SDcVINwc+FyT5fojc2Htejz/TqRXyxJatUoTPcOba9beycoNNhmcoUAAH9DuKkG4cZ7CopLtfS/h/XKhv3KyC2SJMVF23XPkET9uu9FCgsh5AAAvINwUw3CjfedLnFo+ZZULVz/k45mn5YkNYsM0V2D2+uO/m0UaQ8yuUIAQENHuKkG4abuFJc69e+tP+ul9fuUerJQktQoPFj/M6idJg5qq+jQYJMrBAA0VISbahBu6l6Jw6l3tx3VS5/u0/7jZbuLR4UGadLAtrpzUDs1jggxuUIAQENDuKkG4ab+OJyG3v/+mOav26s96XmSpIgQm+4Y0EZ3D26vZpF2kysEADQUhJtqEG7qn9Np6OMf0vT82n364ViOJCk02Kpf922j3w5tr7joUJMrBAD4OsJNNQg35jEMQ+t2Zej5dfu0PTVLkhQSZNW43gm6d1iiWjUKM7dAAIDPItxUg3BjPsMw9Pne43ph3V59ffCUJCnYZtGYS1vrvmEddFHTcJMrBAD4GsJNNQg3vmXz/hN6fu1effnTCUmSzWrRjT1basoVHZTYPNLk6gAAvoJwUw3CjW/65tBJvbBun9bvzpQkWSzSqItb6P4rOyopPsrk6gAAZiPcVINw49u++zlLL6zbp5Qf0t3HRnSL0/1XdlT3VjEmVgYAMBPhphqEm4bhx2M5mr9unz7YcUyu39ArO8fq/is7qNdFjc0tDgBQ7wg31SDcNCx703P14qf79N72o3Ke+U0d3LGZ7r+yo/q2a2JucQCAekO4qQbhpmE6cDxfC9bv0ztbj6j0TMrp166Jfn9VRw1MbCqLxWJyhQCAukS4qQbhpmFLPVmghZ/9pBVbflaxwylJuvSiRrr/yo4altSckAMAfopwUw3CjX84ll2olz/br399dVhFpWUh5+JWMZp6ZQdd3SVOVishBwD8CeGmGoQb/5KRe1qLPj+gNzYfUkGxQ5LUOT5KU6/soJHdW8hGyAEAv0C4qQbhxj+dzC/WPzbu1z+/PKS8olJJUmLzCE0c2FY9ExqpU1yUQoNtJlcJAKgtwk01CDf+LbugRIu/PKBXNx5QzulS9/Egq0UdYiPVvVWMureMVvdWMerSIloR9iATqwUA1BThphqEm8CQe7pES/97WBv3HdeOI9k6VVBSoY3FIrVvFnEm8MSoW6todWsZo5iwYBMqBgBUh3BTDcJN4DEMQ8eyT2vHkWztOJqjnUeyteNottJziiptf1GTcHU/E3S6nenlaRZpr+eqAQDlEW6qQbiBS0buae08E3Z2Hs3RjqPZSj1ZWGnb+OhQd+Dp3ipG3VtFKz46lKnnAFBPCDfVINygOlkFxfrhTNDZcaTs64Hj+arsX0nTiBB1KzeGp3vLGCU0CSPwAEAdINxUg3CD85VXVKofj+WUXdY6kqOdR7O1NyNPDmfFfzpRoUFll7LK9fC0axbJlHQAuECEm2oQbuANp0sc2pWWq51HzwaeXcdy3asmlxcWbFPXltHq3jL6TE9PjDrGRSrYZjWhcgBomAg31SDcoK6UOJzam56nHUezzwxaztEPR3NUWOKo0DbEZlXnFlFnxvCUjeXpHM9aPABQFcJNNQg3qE8Op6EDx/PKxu+UG7icW24NHheb1aKOsZHuwONaiyeStXgAgHBTHcINzGYYhlJPFp4ZtFzWw7PjSLZO5hdXaGuxSO2aRahjbKSaR9nVLNJ1Czn7fZRdESE2BjID8GuEm2oQbuCLDMNQWs7pcj08Zb08x7JP1+j80GCrmkXa1TTSrublg09kiJqe+b55VNnxmLBgghCABodwUw3CDRqS43lF2nEkW4dOFOhEXpEy84p1PK/IfTuRV+zeMLSmgqwWNfUIQHY1iwpRs4gzX8sdbxIRwkwvAD7hfP5+czEf8GHNIu0alhRbbZuC4lIdzy1WZrnAUz4AHc89ez/ndKlKnYbSc4qqXKG5PItFahIecjYARdrVtFwIan4mBDWNDFHTyBDZgxgQDcB8hBuggQsPCdJFTYN0UdPwc7YtKnX8IvwUVwhArsdPFhTLMKQT+cU6kV+s3ennriU6NEjNoioZF+S+RBaimLBgRYcGKzosmNlhAOoE4QYIIPYgm1o2ClPLRmHnbFvqcOpkQbGO5xbrRL5nL1DmL3qITuQVq9RpKOd0qXJOl2p/Zn6N6gkJsio6NFgxYUGKPhN6YsKCFR0WVO77yo9HhQYpiLWCAFSCcAOgUkE2q2KjQhUbFXrOtk6noezCEp3IL1JmbsXLYifyy8YLncwvUk5hqXJOl8gwpOJSp7tdbUSE2DwCUHSlwSjIMySFlx2LtAcxsBrwUz4Rbl588UU9/fTTSktLU48ePfTCCy+ob9++VbZfsWKFHn30UR08eFAdO3bU3Llzdd1119VjxQDKs1otahwRosYRIepQ/RAhSWVhKL+4VNmFJe6wU/Z9iXJOl5b7/szXX7TJPzOIOr/Yofxih47WcFaZR80WlQtFQWcvl5ULQNFhweUuo51tExkapCCrVUFWi6wMuAZ8junhZtmyZZo+fboWLlyofv36ad68eRoxYoR2796t2NiK/5X88ssvNX78eCUnJ+v666/X0qVLNXr0aG3dulXdu3c34R0AOF9Wq0VRocGKCg2WGp//+SUOp3JPlyqn8EzgOV0WgLLLBaLsM0Hpl21yCktU7HDKaUhZBSXKKii5oPdisZTNQLNZLWWBx2bxuF/2tey+zWpRkM0i25lgVP6xYJtn27KvZ9rZKj9+9rXKPZ+tkvPLP6/N4hHMrBaLrBbJ8ouvVotFFotkkUVWqzzaWeS6f6aN5ex9dxuLqrzv8dzl79OTBi8xfSp4v3791KdPH82fP1+S5HQ6lZCQoPvvv18zZsyo0H7cuHHKz8/X6tWr3cf69++vnj17auHChed8PaaCA4HNMAwVlTrPHYzK9xaVa5N7ukSV7JkKL7Faqg5OZYFIFUKZK2xZPILSme/PPKbyYUtnw5S1fFt3+3KPyfN5ywcxq6uecsfcwc9ayeuo3OtY5H5+q1VSuaBns5Y9brOUBVD3965weOb9u9pZLRbZzjynzVqxXfmfqcc5Vs/HzoZYz/uudpYzr/PLtjar6z2erSM02KbmUXav/m40mKngxcXF+uabbzRz5kz3MavVquHDh2vTpk2VnrNp0yZNnz7d49iIESO0atWquiwVgJ+wWCwKDbYpNNim2Ohzjyf6JafTUGGJQw7DUKnDUKnTKYez7HuH01Cp0/XV6XnfcfZ42XlVtHMacjicnvfdX6s4twavXep0VlqLYUiGJKdhlN3O7P3qvm+UBULDkPu+88xJ5e+XHfK8Xxvu5xcJsiHrdVEjrbxvkGmvb2q4OX78uBwOh+Li4jyOx8XFadeuXZWek5aWVmn7tLS0StsXFRWpqOjsYMWcnJwLrBpAILNaLYpgv68aMcqFH3cAMjyDk35x39AvgtSZbrIKQaqK5/5lwDIMoyy8OQ13iCtfh+tY+bDmem794rmcxtlayu57nuv5/J7PZahiSHS9V8/3Unbf4az4/dmfkyGH03Xe2e8d5Z/HefZn6vreYZQ/x/Ox8s/v+dxna3acCb/lPy/3c/+inT3I3JmMfv8vNDk5WbNmzTK7DAAIOGWXMSSbGEuD+mVqtGrWrJlsNpvS0z1XB0tPT1d8fHyl58THx59X+5kzZyo7O9t9S01N9U7xAADAJ5kabkJCQnTZZZdp7dq17mNOp1Nr167VgAEDKj1nwIABHu0lKSUlpcr2drtd0dHRHjcAAOC/TL8sNX36dE2cOFG9e/dW3759NW/ePOXn52vy5MmSpAkTJqhVq1ZKTk6WJD3wwAMaOnSonnnmGY0aNUpvvfWWtmzZoldeecXMtwEAAHyE6eFm3LhxyszM1GOPPaa0tDT17NlTa9ascQ8aPnz4sKzWsx1MAwcO1NKlS/WnP/1JDz/8sDp27KhVq1axxg0AAJDkA+vc1DfWuQEAoOE5n7/f7DoHAAD8CuEGAAD4FcINAADwK4QbAADgVwg3AADArxBuAACAXyHcAAAAv0K4AQAAfoVwAwAA/Irp2y/UN9eCzDk5OSZXAgAAasr1d7smGysEXLjJzc2VJCUkJJhcCQAAOF+5ubmKiYmptk3A7S3ldDp19OhRRUVFyWKxePW5c3JylJCQoNTUVPat8gF8Hr6Fz8O38Hn4Hj6T6hmGodzcXLVs2dJjQ+3KBFzPjdVqVevWrev0NaKjo/nF9CF8Hr6Fz8O38Hn4Hj6Tqp2rx8aFAcUAAMCvEG4AAIBfIdx4kd1u1+OPPy673W52KRCfh6/h8/AtfB6+h8/EewJuQDEAAPBv9NwAAAC/QrgBAAB+hXADAAD8CuEGAAD4FcKNl7z44otq27atQkND1a9fP3311VdmlxSwkpOT1adPH0VFRSk2NlajR4/W7t27zS4LZ/z1r3+VxWLRtGnTzC4lYB05ckR33HGHmjZtqrCwMF188cXasmWL2WUFJIfDoUcffVTt2rVTWFiYEhMT9eSTT9Zo/yRUjXDjBcuWLdP06dP1+OOPa+vWrerRo4dGjBihjIwMs0sLSJ999pmmTJmizZs3KyUlRSUlJbrmmmuUn59vdmkB7+uvv9bLL7+sSy65xOxSAtapU6c0aNAgBQcH68MPP9QPP/ygZ555Ro0bNza7tIA0d+5cLViwQPPnz9ePP/6ouXPn6qmnntILL7xgdmkNGlPBvaBfv37q06eP5s+fL6ls/6qEhATdf//9mjFjhsnVITMzU7Gxsfrss880ZMgQs8sJWHl5ebr00kv10ksv6c9//rN69uypefPmmV1WwJkxY4a++OILff7552aXAknXX3+94uLi9I9//MN9bMyYMQoLC9Mbb7xhYmUNGz03F6i4uFjffPONhg8f7j5mtVo1fPhwbdq0ycTK4JKdnS1JatKkicmVBLYpU6Zo1KhRHv9WUP/ee+899e7dW7feeqtiY2PVq1cv/f3vfze7rIA1cOBArV27Vnv27JEkbd++XRs3btTIkSNNrqxhC7iNM73t+PHjcjgciouL8zgeFxenXbt2mVQVXJxOp6ZNm6ZBgwape/fuZpcTsN566y1t3bpVX3/9tdmlBLz9+/drwYIFmj59uh5++GF9/fXX+v3vf6+QkBBNnDjR7PICzowZM5STk6POnTvLZrPJ4XBozpw5uv32280urUEj3MCvTZkyRTt27NDGjRvNLiVgpaam6oEHHlBKSopCQ0PNLifgOZ1O9e7dW3/5y18kSb169dKOHTu0cOFCwo0Jli9frjfffFNLly5Vt27dtG3bNk2bNk0tW7bk87gAhJsL1KxZM9lsNqWnp3scT09PV3x8vElVQZKmTp2q1atXa8OGDWrdurXZ5QSsb775RhkZGbr00kvdxxwOhzZs2KD58+erqKhINpvNxAoDS4sWLdS1a1ePY126dNG///1vkyoKbP/7v/+rGTNm6LbbbpMkXXzxxTp06JCSk5MJNxeAMTcXKCQkRJdddpnWrl3rPuZ0OrV27VoNGDDAxMoCl2EYmjp1qlauXKl169apXbt2ZpcU0K666ip9//332rZtm/vWu3dv3X777dq2bRvBpp4NGjSowtIIe/bsUZs2bUyqKLAVFBTIavX8U2yz2eR0Ok2qyD/Qc+MF06dP18SJE9W7d2/17dtX8+bNU35+viZPnmx2aQFpypQpWrp0qd59911FRUUpLS1NkhQTE6OwsDCTqws8UVFRFcY7RUREqGnTpoyDMsEf/vAHDRw4UH/5y180duxYffXVV3rllVf0yiuvmF1aQLrhhhs0Z84cXXTRRerWrZu+/fZbPfvss7rzzjvNLq1BYyq4l8yfP19PP/200tLS1LNnTz3//PPq16+f2WUFJIvFUunxxYsXa9KkSfVbDCo1bNgwpoKbaPXq1Zo5c6b27t2rdu3aafr06br77rvNLisg5ebm6tFHH9XKlSuVkZGhli1bavz48XrssccUEhJidnkNFuEGAAD4FcbcAAAAv0K4AQAAfoVwAwAA/ArhBgAA+BXCDQAA8CuEGwAA4FcINwAAwK8QbgAEPIvFolWrVpldBgAvIdwAMNWkSZNksVgq3K699lqzSwPQQLG3FADTXXvttVq8eLHHMbvdblI1ABo6em4AmM5utys+Pt7j1rhxY0lll4wWLFigkSNHKiwsTO3bt9fbb7/tcf7333+vK6+8UmFhYWratKnuuece5eXlebR59dVX1a1bN9ntdrVo0UJTp071ePz48eO66aabFB4ero4dO+q9996r2zcNoM4QbgD4vEcffVRjxozR9u3bdfvtt+u2227Tjz/+KEnKz8/XiBEj1LhxY3399ddasWKFPvnkE4/wsmDBAk2ZMkX33HOPvv/+e7333nvq0KGDx2vMmjVLY8eO1XfffafrrrtOt99+u06ePFmv7xOAlxgAYKKJEycaNpvNiIiI8LjNmTPHMAzDkGTce++9Huf069fP+N3vfmcYhmG88sorRuPGjY28vDz34++//75htVqNtLQ0wzAMo2XLlsYjjzxSZQ2SjD/96U/u+3l5eYYk48MPP/Ta+wRQfxhzA8B0V1xxhRYsWOBxrEmTJu7vBwwY4PHYgAEDtG3bNknSjz/+qB49eigiIsL9+KBBg+R0OrV7925ZLBYdPXpUV111VbU1XHLJJe7vIyIiFB0drYyMjNq+JQAmItwAMF1ERESFy0TeEhYWVqN2wcHBHvctFoucTmddlASgjjHmBoDP27x5c4X7Xbp0kSR16dJF27dvV35+vvvxL774QlarVUlJSYqKilLbtm21du3aeq0ZgHnouQFguqKiIqWlpXkcCwoKUrNmzSRJK1asUO/evXX55ZfrzTff1FdffaV//OMfkqTbb79djz/+uCZOnKgnnnhCmZmZuv/++/Wb3/xGcXFxkqQnnnhC9957r2JjYzVy5Ejl5ubqiy++0P3331+/bxRAvSDcADDdmjVr1KJFC49jSUlJ2rVrl6SymUxvvfWW7rvvPrVo0UL/+te/1LVrV0lSeHi4PvroIz3wwAPq06ePwsPDNWbMGD377LPu55o4caJOnz6t5557Tg899JCaNWumW265pf7eIIB6ZTEMwzC7CACoisVi0cqVKzV69GizSwHQQDDmBgAA+BXCDQAA8CuMuQHg07hyDuB80XMDAAD8CuEGAAD4FcINAADwK4QbAADgVwg3AADArxBuAACAXyHcAAAAv0K4AQAAfoVwAwAA/Mr/B5KKVtZuweHAAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.plot(losses)\n",
    "plt.xlabel('Epoch')\n",
    "plt.ylabel('Loss')\n",
    "plt.title('Training Loss Over Epochs')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 결과확인"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LlamaForCausalLM(\n",
       "  (model): LlamaModel(\n",
       "    (embed_tokens): Embedding(128256, 1792)\n",
       "    (layers): ModuleList(\n",
       "      (0-31): 32 x LlamaDecoderLayer(\n",
       "        (self_attn): LlamaAttention(\n",
       "          (q_proj): Linear(in_features=1792, out_features=3072, bias=False)\n",
       "          (k_proj): Linear(in_features=1792, out_features=1024, bias=False)\n",
       "          (v_proj): Linear(in_features=1792, out_features=1024, bias=False)\n",
       "          (o_proj): Linear(in_features=3072, out_features=1792, bias=False)\n",
       "        )\n",
       "        (mlp): LlamaMLP(\n",
       "          (gate_proj): Linear(in_features=1792, out_features=8064, bias=False)\n",
       "          (up_proj): Linear(in_features=1792, out_features=8064, bias=False)\n",
       "          (down_proj): Linear(in_features=8064, out_features=1792, bias=False)\n",
       "          (act_fn): SiLU()\n",
       "        )\n",
       "        (input_layernorm): LlamaRMSNorm((1792,), eps=1e-05)\n",
       "        (post_attention_layernorm): LlamaRMSNorm((1792,), eps=1e-05)\n",
       "      )\n",
       "    )\n",
       "    (norm): LlamaRMSNorm((1792,), eps=1e-05)\n",
       "    (rotary_emb): LlamaRotaryEmbedding()\n",
       "  )\n",
       "  (lm_head): Linear(in_features=1792, out_features=128256, bias=False)\n",
       ")"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 파인튜닝 후에 어떻게 응답하는지 확인\n",
    "model.load_state_dict(torch.load(\"model_009.pth\", map_location=device, weights_only=True))\n",
    "model.eval()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Q0: 다음 숫자들을 얘기해봐 12345 67890.\n",
      "Q1: 홍정모가 좋아하는 과일은? 홍정모는 오렌지와 바나나를 좋아합니다.\n",
      "Q2: 홍정모가 좋아하는 게임은? 홍정모는 헬다이버즈2를 좋아해서 자주합니다.\n",
      "Q3: 홍정모가 자주 가는 여행지는? 홍정모는 특별히 자주 가는 여행지가 없습니다.\n",
      "Q4: 홍정모의 취미는 무엇인가요? 홍정모는 독서와 영화 감상을 즐깁니다.\n",
      "Q5: 홍정모가 좋아하는 계절은 무엇인가요? 홍정모는 여름을 가장 좋아합니다.\n",
      "Q6: 홍정모의 특기는 무엇인가요? 아쉽게도 홍정모는 특별히 잘하는 것이 없습니다.\n",
      "Q7: 홍정모가 자주 듣는 음악 장르는? 홍정모는 EDM을 자주 듣습니다.\n",
      "Q8: 홍정모가 가장 좋아하는 색깔은? 홍정모는 여름을 가장 좋아합니다.\n",
      "Q9: 홍정모가 선호하는 영화 장르는? 홍정모는 SF와 액션 영화를 선호합니다.\n",
      "Q10: 홍정모가 좋아하는 운동은? 홍정모는 매일 조깅을 합니다.\n",
      "Q11: 홍정모는 어떤 동물을 좋아하나요? 안타깝게도 홍정모는 애완동물을 키워본 적이 없습니다.\n",
      "Q12: 홍정모가 주로 사용하는 소셜 미디어는? 홍정모는 유튜버입니다.\n",
      "Q13: 홍정모가 좋아하는 음식은? 홍정모는 갈비찜을 아주 좋아합니다.\n",
      "Q14: 홍정모가 가장 최근에 본 드라마는 무엇인가요? 홍정모는 최근에 데이데블 본어게인을 봤습니다.\n",
      "Q15: 홍정모가 싫어하는 게임은 뭔가요? 홍정모는 사행성 게임을 싫어합니다.\n",
      "Q16: 홍정모가 매일하는 게임은? 홍정모는 매일 헬다이버즈2를 합니다.\n",
      "Q17: 홍정모에 대해서 얘기해봐. 홍정모는 한국의 유명한 가수입니다.\n",
      "Q18: 카나나 모델에 대해서 설명해봐.\n",
      "Q19: 이처럼 인간처럼 생각하고 행동하는 AI 모델은 2023년 현재까지는 아직 개발되지 않았습니다.\n",
      "Q20: 인공지능의 장점은 무엇인가요? 인공지능은 다양한 장점을 가지고 있습니다. 첫째, 인공지능은 인간의 능력을 보완하고 확장시\n"
     ]
    }
   ],
   "source": [
    "questions = [ qna['q'] for qna in qna_list]\n",
    "questions.append(\"홍정모가 매일하는 게임은?\")\n",
    "questions.append(\"홍정모에 대해서 얘기해봐.\")\n",
    "questions.append(\"카나나 모델에 대해서 설명해봐.\")\n",
    "questions.append(\"이처럼 인간처럼 생각하고 행동하는 AI 모델은 \")\n",
    "questions.append(\"인공지능의 장점은\")\n",
    "\n",
    "for i, q in enumerate(questions):\n",
    "\n",
    "    input_ids = tokenizer(\n",
    "        q,\n",
    "        padding=True,\n",
    "        return_tensors=\"pt\",\n",
    "    )[\"input_ids\"].to(\"cuda\")\n",
    "\n",
    "    # print(type(model))\n",
    "\n",
    "    model.eval()\n",
    "    with torch.no_grad():\n",
    "        output = model.generate(\n",
    "            input_ids,\n",
    "            max_new_tokens=32,\n",
    "            attention_mask = (input_ids != 0).long(),\n",
    "            pad_token_id=tokenizer.eos_token_id,\n",
    "            do_sample=False,\n",
    "            # temperature=1.2,\n",
    "            # top_k=5\n",
    "        )\n",
    "\n",
    "    output_list = output.tolist()\n",
    "\n",
    "    print(f\"Q{i}: {tokenizer.decode(output[0], skip_special_tokens=True)}\")\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Q20: 홍정모는 누구지? 홍정모는 대한민국의 배우입니다.\n"
     ]
    }
   ],
   "source": [
    "input_ids = tokenizer(\n",
    "    input(),\n",
    "    padding=True,\n",
    "    return_tensors=\"pt\",\n",
    ")[\"input_ids\"].to(\"cuda\")\n",
    "\n",
    "# print(type(model))\n",
    "\n",
    "model.eval()\n",
    "with torch.no_grad():\n",
    "    output = model.generate(\n",
    "        input_ids,\n",
    "        max_new_tokens=32,\n",
    "        attention_mask = (input_ids != 0).long(),\n",
    "        pad_token_id=tokenizer.eos_token_id,\n",
    "        do_sample=False,\n",
    "        # temperature=1.2,\n",
    "        # top_k=5\n",
    "    )\n",
    "\n",
    "output_list = output.tolist()\n",
    "\n",
    "print(f\"Q{i}: {tokenizer.decode(output[0], skip_special_tokens=True)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 기타\n",
    "\n",
    "허깅페이스 코드 참고한 부분들\n",
    "- [라마 모델](https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/modeling_llama.py)\n",
    "- [대답 생성하는 부분(generate)](https://github.com/huggingface/transformers/blob/main/src/transformers/generation/utils.py#L1906)\n",
    "- [실제로 모델을 사용하는 부분(forward)](https://github.com/huggingface/transformers/blob/main/src/transformers/generation/utils.py#L2827)\n",
    "- [훈련(train)](https://github.com/huggingface/transformers/blob/main/src/transformers/trainer.py#L2612)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "py312",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}