Hagaha kaliya ee aad u baahan tahay si aad u hagaajiso Llama 3 ama nooc kasta oo kale oo furan (2024)

Hagaajinta qaababka luqadaha waaweyn (LLMs) sida Llama 3 waxay ku lug leedahay la qabsiga qaabka horay loo tababaray ee hawlo gaar ah iyadoo la adeegsanayo xog-ururin gaar ah. Habkani waxa uu ka faa'iidaysanayaa aqoontii hore ee tusaalaha, isaga oo ka dhigaya mid hufan oo kharash-ool ah marka loo eego tababbarka meel eber ah. Tilmaan-bixiyahan, waxaanu ku dhex mari doonaa tillaabooyinka si loo hagaajiyo Llama 3 anagoo adeegsanayna QLoRA (Quantized LoRA), oo ah hab wax ku ool ah oo xaddidan oo yareeya isticmaalka xusuusta iyo kharashyada xisaabinta.

Dulmarka Fine-Tuning

Hagaajinta hagaajintu waxay ku lug leedahay dhowr tillaabo oo muhiim ah:

Doorashada Model horay loo tababaray: Dooro qaab sal ah oo ku habboon qaab-dhismeedka aad rabto.
Soo Ururinta Xogta La Xidhiidha: Ururi oo sii diyaarso xog-ururinta u gaarka ah hawshaada.
Toosinta WanaagsanLa qabso moodeelka adigoo isticmaalaya xog-ururinta si loo horumariyo waxqabadkiisa hawlo gaar ah.
Qiimaynta: Qiimee qaabka si fiican loo habeeyey adoo isticmaalaya cabbir tayo iyo tiro leh.

Fikradaha iyo Farsamooyinka

Habayn Wanaagsan Qaababka Luuqadaha Waaweyn

Buuxi Fine-Tuning

hagaajin buuxda cusboonaysiiyaa dhammaan cabbiraadaha tusaalaha, taasoo ka dhigaysa mid gaar ah hawsha cusub. Habkani wuxuu u baahan yahay ilo xisaabeed oo muhiim ah oo inta badan aan waxtar u lahayn moodooyinka aadka u waaweyn.

Qalabka-Tuning Fine-Fiican (PEFT)

PEFT waxay cusboonaysiisaa qayb ka mid ah cabirrada moodeelka, iyadoo yaraynaysa shuruudaha xusuusta iyo kharashka xisaabinta. Farsamadan waxay ka hortagtaa hilmaanka musiibada waxayna ilaalisaa aqoonta guud ee moodeelka.

Laqabsiga Darajada Hoose (LoRA) iyo LoRA la Tiriyay (QLoRA)

LoRA-tu waxay si fiican u garaacdaa dhawr shay oo darajo hoose ah, halka QLoRA ay qiyaasto jaantusyadan si ay u yaraato raadadka xusuusta.

Hababka Hagaajinta

Buuxi Fine-Tuning: Tani waxay ku lug leedahay tababbarka dhammaan cabbiraadaha tusaalaha ee xog-ururinta hawsha gaarka ah. Inkasta oo habkani uu noqon karo mid aad waxtar u leh, sidoo kale waa qaali xisaabeed wuxuuna u baahan yahay xasuus weyn.
Farsamaynta Wanaajinta Fiican ee Parameter (PEFT)PEFT waxay cusboonaysiisaa qayb yar oo ka mid ah cabbiraadaha moodeelka, taasoo ka dhigaysa mid aad u wanaagsan xusuusta. Farsamooyinka sida La qabsiga Darajada Hoose (LoRA) iyo LoRA la tiriyey (QLoRA) waxay ku dhacaan qaybtan.

Waa maxay LoRA?

Isbarbardhigga hababka hagaajinta: QLORA waxay ku xoojisaa LoRA tiro sax ah 4-bit iyo kor uqaadayaal bogag leh oo loogu talagalay maaraynta sarraynta xusuusta

LoRA waa hab habayn wanaagsan oo la hagaajiyay, halkii la hagaajin lahaa dhammaan miisaannada qaabkii hore loo tababbaray, laba qaybood oo yaryar oo qiyaas ahaan shaxanka waaweyn si fiican loo hagaajiyay. Matrisyadani waxay ka kooban yihiin adabtarada LoRA. Adapter-kan si fiican loo habeeyey ayaa markaa lagu shubaa qaabka horay loo tababaray waxaana loo isticmaalaa fikradda.

Faa'iidooyinka muhiimka ah ee LoRA:

Waxtarka xusuustaLoRA waxay yaraynaysaa raadadka xusuusta iyada oo si fiican u hagaajinaysa matrix yar yar halkii ay ka ahaan lahayd moodelka oo dhan.
Dib u adeegsiga: Qaabka asalka ah ayaa weli ah mid aan isbeddelin, waxaana loo isticmaali karaa qalab badan oo LoRA ah, oo fududeynaya qabashada hawlo badan oo leh shuruudaha xusuusta hoose.

Waa maxay LoRA (QLoRA) la tiriyay?

QLoRA waxay qaadaysaa LoRA tallaabo dheeraad ah iyadoo xisaabinaysa culeyska adaabiyeyaasha LoRA si ay u hooseyso saxnaanta (tusaale, 4-bit halkii 8-bit). Tani waxay sii yaraynaysaa isticmaalka xusuusta iyo shuruudaha kaydinta iyadoo la ilaalinayo waxtarka la midka ah.

Faa'iidooyinka muhiimka ah ee QLoRA:

Xataa Waxtarka Xusuusta WeynMarka la qiyaaso miisaanka, QLoRA waxay si weyn u yaraynaysaa xusuusta moodeelka iyo shuruudaha kaydinta.
Waxay ilaalisaa Waxqabadka: Inkasta oo saxnaanta la dhimay, QLoRA waxay ilaalisaa heerarka waxqabad ee u dhow kuwa moodooyinka saxda ah.

Laqabsiga Gaarka ah ee Hawsha

Inta lagu jiro hagaajinta, cabbirada moodeelka ayaa la hagaajiyaa iyadoo lagu salaynayo xog-ururinta cusub, taasoo ka caawinaysa inay si fiican u fahamto oo ay dhaliso nuxurka khuseeya hawsha gaarka ah. Habkani waxa uu xajiyaa aqoonta guud ee luqadda ee la kasbaday intii lagu jiray tababbarka ka hor iyada oo la waafajinayo tusaalaha nuxurka barta bartilmaameedka.

Fine-Tuning in Practice

Fine-Tuning Full vs. PEFT

Buuxi Fine-Tuning: Waxay ku lug leedahay tababarka qaabka oo dhan, kaas oo noqon kara mid xisaabeed qaali ah una baahan xasuus weyn.
PEFT (LoRA iyo QLoRA):- Laxan-qaabeeya oo kaliya qayb-hoosaadyo, yaraynta shuruudaha xusuusta iyo ka-hortagga illowsiinta musiibada, taas oo ka dhigaysa beddel wax ku ool ah.

Talaabooyinka Hirgelinta

Dejinta Degaanka: Ku rakib maktabadaha lagama maarmaanka ah oo deji jawiga xisaabinta.
Soo deji oo ka hor-u-socodka Xogta: Ku shub kaydka xogta oo u sii diyaarso qaab ku habboon moodeelka.
Raad Model horay loo tababaray: Ku shub qaabka saldhiga oo leh qaabaynta qiyaasaha haddii la isticmaalayo QLoRA.
CalaamadayntaCalaamadee xogta xogta si aad ugu diyaariso tababarka.
Tababarka: Si fiican u hagaaji qaabka adoo isticmaalaya xogta la diyaariyey.
Qiimaynta: Ku qiimee waxqabadka moodeelka hawlo gaar ah adiga oo isticmaalaya cabbiro tayo iyo tiro leh.

Steo by Tusaha Talaabada ee Fine Tune LLM

Dejinta Deegaanka

Waxaan u isticmaali doonaa buug-yaraha Jupyter casharkan. Nidaamyada sida Kaggle, oo bixiya isticmaalka GPU-ga bilaashka ah, ama Google Colab ayaa ku habboon socodsiinta tijaabooyinkan.

1. Ku rakib maktabadaha loo baahan yahay

Marka hore, hubi inaad haysatid maktabadaha lagama maarmaanka ah:

!pip install -qqq -U bitsandbytes transformers peft accelerate datasets scipy einops evaluate trl rouge_score</div>

2. La soo dego Maktabadaha oo Deji Deegaan

import osimport torchfrom datasets import load_datasetfrom transformers import ( AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig, TrainingArguments, pipeline, HfArgumentParser)from trl import ORPOConfig, ORPOTrainer, setup_chat_format, SFTTrainerfrom tqdm import tqdmimport gcimport pandas as pdimport numpy as npfrom huggingface_hub import interpreter_login# Disable Weights and Biases loggingos.environ['WANDB_DISABLED'] = "true"interpreter_login()

3. Ku shub Xog-ururinta

Waxaan u isticmaali doonaa xogta DialogSum casharkan:

4. Samee isku xidhka BitsAndBytes

Si aad moodelka ugu shubto qaab 4-bit ah:

compute_dtype = getattr(torch, "float16")bnb_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type='nf4', bnb_4bit_compute_dtype=compute_dtype, bnb_4bit_use_double_quant=False,)

5. Soo rar qaabka horay loo tababaray

Isticmaalka qaabka Microsoft's Phi-2 ee casharkan:

model_name = 'microsoft/phi-2'device_map = {"": 0}original_model = AutoModelForCausalLM.from_pretrained( model_name, device_map=device_map, quantization_config=bnb_config, trust_remote_code=True, use_auth_token=True)

6. Tokenization

Habee calaamadeeyaha:

tokenizer = AutoTokenizer.from_pretrained( model_name, trust_remote_code=True, padding_side="left", add_eos_token=True, add_bos_token=True, use_fast=False)tokenizer.pad_token = tokenizer.eos_token

Fine-Tuning Llama 3 ama Noocyo kale

Marka moodooyinka la hagaajinayo sida Llama 3 ama ilo kale oo casri ah oo furan LLMs, waxaa jira tixgelinno gaar ah iyo hagaajin loo baahan yahay si loo hubiyo waxqabadka ugu fiican. Waa kuwan tillaabooyinka faahfaahsan iyo fikradaha ku saabsan sida tan loo wajaho noocyo kala duwan, oo ay ku jiraan Llama 3, GPT-3, iyo Mistral.

5.1 Isticmaalka Llama 3

Xulka Model:

Hubi inaad haysato tilmaanta moodeelka saxda ah ee xarunta moodeelka Hugging Face. Tusaale ahaan, qaabka Llama 3 waxaa loo aqoonsan karaa inuu yahay meta-llama/Meta-Llama-3-8B wejiga laabta
Hubi inaad codsato gelitaanka oo aad gasho akoonkaaga Hugging Wejiga haddii loo baahdo moodooyinka sida Llama 3 (Wajiga isku duuban)I do not know

Calaamadaynta:

U isticmaal calaamadeeyaha ku habboon Llama 3, adoo hubinaya inuu la jaanqaadi karo moodalka oo ay taageerto astaamaha loo baahan yahay sida suufka iyo calaamadaha gaarka ah.

Xusuusta iyo Xisaabinta:

Hagaajinta qaababka waaweyn sida Llama 3 waxay u baahan tahay ilo xisaabeed oo muhiim ah. Hubi in deegaankaaga, sida habaynta GPU ee xoogga badan, uu xamili karo xusuusta iyo shuruudaha habaynta. Hubi in deegaanku uu xamili karo shuruudaha xusuusta, kaas oo lagu yarayn karo iyadoo la adeegsanayo farsamooyinka sida QLoRA si loo yareeyo raadadka xusuusta (Isku-duubnida Golayaasha Wajiga)

Tusaale:

model_name = 'meta-llama/Meta-Llama-3-8B'device_map = {"": 0}bnb_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.float16, bnb_4bit_use_double_quant=True,)original_model = AutoModelForCausalLM.from_pretrained( model_name, device_map=device_map, quantization_config=bnb_config, trust_remote_code=True, use_auth_token=True)

Calaamadaynta:

Iyada oo ku xidhan kiiska isticmaalka gaarka ah iyo shuruudaha moodeelka, hubi qaabaynta tokenizer sax ah iyada oo aan la haynin goobo badan. Tusaale ahaan, use_fast=True waxaa lagula talinayaa waxqabad wanaagsan (Wajiga isku duuban)I do not know (Miisaanka & Eexda).

tokenizer = AutoTokenizer.from_pretrained( model_name, trust_remote_code=True, padding_side="left", add_eos_token=True, add_bos_token=True, use_fast=False)tokenizer.pad_token = tokenizer.eos_token

5.2 Isticmaalka Qaababka kale ee caanka ah (tusaale, GPT-3, Mistral)

Xulka Model:

Moodooyinka sida GPT-3 iyo Mistral, hubi inaad isticmaashid magaca moodeelka saxda ah iyo aqoonsiga Hubinta Wajiga Hubinta ama ilo kale.

Calaamadaynta:

Si la mid ah Llama 3, hubi in calaamadeeyaha si sax ah loo dejiyay oo la jaan qaadaya moodalka.

Xusuusta iyo Xisaabinta:

Nooc kastaa wuxuu yeelan karaa shuruudo xusuuseed oo kala duwan. U hagaaji habaynta deegaankaaga si waafaqsan.

Tusaale GPT-3:

model_name = 'openai/gpt-3'device_map = {"": 0}bnb_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.float16, bnb_4bit_use_double_quant=True,)original_model = AutoModelForCausalLM.from_pretrained( model_name, device_map=device_map, quantization_config=bnb_config, trust_remote_code=True, use_auth_token=True)

Tusaale ahaan Mistral:

model_name = 'mistral-7B'device_map = {"": 0}bnb_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.float16, bnb_4bit_use_double_quant=True,)original_model = AutoModelForCausalLM.from_pretrained( model_name, device_map=device_map, quantization_config=bnb_config, trust_remote_code=True, use_auth_token=True)

Tixgelinta Tokenization: Nooc kastaa wuxuu lahaan karaa shuruudo calaamo gaar ah. Hubi in tokenizer-ku uu ku habboon yahay qaabka oo si sax ah loo habeeyey.

Tusaalaha Tokenizer Llama 3:

tokenizer = AutoTokenizer.from_pretrained( model_name, trust_remote_code=True, padding_side="left", add_eos_token=True, add_bos_token=True, use_fast=False)tokenizer.pad_token = tokenizer.eos_token

GPT-3 iyo Mistral Tokenizer Tusaalaha:

tokenizer = AutoTokenizer.from_pretrained( model_name, use_fast=True)

7. Tijaabi Modelka adigoo isticmaalaya Eber-Shot Inferencing

Ku qiimee qaabka saldhiga oo leh muunad gelin:

from transformers import set_seedset_seed(42)index = 10prompt = dataset['test'][index]['dialogue']formatted_prompt = f"Instruct: Summarize the following conversation.\n{prompt}\nOutput:\n"# Generate outputdef gen(model, prompt, max_length): inputs = tokenizer(prompt, return_tensors="pt").to(model.device) outputs = model.generate(**inputs, max_length=max_length) return tokenizer.batch_decode(outputs, skip_special_tokens=True)res = gen(original_model, formatted_prompt, 100)output = res[0].split('Output:\n')[1]print(f'INPUT PROMPT:\n{formatted_prompt}')print(f'MODEL GENERATION - ZERO SHOT:\n{output}')

8. Horay u sii habayn xogta kaydinta

U beddelo lamaanaha wada-hadalka oo kooban una beddelo dareen-celin:

def create_prompt_formats(sample): blurb = "Below is an instruction that describes a task. Write a response that appropriately completes the request." instruction = "### Instruct: Summarize the below conversation." input_context = sample['dialogue'] response = f"### Output:\n{sample['summary']}" end = "### End" parts = [blurb, instruction, input_context, response, end] formatted_prompt = "\n\n".join(parts) sample["text"] = formatted_prompt return sampledataset = dataset.map(create_prompt_formats)

Calaamadee xogta la qaabeeyey:

def preprocess_batch(batch, tokenizer, max_length): return tokenizer(batch["text"], max_length=max_length, truncation=True)max_length = 1024train_dataset = dataset["train"].map(lambda batch: preprocess_batch(batch, tokenizer, max_length), batched=True)eval_dataset = dataset["validation"].map(lambda batch: preprocess_batch(batch, tokenizer, max_length), batched=True)

9. U diyaari Qaabka QLoRA

U diyaari tusaalaha hagaajinta wanaagsan ee cabbiraadda:

original_model = prepare_model_for_kbit_training(original_model)

Hyperparameters iyo Saamayntooda

Hyperparameters ayaa door muhiim ah ka ciyaara hagaajinta waxqabadka moodeelkaaga. Waa kuwan qaar ka mid ah hyperparameters muhiimka ah ee la tixgeliyo:

Heerka Barashada: Waxay xakameysaa xawaaraha uu moodelku ku cusbooneysiinayo cabbirkiisa. Heerka waxbarasho sare wuxuu u horseedi karaa isku xirnaan degdeg ah laakiin wuxuu dhaafi karaa xalka ugu fiican. Heerarka waxbarasho ee hooseeya waxay hubisaa isku xirnaanta joogtada ah laakiin waxay u baahan kartaa waqtiyo badan.
Cabbirka Dufcada: Tirada shaybaarada la farsameeyay ka hor inta aanu moodelku cusboonaysiin xuduudihiisa. Cabbirrada dufcadaha waaweyn waxay hagaajin karaan xasilloonida laakiin waxay u baahan yihiin xusuus badan. Cabbirrada dufcooyinka yaryar waxay keeni karaan buuq badan habka tababarka.
Tillaabooyinka Isku-ururinta JahannamadaHalbeeggani waxa uu ka caawinayaa in la isu ekaysiyo cabbirrada dufcadaha waaweyn iyada oo la ururinayo jaangooyooyin dhawr tallaabo ah ka hor inta aan la samayn casriyaynta cabbirka.
Tirada Epochs: Tirada jeer ee xogta oo dhan la dhex mariyo moodeelka. Xilliyo badan ayaa hagaajin kara waxqabadka laakiin waxay u horseedi karaan si xad dhaaf ah haddii aan si habboon loo maamulin.
Miisaanka oo xumaada: Farsamada habaynta si looga hortago ku-habboonaanta xad dhaafka ah iyada oo la ciqaabayo miisaanka waaweyn.
Jadwalka Heerka Waxbarashada: Wuxuu hagaajiyaa heerka waxbarashada inta lagu jiro tababarka si loo horumariyo waxqabadka iyo isku xirnaanta.

Habee qaabeynta tababarka adiga oo hagaajinaya cabbirada sare sida heerka barashada, cabbirka dufcada, iyo tillaabooyinka ururinta gradient ee ku saleysan moodalka gaarka ah iyo shuruudaha shaqada. Tusaale ahaan, moodooyinka Llama 3 waxay u baahan karaan heerar waxbarasho oo kala duwan marka loo eego moodooyinka yaryar (Miisaanka & Eexda)I do not know (GitHub)

Tusaalaha Habaynta Tababarka

orpo_args = ORPOConfig(learning_rate=8e-6,lr_scheduler_type="linear",max_length=1024,max_prompt_length=512,beta=0.1,per_device_train_batch_size=2,per_device_eval_batch_size=2,gradient_accumulation_steps=4,optim="paged_adamw_8bit",num_train_epochs=1,evaluation_strategy="steps",eval_steps=0.2,logging_steps=1,warmup_steps=10,report_to="wandb",output_dir="./results/",)

10. Tabobar Qaabka

Diyaarso tababaraha oo bilow tababarka:

trainer = ORPOTrainer(model=original_model,args=orpo_args,train_dataset=train_dataset,eval_dataset=eval_dataset,tokenizer=tokenizer,)trainer.train()trainer.save_model("fine-tuned-llama-3")

Qiimaynta Qaabka Wanaagsan ee La Hagaajiyay

Tababarka ka dib, qiimee waxqabadka tusaalaha adiga oo isticmaalaya hababka tayada iyo tirada labadaba.

1. Qiimaynta Aadanaha

Is barbar dhig qoraallada kooban ee la soo saaray iyo kuwa bini'aadmigu qoray si loo qiimeeyo tayada.

2. Qiimaynta Tirada

Isticmaal cabbirada sida ROUGE si aad u qiimeyso waxqabadka:

from rouge_score import rouge_scorerscorer = rouge_scorer.RougeScorer(['rouge1', 'rouge2', 'rougeL'], use_stemmer=True)scores = scorer.score(reference_summary, generated_summary)print(scores)

Caqabadaha iyo Xalka Caadiga ah

1. Xaddidaadaha xusuusta

Isticmaalka QLoRA waxay gacan ka geysataa yaraynta arrimaha xusuusta iyadoo lagu qiyaaso miisaanka moodeelka 4-bit. Hubi inaad haysato xusuusta GPU-da kugu filan si aad u xakamayso cabbirka dufankaaga iyo cabbirka moodeelkaaga.

2. Kusoo kabashada

La soco halbeegyada xaqiijinta si aad uga hortagto ku habboonaanta. Isticmaal farsamooyinka sida joojinta hore iyo qudhunka miisaanka.

3. Tabobar tartiib ah

Kor u qaad xawaaraha tababbarka adiga oo hagaajinaya cabbirka dufcadda, heerka waxbarashada, iyo adeegsiga isugeynta isugeynta.

4. Tayada Data

Hubi in kaydka xogtaagu uu nadiif yahay oo si wanaagsan loo sii diyaariyay. Tayada xogta liidata waxay si weyn u saameyn kartaa waxqabadka moodeelka.

Ugu Dambeyn

Hagaajinta LLM-yada iyadoo la adeegsanayo QLoRA waa hab wax ku ool ah oo lagu waafajiyo moodooyinka waaweyn ee horay loo tababaray hawlo gaar ah oo leh kharashyo xisaabeed oo la dhimay. Markaad raacdo hagahan, waxaad hagaajin kartaa PHI, Llama 3 ama nooc kasta oo kale oo furan si aad u gaadho waxqabad sare oo ku saabsan hawlahaaga gaarka ah.