@yanghaojin on Hugging Face: "Full parameter fine-tuning of the LLaMA-3 8B model using a single GTX 3090 GPU…"

Hugging Face

Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Back to feed

yanghaojin

posted an update 15 days ago

Post

1479

Full parameter fine-tuning of the LLaMA-3 8B model using a single GTX 3090 GPU with 24GB of graphics memory?

Please check out our tool for fine-tuning, inferencing, and evaluating GreenBitAI's low-bit LLMs:
https://github.com/GreenBitAI/green-bit-llm
Model Zoo:
https://huggingface.co/GreenBitAI

yanghaojin

15 days ago

Command for reproducing this run 😉 :
CUDA_VISIBLE_DEVICES=0 WANDB_DISABLED=true python -m sft.finetune --model GreenBitAI/Llama-3-8B-layer-mix-bpw-2.2 --tune-qweight-only --galore --galore-rank 64 --optimizer adamw8bit --batch-size 1 --seqlen 96

jqodiriy

7 days ago

How you prepare dataset for finetuning llama3?
Could you show the structure of your dataset and how you fine-tune using that dataset?

jqodiriy

7 days ago

How you prepare dataset for finetuning llama3?
Could you show the structure of your dataset and how you fine-tune using that dataset?

In this post