Vault-8b-Instruct

A QLoRA fine-tune of meta-llama/Meta-Llama-3-8B-Instruct for answering questions about financial 10-K filings, given the relevant passage as context.

Model Details

Model Description

Vault-8b-Instruct is Llama-3-8B-Instruct fine-tuned with 4-bit QLoRA on 4,000 examples built from virattt/financial-qa-10K: each example pairs a question about a 10-K filing with the specific passage that answers it, and the model is trained to answer using only that provided passage.

• Developed by: harshil30402
• Model type: Causal decoder-only LM, LoRA adapter over Llama-3-8B-Instruct
• Language(s): English
• License: MIT
• Finetuned from model: meta-llama/Meta-Llama-3-8B-Instruct

Model Sources

• Repository: https://huggingface.co/harshil30402/Vault-8b-Instruct
• Training dataset: https://huggingface.co/datasets/virattt/financial-qa-10K

Uses

Direct Use

Answering a question about a financial filing when the relevant passage is supplied directly in the prompt (closed-book, context-conditioned QA) — e.g. as the answer-generation step in a larger document QA system that has already located the relevant passage some other way.

Out-of-Scope Use

This model does not perform retrieval. It was not trained or evaluated as part of a retrieval-augmented generation (RAG) pipeline — there is no embedding/vector-search/chunking component. If you need to find the relevant passage in a large filing before answering, that retrieval step has to be built separately; this model only handles the answer-generation half, given the passage already in hand.

Training Details

Training Data

• 4,000 training / 500 validation / 100 test examples, each a (question, context, answer) triple built from real 10-K filing excerpts (e.g. segment breakdowns, net earnings figures, loan interest rates).
• Formatted with the Llama-3 chat template and a fixed system prompt instructing the model to answer using only the given passage ("Use only the information to answer the question").

Training Procedure

QLoRA: 4-bit-quantized base model with a LoRA adapter trained on top.

Training Hyperparameters

Hyperparameter	Value
LoRA rank (r)	32
LoRA alpha	16
LoRA dropout	0.005
LoRA target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
LoRA bias	none
Learning rate	1e-4
LR scheduler	constant, 10% warmup
Optimizer	paged AdamW 8-bit
Effective batch size	8 (per-device 2 × gradient accumulation 4)
Epochs	1
Precision	fp16
Weight decay	0.0
Max sequence length	512
Seed	42

Evaluation

Not available. No retrieval-quality or answer-quality evaluation was run for this model — there is no before/after accuracy comparison, and no benchmark score to report. An earlier, unpublished claim that this fine-tune "reduced VRAM usage by ~60%" could not be traced to any actual memory measurement and is not repeated here; if you need a real efficiency number, it would need to be measured fresh (e.g. comparing peak memory for 4-bit-quantized vs. full-precision inference on the same hardware).

How to Get Started with the Model

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel

base_model_name = "meta-llama/Meta-Llama-3-8B-Instruct"
adapter_name = "harshil30402/Vault-8b-Instruct"

bnb_config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=torch.float16)

tokenizer = AutoTokenizer.from_pretrained(base_model_name)
base_model = AutoModelForCausalLM.from_pretrained(base_model_name, quantization_config=bnb_config, device_map="auto")
model = PeftModel.from_pretrained(base_model, adapter_name)
model.eval()

question = "How much did the company's net earnings amount to in fiscal 2022?"
context = "Net earnings were $17.1 billion in fiscal 2022."
messages = [
    {"role": "system", "content": "Use only the information to answer the question"},
    {"role": "user", "content": f"{question}\n\nInformation:\n\n```\n{context}\n```"},
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Requires a CUDA GPU for 4-bit quantized loading (bitsandbytes).

Bias, Risks, and Limitations

• No quantitative evaluation exists for this model — accuracy, faithfulness to the provided context, and hallucination rate are all unmeasured.
• Trained on general 10-K excerpts across unspecified companies/sectors; not validated on any specific company's filings or regulatory context.
• Not a retrieval system — cannot locate relevant passages in a full filing on its own.

Technical Specifications

Compute Infrastructure

• Software: PyTorch, Transformers, PEFT, bitsandbytes (4-bit QLoRA).

Model Card Contact

harshil30402