MiniOneRec-1.5B-SFT-GDPO

Generative Recommendation Model — fine-tuned from Qwen2.5-1.5B-Instruct on Amazon Industrial & Scientific reviews.

Repository Contents

Directory	Description	Usage
`sft_model/`	RQ-VAE + SFT model weights	`AutoModelForCausalLM.from_pretrained()`
`gdpo_best_checkpoint-1155/`	Best GDPO RL checkpoint (step 1155)	Best RL model for evaluation
`data/`	Dataset files (.npy, .npz, .csv, .inter)	Required for training reproduction

Model Description

This model transforms item recommendation into a sequence generation task using Semantic IDs (SIDs):

User History → Qwen2.5-1.5B → Semantic ID Sequence → Item Recommendation

Pipeline

SID Construction: RQ-VAE compresses item text embeddings into 3-level discrete codes (256 codes × 3 levels = 16.7M capacity)
SFT: Full-parameter fine-tuning on Amazon Industrial & Scientific interaction sequences
RL (GRPO/GDPO): Reinforcement learning with constrained beam search and ranking-aware rewards

Key Details

Base Model: Qwen2.5-1.5B-Instruct
SID Method: RQ-VAE (M=3 levels, K=256 codes/level, e_dim=2560)
Vocabulary: 151,665 base + 521 SID tokens = 152,186 total
Dataset: Amazon Industrial & Scientific (36,259 train / 4,532 valid / 4,533 test)
Items: 3,686 unique items
Hardware: Single GPU ≥ 48GB VRAM

Performance

Evaluation on Amazon Industrial & Scientific test set (beam=50):

SFT Model

HR@3	NDCG@3	HR@5	NDCG@5	HR@10	NDCG@10
0.0904	0.0792	0.1061	0.0856	0.1337	0.0945

Best RL Checkpoints

Method	Step	HR@5	NDCG@5	HR@10	NDCG@10	vs SFT
GRPO	1320	0.1136	0.0926	0.1379	0.1005	+6.4%
GDPO	1155	0.1050	0.0880	0.1253	0.0944	-0.1%

GRPO significantly outperforms SFT (+6.4% NDCG@10). GDPO shows limited gains in this sparse-reward recommendation setting.

Comparison with Baselines

Method	HR@5	NDCG@10
SASRec	0.0909	0.0806
TIGER	0.1010	0.0908
D3	0.1213	0.1082
MiniOneRec (7B, paper)	0.1321	0.1167
Ours (1.5B, GRPO)	0.1136	0.1005

Usage

Inference

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "onesfour/MiniOneRec-1.5B-SFT-GDPO",
    subfolder="sft_model",          # or "gdpo_best_checkpoint-1155"
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(
    "onesfour/MiniOneRec-1.5B-SFT-GDPO",
    subfolder="sft_model",
    trust_remote_code=True,
)

prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
Can you predict the next possible item that the user may expect?

### User Input:
The user has interacted with items <a_115><b_58><c_12>, <a_86><b_17><c_25> in chronological order. Can you predict the next possible item that the user may expect?

### Response:
"""
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=10, num_beams=50)
predicted = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(f"Predicted SID: {predicted}")

SID Format

Items are represented as 3-level Semantic IDs with an optional deduplication suffix:

<a_X>: Level 0 code (0-255)
<b_Y>: Level 1 code (0-255)
<c_Z>: Level 2 code (0-255)
<d_W>: Deduplication suffix (if collision exists)

Example: <a_115><b_58><c_12> represents a unique item in the catalog.

Item Lookup

To map predicted SIDs back to item titles, use data/Amazon/index/Industrial_and_Scientific.item.json and data/Amazon/info/Industrial_and_Scientific_5_2016-10-2018-11.txt from the repository.

Requirements

transformers>=4.57.1
torch>=2.6.0

Training Reproduction

See YuyaoFan/MiniOneRec-1.5B for full source code, scripts, and documentation.

Quick start:

git clone https://github.com/YuyaoFan/MiniOneRec-1.5B.git
cd MiniOneRec-1.5B

# Download data files from this HF repo → data/Amazon/
# Download model from this HF repo/sft_model → output/sft/.../final_checkpoint/

conda create -n MiniOneRec python=3.11 -y && conda activate MiniOneRec
bash scripts/install_deps.sh
bash scripts/run_rqvae_sft_grpo.sh

Limitations

Domain Specific: Trained exclusively on Amazon Industrial & Scientific (3,686 items)
Cold Start: Cannot generate SIDs for items outside the trained catalog
Single Positive: Each test sample has only one ground-truth item (HR@K = Recall@K)
English Only: Item descriptions are English-only

Citation

@misc{kong2025minionerec,
    title={MiniOneRec: An Open-Source Framework for Scaling Generative Recommendation},
    author={Xiaoyu Kong and Leheng Sheng and Junfei Tan and Yuxin Chen and
            Jiancan Wu and An Zhang and Xiang Wang and Xiangnan He},
    year={2025},
    eprint={2510.24431},
    archivePrefix={arXiv},
    primaryClass={cs.IR},
}

@misc{liu2026gdpo,
    title={GDPO: Group reward-Decoupled Normalization Policy Optimization},
    author={Shih-Yang Liu et al.},
    year={2026},
    eprint={2601.05242},
    archivePrefix={arXiv},
}