ZeyuLing commited on
Commit
ff4d666
Β·
verified Β·
1 Parent(s): 8775565

Normalize Motius Pipeline.from_pretrained metadata

Browse files
Files changed (2) hide show
  1. README.md +21 -13
  2. model_index.json +24 -8
README.md CHANGED
@@ -1,5 +1,5 @@
1
  ---
2
- library_name: hftrainer
3
  pipeline_tag: other
4
  tags:
5
  - motion-generation
@@ -13,20 +13,20 @@ license: other
13
 
14
  # MoMask β€” Generative Masked Modeling of 3D Human Motions
15
 
16
- Text-to-motion baseline integrated into the hftrainer Model Zoo. Our
17
  reproduction is **fully self-contained and independent of `ref_repo`**: the
18
  RVQ-VAE tokenizer, the masked generative transformer, the residual transformer
19
  and the length estimator are all vendored into
20
- `hftrainer.models.motion.momask._momask`, preserving numerical parity with the
21
  released HumanML3D checkpoints. The CLIP ViT-B/32 text encoder is reloaded by
22
- name only for legacy lightweight artifacts; new hftrainer artifacts include
23
  `clip.safetensors`.
24
 
25
  | | |
26
  |---|---|
27
  | **Task** | Text-to-Motion (T2M) |
28
  | **Bundle / Pipeline** | `MoMaskBundle` / `MoMaskPipeline` |
29
- | **Processed HF artifact** | [`ZeyuLing/hftrainer-momask-humanml3d`](https://huggingface.co/ZeyuLing/hftrainer-momask-humanml3d) |
30
  | **Motion representation** | **HumanML3D-263** (263-dim, 20 fps, 22 joints) |
31
  | **Tokenizer** | RVQ-VAE, **6 residual quantizers**, codebook 512Γ—512 |
32
  | **Generator** | MaskTransformer (masked iterative decoding) + ResidualTransformer |
@@ -38,20 +38,20 @@ name only for legacy lightweight artifacts; new hftrainer artifacts include
38
 
39
  ## Weights
40
 
41
- Self-contained hftrainer artifact (diffusers-style `from_pretrained`):
42
 
43
  | Artifact | Location | Contents | Status |
44
  |---|---|---|---|
45
- | MoMask HumanML3D | [`ZeyuLing/hftrainer-momask-humanml3d`](https://huggingface.co/ZeyuLing/hftrainer-momask-humanml3d) | `vq.safetensors` + `t2m_trans.safetensors` + `res_trans.safetensors` + `length_est.safetensors` + `clip.safetensors` + `momask_config.json` + `Mean.npy` / `Std.npy` | public Hub artifact |
46
  | local mirror | `checkpoints/momask/humanml3d` | same layout (produced by `convert_momask_checkpoint.py`, see below) | optional local cache |
47
 
48
  Use the published artifact directly from the Hub:
49
 
50
  ```python
51
- from hftrainer.pipelines.momask import MoMaskPipeline
52
 
53
  pipe = MoMaskPipeline.from_pretrained(
54
- "ZeyuLing/hftrainer-momask-humanml3d",
55
  device="cuda",
56
  )
57
  motions = pipe.infer_t2m(
@@ -74,7 +74,7 @@ python3 scripts/eval/convert_momask_checkpoint.py \
74
  **Use it:**
75
 
76
  ```python
77
- from hftrainer.pipelines.momask import MoMaskPipeline
78
 
79
  pipe = MoMaskPipeline.from_pretrained("checkpoints/momask/humanml3d", device="cuda")
80
  # fixed length (frames @ 20 fps):
@@ -144,7 +144,7 @@ python3 scripts/eval/verify_evaluators.py --which hml263 \
144
 
145
  ### HumanML3D-263 evaluator (native space)
146
 
147
- | Metric | hftrainer | MoMask paper |
148
  |---|---|---|
149
  | **FID** ↓ | **0.097** | 0.045 |
150
  | R-Precision Top-1 / 2 / 3 ↑ | **0.516 / 0.709 / 0.804** | 0.521 / 0.713 / 0.807 |
@@ -168,7 +168,7 @@ native HumanML3D-263 predictions are retargeted through the validated MDM-style
168
  chain: HML263 -> SMPL `motion_135` (IK refine-80, 20 -> 30 fps) ->
169
  MotionStreamer-272 -> `MotionStreamer272Evaluator`.
170
 
171
- | Metric | hftrainer | MS-272 GT/Real |
172
  |---|---:|---:|
173
  | **FID** ↓ | **114.869** | 0.000 |
174
  | R-Precision Top-1 ↑ | **0.485** | 0.706 |
@@ -186,7 +186,7 @@ Run details: `n_repeats = 20`, `n_samples_used = 7392`,
186
 
187
  ## Implementation notes
188
 
189
- - **Vendored, ref_repo-independent**: `hftrainer/models/motion/momask/_momask/`
190
  holds the RVQ-VAE (`vq/`), the masked / residual transformers
191
  (`mask_transformer/`) and the masked iterative decoding entry point
192
  (`inference.py`). Imports are package-relative; training-only code paths are
@@ -200,3 +200,11 @@ Run details: `n_repeats = 20`, `n_samples_used = 7392`,
200
  - **Normalization travels with the checkpoint**: `Mean.npy` / `Std.npy` are the
201
  RVQ-VAE training stats, embedded in the artifact.
202
  - **Guidance**: classifier-free, base `cond_scale=4`, residual `cond_scale=5`.
 
 
 
 
 
 
 
 
 
1
  ---
2
+ library_name: motius
3
  pipeline_tag: other
4
  tags:
5
  - motion-generation
 
13
 
14
  # MoMask β€” Generative Masked Modeling of 3D Human Motions
15
 
16
+ Text-to-motion baseline integrated into the motius Model Zoo. Our
17
  reproduction is **fully self-contained and independent of `ref_repo`**: the
18
  RVQ-VAE tokenizer, the masked generative transformer, the residual transformer
19
  and the length estimator are all vendored into
20
+ `motius.models.motion.momask._momask`, preserving numerical parity with the
21
  released HumanML3D checkpoints. The CLIP ViT-B/32 text encoder is reloaded by
22
+ name only for legacy lightweight artifacts; new motius artifacts include
23
  `clip.safetensors`.
24
 
25
  | | |
26
  |---|---|
27
  | **Task** | Text-to-Motion (T2M) |
28
  | **Bundle / Pipeline** | `MoMaskBundle` / `MoMaskPipeline` |
29
+ | **Processed HF artifact** | [`ZeyuLing/Motius-MoMask-HumanML3D`](https://huggingface.co/ZeyuLing/Motius-MoMask-HumanML3D) |
30
  | **Motion representation** | **HumanML3D-263** (263-dim, 20 fps, 22 joints) |
31
  | **Tokenizer** | RVQ-VAE, **6 residual quantizers**, codebook 512Γ—512 |
32
  | **Generator** | MaskTransformer (masked iterative decoding) + ResidualTransformer |
 
38
 
39
  ## Weights
40
 
41
+ Self-contained motius artifact (diffusers-style `from_pretrained`):
42
 
43
  | Artifact | Location | Contents | Status |
44
  |---|---|---|---|
45
+ | MoMask HumanML3D | [`ZeyuLing/Motius-MoMask-HumanML3D`](https://huggingface.co/ZeyuLing/Motius-MoMask-HumanML3D) | `vq.safetensors` + `t2m_trans.safetensors` + `res_trans.safetensors` + `length_est.safetensors` + `clip.safetensors` + `momask_config.json` + `Mean.npy` / `Std.npy` | public Hub artifact |
46
  | local mirror | `checkpoints/momask/humanml3d` | same layout (produced by `convert_momask_checkpoint.py`, see below) | optional local cache |
47
 
48
  Use the published artifact directly from the Hub:
49
 
50
  ```python
51
+ from motius.pipelines.momask import MoMaskPipeline
52
 
53
  pipe = MoMaskPipeline.from_pretrained(
54
+ "ZeyuLing/Motius-MoMask-HumanML3D",
55
  device="cuda",
56
  )
57
  motions = pipe.infer_t2m(
 
74
  **Use it:**
75
 
76
  ```python
77
+ from motius.pipelines.momask import MoMaskPipeline
78
 
79
  pipe = MoMaskPipeline.from_pretrained("checkpoints/momask/humanml3d", device="cuda")
80
  # fixed length (frames @ 20 fps):
 
144
 
145
  ### HumanML3D-263 evaluator (native space)
146
 
147
+ | Metric | motius | MoMask paper |
148
  |---|---|---|
149
  | **FID** ↓ | **0.097** | 0.045 |
150
  | R-Precision Top-1 / 2 / 3 ↑ | **0.516 / 0.709 / 0.804** | 0.521 / 0.713 / 0.807 |
 
168
  chain: HML263 -> SMPL `motion_135` (IK refine-80, 20 -> 30 fps) ->
169
  MotionStreamer-272 -> `MotionStreamer272Evaluator`.
170
 
171
+ | Metric | motius | MS-272 GT/Real |
172
  |---|---:|---:|
173
  | **FID** ↓ | **114.869** | 0.000 |
174
  | R-Precision Top-1 ↑ | **0.485** | 0.706 |
 
186
 
187
  ## Implementation notes
188
 
189
+ - **Vendored, ref_repo-independent**: `motius/models/motion/momask/_momask/`
190
  holds the RVQ-VAE (`vq/`), the masked / residual transformers
191
  (`mask_transformer/`) and the masked iterative decoding entry point
192
  (`inference.py`). Imports are package-relative; training-only code paths are
 
200
  - **Normalization travels with the checkpoint**: `Mean.npy` / `Std.npy` are the
201
  RVQ-VAE training stats, embedded in the artifact.
202
  - **Guidance**: classifier-free, base `cond_scale=4`, residual `cond_scale=5`.
203
+
204
+ ## Direct Loading
205
+
206
+ ```python
207
+ from motius import Pipeline
208
+
209
+ pipeline = Pipeline.from_pretrained("ZeyuLing/Motius-MoMask-HumanML3D")
210
+ ```
model_index.json CHANGED
@@ -1,10 +1,10 @@
1
  {
2
  "_class_name": "MoMaskPipeline",
3
- "_library_name": "hftrainer",
4
  "model_type": "momask",
5
- "format": "hftrainer-momask-v1",
6
- "bundle_class": "hftrainer.models.motion.momask.bundle.MoMaskBundle",
7
- "pipeline_class": "hftrainer.pipelines.momask.pipeline.MoMaskPipeline",
8
  "artifacts": {
9
  "vq": "vq.safetensors",
10
  "t2m_transformer": "t2m_trans.safetensors",
@@ -29,7 +29,23 @@
29
  }
30
  },
31
  "api": {
32
- "from_pretrained": "hftrainer.models.motion.momask.MoMaskBundle.from_pretrained",
33
- "from_config": "hftrainer.models.motion.momask.MoMaskBundle.from_config"
34
- }
35
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  {
2
  "_class_name": "MoMaskPipeline",
3
+ "_library_name": "motius",
4
  "model_type": "momask",
5
+ "format": "motius-momask-v1",
6
+ "bundle_class": "motius.models.momask.MoMaskBundle",
7
+ "pipeline_class": "motius.pipelines.momask.MoMaskPipeline",
8
  "artifacts": {
9
  "vq": "vq.safetensors",
10
  "t2m_transformer": "t2m_trans.safetensors",
 
29
  }
30
  },
31
  "api": {
32
+ "loader": "motius.Pipeline.from_pretrained",
33
+ "task_methods": [
34
+ "infer_text_to_motion"
35
+ ]
36
+ },
37
+ "format_version": 1,
38
+ "tasks": [
39
+ "text_to_motion"
40
+ ],
41
+ "required_files": [
42
+ "Mean.npy",
43
+ "Std.npy",
44
+ "clip.safetensors",
45
+ "length_est.safetensors",
46
+ "momask_config.json",
47
+ "res_trans.safetensors",
48
+ "t2m_trans.safetensors",
49
+ "vq.safetensors"
50
+ ]
51
+ }