Normalize Motius Pipeline.from_pretrained metadata
Browse files- README.md +21 -13
- model_index.json +24 -8
README.md
CHANGED
|
@@ -1,5 +1,5 @@
|
|
| 1 |
---
|
| 2 |
-
library_name:
|
| 3 |
pipeline_tag: other
|
| 4 |
tags:
|
| 5 |
- motion-generation
|
|
@@ -13,20 +13,20 @@ license: other
|
|
| 13 |
|
| 14 |
# MoMask β Generative Masked Modeling of 3D Human Motions
|
| 15 |
|
| 16 |
-
Text-to-motion baseline integrated into the
|
| 17 |
reproduction is **fully self-contained and independent of `ref_repo`**: the
|
| 18 |
RVQ-VAE tokenizer, the masked generative transformer, the residual transformer
|
| 19 |
and the length estimator are all vendored into
|
| 20 |
-
`
|
| 21 |
released HumanML3D checkpoints. The CLIP ViT-B/32 text encoder is reloaded by
|
| 22 |
-
name only for legacy lightweight artifacts; new
|
| 23 |
`clip.safetensors`.
|
| 24 |
|
| 25 |
| | |
|
| 26 |
|---|---|
|
| 27 |
| **Task** | Text-to-Motion (T2M) |
|
| 28 |
| **Bundle / Pipeline** | `MoMaskBundle` / `MoMaskPipeline` |
|
| 29 |
-
| **Processed HF artifact** | [`ZeyuLing/
|
| 30 |
| **Motion representation** | **HumanML3D-263** (263-dim, 20 fps, 22 joints) |
|
| 31 |
| **Tokenizer** | RVQ-VAE, **6 residual quantizers**, codebook 512Γ512 |
|
| 32 |
| **Generator** | MaskTransformer (masked iterative decoding) + ResidualTransformer |
|
|
@@ -38,20 +38,20 @@ name only for legacy lightweight artifacts; new hftrainer artifacts include
|
|
| 38 |
|
| 39 |
## Weights
|
| 40 |
|
| 41 |
-
Self-contained
|
| 42 |
|
| 43 |
| Artifact | Location | Contents | Status |
|
| 44 |
|---|---|---|---|
|
| 45 |
-
| MoMask HumanML3D | [`ZeyuLing/
|
| 46 |
| local mirror | `checkpoints/momask/humanml3d` | same layout (produced by `convert_momask_checkpoint.py`, see below) | optional local cache |
|
| 47 |
|
| 48 |
Use the published artifact directly from the Hub:
|
| 49 |
|
| 50 |
```python
|
| 51 |
-
from
|
| 52 |
|
| 53 |
pipe = MoMaskPipeline.from_pretrained(
|
| 54 |
-
"ZeyuLing/
|
| 55 |
device="cuda",
|
| 56 |
)
|
| 57 |
motions = pipe.infer_t2m(
|
|
@@ -74,7 +74,7 @@ python3 scripts/eval/convert_momask_checkpoint.py \
|
|
| 74 |
**Use it:**
|
| 75 |
|
| 76 |
```python
|
| 77 |
-
from
|
| 78 |
|
| 79 |
pipe = MoMaskPipeline.from_pretrained("checkpoints/momask/humanml3d", device="cuda")
|
| 80 |
# fixed length (frames @ 20 fps):
|
|
@@ -144,7 +144,7 @@ python3 scripts/eval/verify_evaluators.py --which hml263 \
|
|
| 144 |
|
| 145 |
### HumanML3D-263 evaluator (native space)
|
| 146 |
|
| 147 |
-
| Metric |
|
| 148 |
|---|---|---|
|
| 149 |
| **FID** β | **0.097** | 0.045 |
|
| 150 |
| R-Precision Top-1 / 2 / 3 β | **0.516 / 0.709 / 0.804** | 0.521 / 0.713 / 0.807 |
|
|
@@ -168,7 +168,7 @@ native HumanML3D-263 predictions are retargeted through the validated MDM-style
|
|
| 168 |
chain: HML263 -> SMPL `motion_135` (IK refine-80, 20 -> 30 fps) ->
|
| 169 |
MotionStreamer-272 -> `MotionStreamer272Evaluator`.
|
| 170 |
|
| 171 |
-
| Metric |
|
| 172 |
|---|---:|---:|
|
| 173 |
| **FID** β | **114.869** | 0.000 |
|
| 174 |
| R-Precision Top-1 β | **0.485** | 0.706 |
|
|
@@ -186,7 +186,7 @@ Run details: `n_repeats = 20`, `n_samples_used = 7392`,
|
|
| 186 |
|
| 187 |
## Implementation notes
|
| 188 |
|
| 189 |
-
- **Vendored, ref_repo-independent**: `
|
| 190 |
holds the RVQ-VAE (`vq/`), the masked / residual transformers
|
| 191 |
(`mask_transformer/`) and the masked iterative decoding entry point
|
| 192 |
(`inference.py`). Imports are package-relative; training-only code paths are
|
|
@@ -200,3 +200,11 @@ Run details: `n_repeats = 20`, `n_samples_used = 7392`,
|
|
| 200 |
- **Normalization travels with the checkpoint**: `Mean.npy` / `Std.npy` are the
|
| 201 |
RVQ-VAE training stats, embedded in the artifact.
|
| 202 |
- **Guidance**: classifier-free, base `cond_scale=4`, residual `cond_scale=5`.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
library_name: motius
|
| 3 |
pipeline_tag: other
|
| 4 |
tags:
|
| 5 |
- motion-generation
|
|
|
|
| 13 |
|
| 14 |
# MoMask β Generative Masked Modeling of 3D Human Motions
|
| 15 |
|
| 16 |
+
Text-to-motion baseline integrated into the motius Model Zoo. Our
|
| 17 |
reproduction is **fully self-contained and independent of `ref_repo`**: the
|
| 18 |
RVQ-VAE tokenizer, the masked generative transformer, the residual transformer
|
| 19 |
and the length estimator are all vendored into
|
| 20 |
+
`motius.models.motion.momask._momask`, preserving numerical parity with the
|
| 21 |
released HumanML3D checkpoints. The CLIP ViT-B/32 text encoder is reloaded by
|
| 22 |
+
name only for legacy lightweight artifacts; new motius artifacts include
|
| 23 |
`clip.safetensors`.
|
| 24 |
|
| 25 |
| | |
|
| 26 |
|---|---|
|
| 27 |
| **Task** | Text-to-Motion (T2M) |
|
| 28 |
| **Bundle / Pipeline** | `MoMaskBundle` / `MoMaskPipeline` |
|
| 29 |
+
| **Processed HF artifact** | [`ZeyuLing/Motius-MoMask-HumanML3D`](https://huggingface.co/ZeyuLing/Motius-MoMask-HumanML3D) |
|
| 30 |
| **Motion representation** | **HumanML3D-263** (263-dim, 20 fps, 22 joints) |
|
| 31 |
| **Tokenizer** | RVQ-VAE, **6 residual quantizers**, codebook 512Γ512 |
|
| 32 |
| **Generator** | MaskTransformer (masked iterative decoding) + ResidualTransformer |
|
|
|
|
| 38 |
|
| 39 |
## Weights
|
| 40 |
|
| 41 |
+
Self-contained motius artifact (diffusers-style `from_pretrained`):
|
| 42 |
|
| 43 |
| Artifact | Location | Contents | Status |
|
| 44 |
|---|---|---|---|
|
| 45 |
+
| MoMask HumanML3D | [`ZeyuLing/Motius-MoMask-HumanML3D`](https://huggingface.co/ZeyuLing/Motius-MoMask-HumanML3D) | `vq.safetensors` + `t2m_trans.safetensors` + `res_trans.safetensors` + `length_est.safetensors` + `clip.safetensors` + `momask_config.json` + `Mean.npy` / `Std.npy` | public Hub artifact |
|
| 46 |
| local mirror | `checkpoints/momask/humanml3d` | same layout (produced by `convert_momask_checkpoint.py`, see below) | optional local cache |
|
| 47 |
|
| 48 |
Use the published artifact directly from the Hub:
|
| 49 |
|
| 50 |
```python
|
| 51 |
+
from motius.pipelines.momask import MoMaskPipeline
|
| 52 |
|
| 53 |
pipe = MoMaskPipeline.from_pretrained(
|
| 54 |
+
"ZeyuLing/Motius-MoMask-HumanML3D",
|
| 55 |
device="cuda",
|
| 56 |
)
|
| 57 |
motions = pipe.infer_t2m(
|
|
|
|
| 74 |
**Use it:**
|
| 75 |
|
| 76 |
```python
|
| 77 |
+
from motius.pipelines.momask import MoMaskPipeline
|
| 78 |
|
| 79 |
pipe = MoMaskPipeline.from_pretrained("checkpoints/momask/humanml3d", device="cuda")
|
| 80 |
# fixed length (frames @ 20 fps):
|
|
|
|
| 144 |
|
| 145 |
### HumanML3D-263 evaluator (native space)
|
| 146 |
|
| 147 |
+
| Metric | motius | MoMask paper |
|
| 148 |
|---|---|---|
|
| 149 |
| **FID** β | **0.097** | 0.045 |
|
| 150 |
| R-Precision Top-1 / 2 / 3 β | **0.516 / 0.709 / 0.804** | 0.521 / 0.713 / 0.807 |
|
|
|
|
| 168 |
chain: HML263 -> SMPL `motion_135` (IK refine-80, 20 -> 30 fps) ->
|
| 169 |
MotionStreamer-272 -> `MotionStreamer272Evaluator`.
|
| 170 |
|
| 171 |
+
| Metric | motius | MS-272 GT/Real |
|
| 172 |
|---|---:|---:|
|
| 173 |
| **FID** β | **114.869** | 0.000 |
|
| 174 |
| R-Precision Top-1 β | **0.485** | 0.706 |
|
|
|
|
| 186 |
|
| 187 |
## Implementation notes
|
| 188 |
|
| 189 |
+
- **Vendored, ref_repo-independent**: `motius/models/motion/momask/_momask/`
|
| 190 |
holds the RVQ-VAE (`vq/`), the masked / residual transformers
|
| 191 |
(`mask_transformer/`) and the masked iterative decoding entry point
|
| 192 |
(`inference.py`). Imports are package-relative; training-only code paths are
|
|
|
|
| 200 |
- **Normalization travels with the checkpoint**: `Mean.npy` / `Std.npy` are the
|
| 201 |
RVQ-VAE training stats, embedded in the artifact.
|
| 202 |
- **Guidance**: classifier-free, base `cond_scale=4`, residual `cond_scale=5`.
|
| 203 |
+
|
| 204 |
+
## Direct Loading
|
| 205 |
+
|
| 206 |
+
```python
|
| 207 |
+
from motius import Pipeline
|
| 208 |
+
|
| 209 |
+
pipeline = Pipeline.from_pretrained("ZeyuLing/Motius-MoMask-HumanML3D")
|
| 210 |
+
```
|
model_index.json
CHANGED
|
@@ -1,10 +1,10 @@
|
|
| 1 |
{
|
| 2 |
"_class_name": "MoMaskPipeline",
|
| 3 |
-
"_library_name": "
|
| 4 |
"model_type": "momask",
|
| 5 |
-
"format": "
|
| 6 |
-
"bundle_class": "
|
| 7 |
-
"pipeline_class": "
|
| 8 |
"artifacts": {
|
| 9 |
"vq": "vq.safetensors",
|
| 10 |
"t2m_transformer": "t2m_trans.safetensors",
|
|
@@ -29,7 +29,23 @@
|
|
| 29 |
}
|
| 30 |
},
|
| 31 |
"api": {
|
| 32 |
-
"
|
| 33 |
-
"
|
| 34 |
-
|
| 35 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
{
|
| 2 |
"_class_name": "MoMaskPipeline",
|
| 3 |
+
"_library_name": "motius",
|
| 4 |
"model_type": "momask",
|
| 5 |
+
"format": "motius-momask-v1",
|
| 6 |
+
"bundle_class": "motius.models.momask.MoMaskBundle",
|
| 7 |
+
"pipeline_class": "motius.pipelines.momask.MoMaskPipeline",
|
| 8 |
"artifacts": {
|
| 9 |
"vq": "vq.safetensors",
|
| 10 |
"t2m_transformer": "t2m_trans.safetensors",
|
|
|
|
| 29 |
}
|
| 30 |
},
|
| 31 |
"api": {
|
| 32 |
+
"loader": "motius.Pipeline.from_pretrained",
|
| 33 |
+
"task_methods": [
|
| 34 |
+
"infer_text_to_motion"
|
| 35 |
+
]
|
| 36 |
+
},
|
| 37 |
+
"format_version": 1,
|
| 38 |
+
"tasks": [
|
| 39 |
+
"text_to_motion"
|
| 40 |
+
],
|
| 41 |
+
"required_files": [
|
| 42 |
+
"Mean.npy",
|
| 43 |
+
"Std.npy",
|
| 44 |
+
"clip.safetensors",
|
| 45 |
+
"length_est.safetensors",
|
| 46 |
+
"momask_config.json",
|
| 47 |
+
"res_trans.safetensors",
|
| 48 |
+
"t2m_trans.safetensors",
|
| 49 |
+
"vq.safetensors"
|
| 50 |
+
]
|
| 51 |
+
}
|