Text Classification
PyTorch
Safetensors
Transformers
English
janegpt_v2_janus
janegpt
janus
react
interactive
vite
intent-classification
transformer
virtual-assistant
nlp
voice-assistant
offline-ai
edge-deployment
nlu
slot-filling
multitask-learning
assistant-runtime
Eval Results (legacy)
Instructions to use RavinduSen/JaneGPT-v2-Janus with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use RavinduSen/JaneGPT-v2-Janus with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="RavinduSen/JaneGPT-v2-Janus")# pip install -U transformers accelerate # Load model directly from transformers import JaneGPTv3NLU model = JaneGPTv3NLU.from_pretrained("RavinduSen/JaneGPT-v2-Janus", device_map="auto") - Notebooks
- Google Colab
- Kaggle
|
Download README.md from RavinduSen/JaneGPT-v2-Janus: direct link, hf CLI and curl.
- Browser
- Download file 58.6 kB
-
https://huggingface.co/RavinduSen/JaneGPT-v2-Janus/resolve/main/README.md
- Command line
-
hf download hf://RavinduSen/JaneGPT-v2-Janus/README.md
-
curl -L -o README.md https://huggingface.co/RavinduSen/JaneGPT-v2-Janus/resolve/main/README.md
58.6 kB
| language: | |
| - en | |
| license: apache-2.0 | |
| library_name: pytorch | |
| tags: | |
| - safetensors | |
| - janegpt | |
| - janus | |
| - react | |
| - interactive | |
| - vite | |
| - intent-classification | |
| - transformer | |
| - virtual-assistant | |
| - nlp | |
| - voice-assistant | |
| - offline-ai | |
| - edge-deployment | |
| - nlu | |
| - transformers | |
| - slot-filling | |
| - multitask-learning | |
| - assistant-runtime | |
| demo: https://janus-web-demo.vercel.app/ | |
| pipeline_tag: text-classification | |
| model-index: | |
| - name: JaneGPT-v2-Janus | |
| results: | |
| - task: | |
| type: text-classification | |
| dataset: | |
| name: BANKING77 | |
| type: mteb/banking77 | |
| metrics: | |
| - type: precision | |
| value: 1.0 | |
| name: OOD Precision | |
| - type: f1 | |
| value: 0.878 | |
| name: OOD F1 | |
| - type: recall | |
| value: 0.7825 | |
| name: OOD Recall | |
| - task: | |
| type: text-classification | |
| dataset: | |
| name: CLINC OOS | |
| type: clinc/clinc_oos | |
| metrics: | |
| - type: precision | |
| value: 1.0 | |
| name: OOD Precision | |
| - type: f1 | |
| value: 0.7923 | |
| name: OOD F1 | |
| - type: recall | |
| value: 0.656 | |
| name: OOD Recall | |
| - task: | |
| type: text-classification | |
| metrics: | |
| - type: accuracy | |
| value: 0.9983 | |
| name: Validation Domain Accuracy | |
| - type: accuracy | |
| value: 0.9987 | |
| name: Validation Action Accuracy | |
| - type: f1 | |
| value: 1.0 | |
| name: Slot Extraction F1 | |
| # JaneGPT v2 Janus - Intent Classification Model | |
| --- | |
|  | |
|  | |
| --- | |
| <p align="center"> | |
| <a href="https://janus-web-demo.vercel.app/"> | |
| <img src="https://img.shields.io/badge/Launch-Temple_of_Janus-6366f1?style=for-the-badge&logo=vercel" alt="Experience Janus"> | |
| </a> | |
| </p> | |
| <style> | |
| /* ---- Hero section ---- */ | |
| .hero-banner { text-align: center; margin-bottom: 12px; } | |
| .hero-banner img { max-width: 100%; border-radius: 12px; } | |
| .subtitle { | |
| text-align: center; | |
| font-size: 1.15em; | |
| color: #6b7a90; | |
| margin-bottom: 28px; | |
| } | |
| /* ---- Stat cards (mini key-value row) ---- */ | |
| .stat-row { | |
| display: flex; | |
| flex-wrap: wrap; | |
| gap: 14px; | |
| justify-content: center; | |
| margin-bottom: 32px; | |
| } | |
| .stat-card { | |
| background: #0f1923; | |
| border: 1px solid #1e3048; | |
| border-radius: 10px; | |
| padding: 14px 22px; | |
| min-width: 150px; | |
| text-align: center; | |
| cursor: default; | |
| } | |
| .stat-card .stat-value { | |
| font-size: 1.65em; | |
| font-weight: 700; | |
| color: #58a6ff; | |
| line-height: 1.15; | |
| } | |
| .stat-card .stat-label { | |
| font-size: 0.82em; | |
| color: #8b9ab5; | |
| margin-top: 4px; | |
| text-transform: uppercase; | |
| letter-spacing: 0.5px; | |
| } | |
| /* ---- Section headings ---- */ | |
| .section-heading { | |
| font-size: 1.35em; | |
| font-weight: 700; | |
| color: #e6edf3; | |
| border-bottom: 1px solid #1e3048; | |
| padding-bottom: 6px; | |
| margin-top: 36px; | |
| margin-bottom: 16px; | |
| } | |
| /* ---- Horizontal bar chart (interactive) ---- */ | |
| .chart-container { | |
| background: #0d1520; | |
| border: 1px solid #1a2a3e; | |
| border-radius: 12px; | |
| padding: 24px 28px 60px 28px; | |
| margin-bottom: 24px; | |
| display: flex; | |
| flex-direction: column; | |
| } | |
| .chart-title { | |
| font-size: 1.1em; | |
| font-weight: 700; | |
| color: #e2e8f0; | |
| margin-bottom: 4px; | |
| } | |
| .chart-subtitle { | |
| font-size: 0.88em; | |
| color: #7a8ba3; | |
| margin-bottom: 14px; | |
| line-height: 1.35; | |
| position: relative; | |
| z-index: 3; | |
| } | |
| .bar-group { | |
| margin-bottom: 18px; | |
| } | |
| .bar-label { | |
| font-size: 0.92em; | |
| color: #c5d0de; | |
| margin-bottom: 5px; | |
| font-weight: 600; | |
| } | |
| /* NEW โ replace with this */ | |
| .bar-group { | |
| display: flex; | |
| flex-direction: column; | |
| align-items: center; | |
| flex: 1; | |
| } | |
| .bar-label { | |
| font-size: 0.78em; | |
| color: #8ea2b8; | |
| margin-top: 6px; | |
| text-align: center; | |
| font-weight: 600; | |
| } | |
| .bar-track { | |
| background: #162030; | |
| border-radius: 8px 8px 0 0; | |
| width: 100%; | |
| height: 120px; | |
| position: relative; | |
| display: flex; | |
| align-items: flex-end; | |
| overflow: hidden; | |
| } | |
| .bar-fill { | |
| width: 100%; | |
| border-radius: 6px 6px 0 0; | |
| transition: height 0.8s cubic-bezier(.22,.61,.36,1), box-shadow 0.3s ease; | |
| position: relative; | |
| min-height: 24px; | |
| display: flex; | |
| align-items: flex-start; | |
| justify-content: center; | |
| } | |
| .bar-fill:hover { | |
| box-shadow: 0 0 20px rgba(59, 130, 246, 0.3), inset 0 0 0 999px rgba(255,255,255,0.06); | |
| } | |
| .bar-value { | |
| position: absolute; | |
| top: 7px; | |
| left: 50%; | |
| transform: translateX(-50%); | |
| font-size: 0.78em; | |
| font-weight: 700; | |
| color: #000000; | |
| white-space: nowrap; | |
| line-height: 1; | |
| text-shadow: 0 1px 2px rgba(0,0,0,0.45); | |
| z-index: 2; | |
| pointer-events: none; | |
| } | |
| .bar-note { | |
| font-size: 0.78em; | |
| color: #5c6d82; | |
| margin-top: 6px; | |
| font-style: italic; | |
| } | |
| /* NEW โ add this */ | |
| .charts-row { | |
| display: flex; | |
| gap: 16px; | |
| align-items: stretch; | |
| margin-bottom: 24px; | |
| } | |
| .chart-container-sm { | |
| background: #0d1520; | |
| border: 1px solid #1a2a3e; | |
| border-radius: 12px; | |
| padding: 18px 16px; | |
| flex: 1; | |
| display: flex; | |
| flex-direction: column; | |
| } | |
| .bars-row { | |
| display: flex; | |
| gap: 8px; | |
| align-items: flex-end; | |
| min-height: 150px; | |
| height: 150px; | |
| border-bottom: 1px solid #1e3048; | |
| margin-bottom: 8px; | |
| margin-top: 10px; | |
| } | |
| .bars-row-wide { | |
| gap: 16px; | |
| } | |
| .ood-filter-row { | |
| margin-bottom: 14px; | |
| position: relative; | |
| z-index: 4; | |
| } | |
| .chart-bars { | |
| margin-top: 8px; | |
| position: relative; | |
| z-index: 1; | |
| } | |
| @media (max-width: 900px) { | |
| .charts-row { | |
| flex-direction: column; | |
| align-items: stretch; | |
| } | |
| } | |
| /* ---- Filter buttons (CSS-only via radio hack) ---- */ | |
| .filter-row { | |
| display: flex; | |
| flex-wrap: wrap; | |
| gap: 8px; | |
| margin-bottom: 18px; | |
| } | |
| .filter-row input[type="radio"] { | |
| display: none; | |
| } | |
| input[name="ood-filter"], | |
| input[name="cm-toggle"] { | |
| position: absolute; | |
| opacity: 0; | |
| width: 0; | |
| height: 0; | |
| pointer-events: none; | |
| } | |
| .filter-btn { | |
| display: inline-block; | |
| padding: 5px 14px; | |
| font-size: 0.82em; | |
| font-weight: 600; | |
| border-radius: 20px; | |
| border: 1px solid #253649; | |
| background: #111c2a; | |
| color: #8ea2b8; | |
| cursor: pointer; | |
| transition: background 0.2s, color 0.2s, border-color 0.2s; | |
| user-select: none; | |
| } | |
| .filter-btn:hover { | |
| background: #1a2d42; | |
| color: #c5d8e8; | |
| } | |
| #ood-both:checked ~ .ood-filter-row label[for="ood-both"], | |
| #ood-banking77:checked ~ .ood-filter-row label[for="ood-banking77"], | |
| #ood-clinc:checked ~ .ood-filter-row label[for="ood-clinc"] { | |
| background: #1f6feb; | |
| color: #fff; | |
| border-color: #1f6feb; | |
| } | |
| /* Bar groups visibility controlled by radio state */ | |
| .bar-group[data-dataset] { display: flex; } | |
| #ood-banking77:checked ~ .chart-bars .bar-group[data-dataset="clinc_oos"] { display: none; } | |
| #ood-clinc:checked ~ .chart-bars .bar-group[data-dataset="banking77"] { display: none; } | |
| #ood-both:checked ~ .chart-bars .bar-group[data-dataset] { display: flex; } | |
| /* ---- Comparison table ---- */ | |
| .comparison-table { | |
| width: 100%; | |
| border-collapse: collapse; | |
| font-size: 0.9em; | |
| margin-bottom: 16px; | |
| } | |
| .comparison-table th { | |
| background: #111c2a; | |
| color: #8ea2b8; | |
| font-weight: 700; | |
| text-transform: uppercase; | |
| font-size: 0.78em; | |
| letter-spacing: 0.6px; | |
| padding: 10px 14px; | |
| border-bottom: 2px solid #1e3048; | |
| text-align: left; | |
| } | |
| .comparison-table td { | |
| padding: 9px 14px; | |
| border-bottom: 1px solid #152232; | |
| color: #c5d0de; | |
| } | |
| .comparison-table tr:hover td { | |
| background: #111e2e; | |
| } | |
| .tag-janus { | |
| display: inline-block; | |
| background: #1a3a2a; | |
| color: #4ade80; | |
| border-radius: 6px; | |
| padding: 1px 8px; | |
| font-size: 0.85em; | |
| font-weight: 600; | |
| } | |
| .tag-v2 { | |
| display: inline-block; | |
| background: #1a2a3a; | |
| color: #60a5fa; | |
| border-radius: 6px; | |
| padding: 1px 8px; | |
| font-size: 0.85em; | |
| font-weight: 600; | |
| } | |
| /* ---- Segmented stacked bar (single bar per panel) ---- */ | |
| .stacked-bar-wrapper { | |
| margin-bottom: 12px; | |
| position: relative; | |
| } | |
| .stacked-bar-label { | |
| font-size: 0.85em; | |
| font-weight: 600; | |
| color: #8b9ab5; | |
| margin-bottom: 6px; | |
| } | |
| .stacked-bar { | |
| display: flex; | |
| height: 38px; | |
| border-radius: 8px; | |
| overflow: hidden; | |
| background: #162030; | |
| cursor: default; | |
| } | |
| .stacked-seg { | |
| height: 100%; | |
| transition: filter 0.2s ease, flex 0.3s ease; | |
| position: relative; | |
| } | |
| .stacked-seg:hover { | |
| filter: brightness(1.4) saturate(1.3); | |
| outline: 2px solid #fff; | |
| outline-offset: -2px; | |
| z-index: 5; | |
| } | |
| /* Tooltip for stacked bar segments */ | |
| .stacked-tooltip { | |
| display: none; | |
| position: absolute; | |
| left: 50%; | |
| transform: translateX(-50%); | |
| top: calc(100% + 6px); | |
| background: #0f1923; | |
| border: 1px solid #253649; | |
| border-radius: 8px; | |
| padding: 8px 14px; | |
| font-size: 0.82em; | |
| color: #e2e8f0; | |
| z-index: 20; | |
| white-space: nowrap; | |
| box-shadow: 0 6px 20px rgba(0,0,0,0.5); | |
| pointer-events: none; | |
| } | |
| .stacked-tooltip::before { | |
| content: ''; | |
| position: absolute; | |
| top: -6px; | |
| left: 50%; | |
| transform: translateX(-50%); | |
| border-left: 6px solid transparent; | |
| border-right: 6px solid transparent; | |
| border-bottom: 6px solid #253649; | |
| } | |
| .stacked-seg:hover .stacked-tooltip { | |
| display: block; | |
| } | |
| .stacked-tooltip .st-name { font-weight: 700; color: #4ade80; } | |
| .stacked-tooltip .st-val { color: #8ea2b8; } | |
| /* ---- Confusion matrix panel toggle ---- */ | |
| #cm-domain:checked ~ .confusion-toggle label[for="cm-domain"], | |
| #cm-action:checked ~ .confusion-toggle label[for="cm-action"] { | |
| background: #1f6feb; | |
| color: #fff; | |
| border-color: #1f6feb; | |
| } | |
| .confusion-toggle { | |
| display: flex; | |
| gap: 8px; | |
| margin-bottom: 18px; | |
| } | |
| .cm-panel[data-cm="domain"] { display: block; } | |
| .cm-panel[data-cm="action"] { display: none; } | |
| #cm-action:checked ~ .cm-panel[data-cm="domain"] { display: none; } | |
| #cm-action:checked ~ .cm-panel[data-cm="action"] { display: block; } | |
| #cm-domain:checked ~ .cm-panel[data-cm="domain"] { display: block; } | |
| #cm-domain:checked ~ .cm-panel[data-cm="action"] { display: none; } | |
| /* ---- Confusion stacked bar ---- */ | |
| .cm-stacked-bar { | |
| display: flex; | |
| height: 44px; | |
| border-radius: 10px; | |
| overflow: visible; | |
| background: #162030; | |
| position: relative; | |
| margin-bottom: 8px; | |
| } | |
| .cm-seg { | |
| height: 100%; | |
| position: relative; | |
| transition: filter 0.2s ease; | |
| cursor: default; | |
| /* thin separator between segments */ | |
| border-right: 1px solid rgba(0,0,0,0.35); | |
| } | |
| .cm-seg:last-child { border-right: none; } | |
| .cm-seg:hover { | |
| filter: brightness(1.45) saturate(1.3); | |
| z-index: 10; | |
| } | |
| /* Tooltip on hover */ | |
| .cm-seg .cm-tip { | |
| display: none; | |
| position: absolute; | |
| bottom: calc(100% + 8px); | |
| left: 50%; | |
| transform: translateX(-50%); | |
| background: #0f1923; | |
| border: 1px solid #253649; | |
| border-radius: 8px; | |
| padding: 10px 16px; | |
| font-size: 0.82em; | |
| color: #e2e8f0; | |
| z-index: 30; | |
| white-space: nowrap; | |
| box-shadow: 0 6px 24px rgba(0,0,0,0.55); | |
| pointer-events: none; | |
| } | |
| .cm-seg .cm-tip::after { | |
| content: ''; | |
| position: absolute; | |
| bottom: -6px; | |
| left: 50%; | |
| transform: translateX(-50%); | |
| border-left: 6px solid transparent; | |
| border-right: 6px solid transparent; | |
| border-top: 6px solid #253649; | |
| } | |
| .cm-seg:hover .cm-tip { display: block; } | |
| .cm-tip .tip-name { font-weight: 700; color: #58a6ff; } | |
| .cm-tip .tip-samples { color: #4ade80; } | |
| .cm-tip .tip-acc { color: #fbbf24; } | |
| .cm-tip .tip-miss { color: #f87171; } | |
| .cm-legend { | |
| display: flex; | |
| flex-wrap: wrap; | |
| gap: 10px 18px; | |
| margin-top: 10px; | |
| margin-bottom: 6px; | |
| } | |
| .cm-legend-item { | |
| display: flex; | |
| align-items: center; | |
| gap: 6px; | |
| font-size: 0.78em; | |
| color: #8b9ab5; | |
| } | |
| .cm-legend-swatch { | |
| width: 12px; | |
| height: 12px; | |
| border-radius: 3px; | |
| display: inline-block; | |
| } | |
| /* ---- Code blocks ---- */ | |
| details { | |
| background: #0d1520; | |
| border: 1px solid #1a2a3e; | |
| border-radius: 8px; | |
| margin-bottom: 10px; | |
| padding: 0; | |
| } | |
| details summary { | |
| cursor: pointer; | |
| padding: 12px 18px; | |
| font-weight: 600; | |
| color: #c5d0de; | |
| list-style: none; | |
| } | |
| details summary::-webkit-details-marker { display: none; } | |
| details summary::before { content: "โธ "; color: #58a6ff; } | |
| details[open] summary::before { content: "โพ "; } | |
| details > div { | |
| padding: 0 18px 16px; | |
| } | |
| </style> | |
| <!-- ===== Hero ===== --> | |
| <div class="hero-banner"> | |
| <img src="assets/jane-janus-glitch.webp" alt="Jane Janus animated hero banner" width="980" /> | |
| </div> | |
| <p class="subtitle">Hierarchical command understanding with state-aware runtime behavior for practical assistant workflows.</p> | |
| <!-- ===== Stat Cards ===== --> | |
| <div class="stat-row"> | |
| <div class="stat-card"> | |
| <div class="stat-value">7.95M</div> | |
| <div class="stat-label">Parameters</div> | |
| </div> | |
| <div class="stat-card"> | |
| <div class="stat-value">82</div> | |
| <div class="stat-label">Runtime Turns</div> | |
| </div> | |
| <div class="stat-card"> | |
| <div class="stat-value">0</div> | |
| <div class="stat-label">Errors</div> | |
| </div> | |
| <div class="stat-card"> | |
| <div class="stat-value">25.3 ms</div> | |
| <div class="stat-label">Mean Latency</div> | |
| </div> | |
| <div class="stat-card"> | |
| <div class="stat-value">100%</div> | |
| <div class="stat-label">OOD Precision</div> | |
| </div> | |
| <div class="stat-card"> | |
| <div class="stat-value">30.6 MB</div> | |
| <div class="stat-label">Checkpoint</div> | |
| </div> | |
| </div> | |
| --- | |
| ## ๐๏ธ The Temple of Janus (Web Experience) | |
| We have deployed a dedicated interactive environment to showcase the essence of JaneGPT-v2 Janus. | |
| **Note:** This is a visual and technical walkthrough; it does not feature a live chat interface. | |
| * **[๐ Enter the Experience](https://janus-web-demo.vercel.app/)** | |
| * **Best Viewed On:** Desktop (Chrome/Edge) for full hardware-accelerated 3D effects. | |
| --- | |
| ## Quickstart (2 minutes) | |
| <details open> | |
| <summary><strong>Install + first prediction</strong></summary> | |
| <div> | |
| ```bash | |
| pip install -r requirements.txt | |
| ``` | |
| ```python | |
| from janegpt_v2_janus.inference import JaneGPTv3NLU | |
| nlu = JaneGPTv3NLU( | |
| model_path="weights/janegpt_v2_janus.pt", | |
| tokenizer_path="weights/tokenizer.json", | |
| ) | |
| state = {} | |
| result = nlu.predict("set volume", state=state) | |
| print(result) | |
| if result.get("type") == "command": | |
| state = nlu.update_state(result, state) | |
| ``` | |
| </div> | |
| </details> | |
| <details> | |
| <summary><strong>Runtime wrapper (recommended for assistant flows)</strong></summary> | |
| <div> | |
| ```python | |
| from runtime.jane_nlu_runtime import JaneNLURuntime | |
| rt = JaneNLURuntime(base_dir=".") | |
| state = {} | |
| out, state = rt.handle_turn("set volume", state) | |
| print(out) # expected: clarify prompt for missing VALUE | |
| out, state = rt.handle_turn("55", state) | |
| print(out) # expected: resolved local command | |
| ``` | |
| </div> | |
| </details> | |
| <details> | |
| <summary><strong>Run bundled demos</strong></summary> | |
| <div> | |
| ```bash | |
| python examples/demo_inference.py | |
| python examples/demo_runtime.py | |
| python examples/demo_runtime_suite.py | |
| ``` | |
| </div> | |
| </details> | |
| --- | |
| ## What You Get | |
| - Single-pass multitask prediction: domain + action + BIO slots. | |
| - Runtime-safe clarification loops for missing required slots. | |
| - Stateful follow-ups (for example, "that is not enough" after a volume change). | |
| - Local command routing with controlled chat fallback. | |
| - Compact deployment footprint: ~30.62 MB checkpoint. | |
| --- | |
| ## Model Architecture | |
| ### Interactive Architecture Visualization | |
| <style> | |
| /* ---- Architecture Visualization ---- */ | |
| .arch-container { | |
| background: linear-gradient(135deg, #0a0f1a 0%, #0d1520 100%); | |
| border: 1px solid #1a2a3e; | |
| border-radius: 12px; | |
| padding: 28px; | |
| margin-bottom: 24px; | |
| font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif; | |
| } | |
| .arch-flow { | |
| display: flex; | |
| flex-direction: column; | |
| gap: 12px; | |
| } | |
| .arch-layer { | |
| background: #0d1520; | |
| border: 2px solid #1e3048; | |
| border-radius: 10px; | |
| padding: 18px; | |
| cursor: pointer; | |
| transition: all 0.3s ease; | |
| position: relative; | |
| overflow: hidden; | |
| } | |
| .arch-layer::before { | |
| content: ''; | |
| position: absolute; | |
| top: 0; | |
| left: 0; | |
| right: 0; | |
| height: 2px; | |
| background: linear-gradient(90deg, #3b82f6, #8b5cf6, #ec4899, transparent); | |
| opacity: 0; | |
| transition: opacity 0.3s ease; | |
| } | |
| .arch-layer:hover { | |
| background: #111c2e; | |
| border-color: #2d5aa6; | |
| box-shadow: 0 8px 24px rgba(59, 130, 246, 0.15); | |
| } | |
| .arch-layer:hover::before { | |
| opacity: 1; | |
| } | |
| .arch-layer-title { | |
| font-size: 1.05em; | |
| font-weight: 700; | |
| color: #58a6ff; | |
| margin-bottom: 10px; | |
| display: flex; | |
| align-items: center; | |
| justify-content: center; | |
| gap: 8px; | |
| text-align: center; | |
| } | |
| .arch-layer-icon { | |
| display: none; | |
| } | |
| .arch-layer-desc { | |
| font-size: 0.9em; | |
| color: #8b9ab5; | |
| line-height: 1.5; | |
| max-height: 0; | |
| overflow: hidden; | |
| transition: max-height 0.3s ease; | |
| } | |
| .arch-layer.expanded .arch-layer-desc { | |
| max-height: 500px; | |
| } | |
| .arch-layer-content { | |
| display: grid; | |
| grid-template-columns: repeat(auto-fit, minmax(200px, 1fr)); | |
| gap: 12px; | |
| margin-top: 12px; | |
| max-height: 0; | |
| overflow: hidden; | |
| transition: max-height 0.3s ease; | |
| } | |
| .arch-layer.expanded .arch-layer-content { | |
| max-height: 600px; | |
| } | |
| .arch-spec-item { | |
| background: #162030; | |
| border: 1px solid #253649; | |
| border-radius: 8px; | |
| padding: 10px 12px; | |
| font-size: 0.82em; | |
| text-align: center; | |
| } | |
| .arch-spec-label { | |
| color: #4ade80; | |
| font-weight: 600; | |
| display: block; | |
| margin-bottom: 4px; | |
| } | |
| .arch-spec-value { | |
| color: #c5d0de; | |
| font-family: 'Monaco', 'Courier New', monospace; | |
| } | |
| .arch-arrow { | |
| text-align: center; | |
| color: #4ade80; | |
| font-size: 1.2em; | |
| padding: 4px 0; | |
| opacity: 0.6; | |
| } | |
| .arch-tasks { | |
| display: grid; | |
| grid-template-columns: repeat(auto-fit, minmax(180px, 1fr)); | |
| gap: 12px; | |
| margin-top: 12px; | |
| } | |
| .arch-task { | |
| background: linear-gradient(135deg, #1a2f4a 0%, #0f1f38 100%); | |
| border: 1px solid #253649; | |
| border-left: 3px solid #3b82f6; | |
| border-radius: 8px; | |
| padding: 12px; | |
| font-size: 0.85em; | |
| } | |
| .arch-task.domain { | |
| border-left-color: #3b82f6; | |
| } | |
| .arch-task.action { | |
| border-left-color: #8b5cf6; | |
| } | |
| .arch-task.slot { | |
| border-left-color: #ec4899; | |
| } | |
| .arch-task-name { | |
| font-weight: 700; | |
| color: #e2e8f0; | |
| margin-bottom: 6px; | |
| text-align: center; | |
| } | |
| .arch-task-detail { | |
| color: #8b9ab5; | |
| font-size: 0.78em; | |
| line-height: 1.4; | |
| text-align: center; | |
| } | |
| .arch-loss-box { | |
| background: #1a2a3a; | |
| border: 1px solid #1e3a4f; | |
| border-radius: 8px; | |
| padding: 14px; | |
| margin-top: 12px; | |
| font-family: 'Monaco', 'Courier New', monospace; | |
| font-size: 0.82em; | |
| color: #4ade80; | |
| overflow-x: auto; | |
| } | |
| .arch-decision { | |
| background: #1a3a2a; | |
| border-left: 3px solid #10b981; | |
| border-radius: 6px; | |
| padding: 10px; | |
| margin-top: 8px; | |
| font-size: 0.82em; | |
| color: #a7f3d0; | |
| } | |
| @media (max-width: 900px) { | |
| .arch-layer-content { | |
| grid-template-columns: 1fr; | |
| } | |
| .arch-tasks { | |
| grid-template-columns: 1fr; | |
| } | |
| } | |
| </style> | |
| <div class="arch-container"> | |
| <div class="arch-layer" onclick="this.classList.toggle('expanded')"> | |
| <div class="arch-layer-title"> | |
| <span class="arch-layer-icon">๐ค</span> | |
| 1. Tokenization & Embedding Layer | |
| </div> | |
| <div class="arch-layer-desc"> | |
| Input text is converted to token IDs and projected into a 256-dimensional embedding space. | |
| </div> | |
| <div class="arch-layer-content"> | |
| <div class="arch-spec-item"> | |
| <span class="arch-spec-label">Tokenizer</span> | |
| <span class="arch-spec-value">BPE, vocab=8,192</span> | |
| </div> | |
| <div class="arch-spec-item"> | |
| <span class="arch-spec-label">Max Length</span> | |
| <span class="arch-spec-value">96 tokens</span> | |
| </div> | |
| <div class="arch-spec-item"> | |
| <span class="arch-spec-label">Output Shape</span> | |
| <span class="arch-spec-value">(batch, 96, 256)</span> | |
| </div> | |
| <div class="arch-spec-item"> | |
| <span class="arch-spec-label">Embedding</span> | |
| <span class="arch-spec-value">8192 โ 256 dim</span> | |
| </div> | |
| </div> | |
| </div> | |
| <div class="arch-arrow">โ</div> | |
| <div class="arch-layer" onclick="this.classList.toggle('expanded')"> | |
| <div class="arch-layer-title"> | |
| <span class="arch-layer-icon"></span> | |
| 2. Transformer Backbone (8 Blocks) | |
| </div> | |
| <div class="arch-layer-desc"> | |
| Bidirectional attention layers with residual connections. Each block processes hidden states through grouped query attention and feed-forward networks. | |
| </div> | |
| <div class="arch-layer-content"> | |
| <div class="arch-spec-item"> | |
| <span class="arch-spec-label">Attention Type</span> | |
| <span class="arch-spec-value">Grouped Query (GQA)</span> | |
| </div> | |
| <div class="arch-spec-item"> | |
| <span class="arch-spec-label">Query Heads</span> | |
| <span class="arch-spec-value">8 heads</span> | |
| </div> | |
| <div class="arch-spec-item"> | |
| <span class="arch-spec-label">KV Heads</span> | |
| <span class="arch-spec-value">4 heads (2:1)</span> | |
| </div> | |
| <div class="arch-spec-item"> | |
| <span class="arch-spec-label">Head Dimension</span> | |
| <span class="arch-spec-value">32 (256รท8)</span> | |
| </div> | |
| <div class="arch-spec-item"> | |
| <span class="arch-spec-label">Position Embedding</span> | |
| <span class="arch-spec-value">RoPE</span> | |
| </div> | |
| <div class="arch-spec-item"> | |
| <span class="arch-spec-label">FFN Expansion</span> | |
| <span class="arch-spec-value">256 โ 672 โ 256</span> | |
| </div> | |
| <div class="arch-spec-item"> | |
| <span class="arch-spec-label">FFN Activation</span> | |
| <span class="arch-spec-value">SwiGLU</span> | |
| </div> | |
| <div class="arch-spec-item"> | |
| <span class="arch-spec-label">Normalization</span> | |
| <span class="arch-spec-value">RMSNorm</span> | |
| </div> | |
| <div class="arch-spec-item"> | |
| <span class="arch-spec-label">Causal Masking</span> | |
| <span class="arch-spec-value">OFF (bidirectional)</span> | |
| </div> | |
| <div class="arch-spec-item"> | |
| <span class="arch-spec-label">Dropout Rate</span> | |
| <span class="arch-spec-value">0.1</span> | |
| </div> | |
| </div> | |
| <div class="arch-decision"> | |
| Grouped Query Attention reduces KV cache 50% while maintaining quality | |
| </div> | |
| </div> | |
| <div class="arch-arrow">โ</div> | |
| <div class="arch-layer" onclick="this.classList.toggle('expanded')"> | |
| <div class="arch-layer-title"> | |
| <span class="arch-layer-icon"></span> | |
| 3. Multi-Task Prediction Heads (Parallel) | |
| </div> | |
| <div class="arch-layer-desc"> | |
| Three independent classification heads process the backbone output simultaneously for domain, action, and slot predictions. | |
| </div> | |
| <div class="arch-tasks"> | |
| <div class="arch-task domain"> | |
| <div class="arch-task-name">Domain Head</div> | |
| <div class="arch-task-detail"> | |
| <strong>Input:</strong> Last token (pooled)<br/> | |
| <strong>Arch:</strong> Linear(256) โ GELU โ Dropout โ Linear(10)<br/> | |
| <strong>Output:</strong> 10 classes | |
| </div> | |
| </div> | |
| <div class="arch-task action"> | |
| <div class="arch-task-name">Action Head</div> | |
| <div class="arch-task-detail"> | |
| <strong>Input:</strong> Last token (pooled)<br/> | |
| <strong>Arch:</strong> Linear(256) โ GELU โ Dropout โ Linear(33)<br/> | |
| <strong>Output:</strong> 33 classes | |
| </div> | |
| </div> | |
| <div class="arch-task slot"> | |
| <div class="arch-task-name">Slot Head</div> | |
| <div class="arch-task-detail"> | |
| <strong>Input:</strong> All tokens<br/> | |
| <strong>Arch:</strong> Linear(256) โ Linear(15 BIO)<br/> | |
| <strong>Output:</strong> 15 labels/token | |
| </div> | |
| </div> | |
| </div> | |
| </div> | |
| <div class="arch-arrow">โ</div> | |
| <div class="arch-layer" onclick="this.classList.toggle('expanded')"> | |
| <div class="arch-layer-title"> | |
| <span class="arch-layer-icon"></span> | |
| 4. Output & Post-Processing | |
| </div> | |
| <div class="arch-layer-desc"> | |
| Raw logits are converted to predictions. For slots, BIO tags are decoded into semantic spans. | |
| </div> | |
| <div class="arch-layer-content"> | |
| <div class="arch-spec-item"> | |
| <span class="arch-spec-label">Domain Output</span> | |
| <span class="arch-spec-value">10 classes</span> | |
| </div> | |
| <div class="arch-spec-item"> | |
| <span class="arch-spec-label">Action Output</span> | |
| <span class="arch-spec-value">33 classes</span> | |
| </div> | |
| <div class="arch-spec-item"> | |
| <span class="arch-spec-label">Slots Decoder</span> | |
| <span class="arch-spec-value">BIO โ Spans</span> | |
| </div> | |
| <div class="arch-spec-item"> | |
| <span class="arch-spec-label">Confidence</span> | |
| <span class="arch-spec-value">Softmax scores</span> | |
| </div> | |
| </div> | |
| </div> | |
| </div> | |
| ### Training Objective | |
| <div class="arch-container"> | |
| <div class="arch-layer expanded"> | |
| <div class="arch-layer-title"> | |
| <span class="arch-layer-icon"></span> | |
| Weighted Multi-Task Loss | |
| </div> | |
| <div class="arch-loss-box"> | |
| loss = 1.0 ร L_domain + 1.0 ร L_action + 1.5 ร L_slots | |
| Where: | |
| L_domain = CrossEntropy(domain_logits, domain_labels) | |
| L_action = CrossEntropy(action_logits, action_labels) | |
| L_slots = CrossEntropy(slot_logits, slot_labels) | |
| with ignore_index=-100 (padding) | |
| </div> | |
| <div class="arch-decision"> | |
| Slot weight (1.5ร) reflects higher complexity of sequence tagging vs. classification | |
| </div> | |
| </div> | |
| </div> | |
| ### Architecture Specifications | |
| | Component | Configuration | Details | | |
| |-----------|---|---| | |
| | **Backbone Type** | Transformer (GPT-style) | Bidirectional, non-causal attention | | |
| | **Vocabulary Size** | 8,192 | BPE tokenization | | |
| | **Embedding Dim** | 256 | Token + Rotary Position embeddings | | |
| | **Attention Heads** | 8 Query, 4 KV | Grouped Query Attention (GQA) for efficiency | | |
| | **Head Dimension** | 32 | per head_dim = embed_dim / num_heads | | |
| | **Transformer Blocks** | 8 Layers | Each with Attn + FFN + Residuals | | |
| | **Feed-Forward Hidden** | 672 | SwiGLU gate activation | | |
| | **Position Encoding** | RoPE | Rotary Position Embeddings (theta=10000) | | |
| | **Normalization** | RMSNorm | Pre-layer normalization | | |
| | **Max Sequence Length** | 96 tokens | Approximately 60-80 words | | |
| | **Dropout Rate** | 0.1 | Applied during training | | |
| | **Total Parameters** | 7,949,626 | All trainable | | |
| | **Parameter Breakdown** | Backbone: 7.80M, Task Heads: 146K | Efficient multitask design | | |
| ### Task Configuration | |
| | Task | Type | Classes | Architecture | | |
| |------|---|---|---| | |
| | **Domain Classification** | Sequence-level | 10 domains | Pooled โ Linear(256) โ GELU โ Linear(10) | | |
| | **Action Classification** | Sequence-level | 33 actions | Pooled โ Linear(256) โ GELU โ Linear(33) | | |
| | **Slot Tagging** | Token-level | 15 BIO labels | Per-token โ Linear(256) โ Linear(15) | | |
| --- | |
| ## Benchmark Results | |
| <div class="charts-row"> | |
| <div class="chart-container-sm"> | |
| <div class="chart-title">Runtime reliability</div> | |
| <div class="chart-subtitle">82-turn suite</div> | |
| <div class="bars-row"> | |
| <div class="bar-group"> | |
| <div class="bar-track"><div class="bar-fill" style="height:100%; background:linear-gradient(0deg,#3b82f6,#60a5fa);"><span class="bar-value">82</span></div></div> | |
| <div class="bar-label">Turns</div> | |
| </div> | |
| <div class="bar-group"> | |
| <div class="bar-track"><div class="bar-fill" style="height:81.7%; background:linear-gradient(0deg,#8b5cf6,#a78bfa);"><span class="bar-value">67</span></div></div> | |
| <div class="bar-label">Local</div> | |
| </div> | |
| <div class="bar-group"> | |
| <div class="bar-track"><div class="bar-fill" style="height:14.6%; background:linear-gradient(0deg,#ec4899,#f472b6);"><span class="bar-value">12</span></div></div> | |
| <div class="bar-label">Clarify</div> | |
| </div> | |
| <div class="bar-group"> | |
| <div class="bar-track"><div class="bar-fill" style="height:1%; background:linear-gradient(0deg,#9ca3af,#d1d5db);"><span class="bar-value">0</span></div></div> | |
| <div class="bar-label">Errors</div> | |
| </div> | |
| </div> | |
| <div class="bar-note">fair_benchmarks.json</div> | |
| </div> | |
| <div class="chart-container-sm"> | |
| <div class="chart-title">Predict latency</div> | |
| <div class="chart-subtitle">CUDA ยท batch=1 ยท lower is better</div> | |
| <div class="bars-row"> | |
| <div class="bar-group"> | |
| <div class="bar-track"><div class="bar-fill" style="height:63.3%; background:linear-gradient(0deg,#3b82f6,#60a5fa);"><span class="bar-value">25.3ms</span></div></div> | |
| <div class="bar-label">Pยทmean</div> | |
| </div> | |
| <div class="bar-group"> | |
| <div class="bar-track"><div class="bar-fill" style="height:86.5%; background:linear-gradient(0deg,#8b5cf6,#a78bfa);"><span class="bar-value">34.6ms</span></div></div> | |
| <div class="bar-label">Pยทp95</div> | |
| </div> | |
| <div class="bar-group"> | |
| <div class="bar-track"><div class="bar-fill" style="height:88.4%; background:linear-gradient(0deg,#ec4899,#f472b6);"><span class="bar-value">35.4ms</span></div></div> | |
| <div class="bar-label">Fwdยทmean</div> | |
| </div> | |
| <div class="bar-group"> | |
| <div class="bar-track"><div class="bar-fill" style="height:91.8%; background:linear-gradient(0deg,#a78bfa,#c4b5fd);"><span class="bar-value">36.7ms</span></div></div> | |
| <div class="bar-label">Fwdยทp95</div> | |
| </div> | |
| </div> | |
| <div class="bar-note">janus_model_report.json</div> | |
| </div> | |
| </div> | |
| <div class="chart-container"> | |
| <div class="chart-title">OOD rejection quality</div> | |
| <div class="chart-subtitle">Schema-agnostic ยท hover values</div> | |
| <input type="radio" name="ood-filter" id="ood-both" checked /> | |
| <input type="radio" name="ood-filter" id="ood-banking77" /> | |
| <input type="radio" name="ood-filter" id="ood-clinc" /> | |
| <div class="ood-filter-row filter-row"> | |
| <label class="filter-btn" for="ood-both">Both</label> | |
| <label class="filter-btn" for="ood-banking77">BANKING77</label> | |
| <label class="filter-btn" for="ood-clinc">CLINC</label> | |
| </div> | |
| <div class="chart-bars"> | |
| <div class="bars-row bars-row-wide"> | |
| <div class="bar-group" data-dataset="banking77"> | |
| <div class="bar-track"><div class="bar-fill" style="height:87.8%; background:linear-gradient(0deg,#3b82f6,#60a5fa);"><span class="bar-value">87.8%</span></div></div> | |
| <div class="bar-label">B77 F1</div> | |
| </div> | |
| <div class="bar-group" data-dataset="banking77"> | |
| <div class="bar-track"><div class="bar-fill" style="height:100%; background:linear-gradient(0deg,#ec4899,#f472b6);"><span class="bar-value">100%</span></div></div> | |
| <div class="bar-label">B77 Prec</div> | |
| </div> | |
| <div class="bar-group" data-dataset="banking77"> | |
| <div class="bar-track"><div class="bar-fill" style="height:78.25%; background:linear-gradient(0deg,#8b5cf6,#a78bfa);"><span class="bar-value">78.3%</span></div></div> | |
| <div class="bar-label">B77 Rec</div> | |
| </div> | |
| <div class="bar-group" data-dataset="clinc_oos"> | |
| <div class="bar-track"><div class="bar-fill" style="height:79.23%; background:linear-gradient(0deg,#3b82f6,#60a5fa);"><span class="bar-value">79.2%</span></div></div> | |
| <div class="bar-label">CL F1</div> | |
| </div> | |
| <div class="bar-group" data-dataset="clinc_oos"> | |
| <div class="bar-track"><div class="bar-fill" style="height:100%; background:linear-gradient(0deg,#ec4899,#f472b6);"><span class="bar-value">100%</span></div></div> | |
| <div class="bar-label">CL Prec</div> | |
| </div> | |
| <div class="bar-group" data-dataset="clinc_oos"> | |
| <div class="bar-track"><div class="bar-fill" style="height:65.6%; background:linear-gradient(0deg,#8b5cf6,#a78bfa);"><span class="bar-value">65.6%</span></div></div> | |
| <div class="bar-label">CL Rec</div> | |
| </div> | |
| </div> | |
| </div> | |
| <div class="bar-note">fair_benchmarks.json</div> | |
| </div> | |
| </div> | |
| ### Comprehensive Benchmark Summary | |
| <div class="chart-container"> | |
| <div class="chart-title">Full Benchmark Evidence</div> | |
| <div class="chart-subtitle">All values from real holdout evaluations โ no synthetic or inflated numbers</div> | |
| <table class="comparison-table"> | |
| <thead> | |
| <tr> | |
| <th>Metric</th> | |
| <th>Detail</th> | |
| <th><span class="tag-v2">Jane v2</span></th> | |
| <th><span class="tag-janus">Janus</span></th> | |
| </tr> | |
| </thead> | |
| <tbody> | |
| <tr> | |
| <td><strong>Speed (mean latency)</strong></td> | |
| <td>CUDA, batch=1</td> | |
| <td>31.60 ms</td> | |
| <td><strong>25.31 ms</strong></td> | |
| </tr> | |
| <tr> | |
| <td><strong>Throughput</strong></td> | |
| <td>CUDA, single GPU</td> | |
| <td>32 pred/sec</td> | |
| <td>Stable across 82 turns, 0 errors</td> | |
| </tr> | |
| <tr> | |
| <td><strong>OOD F1</strong></td> | |
| <td>BANKING77</td> | |
| <td><strong>94.31%</strong></td> | |
| <td>87.80%</td> | |
| </tr> | |
| <tr> | |
| <td><strong>OOD F1</strong></td> | |
| <td>CLINC OOS</td> | |
| <td><strong>89.16%</strong></td> | |
| <td>79.23%</td> | |
| </tr> | |
| <tr> | |
| <td><strong>OOD Precision</strong></td> | |
| <td>BANKING77</td> | |
| <td>99.35%</td> | |
| <td><strong>100.00%</strong></td> | |
| </tr> | |
| <tr> | |
| <td><strong>OOD Precision</strong></td> | |
| <td>CLINC OOS</td> | |
| <td>99.14%</td> | |
| <td><strong>100.00%</strong></td> | |
| </tr> | |
| <tr> | |
| <td><strong>OOD Recall</strong></td> | |
| <td>BANKING77</td> | |
| <td><strong>89.75%</strong></td> | |
| <td>78.25%</td> | |
| </tr> | |
| <tr> | |
| <td><strong>OOD Recall</strong></td> | |
| <td>CLINC OOS</td> | |
| <td><strong>81.00%</strong></td> | |
| <td>65.60%</td> | |
| </tr> | |
| <tr> | |
| <td><strong>Validation Accuracy</strong></td> | |
| <td>Domain (best epoch)</td> | |
| <td>โ</td> | |
| <td><strong>99.83%</strong></td> | |
| </tr> | |
| <tr> | |
| <td><strong>Validation Accuracy</strong></td> | |
| <td>Action (best epoch)</td> | |
| <td>โ</td> | |
| <td><strong>99.87%</strong></td> | |
| </tr> | |
| <tr> | |
| <td><strong>Validation Accuracy</strong></td> | |
| <td>Domain+Action pair (best epoch)</td> | |
| <td>โ</td> | |
| <td><strong>99.83%</strong></td> | |
| </tr> | |
| <tr> | |
| <td><strong>Slot Extraction F1</strong></td> | |
| <td>All 15 slot types</td> | |
| <td>โ</td> | |
| <td><strong>1.000 (100%)</strong></td> | |
| </tr> | |
| <tr> | |
| <td><strong>Training Loss</strong></td> | |
| <td>Epoch 1 โ 4</td> | |
| <td>โ</td> | |
| <td>0.060 โ 0.020 โ 0.002 โ 0.001</td> | |
| </tr> | |
| <tr> | |
| <td><strong>Validation Loss</strong></td> | |
| <td>Epoch 1 โ 3</td> | |
| <td>โ</td> | |
| <td>0.0153 โ 0.0116 โ 0.0115 (stable)</td> | |
| </tr> | |
| <tr> | |
| <td><strong>Runtime Reliability</strong></td> | |
| <td>82-turn conversation test</td> | |
| <td>โ</td> | |
| <td><strong>0 errors, 0 crashes</strong></td> | |
| </tr> | |
| <tr> | |
| <td><strong>Domain Confusion</strong></td> | |
| <td>10 domains</td> | |
| <td>โ</td> | |
| <td>99%+ per-domain, minimal cross-confusion</td> | |
| </tr> | |
| <tr> | |
| <td><strong>Action Confusion</strong></td> | |
| <td>33 actions</td> | |
| <td>โ</td> | |
| <td>Perfect diagonal, no action commonly confused</td> | |
| </tr> | |
| </tbody> | |
| </table> | |
| </div> | |
| --- | |
| ## Live Output Shapes (click to expand) | |
| <details> | |
| <summary><strong>Command output</strong></summary> | |
| <div> | |
| ```json | |
| { | |
| "type": "command", | |
| "domain": "apps", | |
| "action": "launch", | |
| "slots": { | |
| "APP_NAME": { | |
| "text": "chrome", | |
| "start": 5, | |
| "end": 11, | |
| "confidence": 0.999 | |
| } | |
| }, | |
| "confidence": 0.97, | |
| "route": "local" | |
| } | |
| ``` | |
| </div> | |
| </details> | |
| <details> | |
| <summary><strong>Clarification output</strong></summary> | |
| <div> | |
| ```json | |
| { | |
| "type": "clarify", | |
| "question": "What value should I set it to?", | |
| "debug": { | |
| "domain": "volume", | |
| "action": "set", | |
| "reason": "missing_VALUE" | |
| } | |
| } | |
| ``` | |
| </div> | |
| </details> | |
| <details> | |
| <summary><strong>Label schema</strong></summary> | |
| <div> | |
| - **Domains (10):** volume, brightness, media, apps, browser, productivity, screen, window, system, conversation | |
| - **Actions (33):** up, down, set, mute, unmute, play, pause, next, previous, launch, close, switch, search, set_reminder, screenshot, read, explain, undo, quit, chat, minimize, maximize, restore, focus, copy, paste, cut, lock, sleep, wifi_on, wifi_off, bluetooth_on, bluetooth_off | |
| - **Slot labels (BIO, 15):** VALUE, APP_NAME, QUERY, DURATION, TIME, WINDOW_NAME, TEXT | |
| </div> | |
| </details> | |
| --- | |
| ## Visual Benchmark Evidence | |
| <p align="center"> | |
| <img src="reports/loss_per_epoch.png" alt="Train and validation loss" width="860" /> | |
| </p> | |
| <p align="center"> | |
| <img src="reports/train_loss_smoothed.png" alt="Smoothed train loss" width="860" /> | |
| </p> | |
| <p align="center"> | |
| <img src="reports/val_slot_f1.png" alt="Validation slot F1" width="860" /> | |
| </p> | |
| ### Confusion Matrix โ Interactive Breakdown | |
| <!-- FIX: radio inputs must be SIBLINGS of both .confusion-toggle AND .cm-panel divs | |
| so the CSS sibling combinator (~) can reach them all from the same parent level --> | |
| <div class="chart-container"> | |
| <div class="chart-title">Per-Class True vs Predicted</div> | |
| <div class="chart-subtitle">Single stacked bar per head โ segment width = sample ratio. Hover any segment for details.</div> | |
| <input type="radio" name="cm-toggle" id="cm-domain" checked /> | |
| <input type="radio" name="cm-toggle" id="cm-action" /> | |
| <div class="confusion-toggle"> | |
| <label class="filter-btn" for="cm-domain">Domains (10)</label> | |
| <label class="filter-btn" for="cm-action">Actions (33)</label> | |
| </div> | |
| <div class="cm-panel" data-cm="domain"> | |
| <div class="stacked-bar-label">Domain Sample Distribution — 3,110 total samples — hover each segment</div> | |
| <div class="cm-stacked-bar"> | |
| <div class="cm-seg" style="flex:430; background:#3b82f6; border-radius:10px 0 0 10px;"><div class="cm-tip"><span class="tip-name">volume</span><br/><span class="tip-samples">430 samples (13.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:250; background:#f59e0b;"><div class="cm-tip"><span class="tip-name">brightness</span><br/><span class="tip-samples">250 samples (8.0%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:340; background:#10b981;"><div class="cm-tip"><span class="tip-name">media</span><br/><span class="tip-samples">340 samples (10.9%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:250; background:#ef4444;"><div class="cm-tip"><span class="tip-name">apps</span><br/><span class="tip-samples">250 samples (8.0%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:120; background:#8b5cf6;"><div class="cm-tip"><span class="tip-name">browser</span><br/><span class="tip-samples">120 samples (3.9%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:340; background:#ec4899;"><div class="cm-tip"><span class="tip-name">productivity</span><br/><span class="tip-samples">340 samples (10.9%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:120; background:#14b8a6;"><div class="cm-tip"><span class="tip-name">screen</span><br/><span class="tip-samples">120 samples (3.9%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:340; background:#f97316;"><div class="cm-tip"><span class="tip-name">window</span><br/><span class="tip-samples">340 samples (10.9%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:580; background:#6366f1;"><div class="cm-tip"><span class="tip-name">system</span><br/><span class="tip-samples">580 samples (18.6%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:340; background:#06b6d4; border-radius:0 10px 10px 0;"><div class="cm-tip"><span class="tip-name">conversation</span><br/><span class="tip-samples">340 samples (10.9%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| </div> | |
| <div class="cm-legend"> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#3b82f6;"></span>volume</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#f59e0b;"></span>brightness</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#10b981;"></span>media</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#ef4444;"></span>apps</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#8b5cf6;"></span>browser</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#ec4899;"></span>productivity</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#14b8a6;"></span>screen</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#f97316;"></span>window</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#6366f1;"></span>system</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#06b6d4;"></span>conversation</div> | |
| </div> | |
| <div class="bar-note">Source: validation set confusion matrix โ segment widths proportional to sample count</div> | |
| </div> | |
| <div class="cm-panel" data-cm="action"> | |
| <div class="stacked-bar-label">Action Sample Distribution — 3,205 total samples — hover each segment</div> | |
| <div class="cm-stacked-bar"> | |
| <div class="cm-seg" style="flex:170; background:#3b82f6; border-radius:10px 0 0 10px;"><div class="cm-tip"><span class="tip-name">up</span><br/><span class="tip-samples">170 samples (5.3%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:165; background:#2563eb;"><div class="cm-tip"><span class="tip-name">down</span><br/><span class="tip-samples">165 samples (5.1%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:170; background:#1d4ed8;"><div class="cm-tip"><span class="tip-name">set</span><br/><span class="tip-samples">170 samples (5.3%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:90; background:#7c3aed;"><div class="cm-tip"><span class="tip-name">mute</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:90; background:#6d28d9;"><div class="cm-tip"><span class="tip-name">unmute</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:90; background:#10b981;"><div class="cm-tip"><span class="tip-name">play</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:90; background:#059669;"><div class="cm-tip"><span class="tip-name">pause</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:90; background:#047857;"><div class="cm-tip"><span class="tip-name">next</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:90; background:#065f46;"><div class="cm-tip"><span class="tip-name">previous</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:90; background:#ef4444;"><div class="cm-tip"><span class="tip-name">launch</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:90; background:#dc2626;"><div class="cm-tip"><span class="tip-name">close</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:90; background:#b91c1c;"><div class="cm-tip"><span class="tip-name">switch</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:90; background:#f59e0b;"><div class="cm-tip"><span class="tip-name">search</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:90; background:#d97706;"><div class="cm-tip"><span class="tip-name">set_reminder</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:90; background:#ec4899;"><div class="cm-tip"><span class="tip-name">screenshot</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:90; background:#db2777;"><div class="cm-tip"><span class="tip-name">read</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:90; background:#be185d;"><div class="cm-tip"><span class="tip-name">explain</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:90; background:#14b8a6;"><div class="cm-tip"><span class="tip-name">undo</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:90; background:#0d9488;"><div class="cm-tip"><span class="tip-name">quit</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:90; background:#0f766e;"><div class="cm-tip"><span class="tip-name">chat</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:90; background:#f97316;"><div class="cm-tip"><span class="tip-name">minimize</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:90; background:#ea580c;"><div class="cm-tip"><span class="tip-name">maximize</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:90; background:#c2410c;"><div class="cm-tip"><span class="tip-name">restore</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:90; background:#6366f1;"><div class="cm-tip"><span class="tip-name">focus</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:90; background:#4f46e5;"><div class="cm-tip"><span class="tip-name">copy</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:90; background:#4338ca;"><div class="cm-tip"><span class="tip-name">paste</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:90; background:#06b6d4;"><div class="cm-tip"><span class="tip-name">cut</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:90; background:#0891b2;"><div class="cm-tip"><span class="tip-name">lock</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:90; background:#0e7490;"><div class="cm-tip"><span class="tip-name">sleep</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:90; background:#84cc16;"><div class="cm-tip"><span class="tip-name">wifi_on</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:90; background:#65a30d;"><div class="cm-tip"><span class="tip-name">wifi_off</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:90; background:#a855f7;"><div class="cm-tip"><span class="tip-name">bluetooth_on</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| <div class="cm-seg" style="flex:90; background:#9333ea; border-radius:0 10px 10px 0;"><div class="cm-tip"><span class="tip-name">bluetooth_off</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div> | |
| </div> | |
| <div class="cm-legend"> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#3b82f6;"></span>up</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#2563eb;"></span>down</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#1d4ed8;"></span>set</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#7c3aed;"></span>mute</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#6d28d9;"></span>unmute</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#10b981;"></span>play</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#059669;"></span>pause</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#047857;"></span>next</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#065f46;"></span>previous</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#ef4444;"></span>launch</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#dc2626;"></span>close</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#b91c1c;"></span>switch</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#f59e0b;"></span>search</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#d97706;"></span>set_reminder</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#ec4899;"></span>screenshot</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#db2777;"></span>read</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#be185d;"></span>explain</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#14b8a6;"></span>undo</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#0d9488;"></span>quit</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#0f766e;"></span>chat</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#f97316;"></span>minimize</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#ea580c;"></span>maximize</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#c2410c;"></span>restore</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#6366f1;"></span>focus</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#4f46e5;"></span>copy</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#4338ca;"></span>paste</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#06b6d4;"></span>cut</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#0891b2;"></span>lock</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#0e7490;"></span>sleep</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#84cc16;"></span>wifi_on</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#65a30d;"></span>wifi_off</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#a855f7;"></span>bluetooth_on</div> | |
| <div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#9333ea;"></span>bluetooth_off</div> | |
| </div> | |
| <div class="bar-note">Source: validation set confusion matrix โ segment widths proportional to sample count</div> | |
| </div> | |
| </div> | |
| <details> | |
| <summary><strong>View original confusion matrix images</strong></summary> | |
| <div> | |
| <p align="center"> | |
| <img src="reports/confusion_domain_val.png" alt="Domain confusion matrix" width="860" /> | |
| </p> | |
| <p align="center"> | |
| <img src="reports/confusion_action_val.png" alt="Action confusion matrix" width="860" /> | |
| </p> | |
| </div> | |
| </details> | |
| <details> | |
| <summary><strong>Additional diagnostics</strong></summary> | |
| <div> | |
| <p align="center"> | |
| <img src="reports/lr_schedule.png" alt="Learning rate schedule" width="860" /> | |
| </p> | |
| <p align="center"> | |
| <img src="reports/epoch_time.png" alt="Epoch time profile" width="860" /> | |
| </p> | |
| <p align="center"> | |
| <img src="reports/train_loss_raw.png" alt="Raw training loss" width="860" /> | |
| </p> | |
| </div> | |
| </details> | |
| --- | |
| ## Upload-Ready Layout | |
| ```text | |
| . | |
| |- README.md | |
| |- .gitattributes | |
| |- LICENSE | |
| |- requirements.txt | |
| |- assets/ | |
| | |- jane-janus-glitch.webp | |
| |- janegpt_v2_janus/ | |
| | |- __init__.py | |
| | |- architecture.py | |
| | |- dataset.py | |
| | |- inference.py | |
| | |- labels.py | |
| | |- multitask.py | |
| |- runtime/ | |
| | |- jane_nlu_runtime.py | |
| |- examples/ | |
| | |- demo_inference.py | |
| | |- demo_runtime.py | |
| | |- demo_runtime_suite.py | |
| |- weights/ | |
| | |- janegpt_v2_janus.pt | |
| | |- tokenizer.json | |
| |- reports/ | |
| | |- fair_benchmarks.json | |
| | |- fair_benchmarks.md | |
| | |- janus_model_report.json | |
| | |- janus_model_report.md | |
| | |- public_benchmarks.json | |
| | |- *.png benchmark visuals | |
| ``` | |
| --- | |
| ## Limitations | |
| - English-focused command language. | |
| - Command NLU model, not an open-domain generative chatbot. | |
| - MASSIVE and SNIPS mapped-intent accuracy is excluded from headline claims because mapping coverage is partial. | |
| --- | |
| ## Use Cases | |
| - Virtual assistant command routing | |
| - Smart home intent classification | |
| - Voice command understanding | |
| - Chatbot intent detection | |
| - Edge device deployment (small enough for embedded systems) | |
| --- | |
| ## Part of the JANE Project | |
| **JANE** โ a fully | |
| offline, privacy-first AI voice assistant. | |
| ๐ [JANE AI Assistant on GitHub](https://github.com/Ravindu-S/JANE-AI-Assistant) | |
| ๐ [JaneGPT-v2 on GitHub](https://github.com/Ravindu-S/JaneGPT-v2) | |
| --- | |
| ## Created By | |
| **Ravindu Senanayake** | |
| Built from scratch โ architecture, tokenizer, and training | |
| pipeline designed and implemented by the author. | |
| [](https://github.com/Ravindu-S) | |
| --- | |
| ## License | |
| Apache-2.0 (see [LICENSE](LICENSE)). |