JaneGPT-v2-Janus / README.md
RavinduSen's picture
Update README.md
ec66b20 verified
|
Raw History Blame Contribute Delete
58.6 kB
---
language:
- en
license: apache-2.0
library_name: pytorch
tags:
- safetensors
- janegpt
- janus
- react
- interactive
- vite
- intent-classification
- transformer
- virtual-assistant
- nlp
- voice-assistant
- offline-ai
- edge-deployment
- nlu
- transformers
- slot-filling
- multitask-learning
- assistant-runtime
demo: https://janus-web-demo.vercel.app/
pipeline_tag: text-classification
model-index:
- name: JaneGPT-v2-Janus
results:
- task:
type: text-classification
dataset:
name: BANKING77
type: mteb/banking77
metrics:
- type: precision
value: 1.0
name: OOD Precision
- type: f1
value: 0.878
name: OOD F1
- type: recall
value: 0.7825
name: OOD Recall
- task:
type: text-classification
dataset:
name: CLINC OOS
type: clinc/clinc_oos
metrics:
- type: precision
value: 1.0
name: OOD Precision
- type: f1
value: 0.7923
name: OOD F1
- type: recall
value: 0.656
name: OOD Recall
- task:
type: text-classification
metrics:
- type: accuracy
value: 0.9983
name: Validation Domain Accuracy
- type: accuracy
value: 0.9987
name: Validation Action Accuracy
- type: f1
value: 1.0
name: Slot Extraction F1
---
# JaneGPT v2 Janus - Intent Classification Model
---
![Total Downloads](https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fmodels%2FRavinduSen%2FJaneGPT-v2-Janus%3Fexpand%3DdownloadsAllTime&query=downloadsAllTime&label=Total%20Downloads&color=blue)
![Monthly Downloads](https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fmodels%2FRavinduSen%2FJaneGPT-v2-Janus&query=downloads&label=Downloads%2FMonth&color=green)
---
<p align="center">
<a href="https://janus-web-demo.vercel.app/">
<img src="https://img.shields.io/badge/Launch-Temple_of_Janus-6366f1?style=for-the-badge&logo=vercel" alt="Experience Janus">
</a>
</p>
<style>
/* ---- Hero section ---- */
.hero-banner { text-align: center; margin-bottom: 12px; }
.hero-banner img { max-width: 100%; border-radius: 12px; }
.subtitle {
text-align: center;
font-size: 1.15em;
color: #6b7a90;
margin-bottom: 28px;
}
/* ---- Stat cards (mini key-value row) ---- */
.stat-row {
display: flex;
flex-wrap: wrap;
gap: 14px;
justify-content: center;
margin-bottom: 32px;
}
.stat-card {
background: #0f1923;
border: 1px solid #1e3048;
border-radius: 10px;
padding: 14px 22px;
min-width: 150px;
text-align: center;
cursor: default;
}
.stat-card .stat-value {
font-size: 1.65em;
font-weight: 700;
color: #58a6ff;
line-height: 1.15;
}
.stat-card .stat-label {
font-size: 0.82em;
color: #8b9ab5;
margin-top: 4px;
text-transform: uppercase;
letter-spacing: 0.5px;
}
/* ---- Section headings ---- */
.section-heading {
font-size: 1.35em;
font-weight: 700;
color: #e6edf3;
border-bottom: 1px solid #1e3048;
padding-bottom: 6px;
margin-top: 36px;
margin-bottom: 16px;
}
/* ---- Horizontal bar chart (interactive) ---- */
.chart-container {
background: #0d1520;
border: 1px solid #1a2a3e;
border-radius: 12px;
padding: 24px 28px 60px 28px;
margin-bottom: 24px;
display: flex;
flex-direction: column;
}
.chart-title {
font-size: 1.1em;
font-weight: 700;
color: #e2e8f0;
margin-bottom: 4px;
}
.chart-subtitle {
font-size: 0.88em;
color: #7a8ba3;
margin-bottom: 14px;
line-height: 1.35;
position: relative;
z-index: 3;
}
.bar-group {
margin-bottom: 18px;
}
.bar-label {
font-size: 0.92em;
color: #c5d0de;
margin-bottom: 5px;
font-weight: 600;
}
/* NEW โ€” replace with this */
.bar-group {
display: flex;
flex-direction: column;
align-items: center;
flex: 1;
}
.bar-label {
font-size: 0.78em;
color: #8ea2b8;
margin-top: 6px;
text-align: center;
font-weight: 600;
}
.bar-track {
background: #162030;
border-radius: 8px 8px 0 0;
width: 100%;
height: 120px;
position: relative;
display: flex;
align-items: flex-end;
overflow: hidden;
}
.bar-fill {
width: 100%;
border-radius: 6px 6px 0 0;
transition: height 0.8s cubic-bezier(.22,.61,.36,1), box-shadow 0.3s ease;
position: relative;
min-height: 24px;
display: flex;
align-items: flex-start;
justify-content: center;
}
.bar-fill:hover {
box-shadow: 0 0 20px rgba(59, 130, 246, 0.3), inset 0 0 0 999px rgba(255,255,255,0.06);
}
.bar-value {
position: absolute;
top: 7px;
left: 50%;
transform: translateX(-50%);
font-size: 0.78em;
font-weight: 700;
color: #000000;
white-space: nowrap;
line-height: 1;
text-shadow: 0 1px 2px rgba(0,0,0,0.45);
z-index: 2;
pointer-events: none;
}
.bar-note {
font-size: 0.78em;
color: #5c6d82;
margin-top: 6px;
font-style: italic;
}
/* NEW โ€” add this */
.charts-row {
display: flex;
gap: 16px;
align-items: stretch;
margin-bottom: 24px;
}
.chart-container-sm {
background: #0d1520;
border: 1px solid #1a2a3e;
border-radius: 12px;
padding: 18px 16px;
flex: 1;
display: flex;
flex-direction: column;
}
.bars-row {
display: flex;
gap: 8px;
align-items: flex-end;
min-height: 150px;
height: 150px;
border-bottom: 1px solid #1e3048;
margin-bottom: 8px;
margin-top: 10px;
}
.bars-row-wide {
gap: 16px;
}
.ood-filter-row {
margin-bottom: 14px;
position: relative;
z-index: 4;
}
.chart-bars {
margin-top: 8px;
position: relative;
z-index: 1;
}
@media (max-width: 900px) {
.charts-row {
flex-direction: column;
align-items: stretch;
}
}
/* ---- Filter buttons (CSS-only via radio hack) ---- */
.filter-row {
display: flex;
flex-wrap: wrap;
gap: 8px;
margin-bottom: 18px;
}
.filter-row input[type="radio"] {
display: none;
}
input[name="ood-filter"],
input[name="cm-toggle"] {
position: absolute;
opacity: 0;
width: 0;
height: 0;
pointer-events: none;
}
.filter-btn {
display: inline-block;
padding: 5px 14px;
font-size: 0.82em;
font-weight: 600;
border-radius: 20px;
border: 1px solid #253649;
background: #111c2a;
color: #8ea2b8;
cursor: pointer;
transition: background 0.2s, color 0.2s, border-color 0.2s;
user-select: none;
}
.filter-btn:hover {
background: #1a2d42;
color: #c5d8e8;
}
#ood-both:checked ~ .ood-filter-row label[for="ood-both"],
#ood-banking77:checked ~ .ood-filter-row label[for="ood-banking77"],
#ood-clinc:checked ~ .ood-filter-row label[for="ood-clinc"] {
background: #1f6feb;
color: #fff;
border-color: #1f6feb;
}
/* Bar groups visibility controlled by radio state */
.bar-group[data-dataset] { display: flex; }
#ood-banking77:checked ~ .chart-bars .bar-group[data-dataset="clinc_oos"] { display: none; }
#ood-clinc:checked ~ .chart-bars .bar-group[data-dataset="banking77"] { display: none; }
#ood-both:checked ~ .chart-bars .bar-group[data-dataset] { display: flex; }
/* ---- Comparison table ---- */
.comparison-table {
width: 100%;
border-collapse: collapse;
font-size: 0.9em;
margin-bottom: 16px;
}
.comparison-table th {
background: #111c2a;
color: #8ea2b8;
font-weight: 700;
text-transform: uppercase;
font-size: 0.78em;
letter-spacing: 0.6px;
padding: 10px 14px;
border-bottom: 2px solid #1e3048;
text-align: left;
}
.comparison-table td {
padding: 9px 14px;
border-bottom: 1px solid #152232;
color: #c5d0de;
}
.comparison-table tr:hover td {
background: #111e2e;
}
.tag-janus {
display: inline-block;
background: #1a3a2a;
color: #4ade80;
border-radius: 6px;
padding: 1px 8px;
font-size: 0.85em;
font-weight: 600;
}
.tag-v2 {
display: inline-block;
background: #1a2a3a;
color: #60a5fa;
border-radius: 6px;
padding: 1px 8px;
font-size: 0.85em;
font-weight: 600;
}
/* ---- Segmented stacked bar (single bar per panel) ---- */
.stacked-bar-wrapper {
margin-bottom: 12px;
position: relative;
}
.stacked-bar-label {
font-size: 0.85em;
font-weight: 600;
color: #8b9ab5;
margin-bottom: 6px;
}
.stacked-bar {
display: flex;
height: 38px;
border-radius: 8px;
overflow: hidden;
background: #162030;
cursor: default;
}
.stacked-seg {
height: 100%;
transition: filter 0.2s ease, flex 0.3s ease;
position: relative;
}
.stacked-seg:hover {
filter: brightness(1.4) saturate(1.3);
outline: 2px solid #fff;
outline-offset: -2px;
z-index: 5;
}
/* Tooltip for stacked bar segments */
.stacked-tooltip {
display: none;
position: absolute;
left: 50%;
transform: translateX(-50%);
top: calc(100% + 6px);
background: #0f1923;
border: 1px solid #253649;
border-radius: 8px;
padding: 8px 14px;
font-size: 0.82em;
color: #e2e8f0;
z-index: 20;
white-space: nowrap;
box-shadow: 0 6px 20px rgba(0,0,0,0.5);
pointer-events: none;
}
.stacked-tooltip::before {
content: '';
position: absolute;
top: -6px;
left: 50%;
transform: translateX(-50%);
border-left: 6px solid transparent;
border-right: 6px solid transparent;
border-bottom: 6px solid #253649;
}
.stacked-seg:hover .stacked-tooltip {
display: block;
}
.stacked-tooltip .st-name { font-weight: 700; color: #4ade80; }
.stacked-tooltip .st-val { color: #8ea2b8; }
/* ---- Confusion matrix panel toggle ---- */
#cm-domain:checked ~ .confusion-toggle label[for="cm-domain"],
#cm-action:checked ~ .confusion-toggle label[for="cm-action"] {
background: #1f6feb;
color: #fff;
border-color: #1f6feb;
}
.confusion-toggle {
display: flex;
gap: 8px;
margin-bottom: 18px;
}
.cm-panel[data-cm="domain"] { display: block; }
.cm-panel[data-cm="action"] { display: none; }
#cm-action:checked ~ .cm-panel[data-cm="domain"] { display: none; }
#cm-action:checked ~ .cm-panel[data-cm="action"] { display: block; }
#cm-domain:checked ~ .cm-panel[data-cm="domain"] { display: block; }
#cm-domain:checked ~ .cm-panel[data-cm="action"] { display: none; }
/* ---- Confusion stacked bar ---- */
.cm-stacked-bar {
display: flex;
height: 44px;
border-radius: 10px;
overflow: visible;
background: #162030;
position: relative;
margin-bottom: 8px;
}
.cm-seg {
height: 100%;
position: relative;
transition: filter 0.2s ease;
cursor: default;
/* thin separator between segments */
border-right: 1px solid rgba(0,0,0,0.35);
}
.cm-seg:last-child { border-right: none; }
.cm-seg:hover {
filter: brightness(1.45) saturate(1.3);
z-index: 10;
}
/* Tooltip on hover */
.cm-seg .cm-tip {
display: none;
position: absolute;
bottom: calc(100% + 8px);
left: 50%;
transform: translateX(-50%);
background: #0f1923;
border: 1px solid #253649;
border-radius: 8px;
padding: 10px 16px;
font-size: 0.82em;
color: #e2e8f0;
z-index: 30;
white-space: nowrap;
box-shadow: 0 6px 24px rgba(0,0,0,0.55);
pointer-events: none;
}
.cm-seg .cm-tip::after {
content: '';
position: absolute;
bottom: -6px;
left: 50%;
transform: translateX(-50%);
border-left: 6px solid transparent;
border-right: 6px solid transparent;
border-top: 6px solid #253649;
}
.cm-seg:hover .cm-tip { display: block; }
.cm-tip .tip-name { font-weight: 700; color: #58a6ff; }
.cm-tip .tip-samples { color: #4ade80; }
.cm-tip .tip-acc { color: #fbbf24; }
.cm-tip .tip-miss { color: #f87171; }
.cm-legend {
display: flex;
flex-wrap: wrap;
gap: 10px 18px;
margin-top: 10px;
margin-bottom: 6px;
}
.cm-legend-item {
display: flex;
align-items: center;
gap: 6px;
font-size: 0.78em;
color: #8b9ab5;
}
.cm-legend-swatch {
width: 12px;
height: 12px;
border-radius: 3px;
display: inline-block;
}
/* ---- Code blocks ---- */
details {
background: #0d1520;
border: 1px solid #1a2a3e;
border-radius: 8px;
margin-bottom: 10px;
padding: 0;
}
details summary {
cursor: pointer;
padding: 12px 18px;
font-weight: 600;
color: #c5d0de;
list-style: none;
}
details summary::-webkit-details-marker { display: none; }
details summary::before { content: "โ–ธ "; color: #58a6ff; }
details[open] summary::before { content: "โ–พ "; }
details > div {
padding: 0 18px 16px;
}
</style>
<!-- ===== Hero ===== -->
<div class="hero-banner">
<img src="assets/jane-janus-glitch.webp" alt="Jane Janus animated hero banner" width="980" />
</div>
<p class="subtitle">Hierarchical command understanding with state-aware runtime behavior for practical assistant workflows.</p>
<!-- ===== Stat Cards ===== -->
<div class="stat-row">
<div class="stat-card">
<div class="stat-value">7.95M</div>
<div class="stat-label">Parameters</div>
</div>
<div class="stat-card">
<div class="stat-value">82</div>
<div class="stat-label">Runtime Turns</div>
</div>
<div class="stat-card">
<div class="stat-value">0</div>
<div class="stat-label">Errors</div>
</div>
<div class="stat-card">
<div class="stat-value">25.3 ms</div>
<div class="stat-label">Mean Latency</div>
</div>
<div class="stat-card">
<div class="stat-value">100%</div>
<div class="stat-label">OOD Precision</div>
</div>
<div class="stat-card">
<div class="stat-value">30.6 MB</div>
<div class="stat-label">Checkpoint</div>
</div>
</div>
---
## ๐Ÿ›๏ธ The Temple of Janus (Web Experience)
We have deployed a dedicated interactive environment to showcase the essence of JaneGPT-v2 Janus.
**Note:** This is a visual and technical walkthrough; it does not feature a live chat interface.
* **[๐Ÿ”— Enter the Experience](https://janus-web-demo.vercel.app/)**
* **Best Viewed On:** Desktop (Chrome/Edge) for full hardware-accelerated 3D effects.
---
## Quickstart (2 minutes)
<details open>
<summary><strong>Install + first prediction</strong></summary>
<div>
```bash
pip install -r requirements.txt
```
```python
from janegpt_v2_janus.inference import JaneGPTv3NLU
nlu = JaneGPTv3NLU(
model_path="weights/janegpt_v2_janus.pt",
tokenizer_path="weights/tokenizer.json",
)
state = {}
result = nlu.predict("set volume", state=state)
print(result)
if result.get("type") == "command":
state = nlu.update_state(result, state)
```
</div>
</details>
<details>
<summary><strong>Runtime wrapper (recommended for assistant flows)</strong></summary>
<div>
```python
from runtime.jane_nlu_runtime import JaneNLURuntime
rt = JaneNLURuntime(base_dir=".")
state = {}
out, state = rt.handle_turn("set volume", state)
print(out) # expected: clarify prompt for missing VALUE
out, state = rt.handle_turn("55", state)
print(out) # expected: resolved local command
```
</div>
</details>
<details>
<summary><strong>Run bundled demos</strong></summary>
<div>
```bash
python examples/demo_inference.py
python examples/demo_runtime.py
python examples/demo_runtime_suite.py
```
</div>
</details>
---
## What You Get
- Single-pass multitask prediction: domain + action + BIO slots.
- Runtime-safe clarification loops for missing required slots.
- Stateful follow-ups (for example, "that is not enough" after a volume change).
- Local command routing with controlled chat fallback.
- Compact deployment footprint: ~30.62 MB checkpoint.
---
## Model Architecture
### Interactive Architecture Visualization
<style>
/* ---- Architecture Visualization ---- */
.arch-container {
background: linear-gradient(135deg, #0a0f1a 0%, #0d1520 100%);
border: 1px solid #1a2a3e;
border-radius: 12px;
padding: 28px;
margin-bottom: 24px;
font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;
}
.arch-flow {
display: flex;
flex-direction: column;
gap: 12px;
}
.arch-layer {
background: #0d1520;
border: 2px solid #1e3048;
border-radius: 10px;
padding: 18px;
cursor: pointer;
transition: all 0.3s ease;
position: relative;
overflow: hidden;
}
.arch-layer::before {
content: '';
position: absolute;
top: 0;
left: 0;
right: 0;
height: 2px;
background: linear-gradient(90deg, #3b82f6, #8b5cf6, #ec4899, transparent);
opacity: 0;
transition: opacity 0.3s ease;
}
.arch-layer:hover {
background: #111c2e;
border-color: #2d5aa6;
box-shadow: 0 8px 24px rgba(59, 130, 246, 0.15);
}
.arch-layer:hover::before {
opacity: 1;
}
.arch-layer-title {
font-size: 1.05em;
font-weight: 700;
color: #58a6ff;
margin-bottom: 10px;
display: flex;
align-items: center;
justify-content: center;
gap: 8px;
text-align: center;
}
.arch-layer-icon {
display: none;
}
.arch-layer-desc {
font-size: 0.9em;
color: #8b9ab5;
line-height: 1.5;
max-height: 0;
overflow: hidden;
transition: max-height 0.3s ease;
}
.arch-layer.expanded .arch-layer-desc {
max-height: 500px;
}
.arch-layer-content {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(200px, 1fr));
gap: 12px;
margin-top: 12px;
max-height: 0;
overflow: hidden;
transition: max-height 0.3s ease;
}
.arch-layer.expanded .arch-layer-content {
max-height: 600px;
}
.arch-spec-item {
background: #162030;
border: 1px solid #253649;
border-radius: 8px;
padding: 10px 12px;
font-size: 0.82em;
text-align: center;
}
.arch-spec-label {
color: #4ade80;
font-weight: 600;
display: block;
margin-bottom: 4px;
}
.arch-spec-value {
color: #c5d0de;
font-family: 'Monaco', 'Courier New', monospace;
}
.arch-arrow {
text-align: center;
color: #4ade80;
font-size: 1.2em;
padding: 4px 0;
opacity: 0.6;
}
.arch-tasks {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(180px, 1fr));
gap: 12px;
margin-top: 12px;
}
.arch-task {
background: linear-gradient(135deg, #1a2f4a 0%, #0f1f38 100%);
border: 1px solid #253649;
border-left: 3px solid #3b82f6;
border-radius: 8px;
padding: 12px;
font-size: 0.85em;
}
.arch-task.domain {
border-left-color: #3b82f6;
}
.arch-task.action {
border-left-color: #8b5cf6;
}
.arch-task.slot {
border-left-color: #ec4899;
}
.arch-task-name {
font-weight: 700;
color: #e2e8f0;
margin-bottom: 6px;
text-align: center;
}
.arch-task-detail {
color: #8b9ab5;
font-size: 0.78em;
line-height: 1.4;
text-align: center;
}
.arch-loss-box {
background: #1a2a3a;
border: 1px solid #1e3a4f;
border-radius: 8px;
padding: 14px;
margin-top: 12px;
font-family: 'Monaco', 'Courier New', monospace;
font-size: 0.82em;
color: #4ade80;
overflow-x: auto;
}
.arch-decision {
background: #1a3a2a;
border-left: 3px solid #10b981;
border-radius: 6px;
padding: 10px;
margin-top: 8px;
font-size: 0.82em;
color: #a7f3d0;
}
@media (max-width: 900px) {
.arch-layer-content {
grid-template-columns: 1fr;
}
.arch-tasks {
grid-template-columns: 1fr;
}
}
</style>
<div class="arch-container">
<div class="arch-layer" onclick="this.classList.toggle('expanded')">
<div class="arch-layer-title">
<span class="arch-layer-icon">๐Ÿ”ค</span>
1. Tokenization & Embedding Layer
</div>
<div class="arch-layer-desc">
Input text is converted to token IDs and projected into a 256-dimensional embedding space.
</div>
<div class="arch-layer-content">
<div class="arch-spec-item">
<span class="arch-spec-label">Tokenizer</span>
<span class="arch-spec-value">BPE, vocab=8,192</span>
</div>
<div class="arch-spec-item">
<span class="arch-spec-label">Max Length</span>
<span class="arch-spec-value">96 tokens</span>
</div>
<div class="arch-spec-item">
<span class="arch-spec-label">Output Shape</span>
<span class="arch-spec-value">(batch, 96, 256)</span>
</div>
<div class="arch-spec-item">
<span class="arch-spec-label">Embedding</span>
<span class="arch-spec-value">8192 โ†’ 256 dim</span>
</div>
</div>
</div>
<div class="arch-arrow">โ†“</div>
<div class="arch-layer" onclick="this.classList.toggle('expanded')">
<div class="arch-layer-title">
<span class="arch-layer-icon"></span>
2. Transformer Backbone (8 Blocks)
</div>
<div class="arch-layer-desc">
Bidirectional attention layers with residual connections. Each block processes hidden states through grouped query attention and feed-forward networks.
</div>
<div class="arch-layer-content">
<div class="arch-spec-item">
<span class="arch-spec-label">Attention Type</span>
<span class="arch-spec-value">Grouped Query (GQA)</span>
</div>
<div class="arch-spec-item">
<span class="arch-spec-label">Query Heads</span>
<span class="arch-spec-value">8 heads</span>
</div>
<div class="arch-spec-item">
<span class="arch-spec-label">KV Heads</span>
<span class="arch-spec-value">4 heads (2:1)</span>
</div>
<div class="arch-spec-item">
<span class="arch-spec-label">Head Dimension</span>
<span class="arch-spec-value">32 (256รท8)</span>
</div>
<div class="arch-spec-item">
<span class="arch-spec-label">Position Embedding</span>
<span class="arch-spec-value">RoPE</span>
</div>
<div class="arch-spec-item">
<span class="arch-spec-label">FFN Expansion</span>
<span class="arch-spec-value">256 โ†’ 672 โ†’ 256</span>
</div>
<div class="arch-spec-item">
<span class="arch-spec-label">FFN Activation</span>
<span class="arch-spec-value">SwiGLU</span>
</div>
<div class="arch-spec-item">
<span class="arch-spec-label">Normalization</span>
<span class="arch-spec-value">RMSNorm</span>
</div>
<div class="arch-spec-item">
<span class="arch-spec-label">Causal Masking</span>
<span class="arch-spec-value">OFF (bidirectional)</span>
</div>
<div class="arch-spec-item">
<span class="arch-spec-label">Dropout Rate</span>
<span class="arch-spec-value">0.1</span>
</div>
</div>
<div class="arch-decision">
Grouped Query Attention reduces KV cache 50% while maintaining quality
</div>
</div>
<div class="arch-arrow">โ†“</div>
<div class="arch-layer" onclick="this.classList.toggle('expanded')">
<div class="arch-layer-title">
<span class="arch-layer-icon"></span>
3. Multi-Task Prediction Heads (Parallel)
</div>
<div class="arch-layer-desc">
Three independent classification heads process the backbone output simultaneously for domain, action, and slot predictions.
</div>
<div class="arch-tasks">
<div class="arch-task domain">
<div class="arch-task-name">Domain Head</div>
<div class="arch-task-detail">
<strong>Input:</strong> Last token (pooled)<br/>
<strong>Arch:</strong> Linear(256) โ†’ GELU โ†’ Dropout โ†’ Linear(10)<br/>
<strong>Output:</strong> 10 classes
</div>
</div>
<div class="arch-task action">
<div class="arch-task-name">Action Head</div>
<div class="arch-task-detail">
<strong>Input:</strong> Last token (pooled)<br/>
<strong>Arch:</strong> Linear(256) โ†’ GELU โ†’ Dropout โ†’ Linear(33)<br/>
<strong>Output:</strong> 33 classes
</div>
</div>
<div class="arch-task slot">
<div class="arch-task-name">Slot Head</div>
<div class="arch-task-detail">
<strong>Input:</strong> All tokens<br/>
<strong>Arch:</strong> Linear(256) โ†’ Linear(15 BIO)<br/>
<strong>Output:</strong> 15 labels/token
</div>
</div>
</div>
</div>
<div class="arch-arrow">โ†“</div>
<div class="arch-layer" onclick="this.classList.toggle('expanded')">
<div class="arch-layer-title">
<span class="arch-layer-icon"></span>
4. Output & Post-Processing
</div>
<div class="arch-layer-desc">
Raw logits are converted to predictions. For slots, BIO tags are decoded into semantic spans.
</div>
<div class="arch-layer-content">
<div class="arch-spec-item">
<span class="arch-spec-label">Domain Output</span>
<span class="arch-spec-value">10 classes</span>
</div>
<div class="arch-spec-item">
<span class="arch-spec-label">Action Output</span>
<span class="arch-spec-value">33 classes</span>
</div>
<div class="arch-spec-item">
<span class="arch-spec-label">Slots Decoder</span>
<span class="arch-spec-value">BIO โ†’ Spans</span>
</div>
<div class="arch-spec-item">
<span class="arch-spec-label">Confidence</span>
<span class="arch-spec-value">Softmax scores</span>
</div>
</div>
</div>
</div>
### Training Objective
<div class="arch-container">
<div class="arch-layer expanded">
<div class="arch-layer-title">
<span class="arch-layer-icon"></span>
Weighted Multi-Task Loss
</div>
<div class="arch-loss-box">
loss = 1.0 ร— L_domain + 1.0 ร— L_action + 1.5 ร— L_slots
Where:
L_domain = CrossEntropy(domain_logits, domain_labels)
L_action = CrossEntropy(action_logits, action_labels)
L_slots = CrossEntropy(slot_logits, slot_labels)
with ignore_index=-100 (padding)
</div>
<div class="arch-decision">
Slot weight (1.5ร—) reflects higher complexity of sequence tagging vs. classification
</div>
</div>
</div>
### Architecture Specifications
| Component | Configuration | Details |
|-----------|---|---|
| **Backbone Type** | Transformer (GPT-style) | Bidirectional, non-causal attention |
| **Vocabulary Size** | 8,192 | BPE tokenization |
| **Embedding Dim** | 256 | Token + Rotary Position embeddings |
| **Attention Heads** | 8 Query, 4 KV | Grouped Query Attention (GQA) for efficiency |
| **Head Dimension** | 32 | per head_dim = embed_dim / num_heads |
| **Transformer Blocks** | 8 Layers | Each with Attn + FFN + Residuals |
| **Feed-Forward Hidden** | 672 | SwiGLU gate activation |
| **Position Encoding** | RoPE | Rotary Position Embeddings (theta=10000) |
| **Normalization** | RMSNorm | Pre-layer normalization |
| **Max Sequence Length** | 96 tokens | Approximately 60-80 words |
| **Dropout Rate** | 0.1 | Applied during training |
| **Total Parameters** | 7,949,626 | All trainable |
| **Parameter Breakdown** | Backbone: 7.80M, Task Heads: 146K | Efficient multitask design |
### Task Configuration
| Task | Type | Classes | Architecture |
|------|---|---|---|
| **Domain Classification** | Sequence-level | 10 domains | Pooled โ†’ Linear(256) โ†’ GELU โ†’ Linear(10) |
| **Action Classification** | Sequence-level | 33 actions | Pooled โ†’ Linear(256) โ†’ GELU โ†’ Linear(33) |
| **Slot Tagging** | Token-level | 15 BIO labels | Per-token โ†’ Linear(256) โ†’ Linear(15) |
---
## Benchmark Results
<div class="charts-row">
<div class="chart-container-sm">
<div class="chart-title">Runtime reliability</div>
<div class="chart-subtitle">82-turn suite</div>
<div class="bars-row">
<div class="bar-group">
<div class="bar-track"><div class="bar-fill" style="height:100%; background:linear-gradient(0deg,#3b82f6,#60a5fa);"><span class="bar-value">82</span></div></div>
<div class="bar-label">Turns</div>
</div>
<div class="bar-group">
<div class="bar-track"><div class="bar-fill" style="height:81.7%; background:linear-gradient(0deg,#8b5cf6,#a78bfa);"><span class="bar-value">67</span></div></div>
<div class="bar-label">Local</div>
</div>
<div class="bar-group">
<div class="bar-track"><div class="bar-fill" style="height:14.6%; background:linear-gradient(0deg,#ec4899,#f472b6);"><span class="bar-value">12</span></div></div>
<div class="bar-label">Clarify</div>
</div>
<div class="bar-group">
<div class="bar-track"><div class="bar-fill" style="height:1%; background:linear-gradient(0deg,#9ca3af,#d1d5db);"><span class="bar-value">0</span></div></div>
<div class="bar-label">Errors</div>
</div>
</div>
<div class="bar-note">fair_benchmarks.json</div>
</div>
<div class="chart-container-sm">
<div class="chart-title">Predict latency</div>
<div class="chart-subtitle">CUDA ยท batch=1 ยท lower is better</div>
<div class="bars-row">
<div class="bar-group">
<div class="bar-track"><div class="bar-fill" style="height:63.3%; background:linear-gradient(0deg,#3b82f6,#60a5fa);"><span class="bar-value">25.3ms</span></div></div>
<div class="bar-label">Pยทmean</div>
</div>
<div class="bar-group">
<div class="bar-track"><div class="bar-fill" style="height:86.5%; background:linear-gradient(0deg,#8b5cf6,#a78bfa);"><span class="bar-value">34.6ms</span></div></div>
<div class="bar-label">Pยทp95</div>
</div>
<div class="bar-group">
<div class="bar-track"><div class="bar-fill" style="height:88.4%; background:linear-gradient(0deg,#ec4899,#f472b6);"><span class="bar-value">35.4ms</span></div></div>
<div class="bar-label">Fwdยทmean</div>
</div>
<div class="bar-group">
<div class="bar-track"><div class="bar-fill" style="height:91.8%; background:linear-gradient(0deg,#a78bfa,#c4b5fd);"><span class="bar-value">36.7ms</span></div></div>
<div class="bar-label">Fwdยทp95</div>
</div>
</div>
<div class="bar-note">janus_model_report.json</div>
</div>
</div>
<div class="chart-container">
<div class="chart-title">OOD rejection quality</div>
<div class="chart-subtitle">Schema-agnostic ยท hover values</div>
<input type="radio" name="ood-filter" id="ood-both" checked />
<input type="radio" name="ood-filter" id="ood-banking77" />
<input type="radio" name="ood-filter" id="ood-clinc" />
<div class="ood-filter-row filter-row">
<label class="filter-btn" for="ood-both">Both</label>
<label class="filter-btn" for="ood-banking77">BANKING77</label>
<label class="filter-btn" for="ood-clinc">CLINC</label>
</div>
<div class="chart-bars">
<div class="bars-row bars-row-wide">
<div class="bar-group" data-dataset="banking77">
<div class="bar-track"><div class="bar-fill" style="height:87.8%; background:linear-gradient(0deg,#3b82f6,#60a5fa);"><span class="bar-value">87.8%</span></div></div>
<div class="bar-label">B77 F1</div>
</div>
<div class="bar-group" data-dataset="banking77">
<div class="bar-track"><div class="bar-fill" style="height:100%; background:linear-gradient(0deg,#ec4899,#f472b6);"><span class="bar-value">100%</span></div></div>
<div class="bar-label">B77 Prec</div>
</div>
<div class="bar-group" data-dataset="banking77">
<div class="bar-track"><div class="bar-fill" style="height:78.25%; background:linear-gradient(0deg,#8b5cf6,#a78bfa);"><span class="bar-value">78.3%</span></div></div>
<div class="bar-label">B77 Rec</div>
</div>
<div class="bar-group" data-dataset="clinc_oos">
<div class="bar-track"><div class="bar-fill" style="height:79.23%; background:linear-gradient(0deg,#3b82f6,#60a5fa);"><span class="bar-value">79.2%</span></div></div>
<div class="bar-label">CL F1</div>
</div>
<div class="bar-group" data-dataset="clinc_oos">
<div class="bar-track"><div class="bar-fill" style="height:100%; background:linear-gradient(0deg,#ec4899,#f472b6);"><span class="bar-value">100%</span></div></div>
<div class="bar-label">CL Prec</div>
</div>
<div class="bar-group" data-dataset="clinc_oos">
<div class="bar-track"><div class="bar-fill" style="height:65.6%; background:linear-gradient(0deg,#8b5cf6,#a78bfa);"><span class="bar-value">65.6%</span></div></div>
<div class="bar-label">CL Rec</div>
</div>
</div>
</div>
<div class="bar-note">fair_benchmarks.json</div>
</div>
</div>
### Comprehensive Benchmark Summary
<div class="chart-container">
<div class="chart-title">Full Benchmark Evidence</div>
<div class="chart-subtitle">All values from real holdout evaluations โ€” no synthetic or inflated numbers</div>
<table class="comparison-table">
<thead>
<tr>
<th>Metric</th>
<th>Detail</th>
<th><span class="tag-v2">Jane v2</span></th>
<th><span class="tag-janus">Janus</span></th>
</tr>
</thead>
<tbody>
<tr>
<td><strong>Speed (mean latency)</strong></td>
<td>CUDA, batch=1</td>
<td>31.60 ms</td>
<td><strong>25.31 ms</strong></td>
</tr>
<tr>
<td><strong>Throughput</strong></td>
<td>CUDA, single GPU</td>
<td>32 pred/sec</td>
<td>Stable across 82 turns, 0 errors</td>
</tr>
<tr>
<td><strong>OOD F1</strong></td>
<td>BANKING77</td>
<td><strong>94.31%</strong></td>
<td>87.80%</td>
</tr>
<tr>
<td><strong>OOD F1</strong></td>
<td>CLINC OOS</td>
<td><strong>89.16%</strong></td>
<td>79.23%</td>
</tr>
<tr>
<td><strong>OOD Precision</strong></td>
<td>BANKING77</td>
<td>99.35%</td>
<td><strong>100.00%</strong></td>
</tr>
<tr>
<td><strong>OOD Precision</strong></td>
<td>CLINC OOS</td>
<td>99.14%</td>
<td><strong>100.00%</strong></td>
</tr>
<tr>
<td><strong>OOD Recall</strong></td>
<td>BANKING77</td>
<td><strong>89.75%</strong></td>
<td>78.25%</td>
</tr>
<tr>
<td><strong>OOD Recall</strong></td>
<td>CLINC OOS</td>
<td><strong>81.00%</strong></td>
<td>65.60%</td>
</tr>
<tr>
<td><strong>Validation Accuracy</strong></td>
<td>Domain (best epoch)</td>
<td>โ€”</td>
<td><strong>99.83%</strong></td>
</tr>
<tr>
<td><strong>Validation Accuracy</strong></td>
<td>Action (best epoch)</td>
<td>โ€”</td>
<td><strong>99.87%</strong></td>
</tr>
<tr>
<td><strong>Validation Accuracy</strong></td>
<td>Domain+Action pair (best epoch)</td>
<td>โ€”</td>
<td><strong>99.83%</strong></td>
</tr>
<tr>
<td><strong>Slot Extraction F1</strong></td>
<td>All 15 slot types</td>
<td>โ€”</td>
<td><strong>1.000 (100%)</strong></td>
</tr>
<tr>
<td><strong>Training Loss</strong></td>
<td>Epoch 1 โ†’ 4</td>
<td>โ€”</td>
<td>0.060 โ†’ 0.020 โ†’ 0.002 โ†’ 0.001</td>
</tr>
<tr>
<td><strong>Validation Loss</strong></td>
<td>Epoch 1 โ†’ 3</td>
<td>โ€”</td>
<td>0.0153 โ†’ 0.0116 โ†’ 0.0115 (stable)</td>
</tr>
<tr>
<td><strong>Runtime Reliability</strong></td>
<td>82-turn conversation test</td>
<td>โ€”</td>
<td><strong>0 errors, 0 crashes</strong></td>
</tr>
<tr>
<td><strong>Domain Confusion</strong></td>
<td>10 domains</td>
<td>โ€”</td>
<td>99%+ per-domain, minimal cross-confusion</td>
</tr>
<tr>
<td><strong>Action Confusion</strong></td>
<td>33 actions</td>
<td>โ€”</td>
<td>Perfect diagonal, no action commonly confused</td>
</tr>
</tbody>
</table>
</div>
---
## Live Output Shapes (click to expand)
<details>
<summary><strong>Command output</strong></summary>
<div>
```json
{
"type": "command",
"domain": "apps",
"action": "launch",
"slots": {
"APP_NAME": {
"text": "chrome",
"start": 5,
"end": 11,
"confidence": 0.999
}
},
"confidence": 0.97,
"route": "local"
}
```
</div>
</details>
<details>
<summary><strong>Clarification output</strong></summary>
<div>
```json
{
"type": "clarify",
"question": "What value should I set it to?",
"debug": {
"domain": "volume",
"action": "set",
"reason": "missing_VALUE"
}
}
```
</div>
</details>
<details>
<summary><strong>Label schema</strong></summary>
<div>
- **Domains (10):** volume, brightness, media, apps, browser, productivity, screen, window, system, conversation
- **Actions (33):** up, down, set, mute, unmute, play, pause, next, previous, launch, close, switch, search, set_reminder, screenshot, read, explain, undo, quit, chat, minimize, maximize, restore, focus, copy, paste, cut, lock, sleep, wifi_on, wifi_off, bluetooth_on, bluetooth_off
- **Slot labels (BIO, 15):** VALUE, APP_NAME, QUERY, DURATION, TIME, WINDOW_NAME, TEXT
</div>
</details>
---
## Visual Benchmark Evidence
<p align="center">
<img src="reports/loss_per_epoch.png" alt="Train and validation loss" width="860" />
</p>
<p align="center">
<img src="reports/train_loss_smoothed.png" alt="Smoothed train loss" width="860" />
</p>
<p align="center">
<img src="reports/val_slot_f1.png" alt="Validation slot F1" width="860" />
</p>
### Confusion Matrix โ€” Interactive Breakdown
<!-- FIX: radio inputs must be SIBLINGS of both .confusion-toggle AND .cm-panel divs
so the CSS sibling combinator (~) can reach them all from the same parent level -->
<div class="chart-container">
<div class="chart-title">Per-Class True vs Predicted</div>
<div class="chart-subtitle">Single stacked bar per head โ€” segment width = sample ratio. Hover any segment for details.</div>
<input type="radio" name="cm-toggle" id="cm-domain" checked />
<input type="radio" name="cm-toggle" id="cm-action" />
<div class="confusion-toggle">
<label class="filter-btn" for="cm-domain">Domains (10)</label>
<label class="filter-btn" for="cm-action">Actions (33)</label>
</div>
<div class="cm-panel" data-cm="domain">
<div class="stacked-bar-label">Domain Sample Distribution &mdash; 3,110 total samples &mdash; hover each segment</div>
<div class="cm-stacked-bar">
<div class="cm-seg" style="flex:430; background:#3b82f6; border-radius:10px 0 0 10px;"><div class="cm-tip"><span class="tip-name">volume</span><br/><span class="tip-samples">430 samples (13.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:250; background:#f59e0b;"><div class="cm-tip"><span class="tip-name">brightness</span><br/><span class="tip-samples">250 samples (8.0%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:340; background:#10b981;"><div class="cm-tip"><span class="tip-name">media</span><br/><span class="tip-samples">340 samples (10.9%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:250; background:#ef4444;"><div class="cm-tip"><span class="tip-name">apps</span><br/><span class="tip-samples">250 samples (8.0%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:120; background:#8b5cf6;"><div class="cm-tip"><span class="tip-name">browser</span><br/><span class="tip-samples">120 samples (3.9%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:340; background:#ec4899;"><div class="cm-tip"><span class="tip-name">productivity</span><br/><span class="tip-samples">340 samples (10.9%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:120; background:#14b8a6;"><div class="cm-tip"><span class="tip-name">screen</span><br/><span class="tip-samples">120 samples (3.9%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:340; background:#f97316;"><div class="cm-tip"><span class="tip-name">window</span><br/><span class="tip-samples">340 samples (10.9%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:580; background:#6366f1;"><div class="cm-tip"><span class="tip-name">system</span><br/><span class="tip-samples">580 samples (18.6%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:340; background:#06b6d4; border-radius:0 10px 10px 0;"><div class="cm-tip"><span class="tip-name">conversation</span><br/><span class="tip-samples">340 samples (10.9%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
</div>
<div class="cm-legend">
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#3b82f6;"></span>volume</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#f59e0b;"></span>brightness</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#10b981;"></span>media</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#ef4444;"></span>apps</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#8b5cf6;"></span>browser</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#ec4899;"></span>productivity</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#14b8a6;"></span>screen</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#f97316;"></span>window</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#6366f1;"></span>system</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#06b6d4;"></span>conversation</div>
</div>
<div class="bar-note">Source: validation set confusion matrix โ€” segment widths proportional to sample count</div>
</div>
<div class="cm-panel" data-cm="action">
<div class="stacked-bar-label">Action Sample Distribution &mdash; 3,205 total samples &mdash; hover each segment</div>
<div class="cm-stacked-bar">
<div class="cm-seg" style="flex:170; background:#3b82f6; border-radius:10px 0 0 10px;"><div class="cm-tip"><span class="tip-name">up</span><br/><span class="tip-samples">170 samples (5.3%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:165; background:#2563eb;"><div class="cm-tip"><span class="tip-name">down</span><br/><span class="tip-samples">165 samples (5.1%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:170; background:#1d4ed8;"><div class="cm-tip"><span class="tip-name">set</span><br/><span class="tip-samples">170 samples (5.3%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:90; background:#7c3aed;"><div class="cm-tip"><span class="tip-name">mute</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:90; background:#6d28d9;"><div class="cm-tip"><span class="tip-name">unmute</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:90; background:#10b981;"><div class="cm-tip"><span class="tip-name">play</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:90; background:#059669;"><div class="cm-tip"><span class="tip-name">pause</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:90; background:#047857;"><div class="cm-tip"><span class="tip-name">next</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:90; background:#065f46;"><div class="cm-tip"><span class="tip-name">previous</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:90; background:#ef4444;"><div class="cm-tip"><span class="tip-name">launch</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:90; background:#dc2626;"><div class="cm-tip"><span class="tip-name">close</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:90; background:#b91c1c;"><div class="cm-tip"><span class="tip-name">switch</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:90; background:#f59e0b;"><div class="cm-tip"><span class="tip-name">search</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:90; background:#d97706;"><div class="cm-tip"><span class="tip-name">set_reminder</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:90; background:#ec4899;"><div class="cm-tip"><span class="tip-name">screenshot</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:90; background:#db2777;"><div class="cm-tip"><span class="tip-name">read</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:90; background:#be185d;"><div class="cm-tip"><span class="tip-name">explain</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:90; background:#14b8a6;"><div class="cm-tip"><span class="tip-name">undo</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:90; background:#0d9488;"><div class="cm-tip"><span class="tip-name">quit</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:90; background:#0f766e;"><div class="cm-tip"><span class="tip-name">chat</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:90; background:#f97316;"><div class="cm-tip"><span class="tip-name">minimize</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:90; background:#ea580c;"><div class="cm-tip"><span class="tip-name">maximize</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:90; background:#c2410c;"><div class="cm-tip"><span class="tip-name">restore</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:90; background:#6366f1;"><div class="cm-tip"><span class="tip-name">focus</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:90; background:#4f46e5;"><div class="cm-tip"><span class="tip-name">copy</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:90; background:#4338ca;"><div class="cm-tip"><span class="tip-name">paste</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:90; background:#06b6d4;"><div class="cm-tip"><span class="tip-name">cut</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:90; background:#0891b2;"><div class="cm-tip"><span class="tip-name">lock</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:90; background:#0e7490;"><div class="cm-tip"><span class="tip-name">sleep</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:90; background:#84cc16;"><div class="cm-tip"><span class="tip-name">wifi_on</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:90; background:#65a30d;"><div class="cm-tip"><span class="tip-name">wifi_off</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:90; background:#a855f7;"><div class="cm-tip"><span class="tip-name">bluetooth_on</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
<div class="cm-seg" style="flex:90; background:#9333ea; border-radius:0 10px 10px 0;"><div class="cm-tip"><span class="tip-name">bluetooth_off</span><br/><span class="tip-samples">90 samples (2.8%)</span><br/><span class="tip-acc">Accuracy: 100%</span><br/><span class="tip-miss">Misclassified: 0</span></div></div>
</div>
<div class="cm-legend">
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#3b82f6;"></span>up</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#2563eb;"></span>down</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#1d4ed8;"></span>set</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#7c3aed;"></span>mute</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#6d28d9;"></span>unmute</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#10b981;"></span>play</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#059669;"></span>pause</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#047857;"></span>next</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#065f46;"></span>previous</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#ef4444;"></span>launch</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#dc2626;"></span>close</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#b91c1c;"></span>switch</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#f59e0b;"></span>search</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#d97706;"></span>set_reminder</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#ec4899;"></span>screenshot</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#db2777;"></span>read</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#be185d;"></span>explain</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#14b8a6;"></span>undo</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#0d9488;"></span>quit</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#0f766e;"></span>chat</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#f97316;"></span>minimize</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#ea580c;"></span>maximize</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#c2410c;"></span>restore</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#6366f1;"></span>focus</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#4f46e5;"></span>copy</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#4338ca;"></span>paste</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#06b6d4;"></span>cut</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#0891b2;"></span>lock</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#0e7490;"></span>sleep</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#84cc16;"></span>wifi_on</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#65a30d;"></span>wifi_off</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#a855f7;"></span>bluetooth_on</div>
<div class="cm-legend-item"><span class="cm-legend-swatch" style="background:#9333ea;"></span>bluetooth_off</div>
</div>
<div class="bar-note">Source: validation set confusion matrix โ€” segment widths proportional to sample count</div>
</div>
</div>
<details>
<summary><strong>View original confusion matrix images</strong></summary>
<div>
<p align="center">
<img src="reports/confusion_domain_val.png" alt="Domain confusion matrix" width="860" />
</p>
<p align="center">
<img src="reports/confusion_action_val.png" alt="Action confusion matrix" width="860" />
</p>
</div>
</details>
<details>
<summary><strong>Additional diagnostics</strong></summary>
<div>
<p align="center">
<img src="reports/lr_schedule.png" alt="Learning rate schedule" width="860" />
</p>
<p align="center">
<img src="reports/epoch_time.png" alt="Epoch time profile" width="860" />
</p>
<p align="center">
<img src="reports/train_loss_raw.png" alt="Raw training loss" width="860" />
</p>
</div>
</details>
---
## Upload-Ready Layout
```text
.
|- README.md
|- .gitattributes
|- LICENSE
|- requirements.txt
|- assets/
| |- jane-janus-glitch.webp
|- janegpt_v2_janus/
| |- __init__.py
| |- architecture.py
| |- dataset.py
| |- inference.py
| |- labels.py
| |- multitask.py
|- runtime/
| |- jane_nlu_runtime.py
|- examples/
| |- demo_inference.py
| |- demo_runtime.py
| |- demo_runtime_suite.py
|- weights/
| |- janegpt_v2_janus.pt
| |- tokenizer.json
|- reports/
| |- fair_benchmarks.json
| |- fair_benchmarks.md
| |- janus_model_report.json
| |- janus_model_report.md
| |- public_benchmarks.json
| |- *.png benchmark visuals
```
---
## Limitations
- English-focused command language.
- Command NLU model, not an open-domain generative chatbot.
- MASSIVE and SNIPS mapped-intent accuracy is excluded from headline claims because mapping coverage is partial.
---
## Use Cases
- Virtual assistant command routing
- Smart home intent classification
- Voice command understanding
- Chatbot intent detection
- Edge device deployment (small enough for embedded systems)
---
## Part of the JANE Project
**JANE** โ€” a fully
offline, privacy-first AI voice assistant.
๐Ÿ”— [JANE AI Assistant on GitHub](https://github.com/Ravindu-S/JANE-AI-Assistant)
๐Ÿ”— [JaneGPT-v2 on GitHub](https://github.com/Ravindu-S/JaneGPT-v2)
---
## Created By
**Ravindu Senanayake**
Built from scratch โ€” architecture, tokenizer, and training
pipeline designed and implemented by the author.
[![GitHub](https://img.shields.io/badge/GitHub-Ravindu--S-black?logo=github)](https://github.com/Ravindu-S)
---
## License
Apache-2.0 (see [LICENSE](LICENSE)).