Spaces:

---

Duplicated from  anotheruserishere/Tokescope

anotheruserishere

/

Tokescope_dev

Like 0

Running on Zero

Hello Hugging Face & @ThomasWolf @giadap @lewtun @akhaliq !

Thank you for the space provided here.

I'm not sure I'm actually asking for a GPU grant here, so let's see where this goes.

First, I'm a little embarrassed to admit this after learning so much from-- and coming to depend on-- your site's practically agalmic services to the commons, but the first few times I came around I wasn't sure it was safe to click anything. Between the name, the large download sizes, and the extreme value and utility of what was on offer, it all felt a little bit like The Pirate Bay or Anna's Archive, so I thought it could be a warez site.

For naming, though, I really do appreciate that sort of nominal direction, because if things go very badly for me and I slip & fall into a YC seed application, the name of the thing is going to be mu10xi!.AI, for "Mechanistic Unicorn 10x Interpretability!" It's got the necessary words, along with the subtle 10x UX and "is that meant to be hex or binary or...?" and, most importantly, it has an unserious ring to it.

I think that may actually be required to get a whole lot more people comfortable plunging into exploratory discovery: spelunking model internals with the same enthusiasm and zeal as digging cheat codes out of a cartridge, knowing you can't hit a button that's going to make anything truly awful happen, and trying bad ideas because a retry only costs a button press. Consider, a wide mass of people clicking download -> install and starting to get a sense of real, actual, model steering and interpretability before needing to go through significant learning gradients of their own, which will very much scaffold and improve understanding later but that will bootstrap things very nicely.
Really, that is meant with a high level of seriousness and gravity.

I was lucky: I came at AI from a background that gave me questions the current tools don't let you ask, and can be hard to even see, meaning hard to think up to begin with. I had no choice but to dig in and build some of my own. And it more than 10x'ed my own gain-of-function in figuring out and understanding how models work when I had a few preliminary ones scrolling through terminals.

That was especially the case when Opus updated less than a year ago. In the verbal formulations of many great frontier models these days: having a minimally comprehensible UX didn't just 10x my rate of understanding; it 10y'ed it too, hyphens and all-- hyphens I acquired honestly during the days I thought Heidegger would be fun (wasn't) and interesting (it was) to read, and the things stuck in my head.

That Opus update suddenly meant stealing hours from sleep to work on these things was no longer stuck on: "How much of my > kids-tucked-in <= 2:00 a.m. can I lose to Torch and still have it be productive?"

Things crystallized, catalyzed, and came together at increasingly higher rates after that. And I don't think I'm especially talented in many of these areas either; I was just suddenly able to productively look for answers to many more of the questions I had, and the small bits of answers fed thoughts I could wander through during more civilized hours while going about my day.

A nice self-reinforcing loop, merely because I could do something like prompt Claude:
"Okay, take my spec for how I'd like to branch lower-probability tokens & their future token-generation paths, calculate each branch’s probability mass relative to the original, and put it into a basic REPL-style CLI with ASCII heat-mapped to argmax confidence on logit-lens-style projections. Also, for each token there, iterate a lookup through Wiktionary on the token's meaning, language-family tree and etymology, keyed to a display option in the CLI."
And all of a sudden that 10x'ed UX shows me all sorts of interesting things. For example: tuned lenses are way overkill for easily seeing that there is vanishingly little in even the more basic logit lens that is truly meaningless noise. Correlated to the semantics of a future token? No, not always.
Correlated to many other things at least as useful and interesting to know? Yes, absolutely. Mostly. (I don't like making absolute claims.)

Really, this is Sapir-Whorf applied to UX, limited horizons of what’s easily explored, and worse: we should know this as a field, that UX matters even at the lowest or most abstract levels to facilitate thinking about concepts, and their ready manipulation inside our heads, when we haven’t fully or at all come to understand a thing. It's why the pure-math people roll their own formalisms when the words and symbols available to exchange meaning through common communications channels don't suffice.

That's absolutely required to allow real momentum-- or at least my epistemic certainty level says it's like 83.282…3%? (Could be some other number I chose that at random) when somebody rolls out a new thing like "calculus", yes that luminary could see it in their heads as they went without the handholding, but the new formalisms they provided to be place holders for what they saw there in their heads meant they could share it, and twink the people around them with all the new gear they'd crafted.

For my part, I have a heck of a lot more ideas than just "UX matters," and I'm pretty sure many of them are very, very awful. Actually, and sadly for all involved, their true awfulness might be inversely proportional to their absurdity. Claude has had to “sit with it” more than once when I explained that I had an idea for chipping away at the reachability problem, but it really wasn’t going to enjoy how I might manage to get a model into some very unlikely places in the manifold.

So, sure, this technically is a GPU grant application? I think? Maybe more of a CV, or just a thank you note. It can be all of those.

It seemed like a not-bad time to ask. Given all the meetings you've probably had with Jensen regarding the acquisition, (tell him James says hi if you see him but don’t be surprised if he pretends not to know who I am. Or I guess even more surprised if he pretends to?) But, assuming some of those meetings were in person I figure there may be a few Blackwell chips that fell out of his pockets and are still stuck in the couch cushions somewhere, so I might as well ask even if I’m not sure I’d have time to do much with it.

Sure, I would try to find a way to make meaningful use of one, cloud-convenient, to do things that aren't easy on a single RTX 4080.

But time is actually the greater constraint, because my current job-- which I really do enjoy-- doesn't involve this kind of work. Meanwhile, modern AI has reignited a fierce determination to understand whatever I can about mind and cognition, something that had been sitting on a shelf giving me the side-eye for more than a decade after grad school ended.

The paths I could see in academia or industry back then looked like they might allow 10% of my time spent on what I think about constantly but the remaining 90% going on tasks that just weren't what I wanted to do and don’t think I had too much insight into.

Now the picture is a little different with AI.

And, as well, the specific nature of AI-- how language has enabled it-- makes an unlikely combination of background experiences a strong fit for coming at things from a direction I'm increasingly sure is useful. There's time I spent in cog sci, analytical philosophy and some NLP. But it turned out that for understanding LLM’s, even though time spent in NLP studying under a former Bell Labs scientist was highly instructive, it was much more the translation theory, pragmatics, semiotics, and Sapir-Whorf that has fueled a lot of the interesting questions whose answers seemed to lie somewhere in the decomposition of models' inference-time calculations, at structural levels above individual activations.

The few Spaces I have show in HF show stripped-down versions of a couple pieces of the tooling I've built. One of them, Tokescope, can identify micro-clusters of semantic meaning based on user-specified tokens monitored in their rank ordering at head or layer hook points, and rank depth specified. Gram-schmidt projections suppress the cluster, just for the current token’s generation, with little impact on model cognition with logits produced as usual. Default targeting is cross-lingual semantic space of the token specified but there’s no reason that rising cluster can’t be the trigger while the target is changed to another known feature w/ known activation coordinates. The model then proceeds normally through its own logits, softmax and token selection rather than having its output distribution patched afterward. Other forms of projecting to the full token vocab that don’t incur the inference compute costs of LM_Head are also effective but not identical in impact. I just don’t have time or resources to explore all avenues of inquiry though.

This work also predates Anthropic's July release of the J-lens, which is some incredible work, and that also uses token ranks much below argmax. Given how much that work dovetails with directions I've been pursuing-- glimpses in Tokescope, but there’s too much to cover here-- there's a lot of fertile ground under argmax still to mine, and Anthropic's work provides at least a little validation that I'm not completely off-base in looking in this.
In addition to the Spaces, the attached 28 seconds shows an early version of Manifographer, a name both accurate and enjoyable for making people consider how it should be pronounced.

It's a preliminary experiment in giving interpretability ann inference-time steering the sort of UX videogames have spent decades refining, better than the earlier things I’d build: continuous visual and haptic feedback loops, spatial navigation, muscle memory, immediate cause-and-effect, and controls designed for exploring complicated dynamic worlds rather than clicking through another dashboard.
It has also led to the realization that Microsoft has an untapped market for Xbox controllers, because they're hands-down better for navigating this stuff.

Everything you'll see in those clips is doing live inference-time steering. Well, the hostile polygons inspired by Geometry Wars aren't themselves hooks into inference. That mode is unlikely to become the daily driver of the very serious p(doom) researchers. I also probably shouldn't distract them too much from swarm management with neon polygons, although that phrase, “swarm management”, already sounds like a process ripe for much easier monitoring with a game mechanic of some sort. This really isn’t about gamification and those things can be left out once the control surface aspect of things is adequately conveyed: it’s complex exploration of many and varied data projections.

Also: everything happening in those modes is or with a toggle can be streaming steering traces to disk. I'm not certain, I have a reasonable level of confidence that some or most of the not-completely-random walk produced by casual gamers steering false negatives and positives-- or other behaviors-- in useful generative bioscience models might generate genuinely useful leads for controlling and improving models that don't receive nearly as much interpretability attention as their immediate utility warrants.

A lot of those deeply complex areas are outside my own knowledge, which is part of why I'd really like to inflict some of these ideas on people who know much more about them. But that's hard to do from outside the industry, with an atypical background, while working in a job that also provides meaningful work but just happens not to be in this area.

I honestly don't know where this style of approach, I mean workplace/role/organization-- and also willingness to embarrass myself with half-formed ideas-- is best suited to land. Although really: I'm not actually half as unserious as all this might imply. Teaching defeasible reasoning to college students for a few years, and having kids, has shown me there's value in simply not caring much about people's perceptions of me if I manage to communicate something useful and important. And it's really a lot of fun to implant false memories in a lecture hall full of students, then have them laugh, and have it hit home that minds have blind spots-- but awareness of those blind spots is at least partial inoculation. And also the ideas don't actually feal that embarrassing when padding one into a sabot, then firing it at high speed into hard questions to see it shatter and how those pieces fell apart, where a shard hit the target.

But I'm trying to figure that out, the landing, for ideas and a place to chase them. And, so, when I saw the GPU grant option something clicked: this was an opportunity whose medium-downside might actually be getting the grant, while the biggest downside is simply continuing what I'm already doing, or proving that Factorio isn’t the only industrial strength software capable of climbing to modest levels in the Steam charts during Summer Sales if all of my thoughts on interpretability & training & alignment & boosting models’ ability to make novel leaps of reasoning all turn out poorly. I’ve always figured the secret back story of Factorio was as a post-Big Data & Cloud industry boom ~2012, intended as an enterprise app for ETL or something, and that got over its skis on scope creep, wandered into skewmorphic design a bit too much until someone realized they couldn’t stop building random things in it and then heck, slap a price tag on it & charge others to do the same.

If Hugging Face happens to have use for somebody who generates this kind of wall of text, I'd really like to talk.
If not, then I'd like to extend a very sincere thank you for the ecosystem you've built.

Thank you, everyone there, sincerely-- @ThomasWolf

James Davison