Hugging Face's logo
Activity Feed Request to join this org

AI & ML interests

None defined yet.

Recent Activity
teagrjohnson  submitted a paper 5 days ago
Characterizing Narrative Content in Web-scale LLM Pretraining Data
teagrjohnson  authored a paper 5 days ago
Characterizing Narrative Content in Web-scale LLM Pretraining Data
tellarin  authored a paper 11 days ago
LectūraAgents: A Multi-Agent Framework for Adaptive Personalized AI-Assisted Learning and Embodied Teaching
View all activity
eienmojiki 
posted an update 2 days ago
mmhamdy 
posted an update 3 days ago
view post
Post
140
It has been more than a decade now since the knowledge distillation paper came out.

Knowledge Distillation (KD) is one of my favorite topics, but I have to confess that I'm not a huge fan of the term because I find it confusing (or at least, it has became so over time).

The idea behind KD is not novel; it was there almost a decade before the paper came out (and arguably even a decade before that, back to 1990-91). But this paper is the one that clicked, the one that made the topic much more popular and introduced it to a broader audience.

First, the timing and the authors played a big role: we have Geoffrey Hinton, Oriol Vinyals, and Jeff Dean here. And second, Geoffrey Hinton is really good at idea branding: Model compression?! No, no, no! Let's call it "Knowledge Distillation" and use evocative terms such as "Dark Knowledge" to describe what is being transferred.

It's a great name, but as time has passed, the term became a bit of a relic. KD is no longer solely about compression (KD used to be introduced as a method for model compression, but now model compression is just one application of KD). And the other thing is that the word "distillation" implies some sort of potency here, that the student is somehow more powerful than the teacher, which is not the case (but many counterarguments could be made, for example, more powerful compared to another model trained with no teacher)

Nevertheless, the paper is incredibly well-written, short, and fun to read. It's one of few papers that I read several times. Check it out, and maybe share your thoughts on the topic with us here!

If you had to choose another name for Knowledge Distillation, what would it be?

  • 2 replies
·
teagrjohnson 
submitted a paper to Daily Papers 5 days ago
teagrjohnson 
authored a paper 5 days ago
Reubencf 
posted an update 8 days ago
view post
Post
3732
Shadows of Tomorrow is finally live on Hugging Face Spaces with Gradio.

It’s a browser-playable RPG built with Godot, set in a post-nuclear future where players explore Magnus Province, collect medicinal plants, craft medicine, and help cure NPCs.

Play it here: Reubencf/Shadows_of_Tomorrow
  • 10 replies
·
Reubencf 
posted an update 9 days ago
view post
Post
131
Tiny Maple is now live on iOS and Android.

The app introduces the @Coherelabs Tiny Aya series of multilingual AI models to mobile devices. This release is significant as it enhances access to multilingual AI from anywhere, particularly for users who prefer offline capabilities.

I invite you to try it and share your feedback.

App Store: https://apps.apple.com/bn/app/tiny-maple/id6774123088

Google Play: https://play.google.com/store/apps/details?id=com.reubencf.tinyaya
tellarin 
authored a paper 11 days ago
tellarin 
submitted a paper to Daily Papers 11 days ago
mmhamdy 
posted an update 12 days ago
view post
Post
2651
What if you could train a model on just 10 images instead of 60,000 and still get close to the same performance?

Traditional machine learning requires thousands, even millions, of data points to achieve high accuracy. But what if we could "distill" the entire dataset into just a few synthetic samples?

This is what Dataset Distillation offers. Unlike traditional knowledge distillation, we keep the model fixed and distill the knowledge contained in a massive training set into a tiny set of synthetic distilled images.

The goal is to train a model on this ultra-small set and achieve performance that almost matches what the same model would get when trained on the massive original dataset.

For example, training on only 10 distilled MNIST images (this is equivalent to a single image per class) yields 94% accuracy, compared to 99% when training on the full 60,000 images.

Interestingly, these distilled images look significantly different (as you can see in the image below) from natural images because they are optimized for model training rather than for matching the correct data distribution.

But that's not all.

Most importantly, this same method opens the door to a potent form of data poisoning. Because distilled images are specifically optimized for rapid learning, an attacker can create a tiny set of adversarial distilled images to cause a well-trained model to forget or misclassify a specific category.

What I find fascinating about dataset distillation is this: it mimics human-like learning by letting a model grasp a concept from a single example, but it does so using alien synthetic images that mean absolutely nothing to a human eye!

What about you? What are your thoughts on it?
  • 2 replies
·
Reubencf 
posted an update 17 days ago
view post
Post
2049
Millions speak Konkani. The internet barely knows it.

Today's major LLMs struggle with regional languages. They can't read, write or even recognize Konkani. So I built one that can.

Here is a working demo of the Konkani LLM I've been training. 👇

https://youtu.be/8K04ylbXh6k
mmhamdy 
posted an update 18 days ago
view post
Post
4970
It was supposed to be a failed experiment. Instead, it led to the discovery of one of the most intriguing phenomena in neural networks, simply because a researcher forgot to turn it off and left it running....for a week!

In 2022, researchers at OpenAI were studying how neural networks generalize from their training data. For this task, they were training small transformer models to perform modular arithmetic.

The thing is, neural networks are weird. When a model has an abundance of parameters (like neural nets), it can easily overfit. It essentially memorizes its training data, scoring a perfect 100% accuracy when tested on it, but remains completely clueless when faced with any new instances not present in the training set (close to 0 accuracy). It is like memorizing 1 + 2 = 3 without understanding the concept of addition, so if 2 + 3 wasn't in the training set, the model fails miserably!

Usually, when a model overfits like this, people just cut their losses, turn off the experiment, and move on with their lives.

But sometimes they forget. And that is exactly what happened to our researchers at OpenAI. A week later, they checked back in, and a miracle had happened!

They discovered Grokking (And no, this has nothing to do with xAI's Grok , the term was originally coined by sci-fi author Robert Heinlein to mean understanding something so deeply that it becomes part of you). Grokking is when a neural network suddenly and abruptly learns to generalize long after it has overfitted. Just take a look at the graph in the image below!

Spooky, right! I told you neural nets are weird!
  • 4 replies
·
mmhamdy 
posted an update 21 days ago
view post
Post
1022
Human brains don't recreate every pixel to understand the world!

Most current models in genomics, proteomics, and single-cell transcriptomics rely on generative objectives like masked language modeling or next token prediction. While effective, these architectures waste significant capacity reconstructing raw, noisy sequence details that may not carry functional biological meaning.

But a promising, more efficient alternative is emerging: Joint-Embedding Predictive Architecture (JEPA)

Originally introduced by Yann LeCun for computer vision, JEPA is a non-generative, self-supervised learning (SSL) framework. Instead of predicting raw inputs, it operates as a world model that predicts abstract semantic embeddings in latent space.

Recently, the JEPA framework (and its more efficient LeJEPA variant) has been adapted into the biological sciences to develop performing foundation models and to improve on already existing ones.

It's interesting how each adaptation modified and tailored JEPA to suit its specific biological domain, whether by experimenting with different backbones or complementing the objective with other loss terms.

For example, JEPA-DNA and ProteinJEPA used JEPA as a continual pre-training framework to enhance existing foundation models without training from scratch, while Cell-JEPA and JEPA-DNA employed a hybrid objective that combines the JEPA loss with a traditional language modeling loss.

The article below provides an overview of these implementations, along with others that came out this year. As always, your thoughts and feedback are welcome and highly appreciated!

Link to the article is in the first comment 👇
  • 3 replies
·
bryanlimy 
authored a paper 22 days ago
mmhamdy 
posted an update 29 days ago
view post
Post
139
Things rarely go as we expect!

In 2017, Google released the Transformer architecture. While it was clear the model was promising, absolutely no one (including its authors) anticipated the pervasive global revolution it would create!

The authors actually viewed the Transformer as just a stepping stone for a much more ambitious project: The MultiModel.

Their ultimate goal was to build a single deep learning architecture capable of jointly learning massive, diverse tasks across entirely different domains (in 2017). A One Model To Learn Them All.

In fact, the MultiModel paper was published in the exact same month as Attention Is All You Need!

But history had other plans. The building block eclipsed the grand design!

So, have you heard about the MultiModel before? 😀
  • 1 reply
·
alielfilali01 
posted an update 29 days ago
view post
Post
552
Plans in HTML > Plans in Markdown
mridul3301 
authored 2 papers about 1 month ago
Reubencf 
posted an update about 1 month ago
view post
Post
4696
I have improved my Portfolio please do check it out
Reubencf/Portfolio
  • 6 replies
·
Tonic 
posted an update about 1 month ago
view post
Post
2963
🙋🏻‍♂️ Hey there folks ,

Turns out : if we predict 🌏 earth we can save a lot of time looking for interesting things and less time looking at things that we expect to see.

Sentinel-2 imagery 🛰️basically takes a long time to download towards earth. so our "near real time" systems are quite far from that in practical terms.

meanwhile , if we "predict" what we will see , based on what we do see , we can send down much less data in a timely way , and prioritize 📡earth-bound response .

I'm talking about illegal fishing , logging , mining or building in nature reserves , the more of that we predict early the more we're able to stop it on time.

At least that's the concept !

check out the blog : https://huggingface.co/blog/Tonic/save-patagonia-by-predicting-earth


- Collection: https://huggingface.co/collections/NuTonic/earth-observation-with-temporal-and-general-understanding
- Code: https://github.com/Josephrp/Nutonic
- Dataset: NuTonic/sat-vl-sft-training-ready-v1
- Model: NuTonic/lspace
- Training: NuTonic/lspace-trackio
- Evals: NuTonic/Patagonia_Eval
  • 2 replies
·
Aurelien-Morgan 
posted an update about 2 months ago
view post
Post
1095
@retrain-pipelines v0.2.0 is out !
I'm at Station F at My booth with GOSIM Paris 2026 today & tomorrow.
Come meet me for a live in-person demo and a chat !
  • 1 reply
·