Hugging Face's logo
AbstractPhila's picture
🤝 Open to Collab

AbstractPhila PRO

AbstractPhil
AbstractPowered
19 6 30
P(doom) Rejects the premise
Sarah54321's profile picture NitrAI's profile picture tyro12's profile picture
·
https://civitai.com/user/AbstractPhila

AI & ML interests

datasets, research papers, experimentation, vision, classification, text encoders, tokenization, llms, diffusion, distillation, and more.

Recent Activity

repliedto their post about 3 hours ago
Now that the Beatrix recall problem will soon be resolved, I can move forward to teaching her useful information related to pragmatism. Beatrix V3 has a lot of trouble with communication, but the model herself is entirely filled with information. With minimal processing and curation, this information will be extractable. It's not the most robust information, but we can say for certain that the information is most definitely represented usefully in a way that can be resurfaced, purposed, utilized, enhanced, expanded, and reduced as necessary. With the AMOE directly hosting and supporting the softmax attention behavioral arms, it brought me a new form of understanding and respect for token recall at depth. I knew Beatrix would be able to represent the data, but what I got was very unexpected. This model when those numbers don't line up, she simply speaks gibberish. Absolute gibberish as shown in the 2s softmax control variant. That model turned into a spinning top that flew off the rails, so if you query that model you will only see gibberish. With those numbers lined up, she speaks... I don't know how to describe it. It's not gibberish, but it's definitely not, expected. Like an understanding of confusion. There are multiple behaviors that her arms are meant to train which are currently devoid, and in some aspects the arms work perfectly like the caption arm. If you use the 3s model with ONLY the caption arm, she will complete a simple caption for you. It may not be very good, and she's not the best model for it, but she most definitely can do it. Don't need all 9 arms, but it doesn't hurt to have them on. Just takes too long. SOFTMAX attention however, inferenced at bf16 this should be fine. Lightning quick you could say. TRAINING I have to train the softmax attention at fp32 forward with some rounding black magic for backward. Otherwise the v3 model simply implodes. There have been a huge series of problems related to her in the coming writeup.
posted an update about 4 hours ago
Now that the Beatrix recall problem will soon be resolved, I can move forward to teaching her useful information related to pragmatism. Beatrix V3 has a lot of trouble with communication, but the model herself is entirely filled with information. With minimal processing and curation, this information will be extractable. It's not the most robust information, but we can say for certain that the information is most definitely represented usefully in a way that can be resurfaced, purposed, utilized, enhanced, expanded, and reduced as necessary. With the AMOE directly hosting and supporting the softmax attention behavioral arms, it brought me a new form of understanding and respect for token recall at depth. I knew Beatrix would be able to represent the data, but what I got was very unexpected. This model when those numbers don't line up, she simply speaks gibberish. Absolute gibberish as shown in the 2s softmax control variant. That model turned into a spinning top that flew off the rails, so if you query that model you will only see gibberish. With those numbers lined up, she speaks... I don't know how to describe it. It's not gibberish, but it's definitely not, expected. Like an understanding of confusion. There are multiple behaviors that her arms are meant to train which are currently devoid, and in some aspects the arms work perfectly like the caption arm. If you use the 3s model with ONLY the caption arm, she will complete a simple caption for you. It may not be very good, and she's not the best model for it, but she most definitely can do it. Don't need all 9 arms, but it doesn't hurt to have them on. Just takes too long. SOFTMAX attention however, inferenced at bf16 this should be fine. Lightning quick you could say. TRAINING I have to train the softmax attention at fp32 forward with some rounding black magic for backward. Otherwise the v3 model simply implodes. There have been a huge series of problems related to her in the coming writeup.
updated a model about 5 hours ago
AbstractPhil/alephllm-mini-beatrix-training
View all activity

Organizations

DeepGHS's profile picture Blog-explorers's profile picture BangumiBase's profile picture Abstract Powered Research's profile picture