KP @kexecv

LLMs, Infosec, Programming, Science & Technology kuldippatel.dev Singularity Joined June 2012

Tweets

989
Followers

575
Following

957
Likes

64K

KP @kexecv

3 days ago

YALM-130M evaluation results are here! The extra time and cost($70) didn't pay off. Results are barely better than YALM-80M. But I am not disappointed as failure is just fuel for the next attempt! 💪 Model: huggingface.co/kp7742/YALM-13…

KP @kexecv

4 days ago

2 0 12 2K 2

Download Image

3 1 15 2K 5

Download Image

KP @kexecv

4 days ago

Finally! 🚀 My long-pending task is complete! YALM-130M training finished in 43 hours. It was a last moment decision to scale up from 80M to 130M. Was it worth the extra time and cost? We'll find out tomorrow after the evaluation results. Stay tuned! (Ignore those spikes 😇)

KP @kexecv

3 months ago

5 1 11 1K 1

2 0 12 2K 2

Download Image

KP @kexecv

4 days ago

Something is cooking; wish me luck :)

1 0 3 194 0

Download Image

KP @kexecv

a month ago

Excited to share a new open-source project I made for quickly testing LLM architectures! It's heavily based on nanoGPT and other OSS repos. Currently supports single GPU only. This is separate from the YALM project, which uses the HuggingFace Trainer. github.com/kp7742/gpt-tra…

0 1 8 269 1

KP @kexecv

a month ago

There are rumours of TikTok coming back to India. Almost everybody is talking about what they will make once it comes. I'm rather interested in the juicy training data it will generate.

0 0 2 130 0

KP @kexecv

2 months ago

I'm so mid.

Yacine Mahdid @yacinelearning

2 months ago

I'm so mid. https://t.co/9OE6ZWC5d1

43 2 303 27K 9

Download Image

1 0 4 362 0

Download Image

KP @kexecv

2 months ago

That’s what we call quality content, a must read.

tokenbender @tokenbender

2 months ago

That’s what we call quality content, a must read.

62 160 2K 296K 2K

Download Image

0 0 8 284 0

KP @kexecv

2 months ago

5-TPG

OpenAI @OpenAI

2 months ago

5-TPG

2K 3K 24K 6.9M 2K

0 0 2 139 0

KP @kexecv

2 months ago

deepmind and openai cooked hard but somebody got sidelined 😏

Bhavya Goyal @777BHAVYA

2 months ago

deepmind and openai cooked hard but somebody got sidelined 😏

2 0 5 425 0

0 0 3 124 0

Neel Nanda @NeelNanda5

2 months ago

Take: Chain of Thought is a misleading name. It's really a "scratchpad". "Thoughts" are internal activations Imagine you're solving a problem and have a scratchpad. Reading the pad gives me info! You *can* avoid writing down key thoughts. But it's a handicap. Real but fallible

33 38 632 50K 167

KP @kexecv

2 months ago

I have been busy with understanding GRPO. Experimenting different ways and tuning parameters to bootstrap reasoning in small models without additional training. Getting some promising results with SmolLM2-135M. Also YALM-80M pretraining is delayed, but still on the roadmap.

2 0 6 338 0

Download Image

KP @kexecv

3 months ago

Just tried Instruction tuning today! I finetuned SmolLM-135M base model using a custom chat format on the SmolTalk dataset for one epoch. I even built a Gradio web chat for easy testing of models. It's great for simple instructions, but not so much for math and tool calls!

0 0 7 348 1

Download Image

KP @kexecv

3 months ago

Tbh, this delay is definitely due to the Kimi K2 release, but not for the reason many people believe. The release of such a SOTA open model would steal the spotlight from their model release. Regardless of its size, everyone would compare it to larger open models. They were…

Sam Altman @sama

3 months ago

2K 1K 19K 3.5M 2K

0 0 3 389 0

KP @kexecv

3 months ago

Attention or linear models alone cannot fix everything; hybrid models are the way forward.

Maxime Labonne @maximelabonne

3 months ago

Attention or linear models alone cannot fix everything; hybrid models are the way forward.

32 107 697 53K 423

Download Image

1 0 6 187 0

KP @kexecv

3 months ago

Tokenizer test results are here! RTL digit tokenization didn't do as well as expected, maybe due to inconsistency. Interestingly, individual digits performed best, likely because of better compression. SuperBPE, despite a higher training loss, did great on benchmarks.

KP @kexecv

3 months ago

2 0 10 1K 0

Download Image

2 2 12 451 1

Download Image

KP @kexecv

3 months ago

Tested Grok-4 on OpenRouter because couldn't afford SuperGrok🫣. Still the same old Grok3 base model, although I expected more. Good at following instructions and takes quite open stand, but I wish I could see its "thinking" process.

0 0 2 270 0

Download Image

KP @kexecv

3 months ago

SuperBPE is quite interesting! It effectively compresses English and Code with just a 32k vocab. But it doesn't compress non-English languages like Hindi that well. Probably I should have tested bigger vocab size but I was resource constraint.

1 0 5 186 0

Download Image

KP @kexecv

3 months ago

Added SuperBPE with digit tokenization tests. It had mixed results; some positive and some negative. I'll share more details soon! Paper: arxiv.org/pdf/2503.13423

KP @kexecv

3 months ago

Added SuperBPE with digit tokenization tests. It had mixed results; some positive and some negative. I'll share more details soon! Paper: arxiv.org/pdf/2503.13423 https://t.co/G9VjNwkHai

2 0 10 1K 0

Download Image

0 0 4 90 0

Download Image

KP @kexecv

3 months ago

Final test run needs 5 more hours, then i need to quickly publish my eval results! Don't want Grok 4 hype to overshadow them. Exciting AI weekend ahead!