KP @kexecv

LLMs, Infosec, Programming, Science & Technology kuldippatel.dev Singularity Joined June 2012

Tweets

976
Followers

574
Following

929
Likes

62K

KP @kexecv

a week ago

Excited to share a new open-source project I made for quickly testing LLM architectures! It's heavily based on nanoGPT and other OSS repos. Currently supports single GPU only. This is separate from the YALM project, which uses the HuggingFace Trainer. github.com/kp7742/gpt-tra…

0 0 5 139 0

KP @kexecv

2 weeks ago

There are rumours of TikTok coming back to India. Almost everybody is talking about what they will make once it comes. I'm rather interested in the juicy training data it will generate.

0 0 2 96 0

KP @kexecv

4 weeks ago

I'm so mid.

Yacine Mahdid @yacinelearning

4 weeks ago

I'm so mid. https://t.co/9OE6ZWC5d1

43 2 307 27K 9

Download Image

1 0 4 324 0

Download Image

KP @kexecv

4 weeks ago

That’s what we call quality content, a must read.

tokenbender @tokenbender

4 weeks ago

That’s what we call quality content, a must read.

60 160 2K 290K 2K

Download Image

0 0 8 253 0

KP @kexecv

a month ago

5-TPG

OpenAI @OpenAI

a month ago

5-TPG

2K 3K 24K 6.9M 2K

0 0 2 123 0

KP @kexecv

a month ago

deepmind and openai cooked hard but somebody got sidelined 😏

Bhavya Goyal @777BHAVYA

a month ago

deepmind and openai cooked hard but somebody got sidelined 😏

2 0 5 406 0

0 0 3 108 0

Neel Nanda @NeelNanda5

a month ago

Take: Chain of Thought is a misleading name. It's really a "scratchpad". "Thoughts" are internal activations Imagine you're solving a problem and have a scratchpad. Reading the pad gives me info! You *can* avoid writing down key thoughts. But it's a handicap. Real but fallible

33 37 635 50K 167

KP @kexecv

2 months ago

I have been busy with understanding GRPO. Experimenting different ways and tuning parameters to bootstrap reasoning in small models without additional training. Getting some promising results with SmolLM2-135M. Also YALM-80M pretraining is delayed, but still on the roadmap.

2 0 6 324 0

Download Image

KP @kexecv

2 months ago

Just tried Instruction tuning today! I finetuned SmolLM-135M base model using a custom chat format on the SmolTalk dataset for one epoch. I even built a Gradio web chat for easy testing of models. It's great for simple instructions, but not so much for math and tool calls!

0 0 7 327 1

Download Image

KP @kexecv

2 months ago

Tbh, this delay is definitely due to the Kimi K2 release, but not for the reason many people believe. The release of such a SOTA open model would steal the spotlight from their model release. Regardless of its size, everyone would compare it to larger open models. They were…

Sam Altman @sama

2 months ago

2K 1K 20K 3.5M 2K

0 0 3 376 0

KP @kexecv

2 months ago

Attention or linear models alone cannot fix everything; hybrid models are the way forward.

Maxime Labonne @maximelabonne

2 months ago

Attention or linear models alone cannot fix everything; hybrid models are the way forward.

32 107 698 52K 424

Download Image

1 0 6 174 0

KP @kexecv

2 months ago

Tokenizer test results are here! RTL digit tokenization didn't do as well as expected, maybe due to inconsistency. Interestingly, individual digits performed best, likely because of better compression. SuperBPE, despite a higher training loss, did great on benchmarks.

KP @kexecv

2 months ago

2 0 10 1K 0

Download Image

2 2 12 435 1

Download Image

KP @kexecv

2 months ago

Tested Grok-4 on OpenRouter because couldn't afford SuperGrok🫣. Still the same old Grok3 base model, although I expected more. Good at following instructions and takes quite open stand, but I wish I could see its "thinking" process.

0 0 2 262 0

Download Image

KP @kexecv

2 months ago

SuperBPE is quite interesting! It effectively compresses English and Code with just a 32k vocab. But it doesn't compress non-English languages like Hindi that well. Probably I should have tested bigger vocab size but I was resource constraint.

1 0 5 177 0

Download Image

KP @kexecv

2 months ago

Added SuperBPE with digit tokenization tests. It had mixed results; some positive and some negative. I'll share more details soon! Paper: arxiv.org/pdf/2503.13423

KP @kexecv

2 months ago

Added SuperBPE with digit tokenization tests. It had mixed results; some positive and some negative. I'll share more details soon! Paper: arxiv.org/pdf/2503.13423 https://t.co/G9VjNwkHai

2 0 10 1K 0

Download Image

0 0 4 82 0

Download Image

KP @kexecv

2 months ago

Final test run needs 5 more hours, then i need to quickly publish my eval results! Don't want Grok 4 hype to overshadow them. Exciting AI weekend ahead!