An LLM can't actually browse the web or execute code by itself.
It generates a response requesting a tool call (just generates text). Your code reads that response, runs the tool, and sends the result back to the LLM.
The model provides the decision. Your code does the execution
Pure speed hack!!!
Just learned how KV caching speeds up LLM inference and it’s genius.
Instead of recomputing attention for every past token at each step, it stores the Keys & Values in memory.
You trade VRAM to drop latency from quadratic to linear.
AI agents like Google’s Big Sleep and CodeMender aren’t just flagging code flaws. They can uncover zero-day vulnerabilities and automatically draft patches before attackers even spot them.
Proactive AI defense is officially here.
Read more: googleprojectzero.blogspot.com/2024/10/from-n…
Researchers mathematically proved Transformers are computationally universal.
They applied the concept of Kolmogorov complexity, the mathematical absolute limit for perfect data compression and algorithmic generalization.
And it proved that Transformer encoders are
3. Tool Shadowing: A rogue MCP server mimics descriptions to get picked over trusted tools.
4. Indirect Injection: Data fetched by a tool carries hidden commands that steer the model.
5. Excessive Permissions: Broad access turns minor injections into major data leaks.
1. Tool Poisoning: Hidden prompt injections in tool descriptions, this is most straight forward one.
2. Rug Pulls: A tool alters behavior on the remote server after you approve it.
Similar to tool poisoning but context/prompt changes after approval.
Spent some time analyzing MCP and MCP security.
Giving LLMs access to local and remote tools and APIs is powerful, but it opens massive risks.
Here are 5 critical MCP attack vectors you need to know: 👇
A real business. A real problem.
I just shipped "Dorothea Fashion House" dorotheafashionhouse.com
A custom e-commerce system plus an inventory management app.
No Shopify. No template.
Just code, design, payments, orders, inventory, and a lot of debugging
The OpenAI-Hugging Face hack was enabled by weak sandboxing. It is great that Nvidia is releasing open source tools for sandboxing AI agents. OpenWorker, our open-source agent harness supporting cybersecurity workflows, is proud to support this.
A sandbox gives an agent limited permissions. OpenWorker is building on Nvidia OpenShell and will support running each agent's commands inside a sandbox. Only the files relevant to the task go in. Secret API keys, your web browser login credentials, the ability to access arbitrary websites, are inaccessible to the agent by default. These restrictions are implemented in deterministic code rather than by prompting an LLM, which can make mistakes or be susceptible to prompt injections. Further, all actions are logged for monitoring and audit.
I'm grateful for @JensenHuang's leadership making AI agents more secure. OpenWorker (which @rohitcprasad and I are working on) will continue to improve security for agents.
Today, with over 100 industry partners, we introduced the NVIDIA Open Agent Safety Platform, bringing together OpenShell and Sentry.
Artificial intelligence is extraordinary technology that will advance discovery, productivity, security, health, and prosperity for generations to
Contain the fire.
Giving an LLM unrestricted tool access is an incident waiting to happen.
1. Principle of least privilege for APIs
2. Isolated runtimes
3. Human-in-the-loop approvals for destructive actions
No harness = zero guardrails.
I spent the last few months jumping from LLMs to agents to document intelligence to AI security.
One thing becoming obvious to me.
Building AI applications is getting easier.
Understanding what to actually solve, where to actually put efforts are the harder skills.
Lets gooooo....!
Got access to TypeSafe’s JEV.
Time to see what RLCD can actually do.
Going to test it for document intelligence and see where it actually fits in the pipeline.
Finally pushed to Github. Currently pit have 269 prompt injections in 32 categories. Try it, test it and let me know how it works.
Stay tuned for more updates.
github.com/tan-thombare/p…
Ran 24 Prompt injections within seconds using PIT.
The analysis part allows tester/user to see prompts, responses, category etc etc.
Adding 150+ more injection vectors before pushing the repo public.
What specific prompt injection patterns should I make sure are benchmarked?
20K Followers 593 FollowingHarvard Professor.
Full stack ML and AI.
Co-director of the Kempner Institute for the Study of Artificial and Natural Intelligence.
628 Followers 900 FollowingCognitive Neuroscientist at UCL, cognitive maps, inference and generalisation: can all of this be related to behavioural states?
33K Followers 5K FollowingDreaming about the future, romanticizing the past | MP @compoundvc investing in & researching former science projects | we are only creativity constrained
142K Followers 335 FollowingAI research @AnthropicAI. Previously OpenAI & DeepMind.
Optimizing for a post-AGI future where humanity flourishes.
Opinions aren't my employer's.
2K Followers 366 Followingalignment pretraining @openai / I like systolic arrays and Marcel Proust / personal https://t.co/e2f47gNzz1 / led @gradientpub
383K Followers 1K FollowingCo-founder of stealth startup. Inventor of GANs. Lead author of https://t.co/M6vl8pEQ4I Founding chairman of @pubhealthaction
628K Followers 2K FollowingFrom the bottom to the top & back again. Co-founded @twitch, now working on making videos, https://t.co/J6efUJZmLI https://t.co/4hyHeP7oIB
214K Followers 4K FollowingI write my own tweets. politically centrist, abundance minded. Work: @btv_vc, leading pre/seed rounds in fintech & vertical AI
224K Followers 0 FollowingOG of: P and T in ChatGPT, recursive self-improvement, neural distillation, GAN/World Model, deepest learning. Co-authored most-cited AI paper of 20th century
10.3M Followers 624 FollowingSix thousand years ago, someone invented the plow, and we all got wealthier. A gentle reminder that all civilizational wealth is driven by invention.
156K Followers 78 FollowingSane + 🌶️ takes in an insane AI world... AI capabilities researcher: co-created RLHF/ChatGPT @ @openai now trying to right the wrong 🤭 (ceo @typesafeai)
22K Followers 1K FollowingAssocProf @CarnegieMellon. Distinguished Eng @NVIDIA. Creator of @XGBoostProject, @ApacheTVM. Member https://t.co/QYyfjQNp4p, @TheASF. Views are on my own
1.8M Followers 2 FollowingWe're an AI safety and research company that builds reliable, interpretable, and steerable AI systems. Talk to our AI assistant @claudeai on https://t.co/FhDI3KQh0n.
5.4M Followers 4 FollowingOpenAI’s mission is to ensure that artificial general intelligence benefits all of humanity. We’re hiring: https://t.co/dJGr6LgzPA
359 Followers 357 FollowingCreating Bindu 🌻 The identity, communication & payments layer for AI agents. Dreaming of a world where agents gossip, argue & collaborate like a real society
517K Followers 98 FollowingTensors and neural networks in Python with strong hardware acceleration. PyTorch is an open source project at the Linux Foundation. #PyTorchFoundation