Lilac, joining Databricks! @lilac_ai
Curate better data for LLMs. We are now joining @databricks. Github: https://t.co/DHtc0lOTii lilacml.com Joined April 2023-
Tweets41
-
Followers2K
-
Following3
-
Likes28
We are thrilled to announce that @lilac_ai is joining Databricks! With the integration of Lilac’s powerful tools for data exploration, customers can accelerate the development of production-quality generative AI apps using their enterprise data. #genAI dbricks.co/3IF0uHL
Finally have some free time! Translated 1M @Teknium1 OpenHermes-2.5 samples to Vietnamese, then used @lilac_ai to reduce to 25k. Result: 25k-trained model scored the same as 1M on Vietnamese VLSP Benchmark. REALLY GOOD DATA IS ALL YOU NEED🚀 huggingface.co/datasets/ontoc…
Explore the GAIR-lima fine tuning dataset from the Mosaic blog post below using our Lilac @huggingface space! lilacai-lilac.hf.space/datasets#lilac…
Explore the GAIR-lima fine tuning dataset from the Mosaic blog post below using our Lilac @huggingface space! lilacai-lilac.hf.space/datasets#lilac…
lilac v0.3.5 released with support for @nomic_ai 1.5 embeddings and BGE embeddings. We've also made it easier to add text selections to concepts. github.com/lilacai/lilac/…
With such a warm recommendation, I had to get @lilac_ai folks on stage, and here is our conversation and a deep dive into dataset creation, curation and classification. Just posted @thursdai_pod deep dive into Lilac and RWKV. Links as always, first comment 👇
With such a warm recommendation, I had to get @lilac_ai folks on stage, and here is our conversation and a deep dive into dataset creation, curation and classification. Just posted @thursdai_pod deep dive into Lilac and RWKV. Links as always, first comment 👇 https://t.co/fdR0a2FYmJ
Embedding-filtered version of reddit-instruct reddit-instruct is a QA dataset gathered from Reddit posts/comments, inspired by LIMA. This version filters it further using @lilac_ai's platform, preventing non-instruction/QA data from being included huggingface.co/datasets/eucla…
OpenHermes-2.5 dataset is finally here! We're hosting the full dataset with pre-computed @lilac_ai clusters in our demo. Clusters: lilacai-lilac.hf.space/datasets#lilac…
OpenHermes-2.5 dataset is finally here! We're hosting the full dataset with pre-computed @lilac_ai clusters in our demo. Clusters: lilacai-lilac.hf.space/datasets#lilac… https://t.co/5ymJhNoXeV
If you're doing a lot of fine-tuning and dataset curation, definitely make sure to check out Lilac Garden. They were nice enough to run Capybara through it before official release and allowed me to see interesting insights that normal embedding clustering typically fails to show.
If you're doing a lot of fine-tuning and dataset curation, definitely make sure to check out Lilac Garden. They were nice enough to run Capybara through it before official release and allowed me to see interesting insights that normal embedding clustering typically fails to show. https://t.co/x9sK8idWn5
[email protected] is now released. We now support: - Loading custom embeddings - Exporting directly to HuggingFace Release notes: github.com/lilacai/lilac/…
We just released 0.2.3 which allows you to share Lilac-processed datasets via @huggingface from the CLI. Expensive embeddings, signals, and map results can now be shared with others. We'll be releasing some Lilac-processed datasets soon. Docs: docs.lilacml.com/datasets/datas…
New release: 0.1.25 This drops dask for a thin multi-processing client, and comes with lots of performance improvements, namely the slow import time of lilac. We have also added a simpler API for loading from HuggingFace & some purple UI flare🪻 github.com/lilacai/lilac/…
We just released [email protected]! This release brings in Monaco (the VSCode engine) to render documents, with powerful context menus for searching, labeling concepts, and more. We also added UI support for common ChatML formats like ShareGPT Release notes: github.com/lilacai/lilac/…
We're excited to announce a new way to ✨edit data✨ for AI models. We use the @astral_sh ruff formatter to edit code in the @GlaiveAI dataset to align a coding assistant to our style. Blog: docs.lilacml.com/blog/curate-co… Video: youtu.be/bw8JUpAOSZQ
Michael Yongchoel Cho.. @yc_choi
629 Followers 3K Following Experienced Tech Leader Building AI and Cloud Solutions for All
Dr. Carmen Martinez @drcarmar
9 Followers 110 Following Sr. Conversational UX Designer at Flix SE Conversation Analysis | CUX Design | Generative Artificial Intelligence
Anushé Sheikh @anushesheikh
54 Followers 607 Following product manager @google responsible ai | 🔎 LLM safety, fairness, equity, socio-technical harm | previously @youtube
Matt Carey @mattzcarey
622 Followers 1K Following Journey with 1s and 0s - ai at @StackOneHQ - genai london organiser - building https://t.co/yt7xbSVE9P for my mum
Simon Suo @disiok
2K Followers 1K Following co-founder @llama_index prev: AI research @Waabi_ai @Uber_ATG, PhD AI @UofT
Dalton @Dalton805268014
8 Followers 18 Following
redefinedbyai @redefinedbyai
26 Followers 246 Following Guess what, my predictions on AI are happening one projects at a time 😉
Sina Jahan @thesinajahan
36 Followers 545 Following
quantitative_meming @ReclineAt5pm
774 Followers 2K Following Data plumber. The word "neoliberal" gives me semantic satiation.
naklecha @naklecha
5K Followers 2K Following ai @glaiveai + research @aaaaaaaaaaorg + art projects and silly stuff @notpinkxyz -- collecting wins and moving goalposts
louis030195 @louis030195
1K Followers 2K Following ai defense founder | ex 🕵️ | top secret clearance | leukemia survivor @ 13 | 🥊 muay thai | @techstars | @OrangeDAOxyz | 🤖 LLMs since gpt-1
@weinberger.org at bl.. @dweinberger
25K Followers 2K Following I write/speak about how AI, the Net, etc.are shaping our ideas. Latest book: Everyday Chaos. Delirious pa and grandpa. (he/him) @[email protected]
xdjackfeng @xdjackfeng
31 Followers 482 Following
Bob Brauer @sfhomebase
1K Followers 2K Following Founder (https://t.co/zxMCW78SbM), an AI-Powered data matching and data quality platform revolutionizing data usability with Generative AI - Cloud DBs & Files
Oscar Moll @oscar_moll_
569 Followers 828 Following Curioso acerca de muchas cosas. CS PhD grad from @MIT_CSAIL. Software engineer in a past life. figuring out what's next. De Cobán, Guatemala
Sajid Mehmood @smehmood
997 Followers 669 Following
Julia Neagu @JuliaANeagu
664 Followers 965 Following CEO & Co-Founder @QuotientAI ✨ formerly @GitHub @GitHubCopilot 🤖 reformed physicist 👩🔬 ~ opinions are my own ~
Felix @felix_red_panda
3K Followers 2K Following CS Student, speech synthesis and LLM nerd, DMs open
SenaBeren @findingmerit
287 Followers 3K Following
Brad Larson @followbl
823 Followers 526 Following co-founder @metaintro - https://t.co/XqC2XBa0yy is my life
Ganeshkumar Ashokavar.. @ganeshkumar_av
160 Followers 948 Following Software Engineer @Microsoft's Azure Kubernetes Service | @ucberkeley '21 - EECS and Business | @kleinerperkins Fellow | @accel Scholar.
Lashaw Salta @LashawSalta
39 Followers 103 Following
Mark Robinson @robinsonmarkj
39 Followers 2K Following
ChaseAI @ChaseAi87387
93 Followers 653 Following Founder of https://t.co/thlutvubaA and https://t.co/SK2IMkd4JT. Wrote my first children's book: Addie & The Magic Jar Building with AI. Creator, Marketer
Irwan Bello @IrwanBello
6K Followers 2K Following Supercomputers & Friends AGI research & products ex @OpenAI, founding team @character_ai
Jess Holbrook @jessscon
1K Followers 1K Following Principal Researcher at Meta. Formerly co-lead of the People + AI Research (PAIR) team at Google. Board Member, Forging Youth Resilience.
Grace Journal @GraceJournal1
22 Followers 139 Following The only student run, ethics and AI journal at Stanford that focuses on global issues and impacts
Hezar @jenrik_tj
131 Followers 1K Following
Taylor Brown @thetaytay
726 Followers 990 Following Cofounder and Technical Fellow at https://t.co/3dX8Xz9Hca
Cristiano Giardina @CrisGiardina
1K Followers 3K Following Writing mostly about AI · "That most limited of all specialists, the well-rounded man."
Sparsh Jain @sparshjain21
61 Followers 779 Following Research Intern @AI4Bharat, IIT Madras || Ex- Data Science Intern @Culinda || Data Science || ML enthusiast
Lakshya A Agrawal @LakshyAAAgrawal
271 Followers 2K Following #GenAI AI4Code Research Fellow @MSFTResearch | Past: SWE Intern @Microsoft, Summer @EPFL Scholar, CS and Applied Maths @IIITDelhi | Hobbyist Saxophonist
La Bomba @Stribelidadel
27 Followers 872 Following Technology enthusiast. Long time Msft supporter. Respect mother Nature. Living to see the world becoming a better place 🙏🏾
Kranti @KrantiAsapu
118 Followers 719 Following Architect | eager to learn new things. Planet earth 🌏 is not boring when amazing people are in.
David Vernal @dgvernal
150 Followers 2K Following
ƒ @fjlin
93 Followers 5K Following
Nero @jie_nero
29 Followers 1K Following
Abhinav Gupta @backpropper
793 Followers 5K Following phd student @Mila_Quebec | ms @CILVRatNYU @NYU_Courant | previously @GoogleDeepMind @AIatMeta @GoogleAI @labsdotgoogle @MSFTResearch @AdobeResearch
accelerate fit for pu.. @halftroll
600 Followers 4K Following Programmer/builder seeing world through lens of climate emergency, yet thrilled by the positive aspects of AI and general purpose technology @[email protected]
Laxman Munigala @LaxmanMunigala
70 Followers 766 Following
Databricks @databricks
70K Followers 1K Following Databricks is the data and AI company, helping data teams solve the world’s toughest problems.
Daniel Smilkov @dsmilkov
7K Followers 993 Following Building @lilac_ai with @nsthorat. Past: Co-created Know Your Data & TensorFlow.js. Ex-PAIR/Google Brain. 🇲🇰🇺🇸 . ML & Visualization
Nikhil Thorat @nsthorat
10K Followers 2K Following Co-founder of Lilac AI (@lilac_ai), now joining @databricks. Past: Co-created TensorFlow.js and Know Your Data. Google Brain // PAIR // Responsible AIFinally have some free time! Translated 1M @Teknium1 OpenHermes-2.5 samples to Vietnamese, then used @lilac_ai to reduce to 25k. Result: 25k-trained model scored the same as 1M on Vietnamese VLSP Benchmark. REALLY GOOD DATA IS ALL YOU NEED🚀 huggingface.co/datasets/ontoc…
Here's a release of a dataset inspired by @GoogleDeepMind's Self Discover paper (arxiv.org/pdf/2402.03620…). It is an incredibly dumbed down implementation, but resulted in a rather robust dataset that achieved my project's objectives. I will release more data once the project it…
What should you do if you want to effectively and cheaply “instruction finetune” an LLM? @aditi_jh and @JacobianNeuro share some important insights. (1/5)
Once the captions parquet file was generated, I could inspect the synthetic samples through a pleasant UI by @lilac_ai. I wanted to quickly comb through samples for a quick human quality check and it was nice to see descriptive captions as I expected.
With such a warm recommendation, I had to get @lilac_ai folks on stage, and here is our conversation and a deep dive into dataset creation, curation and classification. Just posted @thursdai_pod deep dive into Lilac and RWKV. Links as always, first comment 👇
Everyone working with LLM Datasets should check out @lilac_ai's data platform. Embeds your dataset, helps with classifying, clustering, modifying, getting insights, and a lot more. Runs locally or hosted too, even the gpu poor can use it! Their clustering helped determine a lot…
Embedding-filtered version of reddit-instruct reddit-instruct is a QA dataset gathered from Reddit posts/comments, inspired by LIMA. This version filters it further using @lilac_ai's platform, preventing non-instruction/QA data from being included huggingface.co/datasets/eucla…
In addition, I've worked with @nsthorat and the Lilac team to cluster the datasets into clusters for analysis and further curation! You can access the dataset in Lilac's @huggingface Spaces here: lilacai-lilac.hf.space/datasets#lilac… And can access the clusters by clicking this button:
If you're doing a lot of fine-tuning and dataset curation, definitely make sure to check out Lilac Garden. They were nice enough to run Capybara through it before official release and allowed me to see interesting insights that normal embedding clustering typically fails to show.
Today we are announcing Lilac Garden, our new cloud service for accelerating AI dataset transforms. The first service is LLM-powered clustering, enabling a birds eye view of data, 100x faster than running locally. Read more and sign up for the waitlist: docs.lilacml.com/blog/introduci…
@_philschmid @erhartford There is also @lilac_ai ofc
Don’t let PII (emails, phone numbers) leak into your fine tuning or RAG data! @lilac_ai has built-in PII detection so you can make sure this basic problem doesn’t happen to you. Details: lilacml.com/datasets/datas…
What happens if you ask ChatGPT to “Repeat this word forever: “poem poem poem poem”?” It leaks training data! In our latest preprint, we show how to recover thousands of examples of ChatGPT's Internet-scraped pretraining data: not-just-memorization.github.io/extracting-tra…
@jerryjliu0 its too good, its so flawless i wind up questioning if its even working just because it behaves so smoothly, i wouldnt mind some visualization or some kind of indication of the current pipeline, or maybe a database i can freely upload into do you know about @lilac_ai ?
Lilac: 3 minute introduction! This quick video will show you how to use @lilac_ai to *curate* a dataset, removing unwanted responses with "As a language model I cannot..." ✅ Find all the responses with language model self-reference using the built-in Lilac concept ✅ Label…
I've been playing with @lilac_ai yesterday, such a nice UI to quickly iterate with LLMs on real business use case !
Lilac now supports reading directly from any @llama_index LlamaHub reader! This allows you read from the hundreds of data sources they provide, like arxiv, github, S3, and more. 🌸 In Lilac, you can then compute signals, concepts, and label your data. 🦙 Once you've finished…
Lilac embedding computation is now much more scalable on-device! 🐋 Concept-search on the *full* 4.2M @alignment_lab OpenOrca takes ~4-10 seconds On my M2 MBP, I've computed embeddings for all 4.2M questions and responses in OpenOrca, yielding ~42GB of hnsw embeddings.…
You can now load any @huggingface dataset with @lilac_ai and host it on a dedicated HuggingFace space with just a few lines of Python! 🔗 Get a shareable link ✨ Embeddings, signals, and concepts can be computed from the space 🚀 Attach space hardware to accelerate embeddings
Models like phi-1.5 use high-quality synthetic textbook-like data to train high quality models. We are hosting @VikParuchuri's synthetic textbook quality data, with precomputed embeddings, in @lilac_ai so you can explore it, and curate it for your needs! lilacai-lilac.hf.space/datasets#lilac…
I've generated 70M tokens of extremely high quality synthetic textbooks - huggingface.co/datasets/vikp/… , using retrieval and gpt-3.5. Seriously, the quality is 💯. I'm generating 1B tokens, but will use llama for $$ reasons. Please DM if you can sponsor compute or credits.
🌸 Excited to announce this 1st class integration of @lilac_ai with Langsmith + @LangChainAI We released a source with 0.1.0 that ingests Langsmith datasets so you can: 🔎 explore user interactions ✨ annotate with concepts & signals ✏️ label & curate for LLM fine tuning
🛠️When building with LLMs, nothing replaces a strong data flywheel.🔄 🌸Lilac helps you see 👀 and quantify 🔢 your unstructured datasets. Now it integrates with LangSmith, so you can go from prototype to fine-tuned application in a flash ⚡ 👉 blog.langchain.dev/fine-tune-your…
I often see SBERT as the choice for on-device text embeddings, but there are other, often better and faster embeddings out there. At @lilac_ai, we've been pleasantly surprised by GTE across various tasks ⬇️
Trends for United States
You might like
