Sabitlenmiş Tweet
LiteFold
312 posts
LiteFold
@try_litefold
The infrastructure for Drug Discovery. We are here to make AI for Science more accessible.
Earth Katılım Aralık 2024
11 Takip Edilen2.6K Takipçiler
LiteFold retweetledi
A really cool research blog dropping tomorrow at @try_litefold tomorrow, this time it's not gonna be models!
English
LiteFold retweetledi
I’ve been one of the biggest proponents of molecular jackhammers, if not the biggest outside of the team that developed them. I’ve been talking about them for years on here like a crazy person but maybe some of the world is starting to catch on - so there’s some questions that need to be answered.
It’s not expensive at all to make these. Kodak (the camera company) had the large cyanine backbone developed back in the 70’s for printer dyes. So yes, everyone would get them for cheap if they get FDA approved. Unknown how long that will be until then, but we’re also developing new novel ones as well at @try_litefold .
The detection mechanism is very very solid for tumors that are sensitized to it. This means any solid tumor type. The cyanine backbones I investigated without the photoactivation were 450x more potent than doxorubicin at preferentially killing cancer cells. That being said, not all cancer cells are immediately susceptible, although most are, and there’s some literature to show that you can drive tumors into a sensitized phenotype. Non-solid tumors I am unsure about, but solid tumors definitely.
Animal or human studies, won’t change much other than clearance which is already up for concern. The mechanism it relies on is physical, not receptor dependent so cancer cells cannot simply evolve their way out of it. Now whether or not that delocalized lipophilic cation gets out of the kidneys is another question. We’re developing ones that don’t have that issue to begin with.
So for now, yes we need to worry about cancer. In 5-10 years, no, we won’t need to worry about the majority of cancers. And AI wasn’t even a deciding factor in developing these. P.s. this Substack channel is garbage, but it’s the first one I have seen mention this outside of my echo chamber.
English
Well thats the goal, do great quality tasteful work! Thats what we do at LiteFold 🚀
Pratyaksh Patel@baldwin_IVth
okay guys, i read a lot of technical blogs and recently ive been reading a lot from indian orgs. i mostly judge them on how technical they are and how smoothly do they transfer that technical know how to the reader, one of the best ive come across lately are from @c_engines and @try_litefold. smooth, really smooth, especially the litefold guys, i did a bit of bio back in 1st year at iiserb, kinda nudged it finally came to some use. with the concious engines guys, i usually skip the ones on llms and slms and infra, but these guys had me hooked, esp the need for speed one. will now take heavy inspirations from both of these and adapt my own blog (pratyakshpatel.github.io/technical_blog…).
English
LiteFold retweetledi
okay guys,
i read a lot of technical blogs and recently ive been reading a lot from indian orgs.
i mostly judge them on how technical they are and how smoothly do they transfer that technical know how to the reader, one of the best ive come across lately are from @c_engines and @try_litefold.
smooth, really smooth, especially the litefold guys, i did a bit of bio back in 1st year at iiserb, kinda nudged it finally came to some use.
with the concious engines guys, i usually skip the ones on llms and slms and infra, but these guys had me hooked, esp the need for speed one.
will now take heavy inspirations from both of these and adapt my own blog (pratyakshpatel.github.io/technical_blog…).
English
Whatever LiteFold is going to deliver, its going to be the best in class! No compromises
Anindyadeep@anindyadeeps
Since we are now deep into our research, so you guys deserve a better blog! Coming soon
English
LiteFold retweetledi
Since we are now deep into our research, so you guys deserve a better blog! Coming soon
English
LiteFold retweetledi
We benchmarked 5 Protein Language Models on BenchPLM and the results were surprising.
The key takeaway is, Protein Language Models should not be treated as normal texts. Masked Language Model (even Diffusion LMs) works much better than Causal Language Models for learning representation.
We tested thermostability, subcellular localization, metal binding, and FLIP-2 mutation fitness. ESM-C and DPLM were consistently the strongest. ProGen was useful, but it did not match the bidirectional models in the frozen-embedding setup.
The sparse-label results were even more interesting. With lightweight probes and limited training data, ESM-C had the best mean normalized curve score. That matters because most real protein datasets are small, expensive, and noisy.
My read: protein properties depend heavily on long-range constraints, structure, and evolutionary coupling. If the model only reads left to right, it misses a lot of the biology.
Checkout the full blogpost. Link in the comments.
LiteFold@try_litefold
Today LiteFold is introducing BenchPLM. It answered the following question: - Which protein language models learn the best representations, and why? - Which protein language model should be used for downstream tasks? - What kind of representations have these models learned, and how much evolutionary knowledge do they capture? ESM-C (300M) and DPLM (150M) tied for the top. Bidirectional models beat causal ones. At equal size, ESM-2 and DPLM outperformed ProGen2-Small (151M) and ProGen3 (219M). The real difference wasn’t parameter count. It was whether the model could see the full sequence or only a left-to-right prefix. We also evaluated proteins to understand how much evolutionary knowledge it has learned. Two proteins looked similar by sequence but the true evolutionary relative looked different, bidirectional models picked ancestry. Causal models got fooled by surface similarity. Even tiny ESM-2 (35M) did this well. For frozen embedding pipelines: use ESM-C by default and DPLM for membrane proteins or far-mutation fitness. Skip causal models as feature extractors. Full results and data in the blogpost (link in comments).
English
LiteFold retweetledi
Today LiteFold is introducing BenchPLM.
It answered the following question:
- Which protein language models learn the best representations, and why?
- Which protein language model should be used for downstream tasks?
- What kind of representations have these models learned, and how much evolutionary knowledge do they capture?
ESM-C (300M) and DPLM (150M) tied for the top. Bidirectional models beat causal ones. At equal size, ESM-2 and DPLM outperformed ProGen2-Small (151M) and ProGen3 (219M).
The real difference wasn’t parameter count. It was whether the model could see the full sequence or only a left-to-right prefix.
We also evaluated proteins to understand how much evolutionary knowledge it has learned.
Two proteins looked similar by sequence but the true evolutionary relative looked different, bidirectional models picked ancestry. Causal models got fooled by surface similarity. Even tiny ESM-2 (35M) did this well.
For frozen embedding pipelines: use ESM-C by default and DPLM for membrane proteins or far-mutation fitness. Skip causal models as feature extractors.
Full results and data in the blogpost (link in comments).
English
LiteFold retweetledi
Some observations after studying AlphaFold3
- This is very obvious but less the data, complex the architecture
- AF3 has done a beautiful job on getting signals out from the inputs, through continuous residual input using the initial input tensor
- All atom, I had mis-conceptions about all atom before. But essentially AF3 people thought about generalization capability (which was not there in previous generations). So you first start in atomic space (which is a huge input size btw, but AtomTransformer attends within windowed blocks, so it does not affect much) and then merge / pooled into tokens.
- Standard amino acid are considered as single tokens, for ligands, each atom is a token, nucleotide is a token, any non-standard is considered as a ligand.
- It's an O(N^3) algorithm, because of Triangular attention living inside PairFormer
- Torch Compile not supported ughhh
Generalization is still very hard for models like this, and it's heavily dependent on MSA. I did my own experiments where, I saw there is a sharp 20-30% drop in pLDDT when I am not using MSA. Even after full fine-tuning protenix on 50K steps without MSA did not gave us much good results. The model remains MSA dependent.
Still have lot of things to read around this. All I can say is, if you truly can understand the working of AF3/2 then you can potentially every other models in this field and all those other models would feel like a piece of cake.
Anindyadeep@anindyadeeps
Was studying the internals AF3 and all i can say, alphafold 3 is much simpler than af2.
English
LiteFold retweetledi
Was studying the internals AF3 and all i can say, alphafold 3 is much simpler than af2.
English
Designing a biomolecule today means duct-taping 10 disconnected tools into a pipeline that breaks every time you blink.
We're collapsing all of it into one interface. Sequence-first, structure-first, hybrid. Built for humans and agents from day one.
You will see very soon!
Anindyadeep@anindyadeeps
With the autonomy you want. I have been working on this now for nice time now. Will launch it very soon. The one and only one interface to design biomolecules. Be it Sequence first, structure first, hybrid across all modalities.
English
Small molecule
peptide
antibody
enzymes
binders
you name it!! Coming Soon
Anindyadeep@anindyadeeps
As mentioned, ab absolutely new experience for humans and agents to design bio molecules. Coming soon!
English
LiteFold retweetledi
As mentioned, ab absolutely new experience for humans and agents to design bio molecules. Coming soon!
Anindyadeep@anindyadeeps
So, one day in studying about protein / bio-molecule design and I am deeply addicted to that now. Gonna make something interesting in this.
English
LiteFold retweetledi
Meet the members of Proxima:
Anindyadeep (@anindyadeeps) is building Litefold. LiteFold combines physics-based simulation and AI to accelerate the design and validation of drug candidates in silico.
Their leading product is Rosalind: An AI Co-Scientist for Life Sciences.
English