Sabitlenmiş Tweet
ron
1.2K posts
ron
@RonxldWilson
Builder | Autodidact in making.
India Katılım Haziran 2020
312 Takip Edilen753 Takipçiler
ron retweetledi
ron retweetledi
You've implemented FIFO and LIFO data structures, but have you implemented FAFO?
English
@terencebuilds Workers have access to the git repo
they detect change in git commit in 15 min interval and restart the run with updated code
English
@RonxldWilson remote worker control is such a game changer. how are you handling network failures when pushing updates?
English
added in a remote updater to workers
now I can control all of the workers that are online and update their code execution remotely
can also monitor which version of the code they are running on
and see the new changes propogate overtime
ron@RonxldWilson
fully automated end to end pipeline ready and deployed scaled to 22 workers w/ one master now just gonna let it run forever
English
this project has been way too fun to build
ron@RonxldWilson
added in a remote updater to workers now I can control all of the workers that are online and update their code execution remotely can also monitor which version of the code they are running on and see the new changes propogate overtime
English
@ryancarson recently tested out restoring backups of my search engine db and realized its impossible to recover from the backups I was taking so diligently
had to change entire backup strategy due to it
English
Please, pretty please, *actually* test restoring from your db backup (which, for the love of god, should be from an offsite location).
English
fully automated end to end pipeline ready and deployed
scaled to 22 workers w/ one master
now just gonna let it run forever
ron@RonxldWilson
how I built a search engine from scratch here's what I have been building over the course of last month resulting in visiting of over 55 million unique domains 130 GB of sqlite DB, 200 million rows and over 4 million unique Indian B2B businesses 1/n
English
One of the things that I found was most Indian businesses are siloed sites having no outbound/inbound links to other sites in industry
Which made making discovery harder as they couldn't be found via normal hopping
Which lead me to solution I call as search engine distillation
ron@RonxldWilson
how I built a search engine from scratch here's what I have been building over the course of last month resulting in visiting of over 55 million unique domains 130 GB of sqlite DB, 200 million rows and over 4 million unique Indian B2B businesses 1/n
English
These are basically group of 50-100 websites which rapidly linked to each other.
Forming a dense cluster of their own. Think of them like SEO farms which just link to each other to boost each other's popularity
Kinder • Grinder@kinder_grinder
@RonxldWilson what does those isolated groups represent? like PBN?
English
@kinder_grinder Each dot is a site, different colors here represents various TLDs
For my usecase I was mostly concerned with .in .co.in .org .net and .com domains
This graph here is just the top 10k most connected domians, wasn't able to render more as that slows down the device by a lot
English
@RonxldWilson I can imagine. I am very interested in things u do, created some simple crawlers in the past, but always wanted to crawl whole web (ik its huge job).
What does each dot represent? A website or something bigger?
English
@kinder_grinder It does look like that, the interactive view is fun to play around with, can drag around any single one and it causes ripples across the whole landscape
English
@RonxldWilson Very interesting graph, looks like a biological organism or cell.
English
@RonxldWilson Looks impressive man
What do you plan to do with it?
English
@Dmarketsniper this was originally started as a side project to see the feasibility of this becoming a ancillary service to our main service
have reached a point where this alone can be converted into a standalone service but we still need to internally align on where we plan to take this
English
@RonxldWilson Impressive build. Curious how are you planning to turn that into distribution or customers? Feels like that’s where most projects like this stall.
English