@jerrychen @iamveeve @dhruba_rocks @tudor @ShrutiBhat @RocksetCloud @OpenAI Thank you for your support along the way!
English
Igor Canadi
512 posts
Igor Canadi
@igorcanadi
Building big databases at @OpenAI. Previously Rockset and Facebook.
San Francisco Katılım Nisan 2009
1.3K Takip Edilen935 Takipçiler
Congrats to @iamveeve @dhruba_rocks @tudor @ShrutiBhat @igorcanadi! Today @RocksetCloud joins the great team at @OpenAI. It was a privilege to part of the Rockset journey from the start! greylock.com/portfolio-news…
English
This is happening now! Come see what's under the hood of @RocksetCloud.
CMU Database Group@CMUDB
Today's ¡Databases! Seminar Speaker: Ben Hannel will talk about dynamically typed SQL execution in @RocksetCloud. Zoom talk is open to public at 4:30pm ET. YouTube video will be available afterwards: db.cs.cmu.edu/events/databas…
English
Igor Canadi retweetledi
The two major trends in data are speed & scale. The team at @RocksetCloud @iamveeve @dhruba_rocks @tudor @igorcanadi have broken the speed/scale barrier to build modern data applications on a real-time analytics database.
Greylock Partners@GreylockVC
14/ @rocksetCloud + @GreylockVC
English
Igor Canadi retweetledi
@RocksetCloud is on a mission to eliminate all the cost and complexity associated with real-time analytics. Today's Rollup launch is a very important milestone in this journey. Read more👇
English
Very excited about our Series B today! A great milestone on our mission to build some of the finest database tech, optimized for developers. P.S. We are hiring - DMs are open.
English
@narayanarjun @MarkCallaghanDB This is how we use it, exactly - we download all files locally. Still investigating if we can keep a portion of the data in the cloud without a local cache. Likely possible for scan queries - S3's throughput is impressive.
English
@MarkCallaghanDB But shouldn't basically every read be served from your local cache? This way you don't need to replicate your RocksDB - the S3 takes care of that - and your local cache can be on non-persistent SSDs.
English
@MarkCallaghanDB @iamveeve Surprisingly accurate. ;) We allow updates to existing documents, so we keep PK -> rowid index. We use RocksDB's merge operator, but the efficiency of reading chunks that have merge updates is still an open question for us.
English
Igor Canadi retweetledi
The Secret Sauce Behind @RocksetCloud Fast Queries is converged indexing. @igorcanadi goes deeper here: rockset.com/blog/converged…
English
@vfonic Strongly disagree. GraphQL is exposing business logic, which is rarely fully expressible in SQL. Opening up SQL interface to the world is also very risky -- you need strong protection against expensive queries.
English
What's the fastest we can load the data into @RocksDB? New blog post listing some of the optimizations that resulted in 20x performance win at @RocksetCloud : bigger batches, parallel writes, no memtable, no compactions. rockset.com/blog/optimizin…
English
@janicduplessis Hey @janicduplessis I work at Rockset, would love to have you try it out and share your thoughts. First 2GB are free, happy to help if you get stuck with anything.
English
Has anyone tried Rockset? Looks interesting for analytics + search on top of nosql data stores rockset.com
English
Igor Canadi retweetledi
Just added @RocksetCloud's Kafka Connector to the Confluent hub. confluent.io/hub/rockset/ka…
English
@tlipcon @markcallaghan @MongoDBEng @RocksetCloud That’s correct. We have specialized IValue types for arrays of same-type scalars, on which we can do vectorized. For mixed-type arrays (columns) you are out of luck, but when you have an implicit schema the speed should be similar as with explicit schema.
English
@igorcanadi @markcallaghan @MongoDBEng @RocksetCloud Looks like the 128-bit IValue (and dynamic typing in general) would really inhibit your ability to do vectorized query execution unless you specialize operators for runs of same-typed data and avoid materializing IValues in those cases?
English
@markcallaghan @MongoDBEng @RocksetCloud Still early, but we have a project to make columnar better by combining many values into a single key-value in RocksDB through merge operators. This is a user-space approach though; haven't explored improvements to the LSM itself.
English
@tlipcon @markcallaghan @MongoDBEng @RocksetCloud Schemaless is fun, the only trouble is lack of existing schemaless SQL optimizers out there (like Apache Calcite). What made you go the fixed schema route? BTW have you seen rockset.com/blog/ivalue-ef… ?
English
@markcallaghan @MongoDBEng @RocksetCloud Yea still fixed schema. Funny enough, the original design looked more like rockset with schema inference but started with fixed schema and never went and did the schemaless
English
Igor Canadi retweetledi
@FranckPachot @andy_pavlo @markcallaghan @MongoDBEng @RocksetCloud @RocksetCloud's converged indexing is simply indexing almost everything, both at the top level and for nested fields. We detect the type of the value and index it in a type-appropriate manner: lexicographic ordering for strings, arithmetic ordering for numbers, etc. 1/2
English
@FranckPachot @markcallaghan @MongoDBEng @RocksetCloud We build search indexes on columns, which allow us to quickly evaluate conjunction of predicates (intersecting posting lists). There are some queries that would be faster with multi-column indexes, I agree. Future work. :)
English
@igorcanadi @markcallaghan @MongoDBEng @RocksetCloud Indexing all columns is not sufficient. You need to index all combinations of columns that are selective. Or do you build bitmaps to combine?
English