Coiled

1.7K posts

Coiled banner
Coiled

Coiled

@CoiledHQ

Lightweight cloud compute platform for Python people.

Everywhere Katılım Şubat 2020
54 Takip Edilen3.3K Takipçiler
Coiled
Coiled@CoiledHQ·
Easily configure shared memory size for CLI jobs with `--docker-shm-size`. Training PyTorch models on a GPU and need more memory? Ever run into "Error: No space left on device"? Customize Docker shared memory size with `--docker-shm-size`. docs.coiled.io/user_guide/cli…
Coiled tweet media
English
0
0
3
1.2K
Coiled
Coiled@CoiledHQ·
🔨 Job setup option for Coiled Batch Use `--host-setup-script` to configure your VM before your batch job starts. Easily: ✅ Install dependencies ✅ Mount cloud storage ✅ Handle authentication or any other setup your jobs need. docs.coiled.io/user_guide/bat…
English
0
0
0
1.1K
Coiled
Coiled@CoiledHQ·
@black_dot_box 🙌 thanks for the kind words, glad to hear Coiled's been working well for you.
English
0
0
1
523
Coiled retweetledi
black.box
black.box@black_dot_box·
@CoiledHQ ~5 years ago I worked at a startup where we had multiple engineers screwing around for months with Terraform, Kubernetes, EKS, etc. just to get the same capabilities I got after an hour of playing around w/ Coiled. Pretty cool.
English
2
1
3
435
black.box
black.box@black_dot_box·
Recently discovered @CoiledHQ, and it's now one of the most important tools in my stack. If you want to scale Python code on the cloud (w/o DevOps overhead), check it out.
English
2
0
1
80
Coiled retweetledi
Matthew Rocklin
Matthew Rocklin@mrocklin·
Coiled 2024 in Review docs.coiled.io/blog/2024-eoy.… It’s the time when companies issue year-end summaries, acclaiming success (or not), and forecasting incredible growth for the next year (or not). I thought I’d do something similar for Coiled. It’s been quite a year for us ...
English
0
5
27
1.9K
Coiled
Coiled@CoiledHQ·
We're big fans of rich for a nice terminal experience, but have found sometimes folks log things even rich can't handle. In the latest coiled=1.67.0 release, coiled logs automatically falls back to non-rich printing in these situations. Release notes: #v1-67-0" target="_blank" rel="nofollow noopener">docs.coiled.io/user_guide/cha…
Coiled tweet media
English
0
1
3
609
Coiled retweetledi
Matthew Rocklin
Matthew Rocklin@mrocklin·
New Post: Cloud Computing is Broken matthewrocklin.com/cloud-is-broke… Investor asks: "What's next for Data/Cloud Infrastructure?" My answer: "Boring stuff. People struggle with basics." Cloud feels like MP3 players before iPod. In theory everything is good. In practice adoption is low
English
1
3
22
1.9K
Coiled
Coiled@CoiledHQ·
We're now on Bluesky! Should be pretty easy to find us, since bluesky lets us use our coiled.io domain as our handle ☀️
English
0
0
0
437
Coiled retweetledi
Matthew Rocklin
Matthew Rocklin@mrocklin·
New Post: SLURM-Style Job Arrays on the Cloud docs.coiled.io/blog/slurm-job… HPC Job scripts were the first form of parallelism I ever used as a graduate student. They're dead simple and accessible to almost anyone. We replicated the API with Coiled. It feels pretty slick to me 🙂
English
1
5
16
904
Coiled retweetledi
Nazari Goudin
Nazari Goudin@nazari_goudin·
@CoiledHQ is amazing. If you want to have distributed compute and provisioned infrastructure from the code - its easy as that. Forget @ApacheSpark and @awscloud Sagemaker, EMR
Nazari Goudin tweet media
English
0
1
4
224
Coiled retweetledi
Quentin Lhoest 🤗
Quentin Lhoest 🤗@lhoestq·
New blog post: Scale AI-based Data Processing EASY The FineWeb-Edu dataset comes from processing 45TB (🤯) of FineWeb And it uses a Language Model to classify the educational level of the text 😭😭 Still, we reproduced it in a few lines of code ! The key ? HF + Dask 😎
Quentin Lhoest 🤗 tweet media
English
1
25
98
16K
Coiled retweetledi
Arpit Bansal
Arpit Bansal@arpit__bansal·
Implemented @CoiledHQ into our product to offload data syncing from BigQuery to Neo4j 🤯 Works like butter 🧈 Now I don’t have to worry about scaling VMs dynamically to handle variable loads.
English
0
1
3
426
Coiled retweetledi
Matthew Rocklin
Matthew Rocklin@mrocklin·
We're to build a 100-TB scale geospatial benchmark suite docs.coiled.io/blog/geospatia… We've seen an uptick in geospatial users and in challenges of the Xarray/Dask stack to scale beyond ~500-GiB. This post presents a call for benchmark workloads.
English
2
12
45
4.6K
Coiled
Coiled@CoiledHQ·
Run a Python script on a cloud GPU with one line of code. Training a @PyTorch model training takes ~10 minutes and cost ~$0.12 on the NVIDIA T4 GPU on AWS. Coiled handles provisioning hardware, setting up drivers, and installing CUDA-compiled PyTorch. docs.coiled.io/user_guide/gpu…
Coiled tweet media
English
0
0
2
698
Coiled retweetledi
Dask
Dask@dask_dev·
Dask DataFrame is now 20x faster. Some of most prominent changes include: - Apache Arrow support in @pandas_dev - Better shuffling algorithm for faster joins - Automatic query optimization Learn more: docs.coiled.io/blog/dask-data…
Dask tweet media
English
0
8
30
2.8K