Graphsignal

Profiling CUDA in production inference is a different problem than profiling a dev run. What actually works: graphsignal.com/blog/cuda-prof…

GPU builders, inference engineers, and training teams are meeting in SF on July 23. Join engineers @aiand_ @NVIDIAAI, @ByteDanceOSS, @radixark, @liquidai, @zml_ai, @GraphsignalAI, Cosmic Labs, Kernelize, and more for an evening of lightning talks and networking. Hosted by @CrusoeAI and @sgl_project. RSVP: luma.com/rxsn0u0h

RT @dstackai: Working on GPUs, inference, or training? Come hang out with us in SF on July 23. We’re bringing together engineers from @NV…

@GraphsignalAI More on how Graphsignal uses dstack across on-prem and GPU cloud for development and inference: dstack.ai/blog/graphsign…

New: a case study on how @GraphsignalAI uses dstack for development and inference benchmarking. Graphsignal builds tooling to profile model inference, and uses dstack across a fleet of @nvidia DGX Spark devices and @verdacloud to keep the workflow consistent across on-prem and cloud: dstack.ai/blog/graphsign…

@dstackai And we also rely on @dstackai for development and testing of our SDK: github.com/graphsignal/gr…. Can’t wait to share more in another post!

autodebug by @GraphsignalAI is a closed-loop system for inference optimization. It uses @dstackai to provision GPUs and redeploy services on each pass through the loop: benchmark → read profiling telemetry → tweak config → redeploy → repeat. What's interesting here is the combination of agentic optimization and heterogeneous hardware: the system is not tuning a fixed deployment, it is continuously searching across infrastructure and configuration. There's no manual step between iterations. @dmitrimelikyan's writeup: graphsignal.com/blog/autodebug…

@dstackai and here is the repo github.com/graphsignal/au…, inspired by @karpathy authoresearch

autodebug: an autonomous loop that deploys an inference service, benchmarks it, reads profiling telemetry, and redeploys with a better config. Then repeats. Uses @GraphsignalAI for inference profiling, @dstackai for GPU provisioning, Claude Code as the agent. graphsignal.com/blog/autodebug… github.com/graphsignal/au…

Config tuning is just the start. The same loop can optimize inference code and even custom CUDA kernels. It all depends on what tools the agent can use.

Agent orchestration is evolving fast! Agents + orchestration + telemetry → closed-loop systems. Our friends at GraphSignal show how this unlocks continuous inference optimization in production — across heterogeneous hardware. This is where things get interesting.

Now @GraphsignalAI integrates with dstack — add @sgl_project profiling, tracing, and GPU metrics to your inference services. pip install 'graphsignal[cu12]' + wrap with graphsignal-run. That's it. graphsignal.com/docs/integrati…

New post: AI Debugging and Optimization For Production Inference graphsignal.com/blog/ai-debugg… Use Claude Code to debug and optimize AI systems with rich production context from Graphsignal

New post: Traditional Observability Is Blind to Inference graphsignal.com/blog/tradition…

New post: vLLM production observability - from model to hardware. graphsignal.com/blog/vllm-prod…

@dstackai @nvidia Thank you for the mention! We’re big fans of @dstackai

We tested dstack with the new @NVIDIA DGX Spark. If you want to run workloads across DGX Spark, on-prem GPUs, and cloud backends using one workflow, here's how. dstack.ai/blog/nvidia-dg…

Fresh dev setup on #dgx_spark running as a @dstackai fleet⚡️

LLM API Latency Optimization Explained graphsignal.com/blog/llm-api-l…

berlin done, munich next! good crew coming too:

YES, you need to see the prompts! Great article by @HamelHusain hamel.dev/blog/posts/pro…