// project
CompletedCo-Developer
G-Watch: a GPU Profiler with AI-Driven Automatic Optimization
CUDACUPTIC++/Pythonpybind11LLM Inference
Introduction
- G-Watch is a toolbox for agentic GPU kernel optimization.
- It features rich Profiling capabilities on both NVIDIA/AMD GPUs.
- Additionally, G-Watch offers Program Analysis tools for inspecting compiler-generated GPU binaries, facilitating secondary development tasks such as register analysis and binary instrumentation.
- The code will be open-sourced in the future.
Overview
- Designed and implemented a GPU profiling framework in C++/Python (bridged via pybind11), exposing a ProfileContext Python API for seamless integration into existing LLM inference pipelines.
- Instrumented NVIDIA GPU hardware counters via CUPTI to capture fine-grained kernel-level metrics (execution time, memory throughput, compute utilization) across SGLang and vLLM workloads.
- Diagnosed and resolved CUPTI subscriber registration failures caused by SGLang's child-process architecture, requiring process-level isolation and careful GPU resource lifecycle management.
- Built an AI-agent-driven profiling workflow (Gemini, Claude) to automate bottleneck analysis; validated profiler correctness and stability across concurrent multi-request inference sessions.