This OpenClaw Plugin Talks To Your Servers For You
By Beau Johnson Β· Β· 18 min Β· 855 views
Chapters
- 0:00The PromQL problem every developer faces
- 1:30What is Grafana Lens?
- 3:0017 composable agent tools walkthrough
- 5:15grafana_investigate: AI incident response
- 7:00Dashboard creation and alert automation
- 8:30Security monitoring (6 parallel checks)
- 10:15Z-score anomaly detection explained
- 12:00Setup: LGTM stack in one Docker command
- 13:30Custom Data Observatory
- 15:00Why this matters for AI agent builders
About this video
Transcript
Read the full transcript
0:00We're going to be going over something that every single person needs to know about. If you're running AI agents, you're running servers, you're running any type of infrastructure, and we're going to go over it in this video. It's called Grafana Lens, and it's an open-source open cloud plugin that basically gives you an AI SRE. And I'm going to back up and explain what I'm even talking about and how it helps builders build out there. So, for those who don't know what SRE is, it stands for site reliability engineer. That's the person who wakes up at 3:00 a.m. when your server crashes and has to figure out what went wrong. This tool does that job for you. I'll have it linked down below. So, let me paint the picture first because I think the problem here is something a lot of people deal with. You're running your app, maybe you're running your AI agents
0:46like I do, and maybe you have a SaaS product. You know, maybe you have a web app, I don't know. And these customers that you have depend on your app. And if something breaks, uh your first instinct is to jump into your monitoring dashboard, check the logs, check the debugging, and wow, you pull up Grafana, and then you're staring at a wall of dashboards and metrics. You're trying to remember the exact PromQL query to pull the data you need. PromQL, if you've ever tried to write a PromQL query from scratch, you know what I'm talking about. It's like learning a new language just to ask your servers what happened. Rate functions, sum by clauses, histogram qualities, it's crazy. I've spent more time Googling PromQL syntax
1:33than I'd like to admit, and it just hurts my brain. And that's just the metrics part of it. Then you need to check your logs in what is called Loki. Different query language, LogQL, this time. Then maybe you need to trace a specific request through a distributed system using Tempo, another query language, and then TraceQL comes back in. Three different query languages just to figure out why your app is slow. Doesn't this sound exhausting? That is the problem that Grafana Lens solves. So, instead of learning three query languages, you just talk to your agent. Sounds amazing. You say, "What happened to my server in the last hour?" And it goes out and it queries Prometheus, queries Loki, queries Tempo, gathers all the data, and then comes back with an
2:19actual diagnosis. Not just raw data, a diagnosis with a hypothesis about what went wrong and specific follow-up actions you can actually take. Now, let me walk you through the 17 plugin gives your agent because the scope of this thing is pretty impressive. First, you've got a metric query. You can ask your agent to query any Prometheus metric in plain English. No PromQL required. The agent translates your question into the right query and gives you the results. You can ask things like, "Show me the CPU usage over the last 24 hours." Or, "What's my request rate broken down by endpoint?" Then you've got log querying. Same thing, but for Loki. Ask your agent to find the error logs, filter by service, look for specific patterns, and it's all
3:06in a natural language. And trace querying for Tempo, find slow traces, look up specific request traces by ID, filter by duration or status code. Real quick, before we keep going, if you're watching this and you want to actually build with some of these tools, not just watch videos about them, then you're going to want to check out our community down below, Shipping School. We have a full cloud code course, a full open cloud course, and four live bootcamps every single week where we actually help you get set up from scratch. Like actually set this thing up, not just watch a tutorial and figure it out by yourself. And we also provide one-on-one coaching, so you could book a call with me. We could share screens, and I can help you get cloud code or open claw running on your machine. That's it, no
3:51fluff. I built this community because watching YouTube's only gets you so far. We launched it just 3 days ago, and we have over 55 members. You need people around you who are actually building, people who hold you accountable, and coaches who can help you when you get stuck. I'll put the link in the description down below. Get in now before the price goes up. Here's where this gets a whole new level. They built a tool called Grafana Investigate. When you trigger this, it gathers metrics, logs, and traces all of that at the same time, in parallel. It correlates them all together. So, instead of you manually switching between three dashboards trying to line up the timestamps, the AI does all of that and gives you a unified view of what actually happened. And then it generates
4:37a hypothesis. It says, "Hey, it looks like your database query time spiked at 2:47 a.m., which correlates with this error log showing in the connection pool. Here's what I think happened, and here's what you should check." That is pretty powerful. That's not just monitoring anymore. That's actual incident investigation done by AI for you, through OpenClaw. And it doesn't just stop there. You know, Grafana Lens can also create dashboards for you. You know, you could say, you know, "Create a cost dashboard." And it builds a complete cost intelligence dashboard with model attribution, cash savings, and spending trends. All the panels, all the queries, ready to go. And you could basically say, "Alert me if daily spend exceeds 5 cents." And it creates a Grafana native alert rule with the
5:22proper PromQL condition. No clicking through menus, no figuring out alert syntax. You basically just tell it what you want. Like a lot of these agents that we're experienced with building with OpenClaw. Ask your agent, you know, they have 12 pre-built dashboard templates. There's a LLM command center for tracking AI agent performance, a cost intelligence dashboard, a security overview, an SRE operations dashboard, and you can deploy any of these with a single command and have a production-ready monitoring system in just minutes. This would typically take hours and days to get this done. Now, I'm going to talk to you about the security monitoring because I think that this is huge and relevant for a lot of builders that are putting security
6:07first. Nobody else is doing this well, and you know, people are still trying to figure this out. Grafana Lens runs six parallel security checks on your system. It checks for prompt injection attempts, it checks for cost anomalies like, you know, if one of your agents suddenly starts burning way more tokens than normal. It checks for tool loops, which when an agent gets stuck calling the same tool over and over. It checks for session enumeration, webhook errors, and stuck sessions. And it gives you a threat level, green, yellow, or red. If you were running OpenClaw agents like I am, this is exactly the kind of monitoring you need, especially if you're looking to build this as more agents as a service. You know, I run multiple agents on the Mac Mini, and these all fire off 24/7. The Rizza
6:54handles the content pipeline. Inspector Deck handles all the things on X. Jizza does the security audits. Nathan Man does all the clips, clipping the long form into short form. And I need to know if something goes wrong with any of them. And traditionally, that meant manually checking the logs and hoping I notice something weird. But with Grafana Lens, my agent can proactively monitor all that. It can detect anomalies using the Z-score analysis against a 7-day baseline. What does that mean exactly? It means it knows what normal looks like for your system. And when something deviates from normal, it flags it. Not just a simple threshold alert that fires every time a metric goes above a number, like an actual statistical anomaly detection that accounts for patterns and seasonality. It even compares current
7:39values against 1 day ago and 7 days ago just to understand if a spike is unusual or just irregular Tuesday traffic pattern. Now, let me back up and talk about the setup because I know you want to know how hard this is to get going. And it sounds difficult, right? But the recommended approach is using Docker. There's an all-in-one container from Grafana Labs called OTEL IGT M. That stands for Loki, Grafana, Tempo, Mimir, but everyone just calls it LGTM. You pull up one Docker image, you run one command, and you have a Grafana plus Prometheus plus Loki plus Tempo plus OpenTelemetry Collector all running in your machine, one container. Then you
8:24install the Grafana Lens plugin, that's the one command, set your two environment variables, your Grafana URL, and your service account token, restart your OpenClaw gateway, and done. So, you now have a full observatory uh you now have full observability with AI-powered investigation capabilities. Here is what matters the most. How much does this cost? The LGTM stack is completely free and open source. So, Grafana Lens is open source, that's awesome. I love seeing these open source projects, that's why I'm sharing them with you. The only cost is the AI model calls, which you're already paying for anyways, which you're running through OpenClaw, so it's basically nothing extra on top of it. Here's something that I think is really underrated about this tool. They built what they call a
9:09custom data observatory. This means you could push any data into Grafana through conversation. Not just service metrics, but any data. You could say, "Track my daily steps." And it pushes the fitness data to Grafana. You could track revenue, get commits, calendar events, anything you want to monitor over time, essentially. Your agent becomes the interface for pushing data into Grafana and then visualizing it on dashboards. And I love this. For me, my use case, you know, I'm trying to get in the habit of drawing blood every 6 months or every year. I would love to see it on a dashboard. What vitamins, what food do I need to eat? What diet plan do I need to get on to improve my metrics in my blood? You know, that's for me. I know everyone's doing something different, but that would be cool to visualize, you
9:55know, on a live dashboard, right? You know, and if you think about what this means, you know, you could build a personal dashboard that tracks everything essentially. You know, your pets, your kids, whatever. I think being able to visualize this data in real time is amazing and we're still early. You know, we still don't see a ton of this coming out quite yet. You know, you could basically just do like your business metrics, your health data, your productivity, your AI agent costs and it's all in one place. And you could do this just by talking with your agent back and forth. Like, "Hey, I saw this pop up. Tell me about that." And that's the kind of tool that makes me more excited about Open Claw and the entire ecosystem. You know, someone identified a real pain point here. Anyone who runs servers knows the PromQL struggle is
10:40real and they didn't just build a one-off script for this. They built a proper plugin with 17 tools and 12 dashboard templates. All the security monitoring, anomaly detection and then the ability to push custom data through it. That's a pretty good complete solution, I would say. And it's open source, MIT licensed. You can check the code, you can contribute, you can modify it for your own needs. I'm going to break down the real-world scenario where this would actually be applicable for you and me and how it could possibly save your life. If you're struggling to keep up with content, well, I'm about to save you about 40 days worth of work. I [music] built something called Content Machine. It's 10 AI agents that run on the Open Claw orchestration and they handle everything. Scripts, thumbnails,
11:26X posts, [music] blogs, outreach, clips, newsletters, all of it. So, I went from 1,000 subscribers to 4,000 subscribers on YouTube in 7 days using this exact system. Every single morning, I wake up and the content's already done. I spend maybe 15, 20 minutes reviewing and approving them and I move on with my day. It works [music] for any niche, fitness, finance, real estate, marketing, whatever you are building and it is 100% completely customizable to your use case. So, you get the mission control dashboard, all of the cron jobs, everything I've built over the last 40 days helping me gain more and more people to subscribe and join the community. So, you plug in your own thing and it molds it to you, it learns
12:12how you talk and it writes [music] so it doesn't sound like AI slop. $97 one time, it's not a subscription. I'll put the link down below and you'll thank me later. So, let's say it's like 2:00 a.m. You're probably sleeping. Your SaaS app starts throwing errors, you got 200 customers that are freaking out and then your users can't log in. So, without Grafana Lens, here's what would happen. You get paged, you groggily open up your laptop, probably half asleep. You pull up Grafana, you try to remember which dashboard has the relevant metrics, you start writing PromQL queries to figure out what changed and then you check the logs in a separate tab. You cross-reference the timestamps manually. By the time you figure out the problem, it's probably been about an hour and your users have been down the whole time
12:57and they're starting to churn. Bye. They're going somewhere else. With Grafana Lens, your Open Claw agent can be set up to do this proactively, monitor for anomalies, it detects the spike, it runs Grafana to investigate it automatically and it correlates those metrics with the logs and with the traces. So, it can send you a message on Telegram saying, "Hey, your auth service is throwing a connection pool exhaustion error. Database connection spiked at 1:47 a.m. And here's the likely cause and here's what to do about it." That's not a 45-minute investigation, that's 30-second notification with the answer already in it. I'm going to talk about the alert fatigue detection because that's something that also drives every ops person crazy. You know what happens when you set up too many alerts? Everything alerts all the time and then
13:43you start ignoring them. Grafana Lens has that tool that identifies all the noisy alerts. It finds rules that are always firing, rules that are flapping between alert and okay constantly, and rules that are in error or data states. And then it gives you specific optimization suggestions, it tightens this threshold completely. You know, add a four duration to that rule and consolidate these three alerts into one. That's the kind of senior SRE would do during a quarterly review. That's amazing. Now your agent can just do it continuously for no extra cost. The panel sharing feature is pretty cool and very practical. Let's say you're in Slack channel or a Telegram group with your team. Something goes wrong and instead of saying, "Hey, go check the
14:28Grafana dashboard" and then everyone's trying to find the right one, you just tell your agent to render a specific panel in an image. It generates a PNG of that chart and drops it right into the chat. 15 plus messaging channels supported, that's instant visual context for your whole team without anyone having to open Grafana. Let me also touch on the dashboard auditing tool. Over time, dashboards accumulate broken panels. Someone deletes a data source, someone renames a metric, panels show no data and nobody notices because the dashboard has 30 panels and you only look at three of them. So, Grafana Lens can do a health check on all these panels in the dashboard. It finds broken queries, verifies data source connectivity, and flags panels that need attention. So, spring cleaning basically
15:13for your monitoring setup. I think the thing that really sets this apart from other monitoring tool, you know, is the investigation workflow. Most monitoring tools tell you what is happening, CPU is high, memory is low, errors are up, but you know, they don't tell you why. And they diff and they definitely don't tell you what to do about it. But the Grafana Investigate tool does all three. It gathers the data across multiple signals simultaneously, it correlates them, and then it generates a hypothesis with specific text and steps to do following. You know, that's the difference between a dashboard and an actual SRE assistant. Another thing I want to mention, the plugin can track your AI agent costs in Grafana. So, if you're running Open Claw, you probably want to know how much each agent costs per day, which model is
15:59burning the most tokens, whether your prompt caching is actually working. Grafana Lens pushes all of those metrics into Prometheus automatically. So, you could basically build cost dashboards, set spending alerts, and track your AI budget over time. This is huge for big enterprise and small to medium businesses. You know, for anyone running agents in production, that visibility is crucial. I've seen people get surprised by unexpected bills because they didn't have cost monitoring set up. And this solves all of that. And for those of you inside Shipping School who are actually building SaaS products or running AI agents, this is exactly what you need for monitoring. We talk about tools like this all the time in our live build sessions. We have over 110 members in there now building products with AI and that's just in 12 days. So, super thankful for you guys. If you want the
16:45to join a community that's actually building stuff and not just talking about it, I'll put the link down below and I'm also going to drop the link to that GitHub Grafana Lens in the description. Check it out, give it to your agent and ask about it. Say, "Here's the link to this repo. What do you think about it? How will it make it us better and how is it different than what we're doing now?" Take the link to this video and have your agent transcribe it and say, "What the heck is this guy talking about?" You know, whether you're running a personal project or managing anything in production that has heavy infrastructure, having an AI that can talk to your server stack and tell you what's wrong is key. You want to know what's going on. If you haven't subscribed already, please do so because we release about four to six videos every single day and if you want to stay on top of the AI news and cool projects
17:31like this, hit the notification bell and we'll see you in the next one, y'all. Bye-bye.
Build this with us, live
Shipping Skool is a free community with 4 live build sessions a week. Bring your idea.
Join Shipping Skool β Free





