Posted inMonitoring

Automating Core Web Vitals Monitoring with Lighthouse Exporter and Grafana: Lessons from 6 Months in Production

Monitoring tutorial - IT technology blog
Monitoring tutorial - IT technology blog

Why Server Monitoring Alone Isn’t Enough

After over six months of running a monitoring system for an E-commerce project, I learned a hard lesson. The Prometheus dashboard showed extremely low server CPU/RAM usage, and services were all “green,” yet customers still complained that the homepage took 8 seconds to load. The issue is that we often focus too much on Backend infrastructure while neglecting the actual user experience in the browser.

Google uses the Core Web Vitals suite to score websites. If you’re into SEO, you know these metrics determine whether a page lands on the first or tenth page of search results. However, manually checking Chrome DevTools every day is impossible. That’s why I integrated Lighthouse Exporter into the monitoring workflow. This tool transforms dry Lighthouse reports into real-time data on Grafana.

How Does Lighthouse Exporter Work?

Lighthouse Exporter typically runs as a Docker container. When active, it launches a headless Chrome browser to visit your configured URLs. Once the scan is complete, it converts the results into a metrics format that Prometheus can understand.

Here are the “golden” metrics I regularly monitor:

  • Largest Contentful Paint (LCP): Measures the loading speed of the largest element (usually a banner or product image).
  • Cumulative Layout Shift (CLS): Checks if the UI jumps around while images are loading.
  • Total Blocking Time (TBT): The time the browser is “frozen” due to heavy JavaScript processing.
  • Performance Score: An overall score for quick reporting to managers or clients.

The biggest advantage is that the Exporter runs on a fixed server. This eliminates noise factors like a developer’s Wi-Fi speed or local machine specs, ensuring consistent measurement data.

Hands-on: Deploying a Core Web Vitals Monitoring System

Before starting, ensure you have a Prometheus and Grafana cluster ready. If not, take 5 minutes to review the basic installation guides on my blog.

Step 1: Run Lighthouse Exporter with Docker

Using Docker is the fastest way to deploy. A quick note: Headless Chrome is extremely resource-heavy. You should limit the RAM for this container to avoid impacting other services.

docker run -d \
  --name lighthouse-exporter \
  -p 9242:9242 \
  --memory="1g" \
  -e "LIGHTHOUSE_SCRAPE_INTERVAL=300" \
  -e "LIGHTHOUSE_TARGET_URLS=https://itfromzero.com,https://google.com" \
  --restart always \
  femtopixel/google-lighthouse-exporter

In the command above, I configured it to scan 2 websites every 5 minutes. You can add multiple URLs, separated by commas.

Step 2: Configure Prometheus to Scrape Data

Now, let Prometheus know where to fetch the metrics. Open your prometheus.yml file and add the following configuration:

scrape_configs:
  - job_name: 'lighthouse-monitor'
    scrape_interval: 15m # Scanning too frequently will crash the server
    static_configs:
      - targets: ['localhost:9242']

Practical experience shows that you shouldn’t set the scrape_interval too short. Each Lighthouse run takes about 30-60 seconds per URL. If you have 10 URLs and scan every minute, your server CPU will be constantly overloaded.

Step 3: Visualize on a Grafana Dashboard

Once Prometheus receives the data, the rest is just plotting charts. You can use Dashboard ID 12154 on Grafana Labs for a quick import. Here are a few useful PromQL queries:

  • Monitor Performance Score: lighthouse_performance_score{url="https://itfromzero.com"}
  • Monitor LCP (seconds): lighthouse_lcp_seconds{url="https://itfromzero.com"}

Real-world Lessons: Handling Alert Fatigue and Resources

My biggest mistake when first setting this up was setting alerts as soon as the Performance score dropped below 80. As a result, my phone buzzed constantly whenever international network speeds flickered or the server had a minor lag. This is called Alert fatigue—the exhaustion from receiving too many false alarms.

To fix this, I switched to using the avg_over_time function over a 30-minute window. If the average score remains low for half an hour, then there’s likely a real issue with the code or infrastructure. This method eliminated nearly 90% of junk alerts.

Regarding hardware, remember that Headless Chrome is very RAM-hungry. An instance can consume 500MB to 1GB of RAM depending on the page weight. I once crashed a web app because I let the Exporter run on a small VPS (2GB RAM) alongside the main application.

Conclusion

Monitoring Core Web Vitals isn’t just about dry technical numbers. It helps you understand what customers are actually experiencing. When combined with Grafana, you’ll clearly see trends: Did yesterday’s code update slow down the site? Or did a newly uploaded image cause layout shifts?

Detecting a downward trend on a chart before customers have a chance to complain brings immense peace of mind to DevOps engineers. If you’re managing websites that need SEO optimization, start implementing this model today.

Share:
Tags: