Graphite can be considered to be one of the best data center monitoring tools on the market today – and it is free. In this post, we will have a look at how it helps users gain deeper insights into the performance of all their assets in their digital stack.
What is Graphite?
Graphite is a free, open-source software tool that is used to monitor and graph numeric time-series data obtained from the performance of computer systems. Originally developed by Orbitz Worldwide, Inc., Graphite was made available in 2008 as an open-source software for collecting, storing, and displaying time-series data in real-time.
This is an enterprise-ready monitoring tool that runs equally well on modest hardware setups as it does on Cloud infrastructures. Most users rely on it to track the performance of their websites, applications, business services, and networked servers. This is possible because it stores numeric time-series data of the sort produced by monitoring metrics from such technologies, which is then used to render graphs of that data.
Other asset inputs that can be monitored include CPU usage, memory usage, disk usage, network traffic, and application-specific metrics. This makes Graphite an indispensable tool for system administrators, network administrators, and developers who can monitor the performance and health of their systems and applications, and then troubleshoot issues as they arise.
Many organizations also use it to track and analyze business-critical metrics such as website traffic, user engagement, revenue, and other Key Performance Indicators (KPIs) thus allowing them to gain insights into the performance of their business and make accurate data-driven decisions based on it.
In summary, Graphite is a monitoring tool that is used by users who want to collect, store, and analyze time-series data, and is mostly used in IT and DevOps fields, but that also can be used in other industries just as well.
Graphite – in a nutshell
A brief look at the features of Graphite looks like this:
- You can see all of your server, application, and infrastructure data in one place
- When combined with other tools, it sends out detailed alert notifications when your data crosses thresholds or stops reporting
- Users have team sharing and granular access controls
- There are redundant backup and data export tools to ensure offline data security and retrieval capabilities
- It offers advanced statistical and data-slicing tools for the in-depth analysis of data
A more in-depth metrics source list includes data garnered from events like:
- End-to-end application latency
- Disk, CPU, and memory use
- Machine learning pipelines
- Cluster health
- Network traffic including intrusion detection risks
Why choose Graphite?
Graphite is an ideal monitoring tool because it offers many advantages and even some unique features. A few examples include:
- Easy to master Graphite is the choice monitoring tool because it uses simple user interfaces (UIs) and formats which means it has a shallow learning curve for new users. The web interface empowers users to render graphs quickly and immediately, with no need for training or courses.
- Cross-integration Wherever the data metrics come from, it is easy to push to Graphite, regardless if the source is a script, an application, or a command-line terminal.
- Exportable graphs Graphs are created from URL-based APIs, which makes it easy to embed them in dashboards or websites.
- Global support network Graphite has a large community of users from diverse backgrounds and various industries; this means that new and unique functions are always being contributed to help keep the data monitoring current and the users well informed.
The best feature Graphite provides is its simple API that enables everyone – developers, system administrators, network engineers, and security engineers – to store, retrieve, and visualize time-series data at will. They can also correlate data across all these disciplines to understand the full impact of code changes, network infrastructure changes, or security policies in near real-time.
Finally, business decisions are increasingly data-driven, and Graphite provides all the tools to help business-level users collect, store, retrieve, and analyze data quickly and effectively.
Set up and installation of Graphite
Before we move on to the topic of the installation of Graphite, let us have a look at the main components of the tool.
The tool has three main components:
- Carbon This is a Twisted daemon that listens for time-series data
- Whisper A simple database library for storing captured time-series data
- Graphite web app The main code, which is a Django web app, which renders graphs on-demand using the Cairo graphics library.
It is these three components that come together to listen for, capture, store, and render data that is inputted from various systems in a network.
Requirements
The requirements for installing the Graphite monitoring tool vary depending on the specific version and installation method you choose. However, some general requirements include:
- A supported operating system Graphite can be installed on various UNIX and UNIX-like operating systems, including Linux and macOS.
- A web server Graphite requires a web server for visualization and web-based interaction. Common web servers used with Graphite include Apache and Nginx.
- A database Graphite uses a database to store metrics data, such as Whisper, which is a built-in database that ships with Graphite, or other database systems like MySQL or PostgreSQL.
- Python Graphite is written in Python and requires a recent version of Python to be installed.
- Additional Python modules Graphite requires additional Python modules such as Django, Twisted, and pytz.
- Memory and storage Graphite requires a certain amount of memory and storage depending on the number of metrics you will be collecting.
- Internet connectivity Graphite requires internet connectivity to allow clients to send metrics to the server.
As mentioned earlier, additional software like Carbon, which is a service that listens for metrics and stores them in the Whisper database, and Graphite-web, which provides web-based visualization.
There are also several other dependencies required for additional features:
- Render caching Solutions like Memcached and python-memcached
- LDAP authentication python-ldap
- AMQP support txamqp
- RRD support python-rrdtool
Note: It is recommended to check the specific version of Graphite you are using for detailed requirements and installation instructions.
Installation
Installing Graphite is pretty straightforward. All you have to do is go to their installation page and click on the “Install” button.
Installing any software can be a burden when you just want to give it a trial run. That’s why one of the Graphite developers created the Synthesize project.
Synthesize is a fully automated installation and configuration script for the Graphite stack. It was initially designed as a guide for new users, demonstrating how to install Graphite manually, but evolved into a bespoke deployment mechanism in its own right. Although Synthesize targets Ubuntu 14.04 LTS specifically, it includes a Vagrant file to allow anyone to easily and painlessly try out Graphite on their laptop or development system.
Unfortunately, native Graphite on Windows is completely unsupported, but you can run Graphite on Windows in Docker, or the Installing via Synthesize article will help you set up a Vagrant VM that will run Graphite. To leverage this, you will need to install Vagrant.
Alternatively, Graphite can be downloaded from GitHub. For the more tech-savvy user, the entire project, as well as all necessary information about the installation process, can also be obtained from the knowledge repository here.
Creating your first dashboard
Graphite offers a comprehensive instructions page on how you can create, and edit, your dashboards – both new and old.
For those opting for the hosted alternative of Graphite, there are also websites – here is one – to show the creation, configuration, and usage of dashboards.
Using your new Graphite installation
Alerts in Graphite
For a better user experience (UX) Hosted Graphite can be used as an add-on for providing application performance metrics as it offers a simple, scalable way to measure metric data from the application.
Users can get alert notifications instantly when set performance milestones are not met, metrics stop reporting or show unexpected behavior. They can receive notifications via apps like PagerDuty, VictorOps, HipChat, Slack, Email, Webhooks, and more.
What’s more, Graphite sends out full alert histories that help recipients with more information that then allows them to accurately track incidents and troubleshoot them quickly and effectively.
Hosted Graphite alerting breakdown
Users of Hosted Graphite can click on the “Alert” icon to open the alert creation panel. Information that is needed includes:
- Name The notification name to label the alert as well as describe it for easier understanding at a later date.
- Metric Pattern Used to identify the data that is being gauged or tested against a milestone or criteria, respectively.
- Alert Info The message that is sent out with notifications and can contain an arbitrary string as the description of the alert, steps to follow, or references to documentation.
You can check a graph of your desired metric with the “Check Metric Graph” button. When you’re finished, click on “Confirm Metric Choice” to proceed to the Alert Criteria screen.
For the users that opt to forgo Hosted Graphite, there is the option of using other solutions that include:
- graphite-beacon A simple alerting application for Graphite that is asynchronous and sends notification alerts based on the monitoring tool’s metrics. It is light and easy to deploy.
- graphite-to-Zabbix This is a tool that allows for the creation of Zabbix alerts based on Graphite data inputs.
- rearview arguably the best alert solution for Graphite, this real-time monitoring framework sits on top of the monitoring tool’s time series data, allowing users to create monitors and alerts.
Who is Graphite for?
Okay, it is now time to have a look at some companies that have adopted Graphite and use it to monitor their digital infrastructure.
- Booking.com Booking.com is one of the most popular online travel booking sites on the Internet and also happens to run one of the largest Graphite installations. All of Booking.com’s primary system, network, and application performance data is stored in Graphite, along with parts of its security and business intelligence data.
- GitHub Apart from its source code is shared on GitHub, Graphite also provides the platform’s engineering and operations teams with the tools to collect and correlate massive amounts of time-series data. They use Graphite to comprehensively monitor their host, application, service, business, and financial metrics.
- Etsy Etsy, a popular online marketplace that brings artists, craftspeople, and consumers together to buy and sell unique and homemade crafts, also uses Graphite to track and monitor their application metrics as well as, for example, logins or checkouts at any given point. The data is channeled from StatsD, a network daemon that listens for statistics like counters and timers, and that are sent over UDP or TCP.
Graphite Integrations
Graphite can be further enhanced by – or itself be used for enhancing – various popular applications and systems. Some examples include:
- Windows Server Graphite can be configured to run with Windows Server.
- SqlToGraphite SqlToGraphite is an agent for Windows, written in .NET, that is used to collect metrics using database plugins for database solutions like SQL Server and Oracle. It polls an endpoint with an SQL query and pushes the results into Graphite.
- Therry Therry is a simple web service that caches Graphite metrics and exposes an endpoint that can be either dumped or searched against for further in-depth analysis using a third-party solution.
- Logster With Logster we have a tool for reading log files and generating metrics in Graphite. It is ideal for visualizing trends of events that occur in error logs.
On getting started with Graphite
As we have just seen, Graphite is a tool that can be used to monitor any asset in a digital stack – as long as it has any measurable output. This means it can be implemented on a network or in a data center, regardless of its size.
The fact that it is free means anyone can afford it. Any overhead costs are offset by the fact that there is a large user community that can help with any issues that might come up while using the tool.
This all means that Graphite is an ideal tool for anyone – individuals or large businesses – looking for performance and health information on their digital stack.
We would like to hear your opinion, leave us a comment below.
