Synthetic Monitoring

Simulate visitor interaction with your site to monitor the end user experience.

View Product Info

FEATURES

Simulate visitor interaction

Identify bottlenecks and speed up your website.

Learn More

Real User Monitoring

Enhance your site performance with data from actual site visitors

View Product Info

FEATURES

Real user insights in real time

Know how your site or web app is performing with real user insights

Learn More

Infrastructure Monitoring Powered by SolarWinds AppOptics

Instant visibility into servers, virtual hosts, and containerized environments

View Infrastructure Monitoring Info
Comprehensive set of turnkey infrastructure integrations

Including dozens of AWS and Azure services, container orchestrations like Docker and Kubernetes, and more 

Learn More

Application Performance Monitoring Powered by SolarWinds AppOptics

Comprehensive, full-stack visibility, and troubleshooting

View Application Performance Monitoring Info
Complete visibility into application issues

Pinpoint the root cause down to a poor-performing line of code

Learn More

Log Management and Analytics Powered by SolarWinds Loggly

Integrated, cost-effective, hosted, and scalable full-stack, multi-source log management

 View Log Management and Analytics Info
Collect, search, and analyze log data

Quickly jump into the relevant logs to accelerate troubleshooting

Learn More

How we use ZFS to back up 5TB of MySQL data every day

We collect a lot of data every day. In 2012, our monitoring system registered almost 65 million downtimes, we stored 106 billion monitoring results, and we handled other types of data for 300,000+ customers.

Something we’re now using for a small part of all the data we process and store each day is ZFS and its snapshot feature. So far it’s proven to be a reliable, fast, and flexible solution, and we think this is something you also would want to consider for your infrastructure.

Why we like ZFS

zfs on linux

If you’ve ever used ZFS we’re sure you agree with us that there’s plenty to like about it. It’s simply packed with cool features, too many to mention, so here are some of the ones we find especially useful:

Data integrity – To never trust the underlying hardware was one of the stated design principles behind ZFS according to its creators. When data is generated by an application a checksum is created by ZFS. The checksum and the data are stored separately on disk, reducing the risk for both being deleted or damaged in case something would happen. Later, when data is read back, it’s validated against the checksum. If there is no match, it will then attempt to read the data from another disk in the RAID set. All in all, much of the functionality that ensures data integrity is resting with the file system rather than higher-level applications or hardware.

Capacity – What’s not to like about a filesystem that supports virtually unlimited storage? ZFS is a 128-bit file system enabling a maximum size of a single file of 16 exabytes, the equivalent of almost 17M TB.

Snapshots – In ZFS a snapshot is a ready-only copy of a file system. Think if it as an exact copy of the file system as it existed at the time the snapshot was created. At first the snapshot doesn’t take up any additional space, but as data is added or changed, it continues to reference the original data, thereby maintaining a persistent file system. The way snapshots work makes them very fast as well as efficient with storage space. ZFS snapshots can be sent over a network to other machines for backup or other purposes.

How we use ZFS to backup MySQL databases

The scenario that we’ve come the furthest with in terms of using ZFS is backing up MySQL databases. For some servers, we’ve replaced MySQL Dumps, rsync, and storage appliances with ZFS snapshots.

This has essentially made the process of backing up MySQL databases faster and less complicated.

Here’s a brief description of what we do:

  1. We lock the MySQL tables and flush them thereby clearing caches and making sure the latest data is written to disk.

  2. We take a snapshot of the file system with zfs snapshot.

  3. We unlock the MySQL tables.

  4. Finally, in most instances, we send the snapshot to another machine for offsite backups using zfs send.

In this example, we’re using few steps and commands, thereby reducing the complexity and the risk that something would go wrong. Also, the whole process usually takes only a few seconds even for many GB of data.

There’s likely more ZFS in Pingdom’s future

Our use of ZFS so far has been rather limited, but we clearly see that the sky’s the limit. We’re very happy with how ZFS has performed for us as well the functionality it offers us. Right now we’re looking into how we can use it for more applications, and we’re confident we’ll utilize it more in the future.

What’s your experience of ZFS? Are you using it for anything in particular? Let us know in the comments below.

Webpages Are Getting Larger Every Year, and Here’s Why it Matters

Last updated: February 29, 2024 Average size of a webpage matters because it [...]

A Beginner’s Guide to Using CDNs

Last updated: February 28, 2024 Websites have become larger and more complex [...]

The Five Most Common HTTP Errors According to Google

Last updated: February 28, 2024 Sometimes when you try to visit a web page, [...]

Page Load Time vs. Response Time – What Is the Difference?

Last updated: February 28, 2024 Page load time and response time are key met [...]

Can gzip Compression Really Improve Web Performance?

Last updated: February 26, 2024 The size of the web is slowly growing. Over [...]

Monitor your website’s uptime and performance

With Pingdom's website monitoring you are always the first to know when your site is in trouble, and as a result you are making the Internet faster and more reliable. Nice, huh?

START YOUR FREE 30-DAY TRIAL

MONITOR YOUR WEB APPLICATION PERFORMANCE

Gain availability and performance insights with Pingdom – a comprehensive web application performance and digital experience monitoring tool.

START YOUR FREE 30-DAY TRIAL
Start monitoring for free