Synthetic Monitoring

Simulate visitor interaction with your site to monitor the end user experience.

View Product Info

FEATURES

Simulate visitor interaction

Identify bottlenecks and speed up your website.

Learn More

Real User Monitoring

Enhance your site performance with data from actual site visitors

View Product Info

FEATURES

Real user insights in real time

Know how your site or web app is performing with real user insights

Learn More

Infrastructure Monitoring Powered by SolarWinds AppOptics

Instant visibility into servers, virtual hosts, and containerized environments

View Infrastructure Monitoring Info

Comprehensive set of turnkey infrastructure integrations

Including dozens of AWS and Azure services, container orchestrations like Docker and Kubernetes, and more 

Learn More

Application Performance Monitoring Powered by SolarWinds AppOptics

Comprehensive, full-stack visibility, and troubleshooting

View Application Performance Monitoring Info

Complete visibility into application issues

Pinpoint the root cause down to a poor-performing line of code

Learn More

Log Management and Analytics Powered by SolarWinds Loggly

Integrated, cost-effective, hosted, and scalable full-stack, multi-source log management

 View Log Management and Analytics Info

Collect, search, and analyze log data

Quickly jump into the relevant logs to accelerate troubleshooting

Learn More

Further exploring the SysRq support in Linux

The sysrq key

This is a post by guest blogger Wesley David. You can find more information about Wesley at the end of the article.

In my last article “Every sysadmin needs a little SysRq magic” I introduced Linux’s implementation of the SysRq feature that was created years ago in an era of computing history that is rapidly being forgotten. In this follow up article, I’d like to deepen your understanding of Linux’s SysRq support.

Forcing the issue

There are two ways to interact with Linux’s SysRq support. The first and most obvious is through the Magic SysRq keystrokes. In order to use the Magic SysRq keystrokes, you need to enable the SysRq feature in your kernel. However, that’s not the whole truth. While the kernel has to be compiled with SysRq support, it doesn’t need to have that support enabled for SysRq commands to be processed. How is that possible? Through the second way that you can interact with Linux’s SysRq support: by passing a character to the sysrq-trigger node of procfs.

Tucked away in the procfs hierarchy (assuming that you have procfs support in your kernel) is the /proc/sysrq-trigger node. You can send any of the Magic SysRq keys to it with a simple echo command. For example, to forcibly restart your computer in an unpleasant way you can try the following example (be careful!):

echo “b” > /proc/sysrq-trigger

Regardless of if your installation has sysrq enabled, your system will reset. The disabling or enabling of Magic SysRq keys is only referring to the ability to physically trigger them. Go ahead and test this out by verifying that Magic SysRq keys are disabled with the following command:

cat /proc/sys/kernel/sysrq

If the returned value is “1”, then turn off Magic SysRq keystroke support with one of the following commands:

echo “0” > /proc/sys/kernel/sysrq

or

sysctl -w kernel.sysrq=“0”

Before going any further, don’t forget to verify that sysrq is disabled by looking at the value of /proc/sys/kernel/sysrq. If you’re feeling lucky, go ahead and strike a Magic SysRq key combination. Try the innocuous Alt+SysRq+h to see the SysRq help information. Only you won’t see it because you just disabled support for it.

However, you’ll still be able to manually send a command to the sysrq-trigger node of procfs. Don’t believe me? Go ahead and run the command echo “h” > /proc/sysrq-trigger. (That’s virtually the only SysRq command that won’t bring some serious pain to your system.) If all works as it should, you’ll see the SysRq help information sent to your console. (Note: If you still don’t see anything, come back later for the next part in this series because I will address some common reasons why SysRq doesn’t appear to be working.)

Saying that one has “disabled” Magic SysRq Key support is a bit deceiving because the kernel is still capable of responding to SysRq commands, but it’s simply not going to respond to the hardware interrupt signal that is sent to it when pressing the specific Magic SysRq keystrokes on a keyboard. With the kernel.sysrq feature disabled, the only safety you are afforded is that you can safely hammer on any of the Magic SysRq key combinations with impunity.

It should be noted that the ability to manually send SysRq commands to the kernel is less useful than the true hardware interrupt Magic SysRq keys because, typically, if you have shell access to /procfs, then you will have significantly less scenarios where the SysRq commands will be more useful than equivalent standard shell commands.

Go console or go home

Not every sysadmin has a fancy ILO, DRAC or other IPMI interface to their servers. Many of us still use console ports for low level shell access that is independent of a network connection. If you find yourself attempting to use Magic SysRq keystrokes through a console connection, you will be frustrated. Why is that? Because hardware console connections typically require the use of the ‘break’ key to trigger the SysRq interrupt!

Also of note is that triggering the SysRq interrupt using a console connection isn’t quite the same as using a standard locally attached keyboard. Instead of pressing Alt+Break+[key], you only need to press and release the break key once, and then press the appropriate command key within five seconds of the break key being pressed and released. For example, at a physical console connection, pressing and releasing break and then pressing ‘e’ would send a SIGTERM signal to every process (no Alt key required).

If you need to send an actual ‘break’ keystroke through the console connection you’ll need to press the break key twice.

Other architectures

Different processor architectures may use different keys for triggering the SysRq interrupt. For example, Sun SPARC computers will typically cause the SysRq interrupt to trigger when you press Alt+Stop+[key] even though Sun keyboards also have a SysRq key.

The PowerPC uses Alt+Print Screen+[Key] or Alt+F13+[Key]. However, you might also be able to use the Print Screen or F13 key without the Alt key preceding it. For example: Print Screen+k to SIGKILL every process.

If this all sounds confusing and ambiguous, it’s because it is. Even the official documentation for the Magic SysRq keys show some confusion and uncertainty for what may and may not work for different architectures.

I’ll leave it there for now. In the third and final installment in this series of articles about SysRq, I’ll tell you all about how to troubleshoot SysRq and how to limit SysRq’s capabilitites. Stay tuned!

About Wesley David

Wesley David is a systems administrator that works on anything that plugs into a wall. You can follow his misadventures on his blog www.TheNubbyAdmin.com or if you have a short attention span, his Twitter account @Nonapeptide.

End-User Monitoring for IT Operations Monitoring

By Rachel Frnka I’ll be the first to admit one of my weaknesses is public [...]

End-User Monitoring: Best Practices and Tools

Poor application performance, besides being a sign of potential problems, is a [...]

How Website Monitoring Can Help Improve the End-User Experience

Making your customers happy is essential in any industry, but it’s imperative [...]

Monitoring a RESTful API on a Headless CMS Using Pingdom

There are plenty of ways to gain insights on website availability and performan [...]

website statistics
Website Load Time in 2021: How Fast Is Fast Enough?

Today, in the fast-paced digital world, it’s crucial for organizations to dev [...]

Monitor your website’s uptime and performance

With Pingdom's website monitoring you are always the first to know when your site is in trouble, and as a result you are making the Internet faster and more reliable. Nice, huh?

START YOUR FREE 30-DAY TRIAL

MONITOR YOUR WEB APPLICATION PERFORMANCE

Gain availability and performance insights with Pingdom – a comprehensive web application performance and digital experience monitoring tool.

START YOUR FREE 30-DAY TRIAL
Start monitoring for free