Skip to main content

Overview

Hardware issues can occasionally occur where something is not working correctly and our monitoring systems have not picked up on it. This is usually unsatisfactory CPU performance, slow disk performance, or a fault with the server hardware itself. This page explains how to diagnose each of these and what information to collect before opening a ticket. As always, please open a ticket if you run into any problems, even if you are unsure of the cause. We greatly appreciate these reports, as they allow us to find underlying issues that our monitoring may have missed. If you are troubleshooting slow downloads, high latency, or packet loss, see Network Problems instead.

Before You Start

Before assuming a hardware fault, rule out the most common causes, which are almost always inside the server itself:
  • Runaway processes. Run top or htop and check whether a single process is consuming all of the CPU or memory.
  • Memory exhaustion and swapping. Run free -h. If your server is out of memory and swapping heavily, everything will feel slow, including the disk.
  • A full disk. Run df -h and df -i. A disk at 100 percent capacity, or one that is out of inodes, causes failures that look like hardware faults.
  • Recent changes. Consider whether the problem started after a package update, a kernel upgrade, a configuration change, or a new workload being deployed.
  • A reboot. If the server has been running for a long time or is in an unclear state, a clean reboot from the VPS control panel is worth trying first.
It is also worth checking whether the problem is constant or happens only at certain times of day. That distinction is very useful to us when investigating.

CPU Performance

Checking CPU Steal

If you are experiencing unsatisfactory CPU performance on a virtual server, run the top command and check the value labelled st on the CPU line. This indicates CPU steal, which is the percentage of CPU time your VPS is waiting on from the hypervisor.
We generally aim to keep CPU steal at 0 percent across our hypervisors. However, in some cases where CPU contention is higher, you may see values between 5 and 10 percent. This is not unusual in shared environments where multiple virtual servers are competing for CPU resources. Even if the hypervisor still has CPU headroom, some steal can occur if the host is above 50 percent usage and starting to rely on hyperthreading, or due to other low-level factors. If your CPU steal value regularly exceeds 10 percent, please open a support ticket.

Monitoring Steal Over Time

CPU steal often fluctuates, so a single top reading is rarely enough to demonstrate a problem. Tools such as HetrixTools or Netdata can monitor CPU steal continuously without you having to watch top. These tools generate graphs that are very helpful for our team when diagnosing patterns or identifying specific times when performance degrades. If you cannot install a monitoring agent, you can still capture a short sample from the command line:
This prints one line per second for 60 seconds. The st column at the far right is CPU steal. Send us the full output along with the time and timezone it was captured.

If There Is No Steal

If you are not seeing any CPU steal but performance is still poor, please reach out and we can investigate further. A few things worth knowing first:
  • Benchmarks such as Geekbench may report lower scores when the VPS is using hyperthreaded vCPU cores.
  • The clock speed reported inside your VM is frequently inaccurate. See Hardware Information for details on the processors we use and the clock speeds they actually run at.
  • In rare cases, abnormal CPU steal is caused by a hardware fault rather than contention. We have seen situations where faulty memory led to spikes in CPU steal despite otherwise normal conditions.

Dedicated Servers

CPU steal does not exist on dedicated or bare metal machines, since there is no hypervisor scheduling your cores. If you are seeing performance issues on dedicated hardware, install lm-sensors and check the CPU temperatures as a first step:
Sustained temperatures at or near the processor’s thermal limit will cause it to throttle and lose performance. If you see this, please open a ticket so that we can inspect the cooling on the machine.

Disk Performance

Running a Benchmark

We use enterprise Gen3 or Gen4 NVMe SSDs across almost all of our VPS hypervisors, so disk performance issues are extraordinarily rare. To measure your disk, we recommend running the following and checking the fio results it outputs:
Note that yabs.sh is a third-party tool, so use it at your own risk.

Interpreting the Results

As long as the 1m result is above 1 GB/s and the 4k results are above 100 MB/s, your disk performance is within the expected range. Our virtual servers typically exceed these figures by a wide margin, as in the sample output below:
Keep two things in mind when comparing your numbers:
  • Disk throughput is shared between the virtual servers on a hypervisor, so results will vary somewhat between runs.
  • Linux caches disk contents in memory, which can inflate 4k and 1m results well above what the underlying storage can sustain.
If you are consistently seeing under 1 GB/s on 1m or under 100 MB/s on 4k, please open a ticket so that we can investigate.

High I/O Wait

If throughput looks acceptable but the server still feels slow, check the wa (I/O wait) value in top. A persistently high I/O wait combined with low throughput usually points to a workload issuing a large number of small synchronous writes, such as an unoptimised database. Running iostat -x 1 from the sysstat package will show you which device is busy and how long requests are queuing.

Memory Issues

If processes on your server are being killed unexpectedly, check whether the kernel is running out of memory before suspecting faulty hardware:
Entries here mean the OOM killer terminated a process because the server ran out of available memory. That is a capacity issue rather than a hardware fault, and is resolved by reducing memory usage, adding swap, or upgrading your plan. On dedicated servers you can also check for correctable and uncorrectable ECC memory errors:
Any uncorrectable errors, or a steadily increasing count of correctable errors, should be reported to us immediately so that we can arrange a memory replacement. ECC errors are not visible from inside a virtual server, so if you suspect a memory fault on a VPS, please open a ticket and we will check the host.

Reporting a Hardware Issue

Opening a ticket with the information below lets us start investigating right away instead of asking follow-up questions:
  • A clear description of the symptom and how you are measuring it.
  • The date, time, and timezone the problem occurred, and whether it is constant or intermittent.
  • The raw output of any commands you ran, such as top, vmstat 1 60, or the full yabs.sh results. Please paste the text rather than sending a screenshot where possible.
  • Monitoring graphs covering the period in question, if you have them.
  • Anything that changed on the server shortly before the problem started.
Tickets can be opened from the client area. If we find a fault with the underlying hardware, we will migrate your server to healthy hardware or replace the faulty component, and we will keep you updated throughout.