Dell R630 Custom Fan Control

Overview

The Dell PowerEdge R630 server can be quite loud, especially with aftermarket PCI cards or non-standard firmware. The internal fan controller often maintains unnecessarily high RPMs. This guide shows you how to implement intelligent fan control using IPMI to dramatically reduce noise levels while maintaining safe operating temperatures.

How It Works

The fan control system uses a feedback loop that:

• Monitors CPU package temperatures, inlet temperature, and exhaust temperature
• Dynamically adjusts fan speeds to maintain target temperatures
• Implements safety mechanisms to prevent overheating
• Automatically restores default fan control on exit
• Logs all temperature readings and fan speed changes

Key Features

• Intelligent Control Loop: Gradually adjusts fan speeds based on temperature differences
• Safety First: Automatically maxes out fans if alarm temperature is reached
• Critical Protection: Triggers shutdown if critical temperature persists
• Hysteresis Factor: Becomes more aggressive as temperatures approach limits
• Exit Safety: Ensures automatic fan control is restored on exit

Prerequisites

• Dell PowerEdge R630 server
• IPMI tools installed (apt-get install ipmitool)
• Root or sudo access
• Basic understanding of systemd services

Fan Control Script

User-Configurable Parameters

These parameters at the top of the script can be adjusted to suit your environment:

Parameter	Default	Description
DEBUG	0	Enable debugging output (0=off, 1=on)
TARGET_TEMP	35°C	Target CPU temperature
HYSTERESIS_OFFSET	5°C	Temperature buffer for adjustments
LOOP_DELAY	5s	Delay between control loop iterations
START_SPEED	30%	Initial fan speed
MIN_SPEED	5%	Minimum allowed fan speed
MAX_SPEED	70%	Maximum allowed fan speed
PACKAGE_ALARM_TEMP	70°C	Temperature to trigger max fan speed
CRITICAL_TEMP	85°C	Critical temperature threshold
HYSTERESIS_WAIT	300s	Time at critical temp before action
CRITICAL_EVENT_COMMAND	shutdown -h now	Command to run at critical temperature
HYSTERESIS_MULTIPLIER	2.0	Exhaust temperature sensitivity multiplier

Complete Script

Save this as fan-control.sh:

#!/bin/bash
# Copyright 2024 - 2024, Jaremy Hatler
# SPDX-License-Identifier: MIT

## Dell R630 Fan Control Script

# This script controls the fan speed on a Dell r630 server in response to the CPU temperature.
#
# In some cases (nonstandard firmware, aftermarket PCI cards, etc.), the internal fan controller
# will consistently maintain high RPMs. This script uses IPMI to enable manual fan control and set
# the fan speed across all fans. An exit trap is used to ensure automatic fan control is enabled
# upon exiting. On each iteration, the script logs its data and any changes it makes.
#
# A control loop is used which monitors the CPU package temperatures, inlet temp, and exhaust temp
# and adjusts the fan speeds in response. The user can set the parameters of this control loop in
# the section below, including the loop delay. At a high level, the control loop begins by setting
# the fans to the configured start speed. It will then use the difference between the target and
# actual package temperatures to determine whether to increase or decrease the fan speed, and by
# how much.
#
# If the packages are at or below the target temp, the control loop will reduce the fans speeds
# until an equilibrium is reached. If the inlet temperature is higher than the target, it plus
# the configured hysteresis offset will be used as the target.
#
# If the package alarm temperature is met, the fans will immediately be set to their configured
# maximums until the temperature begins to drop. If the configured critical temperature is met
# for longer than the specified hysteresis wait, the user-specified critical event command will
# be run. The script becomes more aggressive as the exhaust temperature and package temperature
# near the package alarm temperature.

# Proof of concept commands:
#   Get Package 0 Temp: cat /sys/devices/platform/coretemp.0/hwmon/hwmon0/temp1_input
#   Get Package 1 Temp: cat /sys/devices/platform/coretemp.1/hwmon/hwmon1/temp1_input
#   Get Inlet Temp: ipmitool sensor get 'Inlet Temp' | grep Reading | cut -f2 -d:
#   Get Exhaust Temp: ipmitool sensor get 'Exhaust Temp' | grep Reading | cut -f2 -d:
#   Enable manual fan control: ipmitool raw 0x30 0x30 0x01 0x00
#   Disable manual fan control: ipmitool raw 0x30 0x30 0x01 0x01
#   Set Fan Speeds to 0%: ipmitool raw 0x30 0x30 0x02 0xff 0x00
#   Set Fan Speeds to 100%: ipmitool raw 0x30 0x30 0x02 0xff 0x64


## User Parameters

DEBUG=0 # Enable debugging
TARGET_TEMP=35 # The target temperature in Celsius
HYSTERESIS_OFFSET=5 # The hysteresis offset in Celsius
LOOP_DELAY=5 # The delay between each loop iteration in seconds
START_SPEED=30 # The starting fan speed in percent
MIN_SPEED=5 # The minimum fan speed in percent
MAX_SPEED=70 # The maximum fan speed in percent
PACKAGE_ALARM_TEMP=70 # The package alarm temperature in Celsius
CRITICAL_TEMP=85 # The critical temperature in Celsius
HYSTERESIS_WAIT=300 # The hysteresis wait time in seconds
CRITICAL_EVENT_COMMAND="shutdown -h now" # The command to run when the critical temperature is met
HYSTERESIS_MULTIPLIER="2.0" # The multiplier for the exhaust temperature hysteresis

## Exit Trap

function _exit_trap() {
    # Ensure automatic fan control is enabled upon exiting
    _disable_manual_fan_control
}


## Functions

function _pkg_temp() {
    # Returns the average of all the platform core temperatures
    local _acc=0
    local _count=0
    for _core in /sys/devices/platform/coretemp.*; do
        _acc=$(( _acc + $(cat $_core/hwmon/hwmon*/temp1_input) ))
        _count=$(( _count + 1000 ))
    done
    echo $((_acc / _count))
}

function _inlet_temp() {
    # Returns the inlet temperature
    ipmitool sensor get 'Inlet Temp' | grep -F 'Sensor Reading' | cut -f2 -d: | cut -f2 -d' ' 2>/dev/null
}

function _exhaust_temp() {
    # Returns the exhaust temperature
    ipmitool sensor get 'Exhaust Temp' | grep -F 'Sensor Reading' | cut -f2 -d: | cut -f2 -d' ' 2>/dev/null
}

function _set_fan_speed() {
    # Sets the fan speed to the given percentage
    local _speed=$1

    # Ensure the speed is within the valid range
    if [[ $_speed -lt $MIN_SPEED ]]; then
        _speed=$MIN_SPEED
    elif [[ $_speed -gt $MAX_SPEED ]]; then
        _speed=$MAX_SPEED
    fi

    # Convert the speed to hex
    local _hex_speed=$(printf "0x%x" $_speed)

    # Set the fan speed
    ipmitool raw 0x30 0x30 0x02 0xff $_hex_speed >&/dev/null
}

function _enable_manual_fan_control() {
    # Enables manual fan control
    ipmitool raw 0x30 0x30 0x01 0x00 >&/dev/null
}

function _disable_manual_fan_control() {
    # Disables manual fan control
    ipmitool raw 0x30 0x30 0x01 0x01 >&/dev/null
}

function _calculate_fan_speed() {
    # Calculates the fan speed based on the target temperature and the actual temperature
    local _target_temp=$1
    local _pkg_temp=$2
    local _exhaust_temp=$3
    local _fan_speed=$4
    local _start_speed=$4

    # Int Rounding Support
    local _rnd_bgn="scale=8; a=("
    local _rnd_end=" + 0.5); scale=0; a/1"

    # Starting factors
    local _hysteresis_factor="1.0"
    local _speed_factor="0"

    # Calculate the hysteresis temperature and round to the nearest integer
    local _hysteresis_offset=$(echo "$_rnd_bgn($HYSTERESIS_OFFSET * $HYSTERESIS_MULTIPLIER)$_rnd_end" | bc)
    local _hysteresis_temp=$(echo "$_rnd_bgn($PACKAGE_ALARM_TEMP - $_hysteresis_offset)$_rnd_end" | bc)

    # Calculate the difference between the actual temperature and target
    local _temp_diff=$(( _pkg_temp - _target_temp ))
    local _temp_diff_abs=${_temp_diff#-}

    # Offset speed factor based on the exhaust temperature
    if [[ $_exhaust_temp -gt $_hysteresis_temp ]]; then
        _speed_factor=$(echo "$_speed_factor + 0.75" | bc -l)
    fi

    # Calculate the fan speed change based on the range of the temperature difference
    if [[ $_temp_diff_abs -gt 32 ]]; then
        _speed_factor="2.0"
    elif [[ $_temp_diff_abs -gt 16 ]]; then
        _speed_factor="1.375"
    elif [[ $_temp_diff_abs -gt 8 ]]; then
        _speed_factor="0.875"
    elif [[ $_temp_diff_abs -gt 4 ]]; then
        _speed_factor="0.5"
    elif [[ $_temp_diff_abs -gt 2 ]]; then
        _speed_factor="0.25"
    fi

    # Update hysteresis factor based on the package temperature compared to the hysteresis temperature
    if [[ $_pkg_temp -gt $_hysteresis_temp ]]; then
        _hysteresis_factor=$(echo "$_hysteresis_factor + 2" | bc)
    else
        _hysteresis_factor=$(echo "$_hysteresis_factor + 1" | bc)
    fi

    # Calculate the speed change based on the speed factor and hysteresis factor, ensure the range
    local _speed_change=$(echo "$_rnd_bgn($_speed_factor * $_hysteresis_factor)$_rnd_end" | bc)
    if [[ $_speed_change -gt 5 ]]; then
        _speed_change=5
    fi

    # Give the speed change a direction
    if [[ $_temp_diff -lt 0 ]]; then
        _speed_change=$(( _speed_change * -1 ))
    fi

    # Ensure the fan speed is reduced if the package temperature at or below the target
    if [[ $_temp_diff -le 0 ]] && [[ $_speed_change -eq 0 ]]; then
        _speed_change=-1
    fi

    # Calculate the new fan speed
    _fan_speed=$(( _fan_speed + _speed_change ))
    if [[ $_fan_speed -lt 1 ]]; then
        _fan_speed=1
    elif [[ $_fan_speed -gt 99 ]]; then
        _fan_speed=99
    fi

    # Debugging
    if [[ "$DEBUG" != "0" ]]; then
        echo "Start Speed: $_start_speed"
        echo "Package Temp: $_pkg_temp"
        echo "Exhaust Temp: $_exhaust_temp"
        echo "Target Temp: $_target_temp"
        echo "Hysteresis Temp: $_hysteresis_temp"
        echo "Temp Diff: $_temp_diff"
        echo "Speed Factor: $_speed_factor"
        echo "Hysteresis Factor: $_hysteresis_factor"
        echo "Fan Speed: $_fan_speed"
    fi >&2

    echo $_fan_speed
}

function _log_data() {
    # Logs the data to a file
    local _pkg_temp=$1
    local _inlet_temp=$2
    local _exhaust_temp=$3
    local _target_temp=$4
    local _fan_speed=$5

    # Log the data
    local _msg="$(date +%s) $(date) - "

    _msg+="Package Temp: $_pkg_temp, "
    _msg+="Inlet Temp: $_inlet_temp, "
    _msg+="Exhaust Temp: $_exhaust_temp, "
    _msg+="Target Temp: $_target_temp, "
    _msg+="Fan Speed: $_fan_speed [$MIN_SPEED - $MAX_SPEED]"

    echo "$_msg" | tee -a /var/log/fanctrl.log >&2
}

function _run_critical_event() {
    # Runs the critical event command
    eval "$CRITICAL_EVENT_COMMAND"
}

## Main
function _main() {
    local critical_event_wait=0 # The time since the critical temperature was met
    local fan_speed=$START_SPEED # The current fan speed
    local new_fan_speed=$START_SPEED # The new fan speed

    # Enable manual fan control and set the initial fan speed
    _enable_manual_fan_control
    _set_fan_speed $fan_speed

    # Main control loop
    while true; do
        # Get the current temperatures
        local pkg_temp=$(_pkg_temp)
        local inlet_temp=$(_inlet_temp)
        local exhaust_temp=$(_exhaust_temp)

        # Calculate the target temperature
        local target_temp=$TARGET_TEMP
        if [[ $inlet_temp -gt $target_temp ]]; then
            target_temp=$((inlet_temp + HYSTERESIS_OFFSET))
        fi

        # Debugging
        if [[ "$DEBUG" != "0" ]]; then
            echo "Start Fan Speed: $fan_speed"
            echo "Crtical Wait: $critical_event_wait"
            echo "Package Temp: $pkg_temp"
            echo "Inlet Temp: $inlet_temp"
            echo "Exhaust Temp: $exhaust_temp"
            echo "Target Temp: $target_temp"
        fi >&2

        # Check for critical temperature
        if [[ $pkg_temp -ge $CRITICAL_TEMP ]]; then
            critical_event_wait=$((critical_event_wait + LOOP_DELAY))
            if [[ $critical_event_wait -ge $HYSTERESIS_WAIT ]]; then
                _run_critical_event
            fi
        else
            critical_event_wait=0
        fi

        # Check for package alarm temperature
        if [[ $pkg_temp -ge $PACKAGE_ALARM_TEMP ]]; then
            new_fan_speed=$MAX_SPEED
        else
            new_fan_speed=$(_calculate_fan_speed $target_temp $pkg_temp $exhaust_temp $fan_speed)
        fi

        # Set the new fan speed if it has changed
        if [[ $new_fan_speed -ne $fan_speed ]]; then
            fan_speed=$new_fan_speed
            _set_fan_speed $fan_speed
        fi

        # Log the data
        _log_data $pkg_temp $inlet_temp $exhaust_temp $target_temp $fan_speed

        # Sleep for the loop delay
        sleep $LOOP_DELAY
    done
}

if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
    trap _exit_trap EXIT
    _main
fi

Understanding the Control Logic

Temperature Difference Ranges

The script adjusts fan speed based on how far the current temperature is from the target:

• >32°C difference: Maximum adjustment (speed factor 2.0)
• 16-32°C difference: Large adjustment (speed factor 1.375)
• 8-16°C difference: Medium adjustment (speed factor 0.875)
• 4-8°C difference: Small adjustment (speed factor 0.5)
• 2-4°C difference: Minimal adjustment (speed factor 0.25)
• <2°C difference: No adjustment

Hysteresis Factor

The script becomes more aggressive as temperatures approach the alarm threshold:

• When package temp > hysteresis temp: Factor increases by 2
• Otherwise: Factor increases by 1
• Additional 0.75 factor added if exhaust temp exceeds hysteresis temp

Systemd Service Setup

To ensure the fan control script runs automatically at boot and restarts if it crashes, we'll create a systemd service.

Step 1: Save the Script

Copy the fan control script to a system location and make it executable:

sudo cp fan-control.sh /usr/local/bin/fan-control.sh
sudo chmod +x /usr/local/bin/fan-control.sh

Step 2: Create the Service File

Create a new systemd service file:

sudo nano /etc/systemd/system/fan-control.service

Add the following configuration:

[Unit]
Description=Dell R630 Fan Control
After=network.target

[Service]
Type=simple
ExecStart=/usr/local/bin/fan-control.sh
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal

# Safety: if script fails 5 times in 2 minutes, stop trying
StartLimitBurst=5
StartLimitIntervalSec=120

[Install]
WantedBy=multi-user.target

Service Configuration Explained

Directive	Purpose
After=network.target	Start after network is available
Type=simple	Service runs in foreground
Restart=always	Automatically restart if it crashes
RestartSec=10	Wait 10 seconds before restarting
StandardOutput=journal	Send output to systemd journal
StartLimitBurst=5	Maximum 5 restart attempts
StartLimitIntervalSec=120	Within a 2-minute window

Step 3: Enable and Start the Service

# Reload systemd to recognize the new service
sudo systemctl daemon-reload

# Enable it to start on boot
sudo systemctl enable fan-control.service

# Start it now
sudo systemctl start fan-control.service

Service Management Commands

# Check service status
sudo systemctl status fan-control.service

# Stop the service
sudo systemctl stop fan-control.service

# Restart the service
sudo systemctl restart fan-control.service

# Disable auto-start on boot
sudo systemctl disable fan-control.service

Monitoring & Logs

Real-Time Monitoring

Service Status

Check if the service is running and view recent log entries:

sudo systemctl status fan-control.service

Live Journal Logs

Follow systemd journal logs in real-time:

sudo journalctl -u fan-control.service -f

Fan Control Log File

Monitor the detailed log file created by the script:

sudo tail -f /var/log/fanctrl.log

Log Analysis Commands

View Last 50 Lines

sudo journalctl -u fan-control.service -n 50

View Logs Since Boot

sudo journalctl -u fan-control.service -b

View Logs from Specific Time

sudo journalctl -u fan-control.service --since "2024-01-01 10:00:00"

View Logs with Priority

# Only errors and critical messages
sudo journalctl -u fan-control.service -p err

# All messages except debug
sudo journalctl -u fan-control.service -p info

Understanding Log Output

The log file (/var/log/fanctrl.log) contains entries in this format:

1704117600 Mon Jan 01 12:00:00 2024 - Package Temp: 42, Inlet Temp: 25, Exhaust Temp: 35, Target Temp: 35, Fan Speed: 28 [5 - 70]

Field	Description
Unix Timestamp	Epoch time for precise sorting
Human Date	Readable timestamp
Package Temp	Average CPU package temperature (°C)
Inlet Temp	Air temperature entering the server
Exhaust Temp	Air temperature leaving the server
Target Temp	Current target temperature (may adjust based on inlet)
Fan Speed	Current fan speed percentage [Min - Max]

Manual Temperature Checks

CPU Package Temperatures

# Package 0
cat /sys/devices/platform/coretemp.0/hwmon/hwmon0/temp1_input

# Package 1
cat /sys/devices/platform/coretemp.1/hwmon/hwmon1/temp1_input

Note: Values are in millidegrees Celsius (divide by 1000)

IPMI Sensor Readings

# Inlet temperature
ipmitool sensor get 'Inlet Temp'

# Exhaust temperature
ipmitool sensor get 'Exhaust Temp'

# All sensors
ipmitool sensor list

Performance Monitoring Script

Create a simple monitoring script to watch key metrics:

#!/bin/bash
while true; do
    clear
    echo "=== Dell R630 Fan Control Monitor ==="
    echo "Time: $(date)"
    echo ""
    systemctl is-active fan-control.service | \
        awk '{print "Service Status: " ($0=="active" ? "✓ Running" : "✗ Stopped")}'
    echo ""
    tail -n 1 /var/log/fanctrl.log 2>/dev/null || echo "No log data yet"
    sleep 5
done

Save as monitor.sh, make executable, and run:

chmod +x monitor.sh
./monitor.sh

Troubleshooting

Common Issues

Service Won't Start

Symptoms: Service fails immediately or shows "failed" status

Solutions:

• Check script permissions: ls -l /usr/local/bin/fan-control.sh
• Verify script has execute permission: sudo chmod +x /usr/local/bin/fan-control.sh
• Check for syntax errors: bash -n /usr/local/bin/fan-control.sh
• View error logs: sudo journalctl -u fan-control.service -n 50

IPMI Commands Not Working

Symptoms: ipmitool commands fail or return errors

Solutions:

• Install ipmitool: sudo apt-get install ipmitool
• Load kernel modules: sudo modprobe ipmi_devintf && sudo modprobe ipmi_si
• Test IPMI: sudo ipmitool sensor list
• Check IPMI service: sudo systemctl status ipmitool

Fans Too Loud

Symptoms: Fans are still running at high speed

Solutions:

• Lower START_SPEED and MAX_SPEED parameters
• Increase TARGET_TEMP (carefully!)
• Check inlet temperature isn't elevated
• Ensure adequate airflow around server

Temperatures Too High

Symptoms: CPU temperatures consistently above target

Solutions:

• Increase MAX_SPEED limit
• Lower TARGET_TEMP
• Decrease LOOP_DELAY for faster response
• Check thermal paste on CPUs
• Verify fan operation with ipmitool sdr type fan

Script Keeps Restarting

Symptoms: Service restarts frequently, hits StartLimitBurst

Solutions:

• Enable debug mode: Set DEBUG=1 in script
• Check for missing dependencies (bc command): sudo apt-get install bc
• Verify temperature sensors exist: ls /sys/devices/platform/coretemp.*
• Review full logs for error patterns

Emergency Fan Restore

If something goes wrong and fans are stuck at low speed:

# Stop the service immediately
sudo systemctl stop fan-control.service

# Manually restore automatic fan control
sudo ipmitool raw 0x30 0x30 0x01 0x01

# Verify fans speed up
sudo ipmitool sdr type fan

Testing Fan Control Manually

Before enabling the service, test the script manually:

# Run script in foreground with debug enabled
sudo DEBUG=1 /usr/local/bin/fan-control.sh

# Press Ctrl+C to stop and restore automatic control

Watch the output for several minutes to ensure:

• Temperatures are being read correctly
• Fan speeds are adjusting appropriately
• No errors or warnings appear
• System stays within safe temperature ranges

Tuning for Your Environment

Every server environment is different. Here are some starting points:

Quiet Home Lab

TARGET_TEMP=40
START_SPEED=20
MIN_SPEED=5
MAX_SPEED=50
LOOP_DELAY=10

Balanced Production

TARGET_TEMP=35
START_SPEED=30
MIN_SPEED=10
MAX_SPEED=70
LOOP_DELAY=5

High Performance

TARGET_TEMP=30
START_SPEED=40
MIN_SPEED=20
MAX_SPEED=90
LOOP_DELAY=3

Getting Help

If you need assistance:

• Collect full logs: sudo journalctl -u fan-control.service --no-pager > fan-control.log
• Include your server specs (CPU model, RAM, installed cards)
• Note your ambient temperature and server location
• Document any modifications you've made to the script
• Check the original GitHub Gist for updates and community discussions