Using Passive OS Fingerprinting (p0f)
Passive OS fingerprinting is the practice of trying to determine the operating system that a connecting network host is running, based on vendor-specific quirks in the TCP/IP stack. It's a technique that can let you know whether the machine sending you email is running Windows, Mac OS X, Linux, FreeBSD, Solaris, etc. More usefully, it can make a pretty educated guess about the version of the OS as well, so you can tell whether a Windows host is running a workstation OS like Windows 98 or server OS like Windows 2003. This can be handy for helping to identify botnet zombies.
To get started you'll need to download and install p0f on your mail gateway--the machine that outside hosts will be connecting to on port 25 to send you mail.
What we intend to do is run p0f as a daemon that monitors port 25 and caches information about all connecting hosts. Using a small Perl script, we'll then tell Postfix to consult that p0f cache when it receives a mail item, and then insert a special X-p0f-OS: header into the email with the pertinent OS information about the client. From there we can use a set of custom SpamAssassin rules that look for that header and adjust the score accordingly.
First we need a small policy script that Postfix can use to consult the p0f cache. Save this file as p0f-policy.pl somewhere (e.g. /var/amavisd/p0f-policy.pl), edit the handful of configurable settings (e.g. your mail server's IP address) and make it executable:
#!/usr/bin/perl
use Sys::Syslog qw(:DEFAULT setlogsock);
use IO::Socket;
use Net::IP;
# p0f-policy.pl is based on the os-greylist.pl script by
# Marmarek <marmarek@staszic.waw.pl>, which can be found at:
#
# http://marmarek.w.staszic.waw.pl/~marmarek/download/os-greylist.pl
#
# This script was modified by Robert LeBlanc <rjl@renaissoft.com> for
# use with Maia Mailguard as a tool for inserting p0f operating system
# information into a mail header called "X-p0f-OS:", which can then
# be used by SpamAssassin for scoring purposes with the rules in the
# associated p0f.cf file.
#
# The p0f-policy.pl script is intended to be used as a delegated
# Postfix smtpd policy server. The script should be installed in your
# amavis/maia user's home directory (e.g. /var/amavis), while the
# p0f.cf file should be installed in the same subdirectory as your
# SpamAssassin local.cf file.
#
# You should run p0f something like this:
#
# p0f -i any -u amavis -Q /var/amavis/p0f-socket -q -l -d -o /dev/null -0 'tcp dst port 25'
#
# In your /etc/postfix/master.cf add:
#
# policy unix - n n - - spawn
# user=amavis argv=/usr/bin/perl /var/amavis/p0f-policy.pl
#
# In your /etc/postfix/main.cf, make the 'check_policy_service' entry
# below the last smtpd_recipient_restriction to be tested:
#
# smtpd_recipient_restrictions =
# ...
# reject_unauth_destination
# check_policy_service unix:private/policy
#
# NOTE: specify check_policy_service AFTER reject_unauth_destination
# or else your system can become an open relay.
#
# To test this script by hand, execute:
#
# % perl p0f-policy.pl
#
# Each query is a bunch of attributes. Order does not matter, and
# the demo script uses only a few of all the attributes shown below:
#
# request=smtpd_access_policy
# protocol_state=RCPT
# protocol_name=SMTP
# client_address=1.2.3.4
# [empty line]
#
# The policy server script will answer in the same style, with an
# attribute list followed by a empty line, e.g.:
#
# action=prepend X-p0f-OS: Windows XP/2000
# [empty line]
######################################################################
# #
# CONFIGURABLE ITEMS (set these) #
# #
######################################################################
# Syslogging options for verbose mode and for fatal errors.
# NOTE: comment out the $syslog_socktype line if syslogging does not
# work on your system.
#
$syslog_socktype = 'unix'; # inet, unix, stream, console
$syslog_facility = 'mail';
$syslog_options = 'pid';
$syslog_priority = 'info';
# The full path to the socket file p0f is listening on.
$p0f_socket = "/var/amavisd/p0f-socket";
# The IP address of your Postfix host.
$mta_ipaddr = "10.0.0.2";
######################################################################
# #
# END OF CONFIGURABLE ITEMS (no need to change anything below here) #
# #
######################################################################
sub p0f_query {
my $QUERY_MAGIC = 0x0defaced;
my $QTYPE_FINGERPRINT = 1;
my $src = new Net::IP ($attr{"client_address"}) or die (Net::IP::Error());
my $dst = new Net::IP ($mta_ipaddr) or die (Net::IP::Error());
my $query = pack("L L L N N S S", $QUERY_MAGIC, $QTYPE_FINGERPRINT, 0x12345678, $src->intip(), $dst->intip(), 0, 25);
my $sock = new IO::Socket::UNIX (Peer => $p0f_socket, Type => SOCK_STREAM);
if (!$sock) {
syslog $syslog_priority, "Could not create p0f socket: $!\n";
return "unknown";
}
# Ask p0f
print $sock $query;
my $response = <$sock>;
close $sock;
# Extract the response from p0f
my ($magic, $id, $type, $genre, $detail, $dist, $link, $tos, $fw, $nat, $real, $score, $mflags, $uptime) =
unpack ("L L C Z20 Z40 c Z30 Z30 C C C s S N", $response);
if ($magic != $QUERY_MAGIC) {
syslog $syslog_priority, "bad p0f query magic";
return "unknown";
}
if ($type == 1) {
syslog $syslog_priority, "malformed p0f query";
return "unknown";
}
if ($type == 2) {
syslog $syslog_priority, "connection is not (no longer?) in the p0f cache" if $verbose;
return "unknown";
}
if ($genre eq "") {
$genre = "unknown";
}
if ($verbose) {
syslog $syslog_priority, "OS detected: %s %s", $genre, $detail;
}
return $genre, $detail;
}
# Demo SMTPD access policy routine. The result is an action just like
# it would be specified on the right-hand side of a Postfix access
# table. Request attributes are available via the %attr hash.
sub smtpd_access_policy {
my($key, $time_stamp, $now);
my ($genre, $detail) = p0f_query();
# The result can be any action that is allowed in a Postfix access(5) map.
#
# To label mail, return ``PREPEND'' headername: headertext
syslog $syslog_priority, "Client OS detected: %s %s", $genre, $detail;
return "prepend X-p0f-OS: $genre $detail";
}
# Log an error and abort.
sub fatal_exit {
my($first) = shift(@_);
syslog "err", "fatal: $first", @_;
exit 1;
}
# This process runs as a daemon, so it can't log to a terminal. Use
# syslog so that people can actually see our messages.
setlogsock $syslog_socktype;
openlog $0, $syslog_options, $syslog_facility;
# We don't need getopt() for now.
while ($option = shift(@ARGV)) {
if ($option eq "-v") {
$verbose = 1;
} else {
syslog $syslog_priority, "Invalid option: %s. Usage: %s [-v]",
$option, $0;
exit 1;
}
}
# Unbuffer standard output.
select((select(STDOUT), $| = 1)[0]);
# Receive a bunch of attributes, evaluate the policy, send the result.
while (<STDIN>) {
if (/([^=]+)=(.*)\n/) {
$attr{substr($1, 0, 512)} = substr($2, 0, 512);
} elsif ($_ eq "\n") {
if ($verbose) {
for (keys %attr) {
syslog $syslog_priority, "Attribute: %s=%s", $_, $attr{$_};
}
}
fatal_exit "unrecognized request type: '%s'", $attr{request}
unless $attr{"request"} eq "smtpd_access_policy";
$action = smtpd_access_policy();
syslog $syslog_priority, "Action: %s", $action if $verbose;
print STDOUT "action=$action\n\n";
%attr = ();
} else {
chop;
syslog $syslog_priority, "warning: ignoring garbage: %.100s", $_;
}
}
Next, tell Postfix where to find this script by adding this to your master.cf file:
...
policy unix - n n - - spawn
user=amavis argv=/usr/bin/perl /var/amavisd/p0f-policy.pl
...
In your main.cf file, add this policy service to your smtpd_recipient_restrictions, so that the p0f cache is consulted only after more basic recipient tests have been performed:
smtpd_recipient_restrictions = ... reject_unauth_destination check_policy_service unix:private/policy
NOTE: specify check_policy_service AFTER reject_unauth_destination or else your system can become an open relay.
Now that we've added the custom p0f header to incoming mail items, we need to assign some score values to different OS results. Save these SpamAssassin rules to a file called p0f.cf in the subdirectory where your local.cf file is located:
################################################################################### # p0f Rules for scoring client operating systems ################################################################################### header P0F_WIN311 X-p0f-OS =~ /^Windows 3.11/ score P0F_WIN311 3.0 describe P0F_WIN311 Client is running Windows 3.11 header P0F_WIN95 X-p0f-OS =~ /^Windows 95/ score P0F_WIN95 3.0 describe P0F_WIN95 Client is running Windows 95 header P0F_WIN98 X-p0f-OS =~ /^Windows 98/ score P0F_WIN98 3.0 describe P0F_WIN98 Client is running Windows 98 header P0F_WINME X-p0f-OS =~ /^Windows ME/ score P0F_WINME 3.0 describe P0F_WINME Client is running Windows ME header P0F_WINNT X-p0f-OS =~ /^Windows NT/ score P0F_WINNT 0.5 describe P0F_WINNT Client is running Windows NT header P0F_WIN2K X-p0f-OS =~ /^Windows 2000(?!.*XP)/ score P0F_WIN2K 1.5 describe P0F_WIN2K Client is running Windows 2000 header P0F_WINXP X-p0f-OS =~ /^Windows XP(?!.*2000)/ score P0F_WINXP 2.5 describe P0F_WINXP Client is running Windows XP header P0F_WINXP2K X-p0f-OS =~ /^Windows (XP.+2000|2000.+XP)/ score P0F_WINXP2K 1.5 describe P0F_WINXP2K Client is running Windows 2000 or XP header P0F_WIN2K3 X-p0f-OS =~ /^Windows 2003/ score P0F_WIN2K3 0.2 describe P0F_WIN2K3 Client is running Windows 2003 header P0F_WINNET X-p0f-OS =~ /^Windows \.NET/ score P0F_WINNET 0.2 describe P0F_WINNET Client is running Windows .NET Enterprise Server header P0F_WINCE X-p0f-OS =~ /^Windows CE/ score P0F_WINCE 0.1 describe P0F_WINCE Client is running Windows CE header P0F_WINVISTA X-p0f-OS =~ /^Windows Vista/ score P0F_WINVISTA 2.5 describe P0F_WINVISTA Client is running Windows Vista header P0F_MACOS X-p0f-OS =~ /^MacOS/ score P0F_MACOS 0.1 describe P0F_MACOS Client is running Mac OS header P0F_FREEBSD X-p0f-OS =~ /^FreeBSD/ score P0F_FREEBSD -0.1 describe P0F_FREEBSD Client is running FreeBSD header P0F_OPENBSD X-p0f-OS =~ /^OpenBSD/ score P0F_OPENBSD -1.0 describe P0F_OPENBSD Client is running OpenBSD header P0F_NETBSD X-p0f-OS =~ /^NetBSD/ score P0F_NETBSD -1.0 describe P0F_NETBSD Client is running NetBSD header P0F_SOLARIS X-p0f-OS =~ /^Solaris/ score P0F_SOLARIS -1.0 describe P0F_SOLARIS Client is running Solaris header P0F_HPUX X-p0f-OS =~ /^HP-UX/ score P0F_HPUX -1.0 describe P0F_HPUX Client is running HP-UX header P0F_TRU64 X-p0f-OS =~ /^Tru64/ score P0F_TRU64 -1.0 describe P0F_TRU64 Client is running Tru64 header P0F_AIX X-p0f-OS =~ /^AIX/ score P0F_AIX -1.0 describe P0F_AIX Client is running AIX header P0F_LINUX X-p0f-OS =~ /^Linux/ score P0F_LINUX -0.1 describe P0F_LINUX Client is running Linux header P0F_SUNOS X-p0f-OS =~ /^SunOS/ score P0F_SUNOS -1.0 describe P0F_SUNOS Client is running SunOS header P0F_IRIX X-p0f-OS =~ /^IRIX/ score P0F_IRIX -1.0 describe P0F_IRIX Client is running IRIX header P0F_OPENVMS X-p0f-OS =~ /^OpenVMS/ score P0F_OPENVMS -1.0 describe P0F_OPENVMS Client is running OpenVMS header P0F_RISCOS X-p0f-OS =~ /^RISC OS/ score P0F_RISCOS -1.0 describe P0F_RISCOS Client is running RISC OS header P0F_BSD X-p0f-OS =~ /^BSD/ score P0F_BSD -1.0 describe P0F_BSD Client is running BSD/OS header P0F_NEWTON X-p0f-OS =~ /^NewtonOS/ score P0F_NEWTON 0.1 describe P0F_NEWTON Client is running NewtonOS header P0F_NEXT X-p0f-OS =~ /^NeXTSTEP/ score P0F_NEXT -1.0 describe P0F_NEXT Client is running NeXTSTEP header P0F_BEOS X-p0f-OS =~ /^BeOS/ score P0F_BEOS -1.0 describe P0F_BEOS Client is running BeOS header P0F_OS400 X-p0f-OS =~ /^OS\/400/ score P0F_OS400 -1.0 describe P0F_OS400 Client is running OS/400 header P0F_ULTRIX X-p0f-OS =~ /^ULTRIX/ score P0F_ULTRIX -1.0 describe P0F_ULTRIX Client is running ULTRIX header P0F_QNX X-p0f-OS =~ /^QNX/ score P0F_QNX -1.0 describe P0F_QNX Client is running QNX header P0F_NETWARE X-p0f-OS =~ /^Novell NetWare/ score P0F_NETWARE 2.0 describe P0F_NETWARE Client is running NetWare header P0F_INTRANETWARE X-p0f-OS =~ /^Novell IntranetWare/ score P0F_INTRANETWARE 2.0 describe P0F_INTRANETWARE Client is running IntranetWare header P0F_BORDERMGR X-p0f-OS =~ /^Novell BorderManager/ score P0F_BORDERMGR 2.0 describe P0F_BORDERMGR Client is running BorderManager header P0F_SCO X-p0f-OS =~ /^SCO/ score P0F_SCO -1.0 describe P0F_SCO Client is running SCO header P0F_DOS X-p0f-OS =~ /^DOS/ score P0F_DOS 3.0 describe P0F_DOS Client is running DOS header P0F_OS2 X-p0f-OS =~ /^OS\/2/ score P0F_OS2 2.0 describe P0F_OS2 Client is running OS/2 header P0F_TOPS20 X-p0f-OS =~ /^TOPS-20/ score P0F_TOPS20 -1.0 describe P0F_TOPS20 Client is running TOPS-20 header P0F_AMIGA X-p0f-OS =~ /^AMIGA/ score P0F_AMIGA 1.0 describe P0F_AMIGA Client is running AMIGAOS header P0F_MINIX X-p0f-OS =~ /Minix/ score P0F_MINIX -1.0 describe P0F_MINIX Client is running Minix header P0F_PLAN9 X-p0f-OS =~ /^Plan9/ score P0F_PLAN9 -1.0 describe P0F_PLAN9 Client is running Plan9 header P0F_FREEMINT X-p0f-OS =~ /^FreeMiNT/ score P0F_FREEMINT 1.0 describe P0F_FREEMINT Client is running FreeMiNT header P0F_NETCACHE X-p0f-OS =~ /^NetCache/ score P0F_NETCACHE -0.1 describe P0F_NETCACHE Client is running NetCache header P0F_CACHEFLOW X-p0f-OS =~ /^CacheFlow/ score P0F_CACHEFLOW -0.1 describe P0F_CACHEFLOW Client is running CacheFlow header P0F_POWERAPP X-p0f-OS =~ /^Dell PowerApp/ score P0F_POWERAPP -0.1 describe P0F_POWERAPP Client is running PowerApp header P0F_PALMOS X-p0f-OS =~ /^PalmOS/ score P0F_PALMOS 0.1 describe P0F_PALMOS Client is running PalmOS header P0F_SYMBIANOS X-p0f-OS =~ /^SymbianOS/ score P0F_SYMBIANOS 0.1 describe P0F_SYMBIANOS Client is running SymbianOS header P0F_ZAURUS X-p0f-OS =~ /^Zaurus/ score P0F_ZAURUS 0.1 describe P0F_ZAURUS Client is running Zaurus header P0F_POCKETPC X-p0f-OS =~ /^PocketPC/ score P0F_POCKETPC 0.1 describe P0F_POCKETPC Client is running PocketPC header P0F_CONTIKI X-p0f-OS =~ /^Contiki/ score P0F_CONTIKI 0.1 describe P0F_CONTIKI Client is running Contiki header P0F_PLAYSTATION X-p0f-OS =~ /^Sony Playstation/ score P0F_PLAYSTATION 3.0 describe P0F_PLAYSTATION Client is running Sony Playstation header P0F_DREAMCAST X-p0f-OS =~ /^Sega Dreamcast/ score P0F_DREAMCAST 3.0 describe P0F_DREAMCAST Client is running Sega Dreamcast header P0F_UNKNOWN X-p0f-OS =~ /^UNKNOWN/ score P0F_UNKNOWN 0.8 describe P0F_UNKNOWN Client OS is unknown
Be sure to run the load-sa-rules.pl script to tell Maia about these new SpamAssassin rules, of course.
Finally, add a line to one of your startup scripts (e.g. rc.local) to make sure the p0f daemon is started at boot time:
... # Start the p0f daemon /usr/sbin/p0f -i any -u amavis -Q /var/amavisd/p0f-socket -q -l -d -o /var/amavisd/p0f.log -0 'tcp dst port 25' ...
Start the p0f daemon with that same command line manually, reload the Postfix daemon, restart amavisd-maia and you should be done. Verify that incoming email now contains the new mail header, and that a "P0F_" rule is being triggered.
