Suricata's file extraction on Debian GNU/Linux

Suricata is a high performance open source IDS/IPS project. I used it a long time ago around 2010 when it was released. I've been playing with Snort recently and then found Suricata has a great feature: File extraction. It'd be helpful to those who want to get malware samples from IDS. Anyway, like old days, I want to test it on my own and see how it works on Debian. First things first, I need to build it and see if it works.

Download a latest version of the small installation ISO image. I need to clarify my testing environment: Debian is running on my virtual machine, which has two NICs are eth0 and eth1. Interface eth0 is running on NAT mode and eth1 is running bridge mode.  Debian don't assign any IP addr to eth1.

Because our Debian is the small installation. So we have to install some dependency packages via simply apt-get:

#apt-get install vim openssh-server ethtool libpcap-dev libnfnetlink-dev libnetfilter-queue-dev libdnet-dev libdumbnet-dev libpcre3-dev libpcre3-dbg bison flex make zlib1g-dev autoconf libtool libnss3-dev libnspr4-dev libjansson4 libjansson-dev libyaml-dev libcap-ng0 libcap-ng-dev libnet1-dev libmagic-dev build-essential

Get the source code of Suricata:
#cd /tmp
#wget wget http://www.openinfosecfoundation.org/download/suricata-2.0.tar.gz
#tar zxvf suricata-2.0.tar.gz
#cd suricata-2.0

Compile and installation:
#./configure --enable-nfqueue --enable-gccprotect --prefix=/usr/local/suricata --localstatedir=/var
#make -j3
#make make-full

Edit suricata.yaml:
1, Set the request/response body a litte bigger:
           request-body-limit: 1gb #3072
           response-body-limit: 1gb #3072

2, Enable file extraction:
  - file-store:
      enabled: yes       # set to yes to enable
      log-dir: files    # directory to store the files
      force-magic: no   # force logging magic on all stored files
      force-md5: no     # force logging of md5 checksums
      waldo: file.waldo # waldo file to store the file_id across runs

  # output module to log files tracked in a easily parsable json format
  - file-log:
      enabled: yes
      filename: files-json.log
      append: yes
      #filetype: regular # 'regular', 'unix_stream' or 'unix_dgram'

3, Add our "test" rule file( test.rules in this case) into the section "default-rule-path:", like:
default-rule-path: /usr/local/suricata/etc/suricata/rules
rule-files:
 - test.rules
 - botcc.rules

Create a rule file:
/usr/local/suricata/etc/suricata/rules/test.rules

Add one line into test.rules( to save any jpg files) :
alert http any any -> any any (msg:"FILESTORE jpg"; fileext:"jpg"; filestore; sid:6; rev:1;)

Enable the eth1:
#ifconfig eth1 up

According to the Suricata's wiki, we should turn off the TCP GSO:
ethtool -K eth1 tso off
ethtool -K eth1 gro off
ethtool -K eth1 lro off
ethtool -K eth1 gso off
ethtool -K eth1 rx off
ethtool -K eth1 tx off
ethtool -K eth1 sg off
ethtool -K eth1 rxvlan off
ethtool -K eth1 txvlan off
ethtool -N eth1 rx-flow-hash udp4 sdfn
ethtool -N eth1 rx-flow-hash udp6 sdfn
ethtool -n eth1 rx-flow-hash udp6
ethtool -n eth1 rx-flow-hash udp4
ethtool -C eth1 rx-usecs 1000
ethtool -C eth1 adaptive-rx off

Run the Suricata with this command:
/usr/local/suricata/bin/suricata -c /usr/local/suricata/etc/suricata//suricata.yaml -i eth1

Use your firefox/chrome on your host machine, and visit some website, like this.

You should see some girl pictures in /var/log/suricata/files  ;-)

btw: Thanks to Suricata community brings us this fuc*ing awesome IDS/IPS project. Special thanks to Peter Manev.