Changelog
02June2017 – Originally posted
28Nov2017  – Updated due to script changes

What is DShield and why would I send them my logs?

According to the SANS Internet Storm Center (ISC), “DShield provides a platform for users of firewalls to share intrusion information. DShield is a free and open service.” While DShield is often referred to generically as a “collaborative firewall log correlation system,” for all practical purposes, it is a bit of threat intelligence well before threat intelligence was cool or an overused buzzword. 😉 The truth is many folks across the internet provide data to the ISC and it really is a community-based effort. DShield aggregates/parses the data and determines different attack vectors and trends. This information then gets filtered back down to the community and it is often found in many of the threat/IP blacklist feeds used by security products. How cool is that? So the bottom line is that if you are a consumer of this data, why not help out the cause and contribute to it? You also get the added bonus of seeing firewall reports on your own last 30 days of data via your account portal. I’ve included one example of the many reports you might expect at the bottom as well. Did I mention it’s free?!?!

DShield account creation

Get started by creating an account on DShield. Once again, it’s free.
https://secure.dshield.org/register.html

After your account is created, go to ‘my account’ and you will see the information we need. Namely, the API key and the user ID. I would recommend just leaving this page up and copying/pasting these when we need them later.

pfSense web config

To get started, we’ll need to add in a notification email address on pfSense. You can do this by going to Systems->Advanced->Notifications. If you already have an account configured here, great! Just remember you will need to use that same email address later when configuring the script. If you don’t have anything configured here, you will need to add one. I chose to use a Gmail address and if you want to do the same, make the modifications as shown in the picture below. Don’t forget to send a test after saving your contents!
Note: I used a different Gmail address than my standard use one for the simple fact you can’t use an account that has 2-factor authentication enabled.

SSH on pfSense is not enabled by default. You can easily turn it on via the System->Advanced menu.

Command line config

Now, SSH to your pfSense box using PuTTY or another SSH client. Don’t forget your username is root, not admin.

Once you are logged in, go to option 8, which drops you to a standard command line. Create a bin directory under root and then change to that directory

# mkdir bin
# cd bin

Next, download the config file from GitHub with the first command and make the file executable with the second command.

# curl https://raw.githubusercontent.com/jullrich/dshieldpfsense/master/dshield.php > /root/bin/dshieldpfsense.php
chmod +x dshieldpfsense.php

Next, download the sample ini file from GitHub.

# curl https://raw.githubusercontent.com/jullrich/dshieldpfsense/master/dshield.sample > /root/bin/dshield.ini

With the latest version of the script, there is no need to edit the actual php script. Instead, you edit the ini by adding your apikey (API key from portal), uid (User ID from portal), and fromaddr (from email address in notifications). As stated earlier, it is far easier to copy/paste the apikey and uid values from the DShield page.
Note: double-check the interface if you are using anything other than WAN. You can use multiple interfaces as long as you delimit them with commas.

# vi dshield.ini

Once those lines are changed, go ahead and exit vi. Perform a quick test run using the command below.

# ./dshieldpfsense.php

If everything is configured correctly, you can take a look at the system log. The command below will grep back any DShield-related content.

# clog /var/log/system.log | grep dshield

The only issue is that if it worked, you will not see anything in the log. 🙁 So go ahead and run the same command again. Assuming no new activity occurs in that time, your system log will now have a log entry similar to the one below the command.

# clog /var/log/system.log | grep dshield
pf php: ./dshieldpfsense.php: no new lines added to log since last run OK

Troubleshooting

If that didn’t work, there are a few other items you can try.

First, you can modify the dshield.ini file and add a ccaddr so the reports go to that email as well. This is extremely useful when you feel the data is collected and correct, but nothing is showing on the DShield website after a fair amount of time.

Another great way to troubleshoot is by changing “debug=0” to “debug=1” which is also found in the dshield.ini file. After making the change, re-run the dshieldpfsense.php script. This time around you should see a fair amount of information returned similar to below. If you see why no logs were sent, e.g. only LAN or ICMP traffic was seen recently, then change debug back to 0.

Reading Nov 29 23:00:54 pf filterlog: 5,,,1000000103,igx0,match,block,in,4,0x0,,64,22100,0,DF,6,tcp,52,192.168.1.2,52.25.125.12,56363,443,0,FA,2164609854,1629822578,1369,,nop;nop;TS
Log was rejected due to wrong interface or flags or because it is ICMP: protocol TCP interface LAN flags FA
Reading Nov 29 23:00:54 pf filterlog: 5,,,1000000103,igx0,match,block,in,4,0x0,,64,3647,0,DF,6,tcp,52,192.168.1.2,52.25.125.12,56361,443,0,FA,3134666011,2501649167,1550,,nop;nop;TS
Log was rejected due to wrong interface or flags or because it is ICMP: protocol TCP interface LAN flags FA
Reading Nov 29 23:00:54 pf filterlog: 5,,,1000000103,igx0,match,block,in,4,0x0,,64,25539,0,DF,6,tcp,52,192.168.1.2,52.25.125.12,56362,443,0,FA,4017349769,959963474,1550,,nop;nop;TS
Log was rejected due to wrong interface or flags or because it is ICMP: protocol TCP interface LAN flags FA
Reading Nov 29 23:00:55 pf filterlog: 5,,,1000000103,igx0,match,block,in,4,0x0,,64,22101,0,DF,6,tcp,52,192.168.1.2,52.25.125.12,56363,443,0,FA,2164609854,1629822578,1369,,nop;nop;TS

A few other file locations worth mentioning for troubleshooting…
/tmp/lastdshieldlog <- contains the last log sent
/var/run/dshieldlastts <- contains the last time (in epoch) the script was completed

Automate it with cron

If everything appears to be working, you just need to automate it. Do this by typing in ‘crontab -e’ from the command line and copying in the cronjob below. I would recommend making these values (25,55) somewhat random and leave them 30 minutes apart from one another. For example, 4,34 or 12,42 or 27,57 would work great.

25,55 * * * * /root/bin/dshieldpfsense.php

Exit as you would with vi and that should give you a message of ‘crontab: installing new crontab’

That’s it!

And now you wait… After your cronjob runs a few times and enough data is collected, you can take a look at your data as aggregated by DShield.

Final note

If you or any feeds you use consume data from DShield, contributing to the cause is really the right thing to do! Keep in mind that other firewalls can send their data as well and it is not just limited to pfSense. For a complete rundown on additional clients and other in’s and out’s of the DShield project, visit the link below.
https://www.dshield.org/howto.html