Block Ads & Malvertising on pfSense Using pfBlockerNG (DNSBL)
This walkthrough uses the DNSBL portion of pfBlockerNG to remove ads/advertising and more importantly, malvertising. It essentially creates a functionality similar to the pi-Hole project except it doesn’t require a separate piece of hardware. Instead, you just use your pfSense + pfBlockerNG! If you’re interested in a write-up on installing/configuring the pi-hole on Ubuntu, I have one here.
Please note this walkthrough is for the devel version of pfBlockerNG. The pfBlockerNG-devel package is in the standard list of available packages and does not require the development/experimental branch of pfSense firmware. Even though the package states “devel,” I have no issues using it in production. First, I was lucky enough to be a beta tester for this release and the number of changes are astounding. Second, the configuration is 10X easier. Last but not least, the package is extremely stable and it has been around since 2018. All that said, if you are still leery about using a “development” package on your pfSense, the older version of this walkthrough is still available at the link below.
<< Old version of this pfBlockerNG DNSBL guide >>
Block Ads & Malvertising on pfSense Using pfBlockerNG (DNSBL) – Old
Warning: DO NOT install the latest version of pfBlockerNG unless you are on the most up-to-date version of pfSense. This is especially important if you are on a pfSense before 2.4.4. Version 2.4.4 introduced PHP 7.2 and it broke a lot of packages, not just pfBlockerNG. I would argue you should upgrade pfSense to the latest version *before* installing any new packages and the “official” pfSense upgrade guide backs up my philosophy. The upgrade guide also emphasizes creating backups, rebooting before updates, etc. which are all fantastic advice.
I love pfSense and if I could only install one package to enhance its capabilities, it is undoubtedly pfBlockerNG. It is the very first package I install after configuring a brand new pfSense and in some cases, it is the only one. pfBlockerNG is a pfSense package maintained by @BBcan177 (on Twitter). It’s worth mentioning that BBCan177 has a Patreon campaign where you can easily donate a few bucks to ensure he continues maintaining and adding to the package. If your using this in a production environment, I highly encourage you to donate. pfBlockerNG is an absolutely amazing package and in my opinion, a pfSense install is not complete without it.
pfBlockerNG can add other security enhancements that I’ve discussed on this site such as blocking known bad IP addresses with blocklists (link below). If you don’t already have the blocklist functionality in place on your pfSense, I would strongly suggest adding it after you’re done with this walkthrough.
<< Link goes to the old version as I’m still working on the new guide >>
Using pfBlockerNG (And Block Lists) On pfSense
Changelog
29May2018 – Originally posted (heavily revised for the new version of pfBlockerNG)
30May2018 – Added TLD feature discussion
4June2018 – Added .cm to TLD block recommendations as well as DNS blocking section
5July 2018 – Added link to Brian Krebs article about TLD ‘badness’
25July2018 – pfBlockerNG-devel no longer requires development firmware
5Sept2018 – Expanded on warning regarding anti-virus and endpoint protection changing DNS settings
27Nov2018 – Added warning about pfSense versions prior to 2.4.4
30Nov2019 – Updated guide to reflect recent changes in pfBlockerNG
13Nov2020 – Added Malwarebytes to DNSBL whitelist due to high CPU
Why remove advertising?
Advertising is great because it pays content creators for their work. After all, even this site utilizes Google Ads. So why would I create a write-up on blocking ads? Because advertisements are known to carry malicious payloads and it’s impossible to distinguish what’s good and what’s bad. Even the background of the featured image (above) for this article was what I received when I was originally writing this up in my lab with no ad blocking, i.e. I visited a site for 30 seconds on a brand new, fully patched Windows system with an up-to-date Google Chrome install. Yes, advertising really is out of hand! And to that end, I’ll happily sacrifice some advertising income for the sake of readers/everyone improving their security. I guess I’ll call that self-deprecating technology. 😉 Granted, you don’t just need to listen to my advice. Even the U.S. National Security Agency (NSA) recommends using ad blockers!
Upgrading from non-devel version
If you installing pfBlockerNG for the first time, skip this step and go to installation. *If* you have quite a few custom settings such as rules, IPv4 lists, and DNSBL lists and you want to keep all of your settings, go to Firewall -> pfBlockerNG (General) and make sure ‘Keep Settings’ is checked. If it’s not, put a check there and click ‘Save’ at the bottom.
Unless you have a very complex setup, my personal opinion is to take the check out of ‘Keep settings’ and setup pfBlockerNG from scratch. As you will see during the setup of the new version, adding a feed is ridiculously easy so don’t assume you are going to spend 20 minutes adding 5 feeds. If you go this route, I would suggest taking screenshots of your various settings as well as the feeds you currently use so you can ensure you add them back in. Trust me when I say that adding feeds in the devel version is point and click! Either way, I’ll proceed through this walkthrough whether settings were kept or not and point out the differences along the way.
Go to System -> Package Manager and delete the package.
Installation of pfBlockerNG-devel
Go to System -> Package Manager -> Available Packages and type ‘pfblocker’ into the search criteria and then click ‘search.’ Make sure you click ‘install’ on the version with ‘-devel’ at the end of it or the package or you will be installing the old one! On the next page, simply click ‘Confirm’ and let the package install. This will take a bit of time as it has to download several files and databases. Wait a few minutes and you should see “success” once the installation is complete.
Note: If you do not see “pfBlockerNG-devel” in the list of available packages, you can also try running ‘pkg update -f’ from the command line. I didn’t need this step on the numerous upgrades/installs I’ve done, but it’s still worth mentioning.
At this point, the package is installed. Now, go to the configuration page (Firewall -> pfBlockerNG). Assuming you didn’t upgrade, you will receive the wizard below instead. The wizard is literally 4 steps and I highly suggest using it to get you started.
Pay special attention to the interface step/page if you have a non-standard setup or if you want to enable pfBlockerNG on multiple interfaces.
Finish up the wizard and you will be automatically directed to the update page. The update will likely take a little bit to complete as it is downloading the various IP and DNSBL feeds associated with the wizard setup. Once the feeds are downloaded, the text in the gray box will stop scrolling and you will see “UPDATE PROCESS ENDED” at the very bottom along with your current date and time. Yay!
Now go back to the main pfBlockerNG page by going to “General” or by clicking Firewall -> pfBlockerNG. The “Enable” and “Keep Settings” checkboxes should already be checked as shown below. So far so good! BTW, just a quick shout out to my buddy, Austin, on the sweet logo he created for the pfBlockerNG project! 😉
Further configuring DNSBL
Next, go to the DNSBL tab and it will take you to the main DNSBL landing page. You should also notice there is already a checkmark in ‘Enable’ next to DNSBL. If you only have one internal interface such as LAN, then you shouldn’t need to do anything else. If you have multiple internal interfaces and you would like to protect them with DNSBL, then you will need to pay attention to the ‘Permit Firewall Rules’ section below. You may have already selected the “extra” interfaces when you went through the wizard above. Either way, keep this in mind should you ever add interfaces or VLANs in the future!
If you do need to add interfaces, place a checkmark in the ‘Enable’ box (red square below). Then, select the various interfaces (to the right) by holding down the ‘Ctrl’ key and left-clicking. Don’t forget to hit ‘Save DNSBL settings’ and move to the DNSBL feeds section.
If your pfSense has plenty of memory, another really amazing feature to consider is TLD (below the DNSBL option in the picture above). This option is required for the TLD blacklists discussed later in the walkthrough. What does the TLD feature provide? Normally, DNSBL (and other DNS blackhole software) block the domains specified in the feeds and that’s that. What TLD does differently is it will block the domain specified in addition to all of a domain’s subdomains. As a result, a bad guy can’t circumvent the blacklist by creating a random subdomain name such as abcd1234.linuxincluded.com (if linuxincluded.com was in a DNSBL feed). That’s really powerful and as far as I know, it is one of the few DNS blackholing software that does it. You can get an idea on memory requirements by clicking on the blue ‘info’ icon next to TLD. If you have less than 2GB of memory on your pfSense, I would skip it. If you’re unsure on your memory, this might be a feature to come back to after you get your feeds and everything else configured. Nonetheless, don’t sleep on this extremely powerful feature because TLD can definitely add several layers of protection.
Configuring DNSBL feeds
Before we go adding additional feeds, we should at least understand what the wizard provided us. Go to DNSBL, DNSBL feeds to see the current (post-wizard) configuration.
Great! But what if you want to add more? Go to ‘Feeds’ (not DNSBL Feeds) at the top. Here you will see all of the pre-configured feeds for the IPv4, IPv6, and DNSBL categories. And yes, there are a bunch of them! You’ll also see custom, user defined feeds at the very bottom if you performed an upgrade and it was unable to match a feed to an existing feed. If you don’t have a “Feeds” sub-menu, that most likely means you are still on the older version of pfBlockerNG. Another way to check is if you have “Alerts” instead of “Reports” along the top row of pfBlockerNG options… That too means you are still on the old version. You can either follow the walkthrough for the older version of pfBlockerNG or better yet, delete the old pfBlockerNG and install the pfBlockerNG-devel package.
Scroll down to the ‘DNSBL Category’ header, which is *after* all of the IPv4 and IPv6 sections. The first DNSBL sub-category you should see is labeled EasyList.
Note that EasyList has a checkbox near the #1. This means the alias/group or category already exists. If you look toward the right near the #2, you will see another checkbox. This means the individual feed is enabled. This subtle distinction is extremely important to understanding how aliases and feeds work. In addition, if a category ever has a problematic feed, you can always disable that feed instead of the entire category, i.e. we do not need to enable every feed for a particular category.
For example, if you want to add the “EasyList Adware Filter” or one of the language specific feeds, you would click the “+” sign to the far right and that would add the individual feed to the already existing “EasyList” group.
On this screen, you need to ensure you switch “OFF” to “ON” and then click “Save” at the bottom of the screen.
If we go back to the Feeds, a category (group) that I always recommend adding is hpHosts. Click the “+” next to the hpHosts header (red arrow below) to add all the feeds related to this category.
After clicking the ‘+’ next to the hpHosts category, you are taken to a DNSBL feeds page with all of the feeds under that category pre-populated. All of the feeds in the list will initially be in the ‘OFF’ state. You can go through and enable each one individually or you can click ‘Enable All’ at the bottom of the list (red box below). Next, make sure you switch the ‘Action’ from Disabled to Unbound (red arrow below). Click ‘Save DNSBL Settings’ at the bottom of the page and you should receive a message at the top along the lines of ‘Saved [ Type:DNSBL, Name:hpHosts ] configuration.’
Click on the ‘DNSBL Feeds’ tab and you will be taken to the DNSBL feeds summary. Assuming everything went as planned, your feeds summary should look similar to the one below.
A couple of other items worth mentioning. If you take a look at the ‘Malicious’ category, you will notice that some feeds have selectable options such as the SANS Internet Storm Center feeds in the purple box (below). I personally recommend switching the feed from ISC_SDH (high) to ISC_SDL (low) as the high feed has under 20 entries and the low feed includes the high feed. In addition, I haven’t seen many false positives when using the expanded (low) list. Also, take note of the door-arrow graphic (in the red boxes below) next to several feeds. The door-arrow graphic means the feed is a subscription feed, which at the very least means you need to register for it. Some subscription feeds also have a fee associated with them. Subscription feeds can have a lower false positive rate and are typically updated on a more frequent basis. That said, I’m not using them for the purposes of this walkthrough. You will see selectable options and subscription feeds throughout the DNSBL feeds so it is important to understand what these graphics mean.
If you ever experience issues with a particular feed, you can go to DNSBL, DNSBL feeds and then click the pencil/edit icon next to that particular category (red arrow).
Once in the category edit screen, simply switch those feeds to ‘OFF’ (as shown below) and then click save at the bottom. You could also delete those feeds, however, I prefer to leave them all in the list so the category retains it’s checkmark when looking through the DNSBL feeds list.
You can add as many feeds as you like, but keep in mind that too many feeds can potentially slow down your firewall. It’s quite possible just adding a few categories by themselves is too much for a resource starved firewall! This is because feeds are periodically downloaded and likewise, unbound is reloaded regularly. If you using a system with limited resources (mainly RAM), you need to be extra careful. When in doubt, add feeds slowly and keep an eye on memory, CPU, etc. If you happen to have an installation of Nagios Core or Nagios XI available, then I’d also recommend heading over to my article on monitoring pfSense with Nagios.
Aside from the EasyList, ADs, and Malicious categories that come as part of the wizard, some other DNSBL categories I use and I have tested quite extensively include the following:
- hpHosts (all of them) – From MalwareBytes
- BBcan177 – From the creator of pfBlockerNG
- BBC (BBC_DGA_Agr) – From Bambenek Consulting <- This feed is extremely large
- Cryptojackers (all of them) – This blocks cryptojacking software and in-browser miners, but it also blocks various coin exchanges.
If you try out some of others and you feel like they worked well for you, please let me know and I’d be happy to make changes to this page based on feedback!
Forcing DNSBL feed updates
Anytime you make changes, you can either wait for the next update or you can force the changes yourself. To force the changes, go over to the Update tab within pfBlockerNG. Heed the warning in the first red box and make sure you are not going to run the updates near the time your cron job would automatically run. If the countdown timer is less than 5 minutes, I would not recommend running it and instead just wait for the system to run it automatically. Assuming you are good on the time, go ahead and click the ‘Run’ button. You will see progress updates in the gray window below including the number of domains downloaded for each list, when the list was last updated, etc. Also note that pfBlockerNG is smart enough to check for and eliminate duplicate DNS (# Dups) entries between the lists.
[ Spam404 ] Reload [ 05/29/18 16:20:07 ] . completed .. ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 7066 7064 62 0 0 7002 ---------------------------------------------------------------------- [ SFS_Toxic_BD ] Reload [ 05/29/18 16:20:08 ] . completed .. ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 14244 14242 6 0 0 14236 ---------------------------------------------------------------------- [ VXVault ] Reload [ 05/29/18 16:20:09 ] . completed .. ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 85 62 58 0 0 4 ---------------------------------------------------------------------- Saving DNSBL database... completed ------------------------------------------------------------------------ Assembling DNSBL database... completed [ 05/29/18 16:20:14 ] Reloading Unbound Resolver..... completed [ 05/29/18 16:20:17 ] DNSBL update [ 158481 | PASSED ]... completed [ 05/29/18 16:20:18 ] ------------------------------------------------------------------------ ===[ GeoIP Process ]============================================ ===[ IPv4 Process ]================================================= ===[ IPv6 Process ]================================================= ===[ Aliastables / Rules ]========================================== No changes to Firewall rules, skipping Filter Reload No Changes to Aliases, Skipping pfctl Update ===[ FINAL Processing ]===================================== [ Original IP count ] [ 0 ] ===[ DNSBL Domain/IP Counts ] =================================== 158481 total 43442 /var/db/pfblockerng/dnsbl/hpHosts_ATS.txt 20749 /var/db/pfblockerng/dnsbl/MDS.txt 14641 /var/db/pfblockerng/dnsbl/EasyList.txt 14597 /var/db/pfblockerng/dnsbl/Cameleon.txt 14236 /var/db/pfblockerng/dnsbl/SFS_Toxic_BD.txt 9660 /var/db/pfblockerng/dnsbl/SWC.txt 8466 /var/db/pfblockerng/dnsbl/CCT_BD.txt 7738 /var/db/pfblockerng/dnsbl/Abuse_URLBL.txt 7002 /var/db/pfblockerng/dnsbl/Spam404.txt 4529 /var/db/pfblockerng/dnsbl/Abuse_urlhaus.txt 2592 /var/db/pfblockerng/dnsbl/MDS_Immortal.txt 2255 /var/db/pfblockerng/dnsbl/D_Me_ADs.txt 1899 /var/db/pfblockerng/dnsbl/Abuse_DOMBL.txt 1470 /var/db/pfblockerng/dnsbl/ISC_SDL.txt 1081 /var/db/pfblockerng/dnsbl/MDL.txt 1071 /var/db/pfblockerng/dnsbl/D_Me_Malv.txt 930 /var/db/pfblockerng/dnsbl/MVPS.txt 611 /var/db/pfblockerng/dnsbl/BBC_DC2.txt 495 /var/db/pfblockerng/dnsbl/SBL_ADs.txt 402 /var/db/pfblockerng/dnsbl/Adaway.txt 312 /var/db/pfblockerng/dnsbl/Yoyo.txt 140 /var/db/pfblockerng/dnsbl/Ponmocup.txt 45 /var/db/pfblockerng/dnsbl/Botvrij_Dom.txt 42 /var/db/pfblockerng/dnsbl/Abuse_Zeus_BD.txt 28 /var/db/pfblockerng/dnsbl/D_Me_Tracking.txt 23 /var/db/pfblockerng/dnsbl/Malc0de.txt 21 /var/db/pfblockerng/dnsbl/H3X_1M.txt 4 /var/db/pfblockerng/dnsbl/VXVault.txt 0 /var/db/pfblockerng/dnsbl/D_Me_Malw.txt ====================[ DNSBL Last Updated List Summary ]============== Jul 31 2015 D_Me_Tracking Mar 9 2016 D_Me_ADs Jan 20 18:32 Adaway
Testing By Browsing
So what does the finished product look like? On many sites, you’ll see gray boxes where an ad normally would have been. A browser add-on like uBlock Origin (discussed below) further cleans this up by removing the gray box entirely and it also provides some secondary protections. Also keep in mind that some ads are still served such as video ads on YouTube. Those ads cannot be blocked via pfBlockerNG since the ad content is served from the same domain names (DNS) as the video content.
If you visit Yahoo.com (why? seriously, find a new news site), our pfBlockerNG configuration eliminates the wasteland of ads that you normally see as well (red box below). Many sites will look similar to this with vast regions of white space where ads normally would show and don’t be surprised to find ads intermingled with news on many sites. <- In advertising, it’s all about improving that click through ratio (CTR)!
How it works – testing from the command line
So you see the end result when browsing, but what’s really going on? How the DNSBL portion of pfBlockerNG works is most easily seen via a command line. Normally, you would ping 302br.net and get back their actual IP address. However, with pfBlockerNG properly setup you will instead see a reply of 10.10.10.1, which is the default virtual IP address DNSBL creates. Basically, the ad/malvertising domain name is blackholed instead of displaying (or resolving). Feel free to test this against any domain in any one of the lists that you added. If you followed my examples/recommendations above, you will likely have a DNSBL list that is well into the hundreds of thousands if not millions.
Integral Ad Science
# ping 302br.net PING 302br.net (10.10.10.1) 56(84) bytes of data. 64 bytes from 10.10.10.1 (10.10.10.1): icmp_seq=1 ttl=64 time=0.684 ms
Yahoo – analytics.yahoo.com
# ping analytics.yahoo.com PING analytics.yahoo.com (10.10.10.1) 56(84) bytes of data. 64 bytes from 10.10.10.1 (10.10.10.1): icmp_seq=1 ttl=64 time=1.18 ms
Statistics and graphs
Wouldn’t it be nice to see which groups and feeds are working? No problem! pfBlockerNG has some really fantastic graphs built-in as shown below. You can even see the top blocked domains, source IPs with the most blocks, blocked user agent strings, TLDs, and much more. Super cool! Also helpful is you need to whittle down the number of feeds you are using, i.e. this feed accounts for 50% of your blocks and it’s a third the size of these other two feeds combined. Just go to Firewall -> pfBlockerNG -> Reports -> DNSBL Stats to see all the DNSBL eye candy, aka graphs/stats.
Troubleshooting/Whitelisting
What happens if/when a website is inadvertantly blocked? Afterall, it is bound to happen. You can either remove the offending list entirely (DNSBL -> DNSBL Feeds -> Edit the list in question) or more preferably, you can just whitelist the domain. The absolute easiest way to do this is by going to the Reports tab and scrolling down to the DNSBL section. Clicking on the red lock (in the orange box below) will temporarily unlock the domain so you can verify if it is indeed the domain that needs to be whitelisted. Clicking the ‘+’ (in the purple box below) will add the domain to the DNSBL whitelist.
When clicking the ‘+’ you will then receive a prompt about whether you want to perform a wildcard whitelist or just a whitelist. Read the explanation, but I typically use whitelist because it is more exact and less prone to letting something past. I would also suggest adding a description so you know what was broken and/or why you fixed it, i.e. today it makes perfect sense, but it might not 6 months from now. In my years of IT/security, I’ve found documentation is as helpful for me as it is for someone else. Maybe I’m just getting old!
If you go back to the main DNSBL tab and expand the DNSBL Whitelist section toward the bottom, you should now see the domain you whitelisted. You might also notice that if the domain you are whitelisting has CNAME records, pfBlockerNG is smart enough to add those too as shown below.
As you might have expected, you can also simply type each domain in on a separate line and then click ‘Save’ if you know which domains to whitelist. If you want the whitelist additions/changes to occur sooner rather than later, you will also need to go back to the ‘Update’ tab and click ‘Run.’ If you don’t want to do the trial and error on your own (and I *really* think you should), I have provided some whitelist recommendations below.
It’s also worth mentioning that if a system already resolved the domain name on your system and it is previously resolved to 10.10.10.1, then you may need to clear your local DNS cache, your browser cache, or both. To clear your machine’s cache, from a command line on Windows, type in ‘ipconfig /flushdns’ and that should take care of it. You can run a similar command on a Linux system, although the commands can vary from one installation to the next. More often than not, simply restarting your network interface will work; on most distributions, ‘service networking restart’ or ‘systemctl restart network’ should take care of it for you. Each browser has a slightly different way to clear the cache, however, all of them allow you to pull a new version of the website if you hold down “Shift” while clicking on the refresh/reload button.
If ads are not getting blocked and the ping commands above don’t return the virtual IP address, it’s also possible your local machine is not using pfSense for its DNS settings. If you are using Windows, check your network settings and make sure it is set to your pfSense IP address. On Linux/*nix, check your /etc/resolv.conf or even Network Manager (if using a GUI). If you are not using pfSense for your DHCP server, you may need to do some digging.
Browsers can also get in the way especially with the advent of DNS over HTTPS. If you find your ping tests work, but your browser doesn’t, then that is most likely your issue. Although somewhat uncommon, some anti-virus packages and endpoint protection can mess with your DNS settings too. Furthermore, those changes may not necessarily be reflected in your operating system’s DNS settings. For example, Avast Premier has a Secure DNS feature that will force your browser to use Avast specified DNS servers in an effort to prevent DNS hijacking. If you find that other devices on your network are blocking ads and one particular device doesn’t, then your anti-virus or endpoint protection very well may be the culprit. When all else fails, you can always fire up Wireshark for a packet capture to ensure your system is querying the DNS server(s) you specify.
Whitelist Recommendations
These are a few domains I’ve seen cause issues if they end up on the various DNSBLs. You can easily copy and paste them into the “custom list” as described above. If you ended up using the pfBlockerNG wizard, BBCan actually incorporated these recommendations already. 😉 If you have no plans to use some of them (based off their name alone), you can and should omit them from your whitelist. Do you have other recommendations beyond the ones I have listed? Let me know and I’ll add them!
s3.amazonaws.com s3-1.amazonaws.com # CNAME for (s3.amazonaws.com) .github.com .githubusercontent.com github.map.fastly.net # CNAME for (raw.githubusercontent.com) .apple.com .sourceforge.net .fls-na.amazon.com # alexa .control.kochava.com # alexa 2 .device-metrics-us-2.amazon.com # alexa 3 .amazon-adsystem.com # amazon app ads .px.moatads.com # amazon app 2 .wildcard.moatads.com.edgekey.net # CNAME for (px.moatads.com) .e13136.g.akamaiedge.net # CNAME for (px.moatads.com) .secure-gl.imrworldwide.com # amazon app 3 .pixel.adsafeprotected.com # amazon app 4 .anycast.pixel.adsafeprotected.com # CNAME for (pixel.adsafeprotected.com) .bs.serving-sys.com # amazon app 5 .bs.eyeblaster.akadns.net # CNAME for (bs.serving-sys.com) .bsla.eyeblaster.akadns.net # CNAME for (bs.serving-sys.com) .adsafeprotected.com # amazon app 6 .anycast.static.adsafeprotected.com # CNAME for (static.adsafeprotected.com) google.com www.google.com youtube.com www.youtube.com youtube-ui.l.google.com # CNAME for (youtube.com) stackoverflow.com www.stackoverflow.com dropbox.com www.dropbox.com www.dropbox-dns.com # CNAME for (dropbox.com) .adsafeprotected.com control.kochava.com secure-gl.imrworldwide.com pbs.twimg.com # twitter images www.pbs.twimg.com # twitter images cs196.wac.edgecastcdn.net # CNAME for (pbs.twimg.com) cs2-wac.apr-8315.edgecastdns.net # CNAME for (pbs.twimg.com) cs2-wac-us.8315.ecdns.net # CNAME for (pbs.twimg.com) cs45.wac.edgecastcdn.net # CNAME for (pbs.twimg.com) aan.amazon.com www.aan.amazon.com mads.amazon.com www.mads.amazon.com aax-us-iad.amazon.com www.aax-us-iad.amazon.com telemetry.malwarebytes.com # causes high cpu otherwise www.telemetry.malwarebytes.com # causes high cpu otherwise elb-telemetry-prod-external-1332413525.us-west-2.elb.amazonaws.com # CNAME for (telemetry.malwarebytes.com)
TLD Blacklisting
TLD (top-level domain) blacklisting is another option in DNSBL. Don’t forget you need to ‘Enable’ the TLD option at the top of the DNSBL configuration page to use the features discussed here. While I don’t normally advocate static blacklisting because the bad guys will simply move around it, TLD blacklisting is a rare instance where you can eliminate some potential attack vectors although its usefulness depends entirely on your situation. TLDs are the characters after the last dot on a domain name, e.g. com, net, and biz are some common ones. The number of TLDs has skyrocketed and there were well over 1,500 in early 2017! Over time, some TLDs have become wastelands for nefarious activity such as command and control servers. If you no plans to connect with a particular TLD and it has shown to be less than reputable, i.e. most sane companies wouldn’t bother trying to use it for legitimate businesses, you can just go to the main DNSBL tab and block it outright using the section below.
Even Brian Krebs got in on talking about the how some TLDs are used extensively for typosquatting — Omitting the “o” in .com Could Be Costly. If you don’t want to read the full article, just understand that instead of typing in remax[dot]com, a user mistakenly types in remax[dot]cm and is directed to a malicious site. There are similar alternative .cm domains for ESPN, Hulu, iTunes, Aetna, AOL, Chase, Facebook, WalMart, etc. and over 1000 others. Needless to say, the .cm TLD is not good.
If you’re looking for a little more guidance of what is ‘bad’ then look no further than Spamhaus and the website link below. Brian Krebs wrote a great article about the badness of TLDs as well. Spamhaus is constantly updating this list and related statistics so check it directly for the most up-to-date information. At the link below, you’ll also find a dropdown to show you the ‘badness’ of every TLD even beyond the top 10 list. At the very least, I would suggest adding the top 3 TLDs in the green box below along with the .cm TLD from the Krebs article. Adding the entire top 10 would likely not cause too many issues, although keep in mind that you will see false positives. For example, note that #8 in the list below is .biz (at the time this image was pulled), which is used by legitimate businesses. I’ve added a textual version of the TLD list below the image so you can easily copy/paste it into your firewall.
https://www.spamhaus.org/statistics/tlds/
cm party click link technology gdn study men biz reise stream
DNS blocking
DNS blacklists are great, but what if a new ransomware botnet pops up, a user gets infected fairly early in the campaign, and it starts calling previously unknown domains? If your DNSBL feeds are set to update every 4 hours and it takes time for them to get included on that list to begin with, it might take awhile before your DNS catches and blocks it. We need something more real-time… To provide another layer of protection, I would also recommend using Quad9 as your primary DNS on pfSense. I wrote up an article some time ago about how to do just that.
Browser side blocking – Ublock Origin
I constantly preach defense-in-depth and this is no different. You could have every malicious advertising domain on the planet included in your configuration, but a new one will inevitably pop-up 5 minutes from now. Aside from some other defenses, I would also strongly suggest using uBlock Origin on all of your browsers. uBlock Origin exists for Chrome, Firefox, etc. so there really isn’t a reason not to have it! While nothing is foolproof, it is another fantastic addition to your overall security.
Using pfSense as an OpenVPN client
Note the underlined client in the previous headline. This change does not apply to you if you use pfSense as an OpenVPN server, but rather when you use it as an OpenVPN client. In these handful of instances, users are redirecting all of their traffic to a VPN service such as Private Internet Access (PIA) or ExpressVPN. In this scenario, users reported back that their DNS was leaking after configuring the solution above.
You can handle this a number of ways. One possible solution would be use to DNS over TLS as described in Configuring Quad9 on pfSense. Another option is to go to Services -> DNS Resolver and switch the outgoing network interface to LAN only instead of all (shown below). As always, don’t forget to click ‘Save’ after making your changes.
Dallas Haselhorst has worked as an IT and information security consultant for over 20 years. During that time, he has owned his own businesses and worked with companies in numerous industries. Dallas holds several industry certifications and when not working or tinkering in tech, he may be found attempting to mold his daughters into card carrying nerds and organizing BSidesKC.
185 thoughts on “Block Ads & Malvertising on pfSense Using pfBlockerNG (DNSBL)”
Thanks for the guide very helpful and everything has a detailed explanation.
You’re very welcome Matthew! Thanks for the feedback!
Hello once I have this setup it seems I am unable to access dropbox. Even access website or sync client. Is there any work around?
Did you add the whitelist recommendations? I am able to access Dropbox without issue. If whitelisting doesn’t work, you can also remove the offending list; simply go to the Reports -> Alerts, find the feed with the Dropbox related domains, and then go back to your feeds to remove it. Don’t forget to force reload after you removing it. You will also probably need to flush your local DNS and/or browser cache too. These items are explained in the troubleshooting/whitelisting section if you need further guidance. Good luck!
great guide! It’s the first time I’ve felt confident in my pfblockerng configuration, so thanks!
One thing, though, I’m having issues updating D_Me_Malw & D_Me_Tracking – is this common? It looks like yours was working just fine….
Russell, thanks for the feedback! It looks like S3 was added to one of the blacklists, which in turn caused those feed downloads to fail (they are hosted at s3.amazonaws.com). Look at your DNSBL alerts (Reports -> Alerts -> DNSBL heading) and then whitelist one of the alerts that say s3.amazonaws.com. Go back to Update and Force/Run and you should see the download goes through without issue for those feeds. Can you verify if you used the whitelist from the guide? I’m just curious if I need to add other hosts to it. Thanks!
I used the pihole for some time and fiddled with the pfsense dnsbl time and again. Before discovering that there was a -devel update to pfblockerng I tested the tld blacklist.
Now you see this is extremely important and it must function like the whitelist. In the older version there was a custom whitelist feature but only the tld blacklist. That bugs me to no end. Blacklisting individual sites is extremely important. I find ad serving sites that get by the blocklists all the time (or just sites that I never want to visit). Without a site blacklist I would not use the tool. I can’t understand why the author doesn’t provide a feature to blacklist sites on the same page or in the same area as the whitelist. It is perplexing.
So I decided to search for a definition of tld blacklisting. I found someone’s answer that indicated that the tld blacklist operated like the custom whitelisting without the use of wild cards. So I tried it by putting the whole sitename in the tld blacklisting box. That worked.
A few days later I saw this post and decided to upgrade. I immediately worried that the tld blacklist feature would be broken. To my surprise it did not fail me. It worked. I did not tick the tld option on the page as you specified.
So, that’s good news yet I’m fearful that since this feature is so poorly documented that he might sneak Nerf it when I’m least looking. Let’s hope not because site blacklisting here is important. And I mean “here” on this page. I am aware of domain overrides. I don’t want to jump around to all over just to do what should be done where everything else blacklisting and whitelisting related is done.
The TLD whitelist is only used in conjunction with the TLD whitelist and the author specifies this several times in the various infobox descriptions. That said, I’m a little confused about the TLD blacklist/whitelist working without the TLD option. I tested this extensively myself (and double/triple-checked as I was writing this walkthrough) and disabling TLD caused the TLD blacklist/whitelist to quit working every time.
FWIW, if you want to block individual sites, you can do this without any feeds… Simply go to DNSBL -> DNSBL Feeds and then click Add. You can then name it “custom_blacklist” (or whatever you want), leave DNSBL source blank/off, select action as unbound, and then then add your domains to the “DNSBL Custom_List” at the bottom. Either way, hopefully this helps!
Great info on pfblocker setup. Thanks! Very useful. Hope you keep updating this page…
Happy to hear it helped and thanks for the feedback! That’s the plan!
I’m running pfBlockerNG 2.2.1 and even with youtube.com and http://www.youtube.com whitelisted, YouTube was not working. I identified H3X, specifically H3X_1M, was blocking it. For now, disabling that list allowed YouTube to start working again. Have you experienced something like this?
I use that particular feed in all of my installs as well. I have youtube.com and http://www.youtube.com added to my whitelists because they do end up on feeds from time-to-time. It appears an additional CNAME is added when whitelisted so you might verify it is present in your whitelist. FWIW, it seems like I was in the alerts -> reports a fair amount when I originally configured DNSBL. Over time, this lessened to the point I honestly don’t know the last time I had to whitelist a domain. Hope this helps!
how to go to blocked youtube.com site, I have added to whitelist list, but still to blocked. And what processes make youtube blocked.
okay, thanks, I try to enter in whitelist, youtube site can open.
i have 2 questions
1. To block facebook can use pfblocker, I tried to put into list TLD Blacklist / Whitelist, still can not
2. to block mobile applications like youtube and facebook. whether to use pfblocker. Maybe it could give clues or something else.
3. if i use m.youtube.com, can still opened
To block Facebook, this is what I’ve done in the past. Go to pfBlockerNG -> DNSBL -> DNSBL Feeds and click add. Use this github repo for the source – https://github.com/jmdugan/blocklists/blob/master/corporations/facebook/all. From there, type in something for the name and header, switch the state to on, and then switch action to unbound. It works extremely well. I haven’t tried finding/creating a YouTube blocklist, but it could be accomplished via the same means. Good luck!
I can block facebook and other social media through TLD. I will try the list you give, thank you.
Do you know how to create a schedule to open a blocked one at a certain time in pfblocker?
Instead of using TLD, I would stick with the blocklist as previously suggested. Once your rules/aliases are created, you could modify the alias so it didn’t have “pfB_” at the beginning, which means future pfBlockerNG changes should leave it unchanged. You could then add a schedule (Firewall -> Schedule) and apply it to the associated rule.
I also have added the list you provide (https://github.com/jmdugan/blocklists/blob/master/corporations/facebook/all) and I have update but still can open facebooknya.
what is missing
Did you perform an update once the list was added? Is the list now showing in your feeds? Last but not least, have you tested via the command line using ping or nslookup to verify the virtual IP is returned instead of the actual IP?
I can block facebook.
it turns out for his links I add ‘raw’ to: (https://raw.githubusercontent.com/jmdugan/blocklists/master/corporations/facebook/all)
Excellent! Yes, anytime you find a blocklist from GitHub, make sure you use the “raw” version in the pfBlockerNG feed configuration. Glad to see it worked!
i will try disable and enable pfblocker..
after that I check in the (Firewallpf->BlockerNG->Alerts) reports tab menu
on DNSBL alerts there is no red key and ‘+’ .. what caused it… just blank
Are you sure you are looking at the DNSBL section and not the IP section on the alerts page? The DNSBL entries should show the + and lock regardless. The IP entries only show the + and lock if you have suppression enabled on the IP tab.
it turns out that DNSBL has not been perfectly synchronized, therefore my RAM is overloaded and red button and sign ‘+’ does not exist
Thanks for the update!
I get an invalid certificate error for both https://youtube.com and https://dropbox.com. It happens on both Chrome and Firefox. Both dropbox and youtube have been whitelisted. I copied your full list.
NET::ERR_CERT_INVALID
Any ideas how to fix this?
It sounds like those 2 sites are getting redirected because they are on a feed/list and causing the SSL cert error. Did you force update, flush your DNS cache, and then clear your browser cache after adding them to the whitelist? If that doesn’t work, then there might be something going on with your whitelist. You could always simply find the offending feed via reports -> alerts and then remove it from the corresponding DNSBL feed. Hope this helps!
Hi,
I just found your site looking for information on PFSENSE, PFBLOCKERNG, and PIHOLE.
Your guides on both of those is excellent. I have been using PIHOLE for a year and a half now and I am very happy with it. However, I would like to ask if you can provide some advice on a situation I have been having recently.
My network consists of the following – Modem, PFSENSE box, PIHOLE, WINDOWS AD DC/DNS, Windows Server FP DNS, and Clients. I originally started with the PFSENSE box doing the DHCP service and setting the PIHOLE as the DNS server for all the clients under DHCP in PFSENSE. The PIHOLE was forwarded to the Windows AD/DNS and the Windows AD/DNS would be forwarded to the PFSENSE box via the forwarders tab. This worked well.
I decided to sign up for a VPN service. I configured the OPENVPN client, downloaded the client configuration file(s) from the provider and set it up in PFSENSE. I created aliases as I only want a couple of devices to go out the VPN connection. I have all my clients setup in statics IP’s on the DHCP server (PFSENSE). I have unbound enabled on PFSENSE and the forwarder option checked, under DNS General, I have a couple of DNS entries, OPENDNS and Google.
I checked many but the one that helped a lot was this one:
https://www.techhelpguides.com/2017/06/12/ultimate-pfsense-openvpn-guide/
For DNS leak prevention, I went with method 1 as the second one I could not get it to work. The DNS leaked. So having set this up with method one, the clients that are supposed to be behind the VPN all work no problem but the clients that use the regular WAN connection, use the same VPN DNS. When the VPN goes down, sometimes all the clients loose internet connection.
So I started searching and asking on forums and I was given advice that it’s best to have the Windows AD/DNS set as the DNS for the clients, then on the Windows DNS under the forwarders tab, set the PIHOLE and on the PIHOLE set the OPEN DNS, Google, or the VPN DNS, whatever. I switched my configuration to this now. I disabled the DHCP from PFSENSE and installed it on the Windows FP/DNS mentioned above. However, this also leaks the DNS for the two VPN devices. On the PIHOLE, if I remove those DNS entries (google, opendns, etc) and I set the PFSENSE, then there is no more DNS leak, however, I am back to square one. The none VPN clients use the VPN DNS.
So now, I started re-searching again and a lot of people suggest that maybe giving PFBLOCKERNG a try might do the trick. This is how I ran into your guide and I would like to give it a try but I think my set up the way I have my network and the way I want it to work, makes it a bit hard to configure. I am missing something somewhere, I just cannot figure it out what it is. It’s frustrating but I don’t give up easily. If you can provide any advice, I’d really appreciate it. If you need any more info, let me know.
Thanks very much and apologies for the long message.
Guicho.
Guicho, sorry I’m just getting back to you. You have quite a bit going on — Windows AD, VPN, and split gateways — and it will take a bit of playing around to get it to work properly. First, I would remove the pi-hole from the setup as you suggested. I love pi-hole, but it is redundant in this scenario. Second, I would disable DHCP on pfSense and have the clients use both DHCP and DNS from the Windows server, which is recommended for AD environments anyway. I would then point my domain controller to the pfSense for the forwarding DNS. This would allow your clients to benefit from pfBlockerNG. Now here is the issue… All of your DNS traffic at this point will go through your standard gateway and the firewall isn’t going to be able to differentiate because all traffic is *originating* from the Windows DC. At this point, the only way to get around this would be to change the DNS entries on individual DHCP static leases, which would then break your AD environment. If those systems don’t need to be on AD, then go that route. Worth mentioning is to remember to not *mix* DNS servers, i.e. don’t have an AD DNS server, OpenDNS, or pfSense in the same client config. DNS servers are not queried in order so you will end up with something working one minute and then not the next. What I would suggest doing instead if you need those systems on AD is to encrypt all your DNS traffic via DNS over TLS and not worry about which gateway DNS traffic goes out. I discuss how to do this in this article, https://linuxincluded.com/configuring-quad9-on-pfsense/ for Quad9, although a similar config would work for Cloudflare or any other DNS provider that supports DNS over TLS. Hopefully that helps and best of luck!
hello, thank you for the guide. i have a question, when pinging site with cmd. i don’t get the 10.10.10.1 like in the picture. i think i am getting the site ip instead. this is on a fresh pfsense 2.4.3 with pfblocker devel 2.2.1. not sure if i miss a step or that is normal. can you shed light on this?
If you’re getting the actual IP of the site, then either a) that site is in your local DNS cache, b) your Windows settings are not quite right, c) DNSBL is not started/enabled, or d) you don’t have DNSBL feeds enabled. From the command line, take a look at ‘ipconfig /all’ for your primary ethernet adapter. Make sure your firewall IP is both the gateway and DNS server. You can also double-check whether DNSBL is working via nslookup and then typing ‘server‘ followed by various hostnames (from the feeds) you want to test. Hopefully that helps! If not, give me a holler back.
Great write-up. Coming from DDWRT, I needed a good walkthrough like this to get me going. Easy to follow and just works unlike a lot of other tutorials I’m reading on the pfSense packages.
Thanks!
Thanks for the feedback! Much appreciated!
I haven’t read an article so detailed and easy to understand that this one! I just put together a new firewall hardware (Xeon processor and 8G RAM) and one of the things I really use is pfBlocker.
This, by far, is the best set of instructions ever.
Easy to follow and well explained.
I appreciate your effort on putting this together.
Keep up!
Thanks for the feedback Juan! Much appreciated!
What an amazing and detailed tutorial. I really learned a lot, thanks so much! Appreciate you spending the time.
So happy to hear you learned from it! I appreciate the feedback!
Dallas thank you for the write up on blocking ads with pf
No problem! Thanks for the feedback!
Hello @Dallas Haselhorst.
I do not know where I did wrong. But my ping results on windows still returns true IP of the server. If I ping on pFsense then it returns true 10.10.10.1. I have removed google’s DNS and open DNS on the DHCP server. I also assign a static IP to the computer and set the DNS on the pFsense’s IP LAN. But it seems that things are not working as I expected.
If you have pfSense responding correctly you are definitely on the right track! The most likely culprit is the local system DNS, which you already corrected to some degree with a static IP and static DNS. My guess is that your local system still has/had the DNS entry in its cache. If you are using Windows, type in ‘ipconfig /flushdns’ (minus the quotes) to clear it. You should see ‘Successfully flushed the DNS Resolver Cache’ on modern versions of Windows.
FWIW, the static IP and static DNS aren’t necessary on the individual machine if you are using DHCP. Simply go to Services -> DHCP Server to change the DNS server assigned to your DHCP clients. You can either specify the DNS or leave it blank to use the pfSense DNS resolver unbound. To verify this setting is correct, you can run ‘ipconfig /all’ from the command line and look for the line that states ‘DNS Servers.’
Also, you were correct in removing your other DNS entries. DNS is a little funny because it doesn’t react as you might expect — primary server, then secondary server, etc. I’ve discussed this before on other posts such as the Configuring Quad9 on pfSense post, https://linuxincluded.com/configuring-quad9-on-pfsense/. Read the red text on that page and it will describe this issue. Assigning “different” vendor DNS works the same at both the client and firewall level, i.e. the DNS servers are *not* queried in order.
Yes. I did all the work you said above. Everything works OK. Just on my computer it does not work as expected. There are all ads of google it blocked very well, other advertisers almost no. I tried adding the host addresses of other advertisers to a host file but it still does not work. Although all the telephone or TV equipment inside my LAN works well.
The file host that I added it can block the majority of advertisers, analyzing the world here: hxxps://ketnoidamme.vn/downloads/hosts.txt
If it’s blocking Google ads I would think it is working. If I ever think something isn’t working quite right, I select a handful of hostnames from the feeds and test them from the command line to ensure they return the pfBlockerNG virtual IP. Also, keep in mind that some sites now utilize “same origin” so you won’t block them.
Last but not least, is it possible an application is using a different DNS. You might also try a Wireshark capture to determine if something is directly querying a different DNS server. For example, if you using Firefox 62, it has DNS over HTTPS support which means it could bypass your local DNS server. It’s an absolute shot in the dark, but Wireshark is a must when something isn’t going quite right. Good luck!
Thank you for all that you have shared. Perhaps I have found something that has interfered with the DNS system on my computer. This is due to Avast’s “Real Site” DNS Custom feature of Avast Premier that I have installed. It has interfered with the DNS on my browsers. Except for Internet Explorer, it has been tested to have discovered this interesting thing.
I want to contribute a share on the VPN Client section.
In this case, you need to activate the DNS Server enable feature and enter the local IP address. Then on the VPN client will work.
I wasn’t familiar with Avast Secure DNS, but after reading about it that makes sense. I did have a one line statement about how some anti-virus packages can mess with DNS configuration settings, but I’ll expand on that a bit and mention Avast as an example. Happy to hear you figured it out!
Your guide is just what I needed. Had to delete my old version and start from scratch to get it working. I have a few question however.
1) My windows computer is blocking adds on yahoo.com fine but my macbook pro is not. I’ve check the DNS setting and it is the IP of my PFSense firewall and have cleared the DNS cache without success. Any ideas?
2) Do you do any geoblocking in PFBlockerNG, for example china and russia? If so do you have recommendations?
Hey Warren! Thanks for the feedback!
1) Since other systems are working properly and you verified the settings are correct, my next check would be some software on that particular system, e.g. VPN, anti-virus, etc. I would also test from the command line and see if those results are different than your browser results. If you can’t seem to find anything, fire up Wireshark and determine where the queries are going.
2) I don’t use geoblocking because I occasionally access sites around the world. Geoblocking is a fantastic addition *if* you know your environment extremely well and you know where your traffic goes. The few times I used it, I would block the usual suspects but I would also watch my logs to see where activity came from. For example, if I saw an increase in activity (WAN block) from France and I knew I wasn’t going to access anything in France, I would add it to my geoblock list. If I saw an increase in activity from Morocco, I would add them, etc. I used to do this by hand, but keep in mind that the new version of pfBlockerNG has the IP Block Stats by Country graph via pfBlockerNG -> Reports -> IP Block Stats to help you. I always found geoblocking ridiculously difficult to troubleshoot which is the reason I only use it in fringe cases at this point and instead opt for “stacking” block lists.
Thanks Dallas, i was reinstating a/my pfsense router and was automatically working towards the ‘old’ version. This is so much easier, your howto but mostly the awesome updates by BBcan177. Loading the updates atm, looking forward to debugging the lists 😉
So happy you were able to use the guide! It’s impossible to overstate the work by BBcan177. pfBlockerNG has went from a country block list to the must have pfSense package (both DNSBL and IP blocking). I refuse to run a pfSense firewall without pfBlockerNG. 😉
Just got into pfSense last week with a purchase of a new XG-7100U and I love it. Thank you for this post I listened to your advice – first thing I did after logging in. I also configured QUAD9 as you suggested. ALMOST all is working perfectly. Only thing is with TLD Blacklisting/Whitelisting. I used the ten TLDs from your post – and cm – but I DO go to one .biz site. The Blacklisting works perfectly. My problem is when I enter cigarplace.biz and/or cigarplace.biz/185.11.187.115 into my Whitelist – I still get blocked. I have repeatedly cleared my Safari cache and used the Mac OS X command “sudo killall -HUP mDNSResponder;sudo killall mDNSResponderHelper;sudo dscacheutil -flushcache” to clear local cache. I can leave biz out of my Blacklist but just bugs me I can’t get it to work. Thanks again!
Awesome! So happy you are making progress! Here are a few things to look at and/or try… First, did you also enable IP blocking? If you did, make sure the IP for the site is not getting blocked. Second, I recommend checking via the command line. There are just too many variables at the browser level — browser cache, possible endpoint protection issues, etc. Third, do you know if the site is aliased? For example, is a whitelist entry required for both www and the main site? Sometimes scripts or external calls are made on sites and those are a source of problems as well. Finally, have you tried the exclusion feature? Give all of that a shot and then see if the site is still showing in the reports/alerts section. Keep an eye on that as it can often give you some insights as well. I’d be happy to hear what you figure out and include it in the guide if it is relevant!
Hi Dallas,
Thank you for this interesting tutorial.
I was not able to finish the complete procedure because I don’t have an option you apparently have on your side. In the DNSBL feeds page, the Unbound action is not an available option. I have several other ones but not “Unbound”.
Could you tell me what am I doing wrong?
Thank you for your feedback.
Hey Denis! If unbound is a missing option, you are either not using the pfSense DNS or you have a different pfSense-based DNS server enabled. To correct this, first disable your other DNS server (since both can’t listen on UDP port 53) if you have one and then enable Unbound via Services -> DNS Resolver. I have the appropriate settings for this and others on the Quad9 pfSense guide (link below). FWIW, you don’t need to use Quad9, but I would recommend it. 😉
https://linuxincluded.com/configuring-quad9-on-pfsense/
Would it be possible to be a bit more specific about how to get the unbound option working? Loved the tutorial, but I’m stuck on this point.
Unbound *should* work. If it doesn’t start, I would double-check if the forwarder is running as they cannot run simultaneously.
I had the same problem with no “unbound” drop down. It was my error as I was in the IPV4 section and NOT the DNSBL. Scroll down and you will see it towards the bottom.
Thanks for the suggestion bluedog!
Hi Dallas,
Thank you for your tutorial.
Could you send me the links of all your ads block and others ?
Hey Daniel! There are honestly too many for me to list here. However, my selection method is pretty straightforward. I use the following DNSBL feed groups: ADs, Malicious, hpHosts, BBcan177, BBC, and Cryptojackers. I don’t use any paid feeds (arrow with exit door icon) in those selections. Last but not least, I also still add the Firehol3 list as a user defined feed although it is mostly redundant to the other feeds. If I ran a web or email server, I would also suggest the phishing group. Hope this helps!
Upgraded pfSense to 2.4.4 today, upgraded to pfBlockerNG-devel, reconfigured the blocklists per your previous guide, configured DNSBL with this guide and switched pfSense DNS servers to Quad9. Seems to be firing on all cylinders. Snort working great too. All of this really makes for a wonderful browsing experience and peace of mind. I still have not dealt with the kids infections on their computers. All kinds of hits coming from the LAN/OPT1 side. Maybe after dinner. Thanks again Mr. Haselhorst!
Fantastic Charlie! It really does make for a much better experience! Congrats on getting it all configured!
Very good Article. I saw BBcan107 link it on Reddit. I’ll bookmark this if anyone I know needs help setting this up. I have never had an issue using pfblocker and was surprised to see you using many of the lists that I use. I did use some of your whitelist entries! Thanks
Thanks for the feedback Ryan! Happy to see it helped you out!
This is a fantastic guide. Bookmarked!
I am a bit confused about the utility of having this and something like suricata running together. Is it as simple as suricata blocks incoming threats and this filters outgoing traffic? I noticed someone mentioned they were using this with snort without a problem. I was using suricata and the old pfblockerng. Had a few problems so wiped my pfsense box clean and started with a fresh 2.4.4 install yesterday. Now pfblockerng is up using this guide, I’d like to move on to getting suricata done ( but not sure if its that necessary). Thoughts? Any btw, do you have a guide for suricata? Just checking because you do it well!!
Hey Victor! Thanks for the feedback! Unfortunately I don’t have a guide on Suricata, but I’ll add it to my list of potential future guides!
As you know, Snort and Suricata are extremely similar as they are both IDS/IPS. I’ve played around with Suricata, but I mostly use Snort so that’s what I’ll reference to answer your question. I personally run pfBlockerNG (both DNSBL & IP blocker) as well as Snort… with a caveat. DNSBL and an IDS/IPS serve very different functions IMO and I would have no concerns running both of them concurrently. What I did find is that IP blocker (also part of pfBlockerNG) does overlap with IDS/IPS a fair amount. In fact, if you have an IDS/IPS on your WAN you’ll likely find that IP blocker handles about 99.9% of the internet cruft. In addition, for the purpose of speed/processing, packets are sent to the firewall rules (what IP blocker adds) and Snort simultaneously resulting in alerts from each of them. To my knowledge, Suricata processes the same way. As such, I’ve disabled Snort on my WAN side and instead, I have Snort running on my internal interfaces such as LAN and I’ve pared back my rulesets quite a bit. If you have numerous open ports on the WAN, I would leave the IDS/IPS enabled and only use rules specific to those open ports.
It was written for the older version of pfBlockerNG, however, I’ve included my guide on configuring IP blocker below. I would highly recommend going through it and getting it configured in addition to DNSBL.
https://linuxincluded.com/using-pfblockerng-on-pfsense/
You hit the nail on the head in regards to potential redundancies between IP Blocker and the IDS/IPS system – hence my question if an IDS/IPS is necessary if I have IP blocker going ( I don’t as of now). Will check out your guide. Thanks!
Hi Dallas
Thank you for writing such an informative and easy to follow article.
I have read it over and over again and followed every step to install on my pfsense 2.4.4
However, when I open a cmd box and type ping 302br.net, i get
Reply from 69.172.216.56: bytes=32 time=331ms TTL=50
instead of a reply from 192.168.57.1 (the virtual ip address i have entered)
Are you able to suggest where things may have gone wrong
Best Regards
Tony
Tony, do other blocked domains return the virtual IP? If so, that would at least tell you that your firewall config is correct. Since you didn’t specify, here are a few items to look at. Is the virtual IP you assigned within your network scope? I don’t know if is part of the issue or not, but the default, 10.10.10.1, is simply created as an alias for the network adapter. The only time I *don’t* use 10.10.10.1 is if I’m working on a 10.0.0.0/8 network. Check the DNS resolver on your pfSense to see if “server:include: /var/unbound/pfb_dnsbl.*conf” is in the custom options. Do you have other devices on your network that you can test? If not and you are familiar with Linux, you might download a live CD and boot it from VMware Player or VirtualBox to see if a different system works. I’ve had multiple readers report back that their endpoint protection was interfering with their pfBlockerNG config. You could also do a packet capture using Wireshark to see if your system is making DNS queries to the firewall or somewhere else. A packet capture in pfSense would accomplish the same thing. Good luck! And please report back what you find. 😉
Hi again Dallas,
Thankyou once again for taking the time to answer me 🙂
When you mentioned, other blocked domains – I thought which ones, how do i know of a blocked domain? I have only enabled ADs and Malicious categories.
I am using 10.x.x.x for my home network, so I have used 192.168.x.x for the virtual ip. The DNS resolver on pfsense does have “server:include: /var/… etc in the custom option, so that’s ok
I have followed the article [I removed the link due to inaccuracies] to use 1.1.1.1 as DNS servers and on the dashboard i see 127.0.0.1 listed first, then 1.1.1.1 and 1.0.0.1 as the DNS servers. When I configured as the article suggests, I placed a tick at DNS Server Override (don’t know if that is causing the trouble)
When I do a ipconfig /all on my windows computer I see
Ethernet adapter Local Area Connection:
Connection-specific DNS Suffix . : myhome.lan
Description . . . . . . . . . . . : Intel(R) I210 Gigabit Network Connection
DHCP Enabled. . . . . . . . . . . : Yes
Autoconfiguration Enabled . . . . : Yes
IPv4 Address. . . . . . . . . . . : 10.1.57.20(Preferred)
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Lease Obtained. . . . . . . . . . : Tuesday, 16 October 2018 6:32:20 PM
Lease Expires . . . . . . . . . . : Tuesday, 16 October 2018 9:32:20 PM
Default Gateway . . . . . . . . . : 10.1.57.1
DHCP Server . . . . . . . . . . . : 10.1.57.1
DHCPv6 IAID . . . . . . . . . . . : 244637312
DNS Servers . . . . . . . . . . . : 1.1.1.1
1.0.0.1
NetBIOS over Tcpip. . . . . . . . : Enabled
So does that mean pfsense box is not used for resolving DNS?
I’ve not used wireshark before so I will download and work out how to use it and report back
Hope the above gives you some clue as to what is happening
Best Regards
Tony
Hey Tony! Yes, your DHCP server is handing out the Cloudflare DNS to the clients directly, which isn’t necessary and won’t work for what you are trying to do. Go to Services -> DHCP Server and remove whatever you have in the DNS Servers section (steps 4 and 5 from the guide you referenced). While you are there, note the comment at the bottom “Leave blank to use the system default DNS servers: this interface’s IP if DNS Forwarder or Resolver is enabled, otherwise the servers configured on the System / General Setup page.” Basically, we want the pfSense to act as the resolver and if we didn’t add those servers into the DHCP config, it would have done that by default. 😉 After you have removed those, perform an ‘ipconfig /release’ and ‘ipconfig /renew’ from the command line and make sure your DNS server is now 10.1.57.1 as well. You can then test it out again via the browser or via the command line. You’re almost there! Holler if you need anything else!
P.S. You actually could have left the default pfBlockerNG virtual IP after all since it would fallen outside of your network range, i.e. your /24 network is 10.1.57.X. If you used a /8 network (10.X.X.X) instead then that would not have been the case.
Hi Dallas
Thank you very much for your comment and advise.
Yes it worked after removing the 1.1.1.1 and 1.0.0.1
But how do I know pfsense is using those DNS servers and not my local ISP DNS server to resolve DNS requests?
Sweet! The easiest way is to perform a packet capture on your WAN interface. If you are *not* doing DNS over TLS, you can simply go to Diagnostics -> Packet Capture, select protocol UDP, port 53, and start a capture. If you’re using DNS over TLS, that traffic occurs over TCP 853 instead so adjust accordingly. You could download the capture for analysis in Wireshark, but you should be able to see what’s going on in the capture window once you stop the capture. Below is my traffic out to Google DNS, which I use as part of my Nagios monitoring. Also, keep in mind that some devices may have hard-coded DNS entries… Google devices are notorious for this. If this annoys you (as it does me), you can add an outbound firewall rule that re-directs all DNS traffic to your firewall instead.
19:09:58.419352 IP [MyIP].63611 > 8.8.8.8.53: UDP, length 37
19:09:58.447900 IP 8.8.8.8.53 > [MyIP].63611: UDP, length 107
Very handy.
Thank you for fantastic tutorial. Issue with some news sites I visit. I’d like to allow ads on some domains only (ie local newspapers…) who block access to their stories when ads are blocked on their site. I’m sure the functionality is there. I’ve whitelisted the sites but ads are still blocked. How do I configure to allow content on particular sites, ignoring the ‘global’ blocking settings?? Thanks again!
I think you are going to have a difficult time doing what you are describing. The reason being is that the news sites aren’t what you would whitelist, but rather the calls to ad networks on those news sites. The glaring issue with that is if you whitelist those ads for that particular site, you whitelist that ad network for every site. This might not be the case if the news site is using their own ad network instead of one of the major ones; in that case, you could visit the site and see what calls it makes and whitelist those. Hopefully that explanation makes sense.
Another alternative I would recommend would be to use a specific browser such as Firefox for those handful of sites and nothing else. Firefox supports DoH (DNS over HTTPS) so you could configure it to use CloudFlare or Quad9 DNS. This effectively bypasses the DNS server configuration of your local machine. Hope that helps!
Hi Dallas. Great tutorial! I followed the steps exactly and it works like a charm. I also learned a great deal about pfsense in the process.
Having said that, I do notice that it blocks ads (very well) on connected networks. I have a few VLAN segments and the ads are blocked on all the ethernet connected ones, but not on the wifi connected ones. Is there a setting I missed?
Thanks again for a great tutorial
Thanks for the feedback! Since it is working on some VLANs and not others, I’m guessing there is an issue with the DNSBL config or possibly a client-related issue. Here are the various items I would check. 1) Under the DNSBL tab, go down to the ‘permit firewall rules’ and ensure all of your VLANs are selected and ‘enable’ is checked. 2) If that is ok, then go to your DHCP server config for that particular VLAN. Does it have something in the DNS servers? If so, I would remove it. 3) If those are both ok (or your not using DHCP), double-check your clients. If you go to ‘ipconfig /all’ [assuming you are using Windows], do you have the firewall listed as the one and only DNS server? Hope that helps!
Also, when I look at the services status, pfb_dnsbl service is disabled. I tried restarting it, but it won’t: it spins for a while then goes back to disabled status. Could this be the problem? If it’s disabled, how are ads being blocked at all?
Here’s a screenshot:
https://imgur.com/a/j3ac5gX
Yes, the pfb_dnsbl service must be running. Try backing out some of your settings changes until it starts back up. I’m guessing some of the hostnames are cached locally. Also, try using nslookup on those separate VLANs to see if it works correctly. Good luck!
It’s fixed now.
I had to do a couple of things:
1) Went to package manager and forced a reinstall of the pfBlockerNG devel package. This finally started the service. However nothing was being blocked.
2) I disabled DNS Forwarder and enabled DNS Resolver. For the Listen Port, I put in 53. Enabled DNS Query Forwarding (I use OpenDNS). Selected all the Network Interfaces.
And Voila! It works fully now! It works on all the Vlans and all devices connected to the networks, including Android devices.
I don’t know if others have similar issues, but would it help to have this info in the tutorial?
Thanks for all your help!
Thanks for the update! I’ll add some verbiage in about re-installing the package if the service doesn’t start as well as disabling the forwarder. FWIW, I’m guessing the forwarder was your issue. At any rate, I’m happy to hear it’s working!
Hi Dallas.
Thanks for getting back to me!
I’m using Windows 10 (I should have mentioned that). 1 & 2 are good. I think the problem may be at point 3. When I checked the DNS for the wireless devices, you were right, they are using the DHCP server/Gateway for that VLAN (192.168.2.1), whereas the network where pfblockerng is working is on 192.168.1.1 (I’m not worried about any of the other VLANS; those are for servers and IoT devices). Is it possible to have devices on 192.168.2.x use a DNS on 192.168.1.x? I’m poking around and I don’t see how to set that up, if it’s possible.
I really appreciate this guide. However, I’m encountering difficulty since I have pfsense configured to use OpenDNS. Do you know if there is a way to use both pfblocker and OpenDNS?
Hey Bill! pfBlockerNG should work fine with OpenDNS. I personally recommend Quad9 for upstream DNS, but Cloudflare, OpenDNS, or anyone should work all the same. If you have issues with unbound (DNS resolver), it is typically in the custom options so I would check there first. Feel free to go through this guide in the link and substitute the IP addresses for OpenDNS. https://linuxincluded.com/configuring-quad9-on-pfsense/
Running into a problem. When I get to this part of the instructions:
“Click on the ‘DNSBL Feeds’ tab and you are taken back to the DNSBL feeds summary. Assuming everything went as planned, you should see the ADs and Malicious entries in the summary list.”
I can never get anything to then show under the “DNSBL Feeds Summary.” It just says “No Alias/Groups are defined. Click Add to define a new Alias/Group.”
I add them on the Feeds tab as instructed.
Interesting. I don’t think I’ve ever seen that issue. If you followed the instructions correctly, I would try reinstalling the pfBlockerNG package. If reinstalling it doesn’t work, try removing it after unchecking ‘keep data’ on the initial pfBlockerNG page. If neither of those work, let me know and I can look into some other possibilities. The author also just recently started a pfBlockerNG subreddit that you can ask questions on too. Good luck!
Had this issue too, thought first it was because the DNS resolver wasn’t running.. but after enabling it, and adding an empty feed manually, the previously “ADs” feed appeared.
I’ve added a comment/note to the post about adding an empty feed if pfb_dnsbl won’t start or if the feeds appear empty. Thanks for the feedback!
Some useful tips for folks with Active Directory / DNS in their environment/homelab:
– add a forwarder to your AD DNS to your pfsense box, set the timeout to lowest (1 sec), default is 3 sec
– on the pfsense box, under DNS resolver, scroll to the bottom and add domain overrides, and add your domain name and the AD DNS, so if the pfsense box needs to query stuff on your domain the querys dont go out to the world
Great tips and thanks for sharing! I already planned to create a post about using this in an AD environment so I appreciate the input/recommendations!
– add a forwarder to your AD DNS to your pfsense box, set the timeout to lowest (1 sec), default is 3 sec?
where is that? and the timeout?, do you can give me some steps or guide or how to from zero?, i want this for my homelab and this caa affecte windows active directory?, pfsense do the DHCP? thanks in advance
First and foremost, I would let the AD server(s) handle both DNS and DHCP. This also assumes DNS services and DNSBL are working properly on pfSense via command line tests… Next, you can modify the respective AD server(s) via the Windows DNS app. Simply right-click on the server name, go to properties, and then go to the forwarders tab. Remove all other servers and leave just the pfSense IP address in the list. This is also where you would add the timeout of 1 second as NeXsGen specified previously. On the same screen, I also remove the checkmark in the “use root hints” because I want to ensure traffic goes through pfSense/pfBlockerNG. Keep in mind that this leaves pfSense as a single point of fail, although I monitor pfSense systems quite extensively so I would know if unbound is down within a few minutes. Regardless, that is something to think about.
Please add a warning in the Install section outlining that you must update Pfsense to the latest version before installing the pfBlockerNG-devel package . i.e. i was on pfsense 2.4.3 ( even though 2.4.4 was available ) and installed pfBlocker-devel. IT BROKE MY PFSENSE INSTALL BEYOND REPAIR. Not even trying to restore from backup config via ssh menu or pkg-static upgrade would work from shell. I had to completely re-install. This package replaces the version of PHP used in pfsense and will cause mayhem if it doesn’t match what already installed.
–
Seb, thanks for the feedback. The new version of pfBlockerNG requires PHP 7.2 and for some reason pfSense doesn’t complain when you install a package requiring it. IMO, the upgrade to PHP 5.6 to 7.2 wasn’t handled quite right by the pfSense devs. Nonetheless, I’ll add a warning to hopefully prevent others from having the same issue. I know you said it was fixed after a complete re-install, but here is a Reddit post describing your issue (and the fix) just in case you come across it again. https://www.reddit.com/r/PFSENSE/comments/9il2u0/pfblockerng_install_broke_my_installation/
Wow, what a complete guide. Good job mate. I was switch to Sophos because I need this features. I didn’t know pfSense has such a great package like this, Thank you sir 😉
Thanks for the feedback! pfBlockerNG is a fantastic package. When combined with a few other packages and extensive monitoring, I tell people you can make it stand on its head if you want. 😉
Amazing guide. Very thorough, yet so easy to follow. Got everything up and running in no time. Thank you for putting this together.
One question, is it possible to whitelist domains/sites only for a specific device? I use DHCP Static Mapping for most of the devices in our home. Would like to be able to whitelist sites for specific devices.
Thanks for the feedback! One option is that you could assign a static DNS on that device, e.g. instead of getting the DNS from DHCP, just assign the DNS statically. Though it might be a little odd, your IP address itself *can* still be handed out via DHCP. If it is a device you don’t have access to the settings, then the answer is ‘no’ if they are on the same network/VLAN. That said, I accomplish what you are trying to do by adding multiple VLANs. This setup has the added benefit of improving security by separating items that should never talk to one another, items they are not as secure, etc. Hope that helps and best of luck!
Thanks for the guide mate! Appreciate it and was awesome. I did it on a newer version. The floating rules has been moved into the IP tab under -> IP Interface/Rules Configuration -> Floating Rules.
Thanks for the feedback! I will update the guide to reflect the latest changes soon!
Thank you for this extremely helpful walkthrough. I got everything working without too much fuss.
I am however, having some issues similar to those above where whitelisting is not working.
When I ping github.com or http://www.google.com on two of my computers on LAN, I get 10.100.100.1 black hole rather than the website.
Those are added to my whitelist which includes:
.github.com
.githubusercontent.com
github.map.fastly.net # CNAME for (raw.githubusercontent.com)
.sourceforge.net
google.com
.google.com
http://www.google.com
I tried flushing dns and rebooting, no change.
Anything else to try before finding & disabling the lists blocking those sites?
Thanks for the feedback! The only times I’ve had issues with the DNSBL whitelist is when I either a) didn’t get an alternative name added or b) I didn’t flush one of the caches properly. In a standard setup, you have DNS cache on your firewall via unbound (restart to flush) and then DNS cache in your browser and OS. The browser cache can be corrected with the shift + refresh or clearing your respective browser’s cache. The Windows OS can be flushed via ipconfig /flushdns. I’m a bit aggressive on my feeds selection so I used to see this a fair amount where a legit site gets added to a list. If you end up finding another issue, let me know and I’ll add it to the guide! Thanks!
Hi Dallas,
Amazing guide as always. I learned a lot about PfSense thanks to you. Thank you a lot for that.
I, however, have a question. You wrote:
“If ads are not getting blocked and the ping commands above don’t return the virtual IP address, it’s also possible your local machine is not using pfSense for its DNS settings.”
Does it mean I have on my PfSense DHCP server settings put my PfSense’s IP Adress in the first DNS field? I, currently, am using the following:
A. 1.0.0.1
B. 1.1.1.1
C. 8.8.4.4
D. 8.8.8.8
I thank you in advance for your possible answer. I wish you to have the best year ever with plenty of success, health, luck, peace, and happiness.
Happy new year,
Syed
Thanks for the feedback!
Yes. You can leave the DHCP servers (Services -> DHCP Server) blank and pfSense will hand out its own IP address to the clients. If you use static IP addresses on your clients, you will obviously need to change the DNS server on them manually.
Keep in mind that pfSense must be the *one* and *only* DNS server in order for DNSBL to work correctly. If, for example, you add Cloudflare as a secondary DNS in the pfSense DHCP settings (or on your client), you will have “unreliable” results. Basically, the DNS queries will constantly bounce back and forth between the two DNS servers so one time your ads will be blocked and the next time they won’t.
I wish you a safe and happy new year as well! Good luck!
Hi,
Thanks for the answer!
I just removed all the DNS entries in my DHCP server as per your recommendation. However, I have a question. If all my home users, clients, are making a DNS query, then they will ask my PfSense directly. However, I am wondering what DNS server my PfSense will use in a Dual Wan setup. Can I change that setting as I like CloudFare ones ? If yes, how?
Furthermore, I loved that guide and the one on PbBlockerNG. They just made my internet life better and my home users” ones as well. Do you have any other one(s) to recommend me? I use PfSense as my central router with firewall on a HP EliteDesk with dual WAN by using Vlans.
Thanks,
Dani
Your pfSense will use the ones found in System -> General Setup. When everything is configured properly with your DNS and resolver/unbound, the ‘System Information’ widget on the main screen (after you login) should read 127.0.0.1, 1.1.1.1, and 1.0.0.1. I personally use Quad9, but I like Cloudflare a lot as well. You can use the guide below and just substitute the Cloudflare DNS rather easily. The dual WAN configuration won’t have any effect on DNS as long as you deselect ‘Allow DNS server list to be overriden by DHCP/PPP on WAN.’
https://linuxincluded.com/configuring-quad9-on-pfsense/
Up to this point, you’ve used the DNSBL portion of pfBlockerNG. I also highly suggest using the IP blocklist portion of pfBlockerNG. It’s much more involved than I’m describing it here, but think of it as blocking known bad IPs instead of blocking by domain names.
https://linuxincluded.com/using-pfblockerng-on-pfsense/
Holler if you need anything else!
Greetings Dallas,
Thanks a lot for your answers. You helped me so much to learn PfSense and all these stuff. You made the difficult things easy. I already read that one as well. 🙂 Have a nice day!
I will keep following your blog and learn more stuffs.
Kind regards,
D.
I love to hear that! I hope to have a little more time to write this year! Take care!
You might consider declaring what version you’re discussing. You should also direct folks to the package manager to determine their version. The rational for this suggestion is that you’ve got:
“Another way to check is if you have ‘Alerts’ instead of ‘Reports’ along the top row of pfBlockerNG options… That too means you are still on the old version.”
This won’t age well. And, it’s murky enough to leave me scratching my head as to whether I should be following you “old” or “new” posting. Your post is dated 9/2018. I have the 11/2018 v2.1.4_14 pfBlockerNG on pfSense 2.4.4-RELEASE. But that “‘Alerts’ instead of ‘Reports'” would mean I’ve somehow got the old version. Which brings me back to the suggestion that you specify what version you consider old vice new.
I appreciate the feedback! Nothing about this ages well with the number of updates the pfBlockerNG package receives! 😉 I actually update the guide on a regular basis and re-read/re-write it when major changes occur so it is never too far out-of-date. The biggest change in the last few months is the initial wizard. Additional changes are coming to the package fairly soon and I’ll re-write for the wizard (and more) when that occurs.
I just looked back over the guide… The reference you cited is well into the guide and it was actually the second point made about versions. The first was in the second paragraph, “Please note this walkthrough is for the new devel version of pfBlockerNG.” Thus, I’m guessing you are using the non-devel version of pfBlockerNG. Simply go to the pfSense package manager, uninstall the one you have installed and reinstall the one with ‘-devel’ instead. Good luck!
I just installed pfSense, and was looking for a comprehensive guide for pfBlockerNG. I ended up reading your guide and all I can say is “THANK YOU SO MUCH” !
Your guide helped me understand the logic of DNSBL and brought me up to speed in less than 30 minutes !
Unless I’ve missed it, I’d also appreciate a guide to block a single country.
So far I’ve been to pfBlockerNG/ IP / IPv4 and added an “IPv4 source definition” using one of the GeoIP’s file (namely /usr/local/share/GeoIP/cc/RU_v4.txt), and set it to “Deny Inbound”. I hope this makes sense…
Thanks so much for the feedback!
I believe a GeoIP guide is in my list of future items, but if not, I’ll add it. In the meantime, simply go to IP -> GeoIP, and then click edit next to the “continent” name. From there, select your country (or multiple countries using ctrl + left-click), change list action to deny inbound or deny both, and then click save. Last but not least, go to Update and click on Run. Once the update process completes, you can go to firewalls rules and you’ll see a new rule on the corresponding interface(s) you chose. Good luck!
Wow, thanks for the swift reply !
I’m going to try it right away !
Hello Sir Dallas:
I installed the new package of pfblockerNG(new version) unfortunately my DNSBL is not working it say (disabled). Please let me know where I got wrong with it. Thank you
Are you on the latest version of pfSense? If not, then I would make sure you are 100% up-to-date. If so, I would try to reinstall the pfBlockerNG package from the package installer menu. Last, go to ‘Update’ from within the package and see if there are any glaring errors. You can also check the logs and especially the pfblockerng.log via the logs menu to see if there are any errors there. Thanks for stopping by!
Hi, thanks for the easy to understand guide.
I followed you up to the point you did easylist.
I don’t have the Configuring DNSBL EasyList option.
It would be great if you can help me with this.
Thanks for stopping by Fred! In the most recent point release, the EasyList was moved to the same location as all other DNSBL feeds so it is no longer a separate tab. Holler back if you need any additional help!
Hi Dallas, great guide (as usual). I have deleted pfBlockerNG and installed pfBlockerNG- net developer 2.2.5_21 on pFsense 2.4.4-RELEASE-p2.
When I select DNSBL I only see DNSBL Feeds and DNSBL Category, no Easylist.
My EasyList is still under DNSBL Feeds. I have tried reboots and reinstalls but no luck. Do you have a clue?
Keep up the good work!
Update: Pls ignore my previous post, I found the latest message.
Glad you found it! I’ll get the guide updated soon too.
does this run on top of DNS Rersolver? Or are we supposed to stop that ?
I enabled it, updated, following your guide, but when I go ping the sites it still pings to public IP and not the fake IP we used.
MY DNS resolver is enabled.
or dns forwarder?
Are they both supposed to be disabled?
nslookup from network clients using my pfsense vm with blockerng installed cannot resolve.
You cannot have the DNS forwarder and DNS resolver running concurrently. In addition, the DNS resolver (unbound) is a must for DNSBL. In fact, double-check your ‘custom options’ and make sure you have an include directive related to pfb_dnsbl — “server:include: /var/unbound/pfb_dnsbl.*conf” in the options as enabling DNSBL should add it automatically for you. Nslookup is great for testing as long as you ensure you are using pfSense as your server. Take special note of the address after you first type the command on a line by itself. You can switch to the correct IP by typing “server” followed by the IP address from the nslookup prompt.
hi dallas followed your tutorials but the ads in youtube and yahoo still shows up, i enabled pfblocker and dnsbl and set it up to defaults and put some feeds like easylists and several ads block, when using nslookup google.com the ip of pfsense shows up and not 10.10.10.1. i have dnsresolver to defaults and not dns ip are put on general setup and dhcp dns servers. but in the dashboard i saw that the dnsbl has blocked some on the list and i can browse the 10.10.10.1 ip or ping it. is it a bug like what i read on pfsense forum?
YouTube is one of the “funny” ads, i.e. ads originate from the same DNS names as content. Thus, they can’t be blocked via any means of DNS blackholes/sinkholes, e.g. DNSBL, pi-hole, etc. Interestingly enough, uBlock Origin (mentioned at the bottom of the post) *does* block YouTube ads. I highly recommend using uBlock Origin not simply for YouTube ads, but to provide another layer of protection.
Google.com will *not* get blocked… That would be bad. 😉 Instead, look at some of the common ad networks DNS names and check those. If you type ‘nslookup analytics.yahoo.com’ you should see 10.10.10.1 returned. If not, go through the standard troubleshooting steps. For example, ensure your checking your pfSense (and not some other IP), verify your devices are using pfSense as the one and only DNS, verify DNS resolver starts without issue, verify DNSBL is started, verify the pfb_dnsbl.conf related line is present in the custom options for DNS resolver, etc. If you are seeing items show up in the DNSBL list (and not the IP list), then I would say there is an issue with which DNS server your devices are querying against and not anything with pfBlockerNG.
You mentioned your general setup and DHCP DNS options. I do have a guide for the general setup and no, you don’t need to setup Quad9 if you prefer something like Cloudflare. This guide also shows the ‘custom options’ I discussed above with the pfb_dnsbl option. Your DHCP/DNS servers do not need any values. If DNS resolver is enabled, it will automatically use the firewall IP address if you leave your DHCP DNS options blank.
https://linuxincluded.com/configuring-quad9-on-pfsense/
Hope that helps!
hi thanks again for the tips and reply, when i do nslookup to analytics.yahoo.com all four dns queries didnt respond maybe some issue in my fwrules as i only allow http,https,8080,dns,and realtime streaming ports maybe i need to put another protocol or ports, but if i where to ping the analytics.yahoo.com it does respond but coming from 10.10.10.1, anyways i will read what you link it through, btw all my limiters rules are in lan, i wonder why some other tutorials and pfsense forums they do the firewall rules on floating rule any thoughts? i read somewhere that the lan rules are for pc clients needing services out in the internet and wan rules are for the internet to view the services in the lan like web server?
You would want to allow DNS (UDP 53) to your firewall and then reject/block all DNS traffic outbound. This will help with ensuring your network clients talk to the pfSense DNS (there are a number of ways to bypass it). It’s odd that the Yahoo address returns 10.10.10.1 because that is expected behavior if everything (including the firewall rules) are working. Are you sure your default DNS is set to the firewall? You can verify this from the command line by typing ‘ipconfig /all’ and looking for your current adapter. Your DNS should be the standard pfSense internal IP and *nothing* else. For example, if you access your firewall by going to 192.168.1.1, then that should be your one and only DNS server listed.
Limiter rules depend on what you are trying to do. If you want to limit any internal client to 2MB, then you need to make sure that is on the LAN because your WAN wouldn’t be able to differentiate the internal client IP addresses. Floating rules simply apply to multiple interfaces. FWIW, I tend to avoid floating rules unless absolutely necessary.
ahh so thanks for the clarification on fw rules, oddly itworks now, i visit some websites that says wherever you have adblock installed in the system, and now thats confirm wherever i have pfblocker or not and youtube has that funny settings that rejected my setup thanks again sir
If you changed your DHCP from something previously, it would take some time for your systems to get the “new” DHCP config and start using pfSense. There’s also the possibility you had some of the domains in cache so they took a little bit to clear out. Happy to hear it is working now and happy browsing. 😉
Can these instruction be used for the latest 2.1.4_16 version. I just installed it today? Thanks!
Yes! The biggest difference is that the wizard installs the PRI1 on the IP block side and a handful of groups for the DNSBL/ad side. In addition, the easylist is now in the feeds like everything else. I highly recommend the walkthrough because it helps you understand a lot of the methodology and testing.
Thanks again! I’m also going through the IP Blocking instructions from your earlier contribution.
A few questions…albeit mushed together so sry about that: I plan to run Snort for IDS along with pfBlocker. I’m looking for a balance in configurations because I know there are IP lists and rules that can be downloaded for both. I plan to subscribe to a paid list in Snort…unless I’m missing something that renders paying unnecessary. Is it best to use Snort on my inside LAN networks to monitor LAN intrusions & outbound rules, and use pfBlocker to run on WAN for Inbound traffic filtering only? I’m trying to avoid redundancy and wasting system resources since my pfSense build is running 24/7. I have 8GB RAM on a dual core 2.5 Ghz Celeron.
I typically run a *lot* less Snort rules as a result of pfBlockerNG. Keep in mind that Snort and pfBlockerNG run simultaneously so an IP blocked by pfBlockerNG may also trigger a Snort event. This is especially the case if you try doubling up the block lists from ET, ISC, etc. If you are just playing around, I would try the free feed first, read about the differences between free and paid, and then make your determination. What interface you run Snort depends on your environment. A lot of times I will run Snort in IPS mode (with limited rules) on the WAN and then Snort in IDS mode on the LAN. It probably goes without saying, but IDS mode does nothing if you are not watching and investigating the alerts Snort generates. It really depends on your traffic on how useful either will be. You may find an overwhelming majority of your traffic is encrypted so Snort isn’t as necessary… And no, that doesn’t mean Snort can’t help with encrypted traffic, but it extremely limited when compared to un-encrypted traffic. Hope that helps!
For the DNSBL procedures here, I notice it’s recomm to enable DNSBL Firewall Rule if there is more than 1 LAN net. I have 2 interfaces. 1 is for my wired trusted devices (lab), and the other is for a wireless access point used by guests/laptops, and other crap devices that access to the internet. These 2 networks are subnetted separately without any crossover traffic between. So is it safe to select both and enable this floating Firewall Rule for DNSBL. I don’t want to inadvertently allow my crap-network access to my trusted LAN.
The description wording on the check-box leads me to ask because it sounds like I might be allowing the crossover?
“This will create a ‘Floating’ Firewall rule to allow traffic from the Selected Interface(s) below to access the DNSBL VIP on the LAN interface. This is only required for multiple LAN Segments.”
The DNSBL virtual IP (VIP) must be accessible to the other VLANs. This will not cause crosstalk between your networks. The easiest way to test this is once you have these in place, try pinging/accessing things from other networks. If DNSBL works and the previous test fails, you should be good. 😉
Excellent. Thanks again!
Thank you. Absolutely awesome write up.
Thank you Keith! I appreciate the feedback!
found out the hard way to get github.com whitelisted being blocked by DNSBL_Malicious2.H3X_1M.
At first I derailed towards HSTS issue’s with the UI webconfigurator https://github.com/pfsense/pfsense/pull/3856
Yeah, there are several domains that are “gotchas” for IT folks. Thus, the whitelist recommendations. BBCan has now added this exact whitelist into the wizard on the latest dev build of pfBlockerNG too so hopefully that keeps future headaches to a minimum. 😉
Very good guide. Is there any way to exclude selected hosts from DNSBL? or use selected DNSBL lists on selected clients. I want to have separate DNSBL for kids and adult computers.
No. Depending on how in-depth you want to get, I would recommend separate vlans and maybe running Pi or something on a secondary network. You could also do something with squid to proxy traffic for certain systems only. Best of luck and I’d love to hear what you come up with!
Best article I’ve found on pfBlockerNG. Thanks!
Thanks for the feedback!
Thank you very much for this great guide. It helped me to set it up in between an hour.
I don’t have the option DNSBL EasyList under DNSBL. I am running the latest version of pfSense 2.4.4 p3.
Do you have any idea why?
Update: I found the answer in the comments. Thx. Great guide!!!
Beautiful
Thanks for what u are doing here with those tutorials. Keep up the good work and thanks once more. Cheers from Germany! 😀
My apologies for the late reply, but thank you so much for the feedback!
Best doc I found for pfBlockerNG-devel! Got the basics up and running within an hour – thx a lot for this excellent work.
The only thing I struggle(d) with is the carp setting for DNSBL, which is still marked as beta. Activating carp instead of virtual IP seems to lead to some confusion between the cluster nodes – they both believe they are the master for this carp if. When using virtual IP again, one has to reboot both nodes to get things in order again.
Thanks for the feedback and tip Michael!
This is a great write up and seems to reflect a great deal of experience and familiarity with the tool, however, some of the steps appear out of date now. Is there any chance you will be updating this guide and keeping it current with the tool?
Hey Sean! The guide has been updated to reflect recent changes. Thanks for the feedback!
@Dallas,
Thank you so much for this write up. I’ve already read it twice, and I will print it out later when I go home, and highlight the key points for me.
I just installed pfSense, and am fairly new to it. I only have Suricata installed, and I will install this later on this week.
If you have any tips other than this great guide, this ‘Ole Marine is all ears.
-Semper Fi
gyrene2083
Excellent and thanks for the feedback! Also, thank you for your service! You’ll likely find you can pare down your IDS/IPS rules due to overlap with some pfBlockerNG feeds. If you haven’t already done so, check out the guide on getting Quad9 configured on pfSense. The combination of those items plus Suricata should go a long way! Take care!
@Dallas,
Thank you for your kind words, patience, and excellent tutorial. This Marine, is happy to know there are people like you willing to help, ‘ole Veterans like me. I will be reading, your Quad9 article thoroughly.
-Semper Fi
gyrene2083
Happy to help!
I am new to pfsense …. Coming from Check Point. I just installed pfblockerNG… stumble upon your post. Thank you very much for taking the time to write up this guide, much appreciated and very helpful!
Thanks for the feedback Dominic! Happy to help!
@Dallas,
Just wanted to thank you so much for this great guide and for pointing me to add Quad9. It all went over without a hitch.
I did have one thing to figure out. I run an UnRaid Server, and I have Sonarr, Radarr and Emby dockers. For some reason Sonarr and Radarr wouldn’t work, so I stopped the pfblockerng services and it still wouldn’t work. I rebooted the dockers and nothing. Then I looked in Radarrs log file, (which I clearly should have read thoroughly, because it was only one line, the first time.) It said something along the lines that the api was being blocked check my adblocker….I had forgotten that I added uBlock, and once I disabled it for Sonarr and Radarr, it worked. So I reenabled the services restarted the dockers and it still worked so I’m thinking, I am ok now. Any thoughts, on how I could have handled that differently, any more tips for an ‘Ole Marine. lol
Thanks again brother, and if I don’t hear from you, have a Merry Christmas.
-Semper Fi
gyreme2083
I don’t use any of those services, but you troubleshot it as expected. You’ll find that after you work through the initial issues of whitelisting, you don’t end up troubleshooting near as much in the future. If I sense pfBlockerNG might be blocking a site, I would look under Reports -> DNSBL. Find your source IP, the domain, and then you can ‘unlock’ the domain temporarily or hit the ‘+’ to add it to the whitelist permanently. Keep in mind that after unlocking or whitelisting, you may need to clear your local OS and/or browser cache too in order for the change to work. You have a Merry Christmas as well!
Thanks for the wonderful writeup. Is there a way to use this to block ads in the Apple News app (on Mac, iPhone, iPad) running on the network?
Thanks for the feedback! I’m not familiar with the Apple News app, however, you should be able to determine what DNS names are getting queried when ads are served. That is also assuming the ads are not served from the same DNS name as content. Given the extensiveness of blocklists used for the tutorial, I’m going to assume the latter is true.
@Dallas,
Merry Christmas Brother! I hope you and your family have an awesome day.
Quick question for you, I am having an issue with BBC and here is the message I am getting:
[ pfB_PRI1_v4 – BBC_C2_v4 ] Download FAIL
I have checked the website and was able to see the text file. Then I checked alerts in pfblocker and Suricata, and to be honest I couldn’t find anything. (Then again, I am not sure exactly what I am looking for.)
Anyway, you enjoy yourself today and your family, and thanks for everything.
-Semper Fi
gyrene2083
Thanks! An occasional error when downloading feeds isn’t too uncommon. I’ve seen a few instances where one feeds block another, but it’s not the norm. More often than not, a feed may be down temporarily. Various feeds also have rate limiting so if you try to download the feed multiple times in a given period, it will stop you from doing so. The good news is that in either of those cases, pfBlockerNG will continue to use the in-place feed until it is accessible again. Best of luck!
Hi Dallas,
Many thanks for your excellent walkthrough, and for freely sharing your knowledge. I was using pfblockerNG within pfSense, and alongside Suricata; but the benefits seemed very limited, and I wasn’t confident I had it configured properly. Certainly, I wouldn’t have expected the ‘devel’ version was for me.
However, I did a clean install of pfblockerNG-devel, and followed your guide to the letter. It works like a charm, and I am really pleased with the results. Lots of free memory, so no constraints.
Thanks again for giving your time to help others.
Best wishes
IC
Fantastic IC! Thanks so much for the feedback!
Thanks a bunch for sharing this with all people you really know what
you’re talking about! Bookmarked.
Thanks so much for the kind words! Much appreciated!
Many thanks for this guide Dallas, simple to follow and informative! It’s not often you find guides as good as this one and I really appreciate the knowledge sharing.
That’s wonderful to hear! Thanks for the feedback Gerald!
@Dallas
I would like to go a step further in safety and block all outbound traffic except for a few whitelisted domains (eg. updates.microsoft.com). Unfortunately, as far as I unterstood, pfBlockerNG is not able to do that, there is no “block all except whitelisted” option. Is that true or do you see a way I could achieve that?
Hey Roland! pfBlockerNG really isn’t well-suited for that. The easiest, yet not 100% complete way would be to use aliases and then create an outbound LAN rule for 443 (or other ports). I’ve utilized this in the past and it works, but expect it to act a little squirrelly at times. The preferred method for this would be to use Squid and SquidGuard. There is a bit more upfront config, but it works extremely well especially for whitelisting. Best of luck!
HI Dallas,
First thank you for such a detailed and in depth guide. I used it to setup my pfSense/pfBlockerNG-Devel with great success and I honestly could not have done it without this guide, excellent work.
It looks like hpHosts & hosts-file.net are no longer usable. They now redirect to Malwarebytes. The statement on the Malwarebytes forum is “We are currently restructuring hpHosts and it will no longer be updated. You could however check out our free browser extension product.”
If you run out of things to spend your time on I vote you update the this post to reflect these new changes. I am sure many, including myself, would like to have your advice on what other lists to use to replace hpHosts and hosts-file.net
Thanks again for all your hard work.
Hey Henry! Thanks for the feedback! I saw hpHosts/hosts-file.net started erring out and yes, apparently it is gone for good. Thanks Malwarebytes! My gut feeling is although it is widely used, most, if not all of the domains are found in other blocklists. I’ll do some testing and get the guide updated to reflect this. If you play around with some of the options and come up with something better, please let me know too. Thanks!
Excellent article! I have been using pfSence for several years now for my home router. I appreciate the in depth detail of this article. It was very helpful adjusting my configurations. I already had pfblockerng installed but some of my configurations needed to be adjusted. Thank you for sharing this!!
Thanks for the feedback Joe! Much appreciated!
How to block l2tp vpn traffic ? Need your kind help
Hey Ace! I’m assuming you are referring to L2TP outbound VPNs? I don’t use L2TP, however, I believe you would need a rule(s) to block destination ports UDP/500, UDP/1701, and UDP/4500 for the LAN going outbound. Keep in mind this would likely block client-based IPSEC tunnels as well. Best of luck!
Hi Dallas,
Great article!!! Thanks for taking the time to create it. I downloaded the pfBlockerNG-devel 2.2.5_32. There are a few differences from your instruction set but I believe I’ve been able to figure it out. Interestingly, the DNSBL_hpHosts will not download. As an example, I receive the following message:
[ DNSBL_hpHosts – hpHosts_EMD ] Download FAIL [ 05/20/20 12:22:33 ]
Firewall and/or IDS (Legacy mode only) are not blocking download.
Any ideas why this may be occurring?
I know the guide has a couple of references that are out of date… I’ll get around to updating those. Unfortunately, hpHosts was removed by MalwareBytes and is no longer available. Hope that helps!
Now that HPHOSTS are no longer updating, are we allowed to use their lists for further updating on github, or anyone willing to maintain these lists.
I happened to have all the needed files with the latest updates. The most recent is from 04.March and some haven’t been updated since back in the middle of 2019.
Hey John! It’s unfortunate that MalwareBytes quit maintaining them when they were so widely used. However, in my own experience, the various lists have a *lot* of overlap so I don’t think we’ve skipped a beat per se. Please let me know if you see anything different in your own testing. Take care and stay healthy!
Hi,
Great article!
Very informative, easy to follow, and kept updated too.
I used this page to initially setup adblocking on my pfsense box, and have been constantly tweaking it. I have since recommended this page to several of my friends as a great how-to reference, so just wanted to drop you an official “thank you!”
Thanks for the feedback Avi! Happy to hear it helped!
Hey Dallas,
As pfBlockerNG-devel development has updated, the whitelisting options have updated as well. As an example, now there is –
Whitelisting Options:
1) Wildcard whitelist [ .cdninstagram.com ]. This will immediately remove the blocked Domain/CNAMES from DNSBL. (CNAMES: Define the external DNS server in Alert settings.)
2) Add [ cdninstagram.com ] to the ‘TLD Exclusion customlist’. A Force Reload-DNSBL is Required! After a Reload any new blocked Domains can be Whitelisted at that time.
Which option would you consider being the safest to choose?
Thanks.
I need to go back through the guide and update it to account for the numerous improvements made. In particular, the recently released version 3 has quite a few updates. To answer your question, I rarely whitelist entire hostnames and I will instead try to limit it as much as possible. Let me know if that doesn’t make sense!