Pragmatic IT Solutions
In this video, I will show you how to migrate from pfSense Plus to pfSense CE when running pfSense on your own hardware.
In this video, I will show you how to set up network-wide ad-blocking and tracking protection using pfSense and pfBlocker-NG. Because it is nobody’s business what you are doing on the internet!
Check my previous blogs:
– Setup pfBlockerNG python mode with pfSense
– How to run pfSense Plus on VMware vSphere 8
Set up network-wide ad-blocking and tracking protection using pfSense and pfBlocker-NG Read More
pfSense is a very powerful firewall and in this video, I will show you how to run pfSense Plus on VMware vSphere 8. By virtualizing pfSense you can now take advantage of the flexibility already there when using VMware vSphere 8.
You can find more information about pfSense in the links below:
– https://docs.netgate.com/pfsense/en/latest/recipes/virtualize-esxi.html
– https://www.pfsense.org/download/
– https://www.netgate.com/pfsense-plus-software/software-types
– https://shop.netgate.com/products/pfsense-software-subscription
I have internet and IPTV from KPN (local ISP) here in the Netherlands. I chose not to use their router but instead use pfsense. The internet configuration part is pretty straight forward but if you want to get routed IPTV working you have to configure specific settings in pfSense. In this video I will show you how I have set this up for my situation. If you need to setup routed IPTV on pfSense as well, use this video as a guideline and check what the specific VLANs and IP ranges are for your ISP.
In this blog post I will show you how to setup pfBlockerNG python mode with pfsense. Nearly a year ago I made a blog post here explaining why I was moving away from pfBlockerNG to Pihole. The main reason was that pfBlockerNG could not show all the blocked DNS requests. This made is difficult to troubleshoot why some app or (iot) device was not working properly. Read my blog post here for all the details.
In recent times, the developer of pfBlockerNG, BBcan177, has given a major update to pfBlockerNG. With the integration of python into pfBlockerNG, it is now possible to show all DNS requests that are blocked. I’ve always loved the combination pfSense and pfBlockerNG. I think it is a great setup for protecting your network and keeping all the ads and tracking away, making a beter and safer internet for all the users on your network (kumbaya-mode). Plus you only need one device for routing and adblocking.
For this blogpost I used the following versions:
Before you start with configuring pfBlockerNG make sure you pfSense firewall runs fine and internet is working as expected for all the devices on your network. If this is the case then continue to make a backup of this running setup. Always a good idea of having a backup before making changes. To do this go to Backup -> Backup & Restore. Click on Download configuration as XML. Safe this file in a secure place.
Then go to System -> Package Manager -> Available Packages. Search for pfBlockerNG. This part is very important. You need to install pfBlockerNG-devel package. Click on the green Install button behind pfBlockerNG-devel to install the package. After installing the package it should be in the list of Installed Packages:
Before you start you should know that using the new python mode you to disable some setting in Unbound DNS Resolver (if you are using that in pfSense). Disable the following options in Unbound Resolver:
pfBlockerNG has some checks to make sure that the options above are disabled when you enable python mode but I would strongly advise to disable them before starting to configure pfBlockerNG.
After you’ve installed you will find it in the menu Firewall -> pfBlockerNG:
When you open pfBlockerNG for the first time you will be presented with a wizard. I just skip this because I like to setup pfBlockerNG manually with my own settings. Let me share them with you 🙂
I will share my production configuration with you so I will blur out some things. Let’s start by the following settings. In the General tab I enable the checkboxes for pfBlockerNG and Keep Settings. The Keep Settings option wil make sure that your pfBlockerNG configuration stays in place when upgrading or when you make a backup of your pfSense configuration. I also change the cron update settings here just be spread the cronjobs load. You can leave this default and everything will just run fine :).
pfBlockerNG can be used for IP blocking (malicious IP’s) and DNSBL (DNS sinkhole). Let’s go over my settings for IP blocking. Go go the IP tab. The first section is IP Configuration. I pretty much the basic settings here. I have a couple of IP blocklists configured and the De-Duplication option will make sure that there are not duplicate IP’s in blocklist pfBlockerNG builds. Make sure that The Placeholder IP Address is not being used in your network. The default of 127.1.7.7 should be fine. Here are my settings:
Next is MaxMind GeoIP configuration. You need to register and get a valid license key and you can register here for free. I use MaxMind GeoIP to block certain countries. If you don’t host services behind your pfSense router (like a webserver) you probably don’t need the country blocking because pfSense will default block all inbound connections. Security is all about layers and having this option is another layer of security. Here are my settings:
Next is Inbound Firewall Rules. These apply to any interface which is used to get internet traffic to you network. Here you select your WAN interface and if you have VPN client connections going to your VPN provider, select those here too. I use several VPN connections to VPN providers and those interface names end with _WAN. The screenshot below shows what I have selected here:
Next is Outbound Firewall Rules. These apply to any interface which you have on your local network. I have several local networks like a guest and a testlab network. I am running a OpenVPN server on pfSense and I treat that network also as a local network. Here are my settings:
I have also enabled the Floating Rules option because I like it that I can find all the firewall rules in one place. Kill States is enabled because IP blocklists are being updated several times a day and you want pfBlockerNG to kill any state to a blocked IP immediately. Don’t forget to hit the Save IP settings when you are done here.
To let pfBlockerNG do block malicious IP’s you need to enable some IP feeds. pfBlockerNG has a nice selection of IP feeds you can enable. Go the Feeds tab and I would suggest to enable the PRI1 feed category and start from there. Remember the more feeds you enable the greater the chances are that you will break the internet :). Proceed with care is my advise and only enable extra feeds after doing extensive testing. This is not a set-and-forget approach! It’s more a set-test-monitor-forget approach. Check my screenshots here:
You just have to click on the + sign and pfBlockerNG will add the feeds. Afterwards you can find the feeds you enabled in the IP -> IPv4 tab. I have done some renaming and sorting here. Feel free to make you own adjustments. Here’s what mine looks like:
I have changed the Action here to Deny Both. This means that pfBlockerNG will block incoming and outgoing communication to a blacklisted IP. I have also changes the Frequency of the updates. The PRI1 category is being updated a few time a day and I want pfBlockerNG to be on top of the changes so I chose to update it every 4 hours. The others in my list have more to do with Public DNS servers and DNS-over-HTTP. In my experience they don’t change that much. I want to make clear that I don’t use IPv6 in my network. I have disabled it in pfSense and blocking it in my Firewall rules. I don’t have need for it at the moment and I don’t want to complicate things.
While you are in the IP -> IPv4 tab click on the GeoIP tab if you want to block specific geographical regions or separate countries.
You will notice here that I have the Action set to Deny Both. But I don’t block the whole region. I block specific countries and you can choose which countries you want to block by clicking on the pencil icon. Then select the countries and enable List Action and Loggin and click on Save:
This is all I configure for the IP blocking part. Let’s move to the DNSBL part.
I assume you know what DNSBL blocking does so I won’t go into the details here. It blocks malicious and/or unwanted adverts domains. Recently pfBlockgerNG got a huge update and DNSBL is now able to use python mode. This new python mode makes it a lot faster and also shows all the DNS requests which are being blocked! This is for me the major option to get rid of my Pihole setup. I want my pfSense doing everything in my network and adding pfBlockerNG to my setup gives me a single place to secure my network and keep ads and malicious traffic out of the door. You can read about all the changes in pfBlockerNG here.
To configure the DNSBL settings click on the DNSBL tab. There are a lot of options here and this can be overwhelming. These are the settings I have enabled or configured and I think this should give you a good starting point:
Scroll down to the DNSBL Configuration section and check Permit Firewall Rules. This will create rules in the Floating in your Firewall. I like having these in one place :). Also select all your internal networks here. This will enable pfBlockerNG for those networks. Here are my settings:
As you can see I have several internal networks (LAN, Guest, DMZ, TestLAB) and I want pfBlockerNG to be enabled on all of those.
Click on the + sign to open this section. In this section you can add domains you don’t want to be blocked. Like when you have a lot of Apple devices in your network you want to whitelist *.apple.com. Adding domains in the whitelist makes sure that even when these domains are in some DNSBL feed you have enabled (see next section) they will still be allowed (whitelisted) by pfBlockerNG. You can add domains here manually or using the Reports tab in pfBlockerNG. In the screenshot below you can see some examples I have added to my whitelist:
Those are all the options I have set for DNSBL. Scroll to the bottom of this page and click on Save DNSBL settings.
Now go to the Feeds tab and scroll down to the DNSBL category. Here you can enable different DNSBL feeds by clicking on the + sign:
I want to remind you again that the more feeds you enable the bigger the chance is that you will break the internet for users on your network :). And you will find that for some services to work you will have to whitelist certain domain names like I explained above. The feeds you enable are listed in the DNSBL Groups section. Here you can review what feeds you have enabled or disable and remove an enabled feed.
By clicking on the pencil icon and the end of the line you can edit those specific groups. The names of the groups will be different for you, I just renamed them for my ease:
This version of pfBlockerNG also has a very extensive list with known public DNS servers who are supporting DNS over HTTPS. DNS over HTTPS is a serious privacy and security risk so you want to enable this because you don’t want devices in your network using these DNS servers and bypassing pfBlockerNG’s adblocking and pfSense’s DNS server. Go to the DNSBL SafeSearch and enable DoH/DoT Blocking. Then select all the DNS servers from the list you want to block and click on Safe:
The Reports tab is very important. It will give you an overview of what IP’s or DNS names are blocked by pfBlockerNG. It will also tell you the source device of the DNS or IP request thus making troubleshooting easy. Here you can investigate if pfBlockerNG is the reason why a certain app or website is not working properly for devices on you network:
The fist section shows you the IP’s being blocked and the section below that will show you DNS requests being blocked. Whitelisting an IP or DNS is simply a matter of clicking on the + sign before the DNS name or IP:
When you click on the + sign pfBlockerNG will ask you if you know for sure you want to whitelist this domain. Click OK:
Then it will ask you if you want to whitelist this domain only or add a wildcard for the domain:
After that you will have the option to add a description. If you don’t want a description just click on No and that’s it. The pfBlockerNG will no longer block that domain:
If you want to review the domains you have whitelisted you can just review them in the DNSBL Whitelist section in the DNSBL tab like I explained above.
With the settings and configuration options explained in this blog you should be off to a great start in keeping those ads, trackers and malicious websites out of your network. This is all you need to setup pfBlockerNG python mode with pfSense. pfBlockerNG has a lot more options but I don’t think you need all the features to be safe. The Reports tab is your friend for troubleshooting. Thank you for reading and good luck!
Setup pfBlockerNG python mode with pfSense Read More
For a while now I have pfSense firewall running at home. I really love the performance, stability and security pfSense provides. It is just rock-solid! But let me tell you why I moved from pfBlockerNG to Pi-Hole. What I also love in pfSense is the ability to install packages and add even more useful features to the platform. So I went ahead and installed the pfBlockerNG-devel package. At the time of writing this blog post the latest version of pfBlockerNG-devel is 2.2.5_29. Note the “devel” in the name because this is the branche of pfBlockerNG which is actively being developed.
Ads on themselves can be OK I think. It all depends on how ads are being used and in the end you need to find funding. After all this site is also using ads. Adding pfBlockerNG allows you not to only block ads but also block web tracking and ransomware. That there is added security and privacy you get when using pfBlockerNG. It will do this for your whole network using something called DNSBL (short for Domain Name System-based Blackhole List). Every device in your network will benefit from this and be protected. But pfBlockerNG does so much more like also giving you the ability to block internet traffic coming from certain IP addresses. These IP addresses translate to specific countries and regions so it can be very handy in protecting your network from all those hackers trying to get in your network.
I went ahead and set up both and for some time everything was working well. I enjoyed ad-free and tracking-free internet on all the devices in my LAN. But then something happened…
I have several iOT devices at home including Ikea Tradfri smart lights. Suddenly these lights because unreachable in the Apple Homekit App on my iPhone. The rest of my Homekit enabled iOT devices were doing fine. The first time this happened I thought it is probably a bug so let’s power cycle the Ikea Tradfri gateway. This was a success and the Ikea smart lights were available again. Nice!
Not so nice when I discovered an hour or so later that the Ikea Tradfri smart light were unreachable again. So now I’m thinking that maybe pfBlockerNG is blocking some hostname (the DNSBL feature). This is possible because maybe one of the DNSBL feeds I am using has got an update and some hostname which Ikea Tradfri gateway uses is bow blacklisted. Luckily pfBlockerNG gives you the ability to whitelist hostnames.
I went into the management interface of my pfSense firewall and selected the Reports tab in pfBlockerNG settings. The Reports tab shows a very nice list of hostnames which have been blocked by pfBlockerNG. There is a nice filtering option as well. See the screenshot below.
My Ikea Tradfri gateway has 192.168.100.51 as IP address. This is static setup in the DHCP server on my pfSense. So I enter this IP address in the Alert filter to see if pfBlockerNG is blocking DNS requests from my Ikea Tradfri gateway. The result was 0 so according to pfBlockerNG nothing from my Ikea Tradfri gateway was blocked. See screenshot below.
But still I had the same behavior. When I power cycle the Ikea Tradfri gateway all is well for a short time and then is just becomes unavailable. I continued my investigation and decided to replace the USB power adapter of the Tradfri gateway. That didn’t help. By now I was thinking that I have tried everything but to replace the unit. I went to Ikea and got a new Tradfri gateway. I set it up and went trough the painful experience of connecting all my Tradfri lights and switches to the new gateway. I was just wrapping up when I saw that all my Ikea lights were unreachable again! Imagine my frustration.
OK now I was furious. Even after replacing the Ikea Tradfri gateway I had the same problem. I was getting more convinced that is has to be something in my network. First step for me now was that I wanted to know all the DNS queries the Ikea Tradfri gateway was making. I tried debugging that in Unbound resolver on my pfSense but there were so many DNS requests flying by that it made troubleshooting nearly impossible.
I needed another DNS server, one specifically for my Ikea Tradfri gateway. And I needed it quick. Since I had a Raspberry Pi lying around I went the Pi-Hole route. Just download the correct image from the Pi-Hole website, extract to the SD-card and startup your new DNS server. Within a couple of minutes I was up and running with Pi-Hole. I loaded the exact same DNSBL lists I was using on pfBlockNG on the Pi-Hole. Using DHCP reservation I managed to set -Pi-Hole as the DNS server on the Tradfri gateway.
Pi-Hole showed me all the DNS queries the Tradfri gateway was doing, which ones were allowed and which ones blocked. I was specifically interested in DNS queries being blocked. I saw immediately that a lot of DNS queries were being blocked to webhook.logentries.com. That DNS query did not came up when I was troubleshooting on pfBlockerNG to find out the blocked queries. I added webhook.logentries.com to the Pi-Hole’s whitelist and waiting a couple of hours. Ikea smart lights were working fine now. Even after 24 hours all my Tradfri lights were now working fine.
Now let’s remove webhook.logentries.com from the Pi-Hole’s whitelist I thought and see what happens. Within the hour my Tradfri lights were offline again. Root cause found :).
I began to investigate why pfBlockerNG was not showing the blocked DNS queries. I discovered that when I did a DNS lookup on pfSense with pfBlockerNG enabled the request for webhook.logentries.com was being “sink holed” to pfBlockerNG, but it was not showing up in the Reports tab as blocked (or allowed). Check the screenshots below what happens on pfSense.
As you can see above the DNS request is blocked by pfBlockerNG because it is “sink-holed” to the DNSBL VIP pfBlockerNG is using (10.10.10.1). But when I check the Reports tab in pfBlockerNG, I don’t see the blocked DNS request.
Now when I do the same DNS lookup against the Pi-Hole I can see the DNS lookup immediately in the Query Log tab:
The gui on the Pi-Hole makes it really easy to troubleshoot as it shows immediately which client is doing what DNS queries and which ones are being blocked. The gui is also very easy in filtering options.
And you can find very easy in which DNSBL feed a certain hostname is so you know what feed is blocking your internet traffic. It even tells you if the dns name is whitelisted. Makes management so much more easy.
This gui compared to pfBlockerNG was refreshing to me. Amazing how much time I spend troubleshooting on pfBlockerNG while the Pi-Hole showed me within minutes what was happening and where the problem was! Great tech :).
I moved from pfBlockerNG to Pi-Hole. Don’t get me wrong, I still love and use pfBlockerNG. But I now only use it to block IP addresses from certain countries and regions. It is still very useful for that.
But I don’t use the DNSBL option anymore because I have no faith in it’s reporting capabilities. And that starts to count very heavy when you are troubleshooting why something is not working in your network. Since I started using Pi-Hole I did find some other dns hostnames which were also blocked and were not reported by pfBlockerNG. One of them was to the download server of Ubiquiti for firmwares. Pretty important to know that sort of stuff.
I just can’t be bothered to make tcp dumps of my network traffic on pfSense and then use some kind of tool to analyze and try to find the needle in the haystack. So I recommend you use Pi-Hole for the DNSBL part as it is amazing at that. From the pragmatic perspective it is blazing fast and has great reporting options about what is happening in your network.
Moved from pfBlockerNG to Pi-Hole Read More