Blocking ads and trackers using HOSTS

If you’ve stumbled across this post, you’re probably familiar with adblocking extensions such as Adblock and uBlock(seriously recommend the latter for a handful of reasons) and most likely you’re in need of a solution to take back your network and system resources as well as a need for less clutter and more privacy in your daily web ventures, however, this method for blocking ads at the browser level only tends to be quite inefficient and fairly limited. Wouldn’t it be cool to also have ads and trackers blocked at the system level, including but not limited to applications like Skype, uTorrent, IE(seriously?) and other browsers or the many shareware/freeware apps that track your usage via mechanisms like Google Analytics(some use exactly that for tracking).

The solution is fairly simple, we’re going to use a simple hostname based block list to map undesirable domain names to either 0.0.0.0 or 127.0.0.1. In my testing on OS X, I found that 0.0.0.0 works best, that might not be the case on different operating systems. The blocking is done via the ages old hosts(5) unix file, but still very useful mechanism for easy static ip-name mappings at the host level.

The current block list that I use is hosted at hosts.neocities.org. I’m not affiliated with that site and don’t know who is providing it, that being said I use git to track and review changes between updates. The list is quite exhaustive, combining lists from several other sources cited in the header. I’d like to see a couple more lists combined like that from several other places(mainly the ones from uBlock would be useful), but you can then add extra lists by modifying the script fairly easily.

Now the script itself, is hosted on Github. Please read the entire script and what I’ve written bellow before running the script on your system.

Before you go on and use the script on your OS X, I really encourage to start using git in your /etc/ directory. The script won’t even work without a git repo in /etc/, unless you know what you’re doing and you’re going to modify it to bypass that. Having a git repo in your etc directory gives you revisioning, rollback, beta-testing, review and scrutiny abilities to whatever you’re doing to your etc. I do this on my workstations, laptops and servers that I manage. The added git overhead on your daily etc routines is insignificant when compared to the benefits you get when you most need them.

The script is smart enough to not break your current system. What it does as part of the first time run initialization is copy your current /etc/hosts to /etc/hosts.d/hosts.1.head. All your existing localhost rules and custom rules will be maintained there. The adblocking rules will go into /etc/hosts.d/hosts.3.adblock. You can add custom mapping rules(for staging servers, local network mappings) to  hosts.2.custom.

Then each time the script updates it will do the following:

  1. Update hosts.3.adblock with the latest rules from upstream;
  2. Concatenate the rules in /etc/hosts.d in the numeric order to your /etc/hosts;
  3. Show you a git diff of the changes and the option to commit those changes or deny to review, undo or commit yourself using git;

The script also has some pfsense blocking rules from www.emergingthreats.net and some custom ip blocking enabled in /etc/pf.rules/ip-block.pf. This is disabled by default, you can enable it by setting the PFSENSE var to “true” or passing -f as argument. If you know of some other worthy and fresh ad/malware ip lists let me know.

Although my script is OS X only, it’s fairly easy to port it to any other UNIX system(I welcome patches to the main script via Github), having such a solution for the Windows platform would be cool too. Maybe someone reading this can weigh in with their solution or insight? Would it work fair enough, is cygwin the only way for automating this? Nonetheless, stay tuned, since I have a similar router solution(AsusWRT, DD-WRT) coming up soon, that steps up the game a notch and provides blocking for your entire network, though it surely doesn’t deprecate this host level solution (on a laptop for e.g. that is frequently switching networks).

Pros for this setup:

  1. Easy setup and update (when compared to a firewall or a custom dns);
  2. Cross-platform and cross-application solution;
  3. Faster and less intrusive(also no https mitm) than proxy solutions(such as Privoxy);
  4. Easy to temporarily disable: just cp /etc/hosts.d/hosts.1.head /etc/hosts and to restore git checkout /etc/hosts;

Caveats:

  1. On some operating systems hosts files with tens of thousands of rules might slow name resolution up to a certain degree. In my usage with over 50000 rules, OS X and Linux is quite fine in that regard. If you find that such is your case, maybe using a dns server or firewall rules is better for you;
  2. Some blank spaces, containers, divs or unresolved error messages will take the place of the ads themselves in sites and apps that don’t handle failure very well. You can get rid of the browser related blanks at least by using uBlock extension with just the cosmetic rules enabled(in the extension Settings);
  3. Related to the previous one, you might experience some failures in certain web related functionality(fairly limited though). Most of them will be social related or news sites that use ad nag pages before they redirect you to the article content itself. Personally I don’t care about them and as soon as I hit such a road block I close it and move on. The benefit of more resources and network bandwidth for my system as well as the increased privacy and less clutter in general, totally trumps any minor drawback like this;
  4. The script relies on the links(1)(or elinks) tool to parse the html page at hosts.neocities.org and extract only the text. On OS X I use homebrew to install additional tools that I need. If you have a better solid solution that relies only on coreutils or other commonly installed shell utilities let me know;

 

 

Tagged with:
 

Conspiracy theories fly around Norton forum ‘Pifts’ purge

Conspiracy theories are running rampant in the absence of a clear explanation of why Symantec deleted threads expressing concern about a file called pifts.exe from its Norton support forums.

Many users running Norton Internet Protection began seeing a popup warning on Monday that a file called PIFTS.exe on their systems was trying to access the internet. The location of the file was given as a non-existent folder buried inside the Symantec LiveUpdate folder.

The appearance of a file in a non-existent folder suggests rootkit-like behaviour. PIFTS.exe attempts to contact a server in Africa, which has been traced to Symantec.

Concerned punters started posting on Norton’s support forums, asking what was going on. That’s all normal enough, but then discussions on the subject were deleted without explanation from Norton’s community pages. Follow-up threads mentioning the issue were deleted even more quickly.

source

Symantec creates havoc with unsigned Norton patch

Symantec released a diagnostic patch for some of its older Norton products on Monday night that did not identify its origin and thus triggered alerts on user firewalls, the company said Tuesday.

The patch for 2006 and 2007 versions of Norton Internet Security and Norton Antivirus, a program dubbed “PFST.exe,” (Product Information Framework Trouble Shooter) was distributed to collect anonymous statistics on matters such as how many computers are using the products and what operating system they are running, Jeff Kyle, group product manager for Symantec consumer products, said Tuesday.

Because it was unsigned–a result of human error–firewalls started prompting users with messages asking them if they trust the patch, Kyle said. Of course, because the patch had no signature indicating it was from Symantec, users didn’t know whether to trust it and many of them went to the Norton user forum for answers.

The company pulled the patch after three hours and then unwittingly laid the groundwork for conspiracy theorists after it started deleting forum posts related to the matter. The company was not censoring the posts, but fighting off a spam attack, according to Kyle.

“At the same time we were pulling down the patch a spammer created a new account on our forum and minutes after that there were 200 new users all targeting the same thread,” he said. “Within the first hour there were like 600 posts to that thread. Obviously it was a bot creating this.”

The posts were written with poor grammar and broken English and some were vulgar and nonsensical. It is possible, though, that Symantec could have inadvertently deleted some legitimate posts while it was purging the spam, Kyle said.

“There is no conspiracy theory. There’s nothing we are hiding at all,” Kyle added.

Meanwhile, Kyle said he isn’t sure whether or when Symantec will redistribute the patch, but if they do, he said, it will be signed.

Symantec has more information on its message board site. The Washington Post reported that hackers were exploiting the situation and had managed to get malicious Web sites into top Google search results for “pifts.exe.”

source

Digg in:

http://chrysler5thavenue.blogspot.com/2009/03/piftsexe.html

http://www.abovetopsecret.com/forum/thread444230/pg1

http://blogs.howstuffworks.com/2009/03/10/what-is-piftsexe/

http://www.theregister.co.uk/2009/03/10/norton_pifts_mystery/

http://news.cnet.com/8301-1009_3-10192899-83.html?part=rss&subj=news&tag=2547-1_3-0-20

http://isc.sans.org/diary.html?storyid=5992

http://it.slashdot.org/article.pl?sid=09/03/10/139229

http://www.tech-linkblog.com/2009/03/conspiracy-theories-run-rampant-due-to-piftsexe.html/

http://voices.washingtonpost.com/securityfix/2009/03/symantec_users_complain_of_mys.html

http://community.norton.com/norton/board/message?board.id=nis_feedback&thread.id=39119

http://www.virustotal.com/analisis/734465e30a6ee6d6c493471d77940f4c

http://www.threatexpert.com/report.aspx?md5=91b564d825a3487ae5b5fafe57260810

http://www.mediafire.com/download.php?iotyqjbrmry – pifts.exe asm code from IDA and C-like decompiled pseudocode from HexRays Decompiler

The disassembly of the binary doesn’t look out of the ordinary and after a quick look at the PIFTS.c decompiled code in the above archive it seems that symantec is telling nothing but the truth.

Snips:

$cat PIFTS.c | grep -i RegOpenKey
// LSTATUS __stdcall RegOpenKeyExW(HKEY hKey, LPCWSTR lpSubKey, DWORD ulOptions, REGSAM samDesired, PHKEY phkResult); idb
v1 = RegOpenKeyExW(HKEY_LOCAL_MACHINE, L”Software\\Symantec\\InstalledApps”, 0, 1u, &hKey);
if ( RegOpenKeyExW(HKEY_LOCAL_MACHINE, L”Software\\Symantec\\InstalledApps”, 0, 1u, &hKey)
if ( RegOpenKeyExW(HKEY_LOCAL_MACHINE, L”Software\\Symantec\\InstalledApps”, 0, 1u, &hKey)
if ( RegOpenKeyExW(HKEY_LOCAL_MACHINE, L”Software\\Symantec\\InstalledApps”, 0, 1u, &hKey)
if ( (RegOpenKeyExW(HKEY_LOCAL_MACHINE, L”Software\\Symantec\\InstalledApps”, 0, 1u, &hKey)
&& (RegOpenKeyExW(HKEY_LOCAL_MACHINE, L”Software\\Symantec\\InstalledApps”, 0, 1u, &hKey)
if ( RegOpenKeyExW(HKEY_LOCAL_MACHINE, L”Software\\Symantec\\InstalledApps”, 0, 1u, &hKey)

$cat PIFTS.c | grep -i http
extern wchar_t aHttpStats_nort[40]; // weak
sub_402210((int)L”http://stats.norton.com/n/p?module=2667″, (int)&lpszUrl);

$cat PIFTS.c | grep -i symantec
v1 = RegOpenKeyExW(HKEY_LOCAL_MACHINE, L”Software\\Symantec\\InstalledApps”, 0, 1u, &hKey);
if ( RegOpenKeyExW(HKEY_LOCAL_MACHINE, L”Software\\Symantec\\InstalledApps”, 0, 1u, &hKey)
if ( RegOpenKeyExW(HKEY_LOCAL_MACHINE, L”Software\\Symantec\\InstalledApps”, 0, 1u, &hKey)
if ( RegOpenKeyExW(HKEY_LOCAL_MACHINE, L”Software\\Symantec\\InstalledApps”, 0, 1u, &hKey)
if ( (RegOpenKeyExW(HKEY_LOCAL_MACHINE, L”Software\\Symantec\\InstalledApps”, 0, 1u, &hKey)
&& (RegOpenKeyExW(HKEY_LOCAL_MACHINE, L”Software\\Symantec\\InstalledApps”, 0, 1u, &hKey)
if ( RegOpenKeyExW(HKEY_LOCAL_MACHINE, L”Software\\Symantec\\InstalledApps”, 0, 1u, &hKey)
L”SOFTWARE\\Symantec\\PIF\\{B8E1DD85-8582-4c61-B58F-2F227FCA9A08}\\PifEngine”,
L”SOFTWARE\\Symantec\\PIF\\{B8E1DD85-8582-4c61-B58F-2F227FCA9A08}\\HbEngine”,
L”SOFTWARE\\Symantec\\PIF\\{96E26A03-A25A-400b-B9B4-564C9BD00F46}”,

For me it doesn’t sound like much of a conspiracy or an uber slick big brother spyware because Symantec’s explanation of pifts.exe is somewhat reasonable although the method they used to get the stats is questionable and unprofessional(don’t they have test systems, labs or networks? why push stuff that should be tested in the “development labs” to clients? clients are not a testing platform) and Symantec removing forum posts when they look like this doesn’t bother me either. People who started this fire are ones who don’t have the skills to reverse engineer an executable because if they did they would know that there isn’t really nothing interesting in pifts code. Another reason for this might be this one too:

Finally, it has also been reported by the Washington Post that hackers are taking advantage of this situation. “Some of the top searches (currently the 3rd and 4th result in a Google search) are Web sites that try to install malicious software when you visit them.” When searching for information on “pifts.exe,” Symantec strongly advises all users to be wary of following links to unknown sites as malicious users are attempting to use this hot topic to distribute malware.

Although it is hard to believe that this was a planned scam because you really have to be a con mastermind in order to think(of it previously and wait for the right moment), spot the apropriate moment(symantec’s pifts.exe unsigned and questionable push update), prepare(create bogus trap websites/page related to the subject and embedd exploit+malware code) and fire the scam(make people believe your conspiracy theory-the symantec forums, the stories, blog posts) then watch your botnet(s) grow. As it is written above it is more probably that bad guys saw the opportunity after the hype has been created and started to take advantage of it.

Tagged with:
 

Well if you have the luck to get a high pagerank and really show up alot in google searches then you automatically get this free gift. Those are http download stats or better called http get commands issued to the server and you should not confuse that page with hack attempts. Do you see the immense amount of bot web attacks, mostly rfi and lfi? Well i’ve seen some of that as soon as google indexed my site and started showing it in searches. The conclusion is, if you don’t know this already, if you want to have your own server under your administration you’d better be having the necessary skills to secure it, because if you don’t, then your website and/or server won’t stand up for too long before getting pwned.

Tagged with:
 

Djbdns bug finder awarded 1000$

From: D. J. Bernstein <djb <at> cr.yp.to>
Subject: djbdns<=1.05 lets AXFRed subdomains overwrite domains
Newsgroups: gmane.network.djbdns
Date: 2009-03-04 01:34:21 GMT (11 hours and 49 minutes ago)

If the administrator of example.com publishes the example.com DNS data
through tinydns and axfrdns, and includes data for sub.example.com
transferred from an untrusted third party, then that third party can
control cache entries for example.com, not just sub.example.com. This is
the result of a bug in djbdns pointed out by Matthew Dempsky. (In short,
axfrdns compresses some outgoing DNS packets incorrectly.)

Even though this bug affects very few users, it is a violation of the
expected security policy in a reasonable situation, so it is a security
hole in djbdns. Third-party DNS service is discouraged in the djbdns
documentation but is nevertheless supported. Dempsky is hereby awarded
$1000.

The next release of djbdns will be backed by a new security guarantee.
In the meantime, if any users are in the situation described above,
those users are advised to apply Dempsky's patch and requested to accept
my apologies. The patch is also recommended for other users; it corrects
the bug without any side effects. A copy of the patch appears below.

---D. J. Bernstein
   Research Professor, Computer Science, University of Illinois at Chicago

--- response.c.orig     2009-02-24 21:04:06.000000000 -0800
+++ response.c  2009-02-24 21:04:25.000000000 -0800
@@ -34,7 +34,7 @@
         uint16_pack_big(buf,49152 + name_ptr[i]);
         return response_addbytes(buf,2);
       }
-    if (dlen <= 128)
+    if ((dlen <= 128) && (response_len < 16384))
       if (name_num < NAMES) {
        byte_copy(name[name_num],dlen,d);
        name_ptr[name_num] = response_len;
source

djbdns
Tagged with:
 

Security fail

fail-owned-anti-theft-fail

I couldn’t help to think about the stunning resemblance of this image with the Internet Security mentality and practices of nowadays.

Tagged with:
 

Read this fine analysis of Raidon Staray-S Series hardware encrypted hdds  on Heise Security:

Budget encryption – Attacking a weak crypto system

So next time think again, read about it, investigate it before buying such a hdd and trusting it with your most precious data.

Tagged with: