autohosts

Auto Update Ad-blocking Hosts file on Linux and Mac Systems

Automate hosts file updates on Linux-based and MacOS systems.

v3.0.0 Changes

Freedom of choice!

• You can now utilize your own choice upstream hosts curators, rather than being content with the (previously) hardcoded host list by Steven Black.

• Logic improvements and bugfixes

Purpose of Hosts Files

Hosts files will reroute unwanted traffic from ad farms, behavioral tracking firms and malware sites to a blackhole; routing to 0.0.0.0 (localhost; your PC) when a request is made to a URL on the blacklist.

Which means any traffic that would have left your system for that destination, is sent inward, to your localhost and then abandoned.

Despite what some may suggest, hosts files are not "1980s technology" and still very useful today, as an additional layer of security.

Hosts file are a useful redundancy when coupled with ad blockers like uBlock Origin and uMatrix - while debugging or 'Temporarily Allow All on this Site' with Noscript can open you up to underlying attacks or privacy intrusions.

In-browser filters won't protect you if the browser itself is phoning home.

If you have an up-to-date hosts file, the risk is severely lessened.

Auto Hosts will automate the setup process for maintaining an up to date hosts file, by:

• Installing a weekly cronjob to pull fresh/updated copies of Steven Black's Host file and LocalFigurez Firefox Telemetry List (default is every Sunday at 7:22pm)
• Appends Facebook trackers, Linkedin ads, Google fonts, Firefox telemetry and other harvester sites that curated lists for whatever reason, have not added to their blacklists
• Refreshes DNS to instantiate the re-routed changes (Mac Only)
• If Devdom is installed, append all local virtualhosts (if it is installed - it is -not- a requirement! Autohosts does a conditional check to ensure your dev environment doesn't break if you're using Devdom. Read the code.)
• Uninstall offers the ability to revert to a fresh hosts file and undo any changes made by Autohosts; a backup will be copied to /tmp
• Add custom filters from ~/autohosts on your desktop

Installing:

Firefox Snooping list, originally by LocalFigurez

For those of us who want our browser to behave like a browser and not a GPS anklet, this list will prove useful by blocking a plethora of tracking URLs. A few of which, some Firefox users have noticed seem to phone home even when the setting is toggled off in about:config.

Custom filters

You can easily append any domain you want to blacklist directly from your home directory. Your custom preferences will be stored with each subsequent update - set it and forget it!

Whitelisting

If the upstream list curators block a domain you want to see, add it to ~/autohosts/whitelist and run sudo autohosts

Be sure to add nothing other than the domain(s) you want to whitelist to this file! I added 3 examples to the default whitelist file; you can regex it (TLD domain with no prefix or suffix, to whitelist subdomains, too: example) or supply a full TLD: www.example.com

To Install

Debian-based Linux distros:

git clone https://github.com/angela-d/autohosts.git && sudo apt install ./autohosts.deb

MacOS or non-Debian-based Linux distros:

git clone https://github.com/angela-d/autohosts.git && cd autohosts && sudo ./autohosts

That's it !

---

To upgrade an existing install

To see which version you're running on your system: grep "VERSION=" /etc/autohosts.conf

Versions prior to 3.0.0:

Due to structural changes in the codebase, an uninstall and reinstall is recommended (unless installed via deb). You can do this without losing your custom filters.

git clone https://github.com/angela-d/autohosts.git /tmp/autohosts &&
cd autohosts &&
cp ~/autohosts/custom_filters /tmp/custom_filters &&
sudo ./prior-v2-uninstall

Run your preferred method of installation to get v3.0.0. Once installation completes, restore your custom filters:

rm ~/autohosts/custom_filters &&
mv /tmp/custom_filters ~/autohosts/custom_filters

Debian users: Simply apt install ./autohosts.deb to upgrade to the latest version.

---

Adding Custom Blacklists or Whitelists

• These files can be edited in your favorite text editor; command-line modifications are not necessary!

Custom filters are loaded to your home directory:

Filter Option	Linux File Location	Mac File Location
Blacklist	~/autohosts/custom_filters or /home/your_username/autohosts/custom_filters	~/autohosts/custom_filters or /Users/your_username/autohosts/custom_filters
Whitelist	~/autohosts/whitelist or /home/your_username/autohosts/whitelist	~/autohosts/whitelist or /Users/your_username/autohosts/whitelist
Hosts List Curator	~/autohosts/hosts_source or /home/your_username/autohosts/hosts_source	~/autohosts/hosts_source or /Users/your_username/autohosts/hosts_source

Choosing Hosts Curators

Use as many as you'd like, to strengthen your filtering. Though it would be wise to keep the total to a reasonable amount; as there is not currently any duplicate removal, so the potential for unneeded overhead is certain.

List each curator on a separate line in ~/autohosts/hosts_source - do not add any comments or whitespace to this file - just a list of the raw hosts source. (ie. the plain-text filters).

When an update is ran, Autohosts will probe each curator to ensure the list is responding with a 200/OK response, so the potential for indexing garbled junk to your hosts file is severely lessened.

Adjust the cron time

If your computer is not powered on when the cron is scheduled, you'll miss the update. Ensure the cronjob is set for a time when you're most likely to have it on. You can adjust it by running:

sudo crontab -e

and modifying the dates to suit.

Cron legend:

* * * * * = minute, hour, day of month, month, day of week (0 = Sunday, 6 = Saturday)

(* = every, so 5 straight stars is equal to every minute of every hour of every day and every month.. which you should never run while pulling 3rd party content!)

Uninstalling:

Note: Because this script has to modify /etc/hosts - it needs elevated privileges (running as root or a sudo user). Scripts that require elevated privileges should be read and analyzed so you know what's being done to your system! Read the source code of this script (and any others requiring such permissions) before you install.

Alternative uses with Hosts files

• If you're running DD-WRT, you can add a cron to pull a hosts file for your entire network or load Pi-Hole to a Raspberry Pi.
• OpenWRT also has network-based adblocking capabilities.
• Rooted Android devices can utilize hosts files, too; simply load the hosts file via adb push to /etc/hosts
• If you have to use Windows, Unified Hosts Autoupdate offers similar capabilities to Autohosts, in a Windows environment