How to meld multiple large huge hosts files on Ubuntu?

[Souman]

Hey experts! I am interested in knowing some ideas from you all about improvement to processing maintenance of large hosts file? Whenever I see a advertisement site with an objectionable, flash content, redirects etc. To generally add my domain I do save the the hosts.txt file to /tmp/hosts.txt and then remove all the redirects such as spaces, comments etc. So my actual question is how we can meld huge large hosts files on Ubuntu?

[Aniela]

The methodology that you are currently using moves the localhost to alternate best-level lines including flatter level which you then need to manually carry back to the top to keep a similarity of the initial accommodates request. Just follow the code below and you will be able to know what I am trying to explain.

127.0.0.1 machine localhost.localdomain localhost
127.0.1.1 machine
# The following lines are desirable for IPv6 capable hosts
::1     localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
# begin parasitic blocks

[Jezriah]

While searching over the internet I found that as per Wikipedia, DynDNS clearly furnishes a few services that are free. It consistently updates your evident IP location. It likewise appears to have a defectively described Internet matter separating utility. Consistent with Wikipedia, OpenDNS obviously gives advertisement-backed utilities. It appears to do unlimited substance separating but its underpinned by notices. The proposed promotions appear to appear in bizarre

[Sudhamay]

The entire indicate is to effortlessly wipe out numerous notices with a lone accommodates document, so there doesn't give off an impression of being much sense in further investigating OpenDNS on account of it appears to be upheld by notices-which is the whole indicate kill. Anyhow the DynDNS utility does appear charming right away look. I as a matter of fact such as BOTH things it does i.e., update your IP location customarily and perform some sort of substance sifting, whatever that intends to it. I'll investigate DynDNS facilitate-but what I was 'quite' searching for were basic UNIX orders to separate out copies from gigantic fused documents.

[Theking]

The script beneath then makes a new accommodates record for you in /and whatnot which looks like:

# hosts file made Mon Nov 21 16:18:44 GMT 2011
# contents of /tmp/hosts.real
127.0.0.1    localhost
127.0.1.1    laptop
192.168.0.32    myserver

# The following lines are desirable for IPv6 capable hosts
::1     localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
# contents of /tmp/hosts.d/*
#  total 1200
#  -rw-rw-r-- 1 jonathan jonathan 612606 2011-11-21 12:37 hosts2.txt
#  -rw-rw-r-- 1 jonathan jonathan 612606 2011-10-13 18:49 hosts.txt
127.0.0.1                             ads.7days.ae
127.0.0.1                             ads.angop.ao
127.0.0.1              smartad.mercadolibre.com.ar
[14,000 more...]

[Lankesh]

As I'm positive some viewers know and some don't: awk will take any number of records, tr doesn't take filenames, and some variants of sort don't permit-u.) If you utilize this tyle of "vast secluded pipeline" you are able to advance things fast, putting short one-liners in tackle each and every part of the riddle. In this exact case, some of the tr was put late, in fix the spurious carriage benefits, and its disturbing to have move the filenames around.

[Kirt]

I think you'll identify that the first script even now sorted the spaces from the right ie, every last trace of the com as one, every last trace of the uk as a single unit and so forth, which is the reason as noted in the remarks of the a few awks around the sort; your increases invalidate that impact.

[Marcya]

I prescribe printf over print and reverberate in diverse dialects. The essential purpose behind this is transportability and adaptability, for the cost of a touch of memorizing. As C, perl, php, awk, java and a significant number of alternate dialects have printf, you could probably memorize it once and utilize it in bunches of situations.

[AbhayD]

Rather than perusing a Wikipedia item on DynDNS, you ought to rather head over to the DynDNS net destination and see for yourself. It is a somewhat exceptional aid. I have arranged it for utilization on the people machines at various household centres. I am attempting to separate out web cruft is an enormous waste of time, most fit done by a hefty utility association like DynDNS. It is not something that a specific individual can do decently.

[Damodaran-G]

Blocking advertisements and trackers is a tricky job. Utilizing /etc/hosts is rough on the grounds that it is a static posting and just bargains with determining space names. Notwithstanding by utilizing an outer piece record you let the maker demarcate what is regretful or unwanted.

[Rajesh4mLA]

Following on your scanning propensities ninety nine percent of its substance would not be able to even be of any utilization to you.

• -incrementally upgrade your piece post,
• -piece promotion-tracking treats,
• -piece promotions dwelling in a way on the same server you visit,
• -piece promotions from a host name of which the space name is the same as the server you visit,
• -square notices on the same server you visit displayed through Javascript or Flash,
• -square notices by host or way substring match,
• -piece just net bugs,
• -square notices server over HTTPS.