Monday 12 March 2018

Newsgroups: I Made a Difference

In 2013, I had the idea of making a newsgroup search engine. All I had to do was download all newsgroup postings and index them.

Well I wrote some Python code which did multiple threads, and got a Giganews Diamond account which allowed a lot of threads and unlimited downloads.

After a month I had most newsgroups - excepting binaries - and it came to 800GB.

Trying to index THAT lot was impossible.

So in mid-2016, I just heaved the 800GB's onto a 1TB 2.5" USB hard drive and sent it to the Internet Archive.

I thought they hadn't done anything with the drive.

After a few weeks of confusion, a friend of mine - connected to the IA - got all it sorted out. They had uploaded all my data here

https://archive.org/details/giganewsnewsgroups2003to2013

(2003 - 2013 because 2013 is when I last indexed them).

There was a mystery of the missing drive. Basically, they mislaid my original drive, so said they couldn't send it back to me in the post. My friend got that sorted out too and a new drive is on its way.

Anyway, it's all there, independent of Google Groups.

To download a newsgroup, get the domain (below, gna-comp is comp.*)

https://archive.org/download/gna-comp/comp.sys.amiga.announce.mbox.gz

And unzip it. You'll be able to load it into your email client or newsgroup reader.

Hoping to index 2013-2018, but I have until 2023 to do so (it goes back only 10 years).

My 1981-1991 newsgroup search engine is at: http://www.dejadejadeja.com/

Finally, the technology behind my newsgroup search engine is called MailXplorer, and it's here:

http://www.sanfransys.com/mailxplorer

I'm looking for some customers who want to search old hard drives for missing emails.

That's all! Thanks for reading.


10 comments:

Tiago Loges said...

It seems you have an mysql issue in http://www.dejadejadeja.com/

Clipping Solutions said...

I am absolutely loving your gorgeous blog!
clipping path service
Photo Retouching
Raster To vector conversion

Nancy Diese said...

Thank You so much for this great article. You can check out our
Photoshop retouching service

Unknown said...

I tried this several years ago and had a terrible time with getting all of it to be straight. I eventually made some shorter, some longer and used that layout.,....
http://www.designercountry.com/

Unknown said...


That was a remarkable and complete article… all of the tips tallied and explained will be valuable for the people who are wise enough to tap from it. Any business nowadays without electronic long range informal communication sign and proximity may not make it to the farthest, and adventure additionally is a bit of the best approach to accomplishment in business. Continue doing amazing.
http://www.designercountry.com/

Tanjina Akhi said...

Great important post, where you make the main focus, really makes me happy for its necessary of importance.

Tanjina Akhi said...

great post
http://www.designercountry.com/

Tanjina Akhi said...

awesome post

Kamruzzaman Babu said...

Excellent article! This post is very informative and useful also help me a lot. Thanks for sharing.Clipping Path Service

Clipping Path View said...

So nice and important blog.Thank you so much for it from ecommerce photo editing service provider