Well I wrote some Python code which did multiple threads, and got a Giganews Diamond account which allowed a lot of threads and unlimited downloads.
After a month I had most newsgroups - excepting binaries - and it came to 800GB.
Trying to index THAT lot was impossible.
So in mid-2016, I just heaved the 800GB's onto a 1TB 2.5" USB hard drive and sent it to the Internet Archive.
I thought they hadn't done anything with the drive.
After a few weeks of confusion, a friend of mine - connected to the IA - got all it sorted out. They had uploaded all my data here
https://archive.org/details/giganewsnewsgroups2003to2013
(2003 - 2013 because 2013 is when I last indexed them).
There was a mystery of the missing drive. Basically, they mislaid my original drive, so said they couldn't send it back to me in the post. My friend got that sorted out too and a new drive is on its way.
Anyway, it's all there, independent of Google Groups.
To download a newsgroup, get the domain (below, gna-comp is comp.*)
https://archive.org/download/gna-comp/comp.sys.amiga.announce.mbox.gz
And unzip it. You'll be able to load it into your email client or newsgroup reader.
Hoping to index 2013-2018, but I have until 2023 to do so (it goes back only 10 years).
My 1981-1991 newsgroup search engine is at: http://www.dejadejadeja.com/
Finally, the technology behind my newsgroup search engine is called MailXplorer, and it's here:
http://www.sanfransys.com/mailxplorer
I'm looking for some customers who want to search old hard drives for missing emails.
That's all! Thanks for reading.
12 comments:
It seems you have an mysql issue in http://www.dejadejadeja.com/
I am absolutely loving your gorgeous blog!
clipping path service
Photo Retouching
Raster To vector conversion
Thank You so much for this great article. You can check out our
Photoshop retouching service
I tried this several years ago and had a terrible time with getting all of it to be straight. I eventually made some shorter, some longer and used that layout.,....
http://www.designercountry.com/
That was a remarkable and complete article… all of the tips tallied and explained will be valuable for the people who are wise enough to tap from it. Any business nowadays without electronic long range informal communication sign and proximity may not make it to the farthest, and adventure additionally is a bit of the best approach to accomplishment in business. Continue doing amazing.
http://www.designercountry.com/
Great important post, where you make the main focus, really makes me happy for its necessary of importance.
great post
http://www.designercountry.com/
awesome post
Excellent article! This post is very informative and useful also help me a lot. Thanks for sharing.Clipping Path Service
So nice and important blog.Thank you so much for it from ecommerce photo editing service provider
Incredible work! Creating a newsgroup search engine and preserving all that data is a massive achievement. Thank you for sharing your journey and for making this valuable resource available on the Internet Archive. Excited to see what you accomplish with the 2013-2018 index. Keep up the fantastic work!
Clipping Path Service provider
This post was packed with actionable advice. Thank you for such a well-thought-out article!
Visit Here: Background Removal Service
Post a Comment