+ open.mp forum (https://forum.open.mp)
-- Forum: SA-MP (https://forum.open.mp/forumdisplay.php?fid=3)
--- Forum: General Discussions (https://forum.open.mp/forumdisplay.php?fid=6)
--- Thread: For those concerned: (/showthread.php?tid=248)
For those concerned: - Archive - 2019-04-15
Project essentially completed. While we couldn't archive the full SA-MP forum website without receiving IP bans,?https://forum.sa-mp.com/archive?has been fully archived, it is basic but it does have full functionality (the main website does display large threads in a different manner). Note: archive version of the forums (forum.sa-mp.com/archive) has the latest threads of sections on the latest pages. The main website has had 30 GB of data downloaded from it and over 1.2 million pages (including pastebins) with roughly at least another 2.5 million pages missing. What has been archived has been uploaded, including some pages from the archive website. This project went from the 14th to 19th April, so you can expect to see multiple archives from this duration.
This project was initiated because Kalcor threatened to delete the forum's last week (according to Y Less) and hasn't refuted this statement in the thread where Y Less' statement was posted.
You can find the archives on archive.org's WayBack Machine. https://archive.org/web/
Shoutout to the Archive Team for doing all of this.?https://archiveteam.org/
Over 40GB of data has been captured and millions of pages.
RE: For those concerned: - IllidanS4 - 2019-04-15
The Archive Team are the best pals. Good initiative!
RE: For those concerned: - hual - 2019-04-15
Awesome!
RE: For those concerned: - nik - 2019-04-15
what an emotional person
RE: For those concerned: - BigETI - 2019-04-15
noice
RE: For those concerned: - Archive - 2019-04-16
Update #1 issued.
RE: For those concerned: - WestCoast - 2019-04-16
(2019-04-16, 05:40 PM)Archive Wrote: Update #1 issued.What update are you talking about?
RE: For those concerned: - Lars - 2019-04-16
(2019-04-16, 06:36 PM)WestCoast Wrote:(2019-04-16, 05:40 PM)Archive Wrote: Update #1 issued.
What update are you talking about?
(2019-04-15, 09:31 AM)Archive Wrote: Update #1 16/APRIL/2019:?files.sa-mp.com and wiki.sa-mp.com have been fully archived. Data may take a while to show up on the WayBack Machine for the wiki; the job ran on a pipeline which has a few 100 GiB of backlog and some network issues which cause the uploads to be very slow at the moment.?
Current progress on main forums:
Quote:In progress. Downloaded 26136.58 MiB, 7600 errors, 2,261,469 (and more) queued (1,049,124 completed). 6 workers, delay: 250.0, 375.0 ms.
RE: For those concerned: - Markski - 2019-04-16
Thanks for taking care and interest in this. Many things have been lost already, with great contributors such as Y_Less having all of their posts removed. (Over 15 thousands!)
RE: For those concerned: - Markski - 2019-04-17
All the workers are timing. It looks like they blocked the archive bots.
RE: For those concerned: - Nutter - 2019-04-17
In hindsight, I think it would have been better to announce your intentions after you had successfully archived all the posts. I feel that the SA-MP developer deleted Y_Less's account as well as the accounts of others after your intentions went public. Nonetheless, I thank you for your initiative.
RE: For those concerned: - Archive - 2019-04-17
(2019-04-17, 02:17 AM)Nutter Wrote: In hindsight, I think it would have been better to announce your intentions after you had successfully archived all the posts. I feel that the SA-MP developer deleted Y_Less's account as well as the accounts of others after your intentions went public. Nonetheless, I thank you for your initiative.
Possibly; but this is nothing new. Kalcor deleted and even censored FCNPC as a word after the developer was banned, and I remember him commenting on it. I can't be certain on what has or hasn't been captured, but already, over 1 million links have. And a lot of the important/popular/useful?stuff that people like Y Less have released have already been archived through Wayback, but the obscure support threads, obscure releases, pages of comments and pages of navigation have not been archived at all really. Archival only has benefits long-term, unless you want something dead, in which case you can pull the plug then and there. While I think Kalcor has acted in a carefree manner, I seriously doubt?he would interfere exclusively with archival by deleting profiles (he probably did it out of distaste for them), as the people involved have only acted against his version of the modification on a recent basis and it was contained to two threads. The good thing is these people already have all their releases on GitHub and these forums. Even if Open-MP doesn't feel a need to release (if Kalcor provides an update), these forums will still have a purpose against the censorship by Kalcor.
As far as the bot being timed out, this issue is being worked on.
RE: For those concerned: - Archive - 2019-04-18
Update #2 18/APRIL/2019:
Currently, the main forums' website (forum.sa-mp.com) pipeline has downloaded 30GB and 1.3 million links/pages (this includes some external pages, images and the like) with at least another 2.5 million links to go. Due to the 6 workers being detected as using up much bandwidth and resources, the pipeline workers have been IP banned. Currently this is being worked on as this archive was started before big profiles were deleted.
Since then, we also started and have successfully completed an archive for http://forum.sa-mp.com/archive. This was done in the meanwhile as it was much less likely to receive any bans. It is very compact and doesn't even have formatting for things like external links and text, it was far easier and far faster. The bad news is the full-scale archive of the archive version was later, after Y Less and Southclaws and the rest have been deleted from the site (and not just banned), so posts by certain people may not be recorded. While this does mean every thread has been archived, the archived version only has the important stuff and lacks a lot of formatting and website interaction.
http://forum.sa-mp.com/archive and the hundreds of thousands of links archived will be pushed to archive.org Wayback Machine in the following days. P.S., if you're visiting that version of the forums for the first time, the latest threads of sections are on the latest pages, not the first pages.