I mean, yeah like another user said, ideally it would be in the interest of groups which allege to have am interest in some form of democracy. But additionally, the ability to set up browsable partial mirrors which could be hosted by miscellaneous nonprofits and individuals both within and outside of the US would be a massive first step to preserving the information that IA stores. The fact that attacks on their servers can eradicate all access to the information they store is troubling given how many enemies they’ve made simply through the work they do.
The actual volume of data is kind of insane for distribution. You start running into many scale problems.
At ~70PB of storage, assumed redundant as well. And at ~$15/TB JUST for HDDs alone, you’re talking $2.1 million in just hard drives.
Installation, hardware, and facility costs will at least pentuple that number, if we’re being crazy conservative. Making the cost to stand up an archive $10.5 million?
During this process I found out that their finances are public and there is more reliable information out there:
$2/GB for permanent storage, overall ( $2000/TB)
The cost to store the data and run the archive is a whopping $36mill/y at the moment.
Which if you consider what they do is incredibly cheap. And easily fundable by even a small municipality never mind a large Nation.
It would be interesting to have encrypted blobs scattered around volunteer computers/servers, like a storage version of BOINC / @HOME.
People tend to have dramatically less spare storage space than space compute time though and it would need to be very redundant to be guaranteed not to lose data.
Oh for sure, that’s quite reasonable, though at some point you just move towards re-creating BitTorrent, which will be the actual effect you want.
You could build an appliance on top of the protocol that enables the distributed storage, that might actually be pretty reasonable 🤔
Ofc you will need your own protocols to break the data up into manageable parts, chunked in a same way, and make it capable of being removed from the network or at least made inaccessible for dmca claims. Things that is completely preventing the internet archive from being too much of a target from government entities.
And funded by who?
It’s nice to say that it should be decentralized, but who is funding the development of that? Are you donating to IA?
TBH this is an important enough resource the UN should fund it.
They won’t but they should.
I mean, yeah like another user said, ideally it would be in the interest of groups which allege to have am interest in some form of democracy. But additionally, the ability to set up browsable partial mirrors which could be hosted by miscellaneous nonprofits and individuals both within and outside of the US would be a massive first step to preserving the information that IA stores. The fact that attacks on their servers can eradicate all access to the information they store is troubling given how many enemies they’ve made simply through the work they do.
The actual volume of data is kind of insane for distribution. You start running into many scale problems.
At ~70PB of storage, assumed redundant as well. And at ~$15/TB JUST for HDDs alone, you’re talking $2.1 million in just hard drives.
Installation, hardware, and facility costs will at least pentuple that number, if we’re being crazy conservative. Making the cost to stand up an archive $10.5 million?
During this process I found out that their finances are public and there is more reliable information out there:
The cost to store the data and run the archive is a whopping $36mill/y at the moment.
Which if you consider what they do is incredibly cheap. And easily fundable by even a small municipality never mind a large Nation.
It would be interesting to have encrypted blobs scattered around volunteer computers/servers, like a storage version of BOINC / @HOME.
People tend to have dramatically less spare storage space than space compute time though and it would need to be very redundant to be guaranteed not to lose data.
Oh for sure, that’s quite reasonable, though at some point you just move towards re-creating BitTorrent, which will be the actual effect you want.
You could build an appliance on top of the protocol that enables the distributed storage, that might actually be pretty reasonable 🤔
Ofc you will need your own protocols to break the data up into manageable parts, chunked in a same way, and make it capable of being removed from the network or at least made inaccessible for dmca claims. Things that is completely preventing the internet archive from being too much of a target from government entities.