200 lines
13 KiB
XML
200 lines
13 KiB
XML
<?xml version="1.0" encoding="utf-8" standalone="yes"?>
|
|
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
|
|
<channel>
|
|
<title>Volatile Mediums on Joel Beckmeyer's Blog</title>
|
|
<link>https://beckmeyer.us/tags/volatile-mediums/</link>
|
|
<description>Recent content in Volatile Mediums on Joel Beckmeyer's Blog</description>
|
|
<generator>Hugo -- gohugo.io</generator>
|
|
<language>en-us</language>
|
|
<managingEditor>joel@beckmeyer.us (Joel Beckmeyer)</managingEditor>
|
|
<webMaster>joel@beckmeyer.us (Joel Beckmeyer)</webMaster>
|
|
<lastBuildDate>Fri, 19 Mar 2021 15:00:00 -0400</lastBuildDate><atom:link href="https://beckmeyer.us/tags/volatile-mediums/index.xml" rel="self" type="application/rss+xml" />
|
|
<item>
|
|
<title>The Generation Ship Problem</title>
|
|
<link>https://beckmeyer.us/posts/the_generation_ship_problem/</link>
|
|
<pubDate>Fri, 19 Mar 2021 15:00:00 -0400</pubDate>
|
|
<author>joel@beckmeyer.us (Joel Beckmeyer)</author>
|
|
<guid>https://beckmeyer.us/posts/the_generation_ship_problem/</guid>
|
|
<description><p>After talking about the hardware and software problems of
|
|
digital permanence, I&rsquo;m struck by a classical Sci-Fi
|
|
motif with a conundrum: the <strong>Generation Ship</strong>; a ship
|
|
outfitted with all of the technology, infrastructure, and
|
|
storage to support lightyear-scale human travel.</p>
|
|
<p>But what about that technology on the ship? If we build
|
|
one of these ships, we need to accomplish one of several
|
|
things in regards to information storage:</p>
|
|
<h3 id="1-innovate-to-the-point-where-the-lifetime-of-the-storage-devices-is-able-to-support-lightyear-scale-travel">1. Innovate to the point where the lifetime of the storage devices is able to support lightyear scale travel.</h3>
|
|
<p>That&rsquo;s a tall order, given where we are right now with
|
|
physical storage devices. As I mentioned in one of my
|
|
previous posts, the average lifetime of physical storage
|
|
devices is less than 100 years, no matter if it is a hard
|
|
drive, solid-state drive, etc.</p>
|
|
<h3 id="2-provide-the-facility-to-create-new-storage-devices-to-replace-the-failing-old-ones">2. Provide the facility to create new storage devices to replace the failing old ones.</h3>
|
|
<p>Again, in my mind a tall order, since it would require
|
|
facilities on the ship to create storage devices. The
|
|
problem of having materials is at least solvable by just
|
|
sending the ship with all of the materials it needs in
|
|
advance.</p>
|
|
<h3 id="3-provide-the-facility-to-revitalize-storage-devices">3. Provide the facility to revitalize storage devices.</h3>
|
|
<p>One of the main reasons I&rsquo;m even thinking about this is
|
|
because I&rsquo;m an individual with limited resources.
|
|
Accordingly, I think about things in terms of
|
|
broken/working, on/off, etc. With enough resources, there
|
|
is a much larger chance of being able to repair, re-purpose,
|
|
and otherwise revitalize storage devices, increasing their
|
|
lifetime. E.g., if the only failure in the hard drive is the
|
|
control circuit, that is an &ldquo;easy enough&rdquo; repair.</p>
|
|
<p>I like to toy with the idea of a generation ship a lot in
|
|
my head, but I think it&rsquo;s really fun to think about the
|
|
technical possibilities and needs of a ship like this.</p>
|
|
</description>
|
|
</item>
|
|
|
|
<item>
|
|
<title>Volatile Formats</title>
|
|
<link>https://beckmeyer.us/posts/volatile_formats/</link>
|
|
<pubDate>Thu, 18 Mar 2021 14:24:00 -0400</pubDate>
|
|
<author>joel@beckmeyer.us (Joel Beckmeyer)</author>
|
|
<guid>https://beckmeyer.us/posts/volatile_formats/</guid>
|
|
<description><p><em>Note: This is a continuation of the thoughts I started
|
|
thinking about in my <a href="https://beckmeyer.us/posts/volatile_mediums/">Volatile Mediums</a> blog post.</em></p>
|
|
<p>The next level up from physical mediums for data storage
|
|
is the <em>way</em> that the data is stored. In the digital age,
|
|
we have a plethora of formats for storing information.
|
|
For me, one of the most interesting areas of information
|
|
storage is the analog-digital space.</p>
|
|
<p>The fundamental problem of storing audio, video, and other
|
|
replications of the physical world is that there is so much
|
|
information that we can collect with sensors
|
|
(think microphones, video cameras, etc.). It would be great
|
|
if we could go get the best camera and microphone out there,
|
|
record whatever people record these days, and have that
|
|
exact physical experience &ldquo;played back&rdquo; for us on a screen
|
|
and speaker/headphones.</p>
|
|
<p>Unfortunately, there are several problems with this. Among
|
|
those is the actual design of the sensor. It takes a lot of
|
|
careful thought, engineering, and the like to create a truly
|
|
good microphone or camera. And after all of that, this sensor
|
|
will cost something. Hopefully, that cost will correspond to
|
|
the actual technical ability of that sensor! In any case,
|
|
not everyone can have the best camera or microphone due to
|
|
any number of constraints, not just those listed above.</p>
|
|
<p>The second problem is the sampling issue. The sensor will
|
|
create some sort of output that can then be measured, or
|
|
<strong>sampled</strong>, by an ADC (analog-to-digital converter). The
|
|
very word &ldquo;sample&rdquo; belies what this nearly magical box is
|
|
doing: it is only looking at certain portions or timestamps
|
|
of the analog signal. Granted, the time between samples
|
|
can be very small (e.g. 44.1 kHz is a fairly common sample
|
|
rate for audio), but there is still some loss of signal.
|
|
Once the ADC creates these samples, it converts them into
|
|
a digital format (something that can be stored on a
|
|
CD, hard drive, thumb drive, etc.).</p>
|
|
<p>The third problem is the encoding issue. The ADC creates all
|
|
of these samples, but we need to start thinking about storage
|
|
limitations. Storing the raw output of a sensor can take a
|
|
lot of space: an average album length (40 minutes) could
|
|
easily take 400MB of space! Now, again, the physical storage
|
|
space is moving in the upward direction to combat this, but
|
|
storing isn&rsquo;t the only problem. One prime issue is internet
|
|
bandwidth.</p>
|
|
<p>The solution to this is compression, like a ZIP file. It
|
|
makes big files smaller by doing some fancy math tricks
|
|
that can be reversed by a computer to reconstruct the
|
|
original file. However, for audio/video files, another level
|
|
of compression exists which actually gets rid of some of the
|
|
information in the original file to save more space. This
|
|
is called &ldquo;lossy&rdquo; compression, as opposed to &ldquo;lossless&rdquo;
|
|
compression.</p>
|
|
<p>Great! We&rsquo;ve found a way to save more space. The problem
|
|
with lossy compression is that we have to decide which
|
|
information to throw away. Usually, this is frequencies
|
|
that the average human ear/eye can&rsquo;t perceive. But, let&rsquo;s
|
|
just say that some compression is a bit too &ldquo;greedy&rdquo; when it
|
|
comes to saving space and starts to cut into the band of
|
|
frequencies that can be perceived. Also note that
|
|
the design of these compression algorithms is an artform
|
|
and takes lots of careful consideration.</p>
|
|
<p>The final problem I want to mention is the codec problem.
|
|
There are many different codecs available today, and for
|
|
each and every one of them to be useful, you need to have a
|
|
way to decode each and every one of them. Unfortunately,
|
|
this is sometimes very difficult.</p>
|
|
<p>It could be a licensing
|
|
issue, where you don&rsquo;t have the correct software installed
|
|
or purchased to actually decode that file on your computer.</p>
|
|
<p>Or it could be a physical constraints issue, where your
|
|
computer isn&rsquo;t powerful enough to decode the file at a fast
|
|
enough rate for you to view it without stuttering,
|
|
buffering, etc.</p>
|
|
<p>Third, it could be a personal preference. Some people
|
|
have much more sensitive eyes/ears and need to have formats
|
|
that are more <strong>transparent</strong>, meaning that the lossy file
|
|
is perceptually identical to the source it was encoded from.</p>
|
|
<p>With all of these issues at play, I think there are several
|
|
key points to make:</p>
|
|
<h3 id="1-codecs-need-to-be-freely-available-for-widespread-use-with-no-strings-attached">1. Codecs need to be freely available for widespread use with no strings attached.</h3>
|
|
<p>Can&rsquo;t stress this one enough: we need to make sure we are
|
|
doing everything possible to not let our information die
|
|
when a corporation or individual makes a decision that
|
|
impacts the &ldquo;who, what, where, when, and how&rdquo; of their codec
|
|
usage.</p>
|
|
<h3 id="2-lossless-compression-is-good-but-it-is-not-the-only-thing-we-need">2. Lossless compression is good, but it is not the only thing we need.</h3>
|
|
<p>We need to remember that not everyone has the ability to use
|
|
lossless codecs, whether that be because of internet
|
|
bandwidth limitations, storage limitation, or the like.
|
|
Instead, we need to continue to innovate in the lossy
|
|
compression space to narrow the perceptual gap between lossy
|
|
and lossless more and more.</p>
|
|
<h3 id="3-a-codec-should-never-become-obsolete">3. A codec should never become obsolete.</h3>
|
|
<p>This one may sound weird, but the fact is, if we&rsquo;re talking
|
|
about long-term storage of information, we can&rsquo;t let codecs
|
|
die, since there may come a day where we need a codec to
|
|
decode great-grandpa&rsquo;s album that never made it big.</p>
|
|
</description>
|
|
</item>
|
|
|
|
<item>
|
|
<title>Volatile Mediums</title>
|
|
<link>https://beckmeyer.us/posts/volatile_mediums/</link>
|
|
<pubDate>Fri, 29 Jan 2021 23:36:00 -0500</pubDate>
|
|
<author>joel@beckmeyer.us (Joel Beckmeyer)</author>
|
|
<guid>https://beckmeyer.us/posts/volatile_mediums/</guid>
|
|
<description><p>I&rsquo;ve recently been thinking a lot about storage mediums [1] &ndash; especially in the long-term.</p>
|
|
<p>Technology has made a lot of progress. Digital storage mediums started out only being
|
|
able to store <a href="https://en.wikipedia.org/wiki/Tape_drive">224KB on a tape drive</a>
|
|
for an average lifetime of <a href="https://blog.storagecraft.com/data-storage-lifespan/"><em>up to</em> 30 years</a>.
|
|
Now, we can store terrabytes of data on hard drives and solid-state drives. However,
|
|
no one ever really answered the question about long-term storage.</p>
|
|
<p>(Note: the following is based off an assumption that the storage medium is only
|
|
being used to make backups or archive data. The device itself could be unplugged and stored
|
|
when no backup is in progress.)</p>
|
|
<p>Even though <em>theoretically</em> hard drives could store data for 20+ years, random bit flips, drive
|
|
failure, etc. all make hard drives too volatile of an option. As always, of course
|
|
redundancy takes away some of these issues.</p>
|
|
<p>SSDs are in an even worse position: they cost significantly more than hard drives
|
|
per TB right now, and last I heard, there were still issues with bit fade when
|
|
unpowered.</p>
|
|
<p>CD/DVD is sounding a lot better, but there are some serious issues here too.
|
|
Variable quality directly impacts the storage lifetime. Physically storing the
|
|
discs is a lot more risky since the disc itself doesn&rsquo;t have as much built-in
|
|
protection as a hard drive or SSD has. You&rsquo;ll need a much larger quantity to
|
|
store the terrabytes of data that you can easily dump on one hard drive. And finally, life
|
|
expectancy is still fairly low &ndash; while manufacturers of recordable discs (the &lsquo;R&rsquo; in CD-R, DVD-R, etc.)
|
|
claim life expectancies of 100-200 (!) years under optimal conditions, others are <em>slightly</em> more conservative,
|
|
<a href="https://www.clir.org/pubs/reports/pub121/sec4/">giving an estimate of 30 years</a>.
|
|
Oh, and remember how I mentioned this is for recordable discs? That means they&rsquo;re single write.
|
|
The random access (RW - CD-RW, DVD-RW, etc.) discs have even lower life expectancies.</p>
|
|
<p>All in all, humanity has not gotten very far with the digital storage medium.
|
|
All of these life expectancies have an inconsequential variance when we zoom out
|
|
to the century view of history.</p>
|
|
<p>[1] And no, I&rsquo;m not talking about the kind you pay to see your dead great-great-aunt to figure out if
|
|
you&rsquo;re actually related to George Washington.</p>
|
|
<p><em>This is intended to be the beginning of a learning series/personal study on the issues surrounding
|
|
information preservation, digital permanence, and their related issues.</em></p>
|
|
</description>
|
|
</item>
|
|
|
|
</channel>
|
|
</rss>
|