<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>Volatile Mediums on Joel Beckmeyer&#39;s Blog</title>
    <link>https://beckmeyer.us/tags/volatile-mediums/</link>
    <description>Recent content in Volatile Mediums on Joel Beckmeyer&#39;s Blog</description>
    <generator>Hugo -- gohugo.io</generator>
    <language>en-us</language>
    <managingEditor>joel@beckmeyer.us (Joel Beckmeyer)</managingEditor>
    <webMaster>joel@beckmeyer.us (Joel Beckmeyer)</webMaster>
    <lastBuildDate>Fri, 19 Mar 2021 15:00:00 -0400</lastBuildDate><atom:link href="https://beckmeyer.us/tags/volatile-mediums/index.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>The Generation Ship Problem</title>
      <link>https://beckmeyer.us/posts/the_generation_ship_problem/</link>
      <pubDate>Fri, 19 Mar 2021 15:00:00 -0400</pubDate>
      <author>joel@beckmeyer.us (Joel Beckmeyer)</author>
      <guid>https://beckmeyer.us/posts/the_generation_ship_problem/</guid>
      <description>&lt;p&gt;After talking about the hardware and software problems of
digital permanence, I&amp;rsquo;m struck by a classical Sci-Fi
motif with a conundrum: the &lt;strong&gt;Generation Ship&lt;/strong&gt;; a ship
outfitted with all of the technology, infrastructure, and
storage to support lightyear-scale human travel.&lt;/p&gt;
&lt;p&gt;But what about that technology on the ship? If we build
one of these ships, we need to accomplish one of several
things in regards to information storage:&lt;/p&gt;
&lt;h3 id=&#34;1-innovate-to-the-point-where-the-lifetime-of-the-storage-devices-is-able-to-support-lightyear-scale-travel&#34;&gt;1. Innovate to the point where the lifetime of the storage devices is able to support lightyear scale travel.&lt;/h3&gt;
&lt;p&gt;That&amp;rsquo;s a tall order, given where we are right now with
physical storage devices. As I mentioned in one of my
previous posts, the average lifetime of physical storage
devices is less than 100 years, no matter if it is a hard
drive, solid-state drive, etc.&lt;/p&gt;
&lt;h3 id=&#34;2-provide-the-facility-to-create-new-storage-devices-to-replace-the-failing-old-ones&#34;&gt;2. Provide the facility to create new storage devices to replace the failing old ones.&lt;/h3&gt;
&lt;p&gt;Again, in my mind a tall order, since it would require
facilities on the ship to create storage devices. The
problem of having materials is at least solvable by just
sending the ship with all of the materials it needs in
advance.&lt;/p&gt;
&lt;h3 id=&#34;3-provide-the-facility-to-revitalize-storage-devices&#34;&gt;3. Provide the facility to revitalize storage devices.&lt;/h3&gt;
&lt;p&gt;One of the main reasons I&amp;rsquo;m even thinking about this is
because I&amp;rsquo;m an individual with limited resources.
Accordingly, I think about things in terms of
broken/working, on/off, etc. With enough resources, there
is a much larger chance of being able to repair, re-purpose,
and otherwise revitalize storage devices, increasing their
lifetime. E.g., if the only failure in the hard drive is the
control circuit, that is an &amp;ldquo;easy enough&amp;rdquo; repair.&lt;/p&gt;
&lt;p&gt;I like to toy with the idea of a generation ship a lot in
my head, but I think it&amp;rsquo;s really fun to think about the
technical possibilities and needs of a ship like this.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Volatile Formats</title>
      <link>https://beckmeyer.us/posts/volatile_formats/</link>
      <pubDate>Thu, 18 Mar 2021 14:24:00 -0400</pubDate>
      <author>joel@beckmeyer.us (Joel Beckmeyer)</author>
      <guid>https://beckmeyer.us/posts/volatile_formats/</guid>
      <description>&lt;p&gt;&lt;em&gt;Note: This is a continuation of the thoughts I started
thinking about in my &lt;a href=&#34;https://beckmeyer.us/posts/volatile_mediums/&#34;&gt;Volatile Mediums&lt;/a&gt; blog post.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;The next level up from physical mediums for data storage
is the &lt;em&gt;way&lt;/em&gt; that the data is stored. In the digital age,
we have a plethora of formats for storing information.
For me, one of the most interesting areas of information
storage is the analog-digital space.&lt;/p&gt;
&lt;p&gt;The fundamental problem of storing audio, video, and other
replications of the physical world is that there is so much
information that we can collect with sensors
(think microphones, video cameras, etc.). It would be great
if we could go get the best camera and microphone out there,
record whatever people record these days, and have that
exact physical experience &amp;ldquo;played back&amp;rdquo; for us on a screen
and speaker/headphones.&lt;/p&gt;
&lt;p&gt;Unfortunately, there are several problems with this. Among
those is the actual design of the sensor. It takes a lot of
careful thought, engineering, and the like to create a truly
good microphone or camera. And after all of that, this sensor
will cost something. Hopefully, that cost will correspond to
the actual technical ability of that sensor! In any case,
not everyone can have the best camera or microphone due to
any number of constraints, not just those listed above.&lt;/p&gt;
&lt;p&gt;The second problem is the sampling issue. The sensor will
create some sort of output that can then be measured, or
&lt;strong&gt;sampled&lt;/strong&gt;, by an ADC (analog-to-digital converter). The
very word &amp;ldquo;sample&amp;rdquo; belies what this nearly magical box is
doing: it is only looking at certain portions or timestamps
of the analog signal. Granted, the time between samples
can be very small (e.g. 44.1 kHz is a fairly common sample
rate for audio), but there is still some loss of signal.
Once the ADC creates these samples, it converts them into
a digital format (something that can be stored on a
CD, hard drive, thumb drive, etc.).&lt;/p&gt;
&lt;p&gt;The third problem is the encoding issue. The ADC creates all
of these samples, but we need to start thinking about storage
limitations. Storing the raw output of a sensor can take a
lot of space: an average album length (40 minutes) could
easily take 400MB of space! Now, again, the physical storage
space is moving in the upward direction to combat this, but
storing isn&amp;rsquo;t the only problem. One prime issue is internet
bandwidth.&lt;/p&gt;
&lt;p&gt;The solution to this is compression, like a ZIP file. It
makes big files smaller by doing some fancy math tricks
that can be reversed by a computer to reconstruct the
original file. However, for audio/video files, another level
of compression exists which actually gets rid of some of the
information in the original file to save more space. This
is called &amp;ldquo;lossy&amp;rdquo; compression, as opposed to &amp;ldquo;lossless&amp;rdquo;
compression.&lt;/p&gt;
&lt;p&gt;Great! We&amp;rsquo;ve found a way to save more space. The problem
with lossy compression is that we have to decide which
information to throw away. Usually, this is frequencies
that the average human ear/eye can&amp;rsquo;t perceive. But, let&amp;rsquo;s
just say that some compression is a bit too &amp;ldquo;greedy&amp;rdquo; when it
comes to saving space and starts to cut into the band of
frequencies that can be perceived. Also note that
the design of these compression algorithms is an artform
and takes lots of careful consideration.&lt;/p&gt;
&lt;p&gt;The final problem I want to mention is the codec problem.
There are many different codecs available today, and for
each and every one of them to be useful, you need to have a
way to decode each and every one of them. Unfortunately,
this is sometimes very difficult.&lt;/p&gt;
&lt;p&gt;It could be a licensing
issue, where you don&amp;rsquo;t have the correct software installed
or purchased to actually decode that file on your computer.&lt;/p&gt;
&lt;p&gt;Or it could be a physical constraints issue, where your
computer isn&amp;rsquo;t powerful enough to decode the file at a fast
enough rate for you to view it without stuttering,
buffering, etc.&lt;/p&gt;
&lt;p&gt;Third, it could be a personal preference. Some people
have much more sensitive eyes/ears and need to have formats
that are more &lt;strong&gt;transparent&lt;/strong&gt;, meaning that the lossy file
is perceptually identical to the source it was encoded from.&lt;/p&gt;
&lt;p&gt;With all of these issues at play, I think there are several
key points to make:&lt;/p&gt;
&lt;h3 id=&#34;1-codecs-need-to-be-freely-available-for-widespread-use-with-no-strings-attached&#34;&gt;1. Codecs need to be freely available for widespread use with no strings attached.&lt;/h3&gt;
&lt;p&gt;Can&amp;rsquo;t stress this one enough: we need to make sure we are
doing everything possible to not let our information die
when a corporation or individual makes a decision that
impacts the &amp;ldquo;who, what, where, when, and how&amp;rdquo; of their codec
usage.&lt;/p&gt;
&lt;h3 id=&#34;2-lossless-compression-is-good-but-it-is-not-the-only-thing-we-need&#34;&gt;2. Lossless compression is good, but it is not the only thing we need.&lt;/h3&gt;
&lt;p&gt;We need to remember that not everyone has the ability to use
lossless codecs, whether that be because of internet
bandwidth limitations, storage limitation, or the like.
Instead, we need to continue to innovate in the lossy
compression space to narrow the perceptual gap between lossy
and lossless more and more.&lt;/p&gt;
&lt;h3 id=&#34;3-a-codec-should-never-become-obsolete&#34;&gt;3. A codec should never become obsolete.&lt;/h3&gt;
&lt;p&gt;This one may sound weird, but the fact is, if we&amp;rsquo;re talking
about long-term storage of information, we can&amp;rsquo;t let codecs
die, since there may come a day where we need a codec to
decode great-grandpa&amp;rsquo;s album that never made it big.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Volatile Mediums</title>
      <link>https://beckmeyer.us/posts/volatile_mediums/</link>
      <pubDate>Fri, 29 Jan 2021 23:36:00 -0500</pubDate>
      <author>joel@beckmeyer.us (Joel Beckmeyer)</author>
      <guid>https://beckmeyer.us/posts/volatile_mediums/</guid>
      <description>&lt;p&gt;I&amp;rsquo;ve recently been thinking a lot about storage mediums [1] &amp;ndash; especially in the long-term.&lt;/p&gt;
&lt;p&gt;Technology has made a lot of progress. Digital storage mediums started out only being
able to store &lt;a href=&#34;https://en.wikipedia.org/wiki/Tape_drive&#34;&gt;224KB on a tape drive&lt;/a&gt;
for an average lifetime of &lt;a href=&#34;https://blog.storagecraft.com/data-storage-lifespan/&#34;&gt;&lt;em&gt;up to&lt;/em&gt; 30 years&lt;/a&gt;.
Now, we can store terrabytes of data on hard drives and solid-state drives. However,
no one ever really answered the question about long-term storage.&lt;/p&gt;
&lt;p&gt;(Note: the following is based off an assumption that the storage medium is only
being used to make backups or archive data. The device itself could be unplugged and stored
when no backup is in progress.)&lt;/p&gt;
&lt;p&gt;Even though &lt;em&gt;theoretically&lt;/em&gt; hard drives could store data for 20+ years, random bit flips, drive
failure, etc. all make hard drives too volatile of an option. As always, of course
redundancy takes away some of these issues.&lt;/p&gt;
&lt;p&gt;SSDs are in an even worse position: they cost significantly more than hard drives
per TB right now, and last I heard, there were still issues with bit fade when
unpowered.&lt;/p&gt;
&lt;p&gt;CD/DVD is sounding a lot better, but there are some serious issues here too.
Variable quality directly impacts the storage lifetime. Physically storing the
discs is a lot more risky since the disc itself doesn&amp;rsquo;t have as much built-in
protection as a hard drive or SSD has. You&amp;rsquo;ll need a much larger quantity to
store the terrabytes of data that you can easily dump on one hard drive. And finally, life
expectancy is still fairly low &amp;ndash; while manufacturers of recordable discs (the &amp;lsquo;R&amp;rsquo; in CD-R, DVD-R, etc.)
claim life expectancies of 100-200 (!) years under optimal conditions, others are &lt;em&gt;slightly&lt;/em&gt; more conservative,
&lt;a href=&#34;https://www.clir.org/pubs/reports/pub121/sec4/&#34;&gt;giving an estimate of 30 years&lt;/a&gt;.
Oh, and remember how I mentioned this is for recordable discs? That means they&amp;rsquo;re single write.
The random access (RW - CD-RW, DVD-RW, etc.) discs have even lower life expectancies.&lt;/p&gt;
&lt;p&gt;All in all, humanity has not gotten very far with the digital storage medium.
All of these life expectancies have an inconsequential variance when we zoom out
to the century view of history.&lt;/p&gt;
&lt;p&gt;[1] And no, I&amp;rsquo;m not talking about the kind you pay to see your dead great-great-aunt to figure out if
you&amp;rsquo;re actually related to George Washington.&lt;/p&gt;
&lt;p&gt;&lt;em&gt;This is intended to be the beginning of a learning series/personal study on the issues surrounding
information preservation, digital permanence, and their related issues.&lt;/em&gt;&lt;/p&gt;
</description>
    </item>
    
  </channel>
</rss>
