My Giant Hard Drive: Building a Storage Box with FreeNAS


UPDATE 2/20/2015: This build failed after about 15 months, due to extensive drive failure. By extensive, I mean there were a total of 9 drive replacements, before three drives gave out over a weekend. This correlates closely to data recently published by Backblaze, which suggested 3 TB Seagate drives are exceptionally prone to failure. I’ve replaced these with 6 HGST Deskstar NAS 4TB drives, which were rated highly, and are better suited for NAS environments.

For many years, I’ve had a lot of hard drives being used for data storage. Movies, TV shows, music, apps, games, backups, documents, and other data have been moved between hard drives and stored in inconsistent places. This has always been the cheap and easy approach, but it has never been really satisfying. And with little to no redundancy, I’ve suffered a non-trivial amount of data loss as drives die and files get lost. Now, I’m not alone to have this problem, and others have figured out ways of solving it. One of the most interesting has been in the form of a computer dedicated to one thing: storing data, and lots of it. These computers are called network-attached storage, or NAS, computers. A NAS is a specialized computer that has lots of hard drives, a fast connection to the local network, and…that’s about it. It doesn’t need a high-end graphics card, or a 20-inch monitor, or other things we typically associate with computers. It just sits on the network and quietly serves and stores files. There are off-the-shelf boxes you can buy to do this, such as machines made by Synology or Drobo, and you can assemble one yourself for the job.

I’ve been considering making a NAS for myself for over a year, but kept putting it off due to expense and difficulty. But a short time ago, I finally pulled the trigger on a custom assembled machine for storing data. Lots of it; almost 11 terabytes of storage, in fact. This machine is made up of 6 hard drives, and is capable of withstanding a failure on two of them without losing a single file. If any drives do fail, I can replace them and keep on working. And these 11 terabytes act as one giant hard drive, not as 6 independent ones that have to be organized separately. It’s an investment in my storage needs that should grow as I need it to, and last several years.

Building a NAS took a lot of research, and other people have been equally interested in building their own NAS storage system, so I have condensed what I learned and built into this post. Doing this yourself is not for the faint of heart; it took at least 12 hours of work to assemble and setup the NAS to my needs, and required knowledge of how UNIX worked in order to make what I wanted. This post walks through a lot of that, but still requires skill in system administration (and no, I probably won’t be able to help you figure out why your system is not working). If you’ve never run your own server before, you may find this to be too overwhelming, and would be better suited with an off-the-shelf NAS solution. However, building the machine yourself is far more flexible and powerful, and offers some really useful automation and service-level tools that turn it from a dumb hard drive to an integral part of your data and media workflows.

Read More

Smartphones have replaced lots of types of small devices. iOS and Android have made it easy to build apps that perform all kinds of functions, replacing other standalone devices like media players and GPS. It’s been wondered if they would replace handheld gaming devices, and for many people they have. For awhile, I thought they had, at least for my needs. But after trying to play games on touchscreen-only devices for years, I’ve largely felt unenthused about the deeper and more engaging games that would come from big studios. These games required a higher level of precision control that touchscreens just couldn’t deliver.

The PS Vita caught my attention about a month before its launch in the US. It combines a lot of the best features of smartphones with the controls of console games. It has a gorgeous, large, high-resolution touchscreen (and a back panel that is touch-sensitive), as well as a tilt sensor and cameras for augmented reality games. But it also has almost all of the buttons of a typical PS3 controller, including two analog sticks. Sony managed to cram all of this functionality into a device that, while large, is not too big to fit into my pocket, and with long enough battery life for a busy day interspersed with some gaming. The combination of apps and games (which I will describe as just “apps” for the sake of this review) is powerful, and the hardware power and display size make it a compelling device.

Read More

Adobe is finally putting an end to Flash Player. They’ve announced they’re stopping development of the mobile Flash Player, which is where the future of tech innovation is heading, and the writing is on the wall for desktop Flash Player as well. This is a good thing for a myriad of reasons, both technical and political.

However, it is important to remember that Flash drove much of the innovation on the web as we know it today. When Flash was conceived over a decade ago, the web was a glimmer of what it is today. Creating something visually impressive and interactive was almost impossible. Flash brought the ability to do animation, sound, video, 3D graphics, and local storage in the browser when nothing else could.

Without Flash, MapQuest would not have been able to provide maps for years before Google did in JavaScript. The juggernaut YouTube would not have been possible until at least 2009, four years after its actual launch. Gaming on the web, which has been around as long as Flash, would only now be possible a decade later. Flash enabled developers to create rich user experiences in a market dominated by slow moving browser developers. Even in 2011 Flash exists to provide those more powerful apps to less tech-savvy people who still use old versions of Internet Explorer.

Flash Player itself seemed like a means to an end. Macromedia, and then Adobe who acquired them, sells the tool that you use to build Flash content. Thus, Adobe’s incentive was not to build a great Flash Player, but a pervasive one that would sell its tools. Its technical stagnation provided a market opportunity for browser developers to fill in the gaps that Flash provided. As a result it has a huge market dominance in tools for building rich apps for the web, tools HTML5 lacks.

This puts Adobe in a unique position. As HTML5 continues to negate the need for Flash Player, Adobe has the tools for implementing Flash within HTML5, and the market eager for those tools. Hopefully this move signals that Adobe will be moving in this direction. Because the web DOES need great HTML5 tools for people who aren’t savvy in JavaScript, especially for the people who used Flash to do it previously.

HTML5 offers developers the ability to build high-performance, low-power apps and experiences. Browser innovation has never been faster; Apple, Google, Microsoft, and Mozilla are all competing to bring the best new features to their browsers in compatible ways. But they’re just now filling in many features Flash Player has had for years. Adobe can harness this to help build a better web, and few others can. Hopefully they seize this moment.

I wrote a guest post for MacStories, covering the history of patent law surrounding patent trolls. While recent lawsuits from Lodsys and Kootol are causing panic and alarm from indie developers, it’s not like this threat is suddenly new. Patent lawsuits have always been on the table, but they were ignored by the majority of small companies. Now it’s clear that patent holders will pursue people who violate their patent. Whether ethical or not, they are legally required to defend their patents, and that means we will see more patent lawsuits pursued by trolls. Meanwhile, none of these small developers can afford to fight, so they settle, perpetuating the cycle.

Link

It seems like every other week I’m reading a blog post from a person who went to a tech conference, or a meetup, or heard a talk, and was rightly offended that someone made a tactless joke about women, either about women in general or about specific women. It is disheartening to hear that any group would be made to feel less worthy of respect in our circle, especially at a time when our industry is undergoing one of the most massive and impactful revolutions in decades, and at a time when we need new blood the most. Every person in our industry should be fighting for inclusivity and should welcome new members with open arms and helpful tutorials. Why there aren’t more people pushing for this, I don’t know.

It is equally saddening to hear so many respectable people jump to conclusions about what your actual motives might be when trying to have an adult discussion about this sensitive subject. I’ll make no qualms about it; I’m a born-middle-class white guy, so right off the top, there will be people who will read this under the pretext that I’m either a misogynist or that I’m some kind of “Internet white knight”. As a middle-class white guy, my exposure to injustice and inequality has been limited. I cannot possibly know how it feels to hear words thrown around that minimize the role of women in tech. But I also have rarely been a presenter; who am I to say that I know what Noah Kagan’s motives were when he put “faceless bitch” on a slide at a recent conference? He could’ve been trying to lighten the mood, he could have a vendetta against women in tech. I don’t know.

I’m inclined to believe that incidents like this, such as where women are mocked by a presenter, are isolated events perpetuated by a non-representative group of a few people. When I go to conferences, I keep an active ear open for slurs against women, and have yet to hear any. But what’s fascinating to me is how women are the group continually called out. A demographic survey created by A List Apart shows that women made up 17.8% of respondents; the same study also showed that Asians, blacks, and Hispanics each represented no more than 6% of the group (which is itself a completely separate topic of inequality that seems to be forgotten in these discussions). Yet women are the demographic so frequently mocked and shamed. It probably boils down to sex and the fact that the people that connect to this industry tend to be more introverted, but I don’t know.

The only two things that unite everyone in this industry are that 1) we are all fascinated with high technology, and that 2) we are all humans. As humans we have cognitive biases which prejudice us towards recognizing things the way we’d like them to be. So when we hear that, over the course of several conferences, jokes were made that denigrate women, we’re biased to believe that these events are misogynistic in nature, and that repeated incidents show a trend of sexist men trying to keep out women. It’s possible that’s what’s happening; I think the truth is that these people generally are poor communicators and entertainers put into a role of communicating and entertaining, and failing. But I don’t know.

I don’t know the solutions to the problems we face, but I do know a few things that we all can do better, no matter what subset of demographics you belong to.

  • Actively call out unacceptable remarks when they’re perpetuated at the expense of any group within our community. Whether that’s at the expense of women, men, Android fans, Windows fans, Apple fans, anyone. There is no logical reason for our fledgling industry to show animosity towards any group.

  • Fight the groupthink mentality to label anyone who screws up . Nobody is perfect. Everyone makes mistakes. Few people are truly evil and rotten. I’m reminded of this YouTube video on race; watch, but replace “racist” with “sexist”. Address what they did, not who they are. Let’s address individual problems without calling into question someone’s motives, unless someone makes the same mistake over and over without remorse.

  • Consider not just on how your message is delivered, but also on how it will be perceived. Your audience will contain not only women, but members of every race, gender, religion, and sexual orientation. Joking at the expense of other groups is juvenile and unbecoming, and will reflect negatively not only on the speaker but on the tech community as a whole.

  • Mentor young people who are interested in high technology, and help them learn how to become successful and open-minded. This is something that we should be doing a better job of as an industry as a whole. A teenager who wants to become a software engineer will learn acceptance if they are accepted into a group dominated by grown-ups.

More non-middle-class-white-guy people in our industry will only benefit everyone, from developers to designers to companies to customers. We must be vigilant to keep prejudice out and embrace every single person who wants to contribute to this revolution. But we must be similarly careful not to vilify people for mistakes; hindsight is, after all, 20/20. Of course, maybe I’m wrong. I just don’t know.

Be excellent to each other.

Thanks to Faruk Ateş, who has spoken at length on this issue, for his feedback on this post.

JailbreakMe.com is a website that offers visitors the ability to jailbreak their iPhone without a computer-based tether. It does this by exploiting the system-wide ability for applications to read PDF files, where an incorrectly-formatted PDF file can lead a hacker to do anything they want to your system. While this bug CAN be used maliciously to steal all the personal data from your phone, the developers in this instance used it to enable jailbreaking.

Others will tell you why you should or should not jailbreak your iPhone. Others will decry the developers for bringing to light a serious vulnerability in the iPhone OS. In this blog post, I won’t do any of that, but will instead point out some things you should and shouldn’t do if you decide to jailbreak.

Read More

iPhone had the first two store UIs; the iTunes Store for content like music and movies, and the App Store for software. The iPad will add a third, the iBookstore, for buying eBooks. These stores all provide content for users to extend the utility of their device. But each has a pretty different user interaction model for accessing, purchasing, and consuming that content.

  • The iTunes Store is a separate app that is completely distinct from the iPod app. When you find something to buy, prompting you for your iTunes account password. It then adds the purchase to the app’s Downloads tab. Once you have purchased the content, you must then switch back to the iPod app to listen to or watch it.
  • The App Store is a separate app. When you purchase something, it prompts you for your iTunes password, and then exits to the home screen, switching to the screen where the app will live. The state of the download is reflected in the app icon. When the download is complete, you tap the icon on the home screen to use it
  • The iBookstore (the one word is the official name as used by Apple) is not a separate app, but lives within the iBooks app on the iPad. Purchasing content prompts for the iTunes password and downloads in-app, which can be directly accessed after it has finished downloading.

Each type of content follows a different workflow when going from access to purchase to use. If a goal of the iPad’s low price is to drive content sales through the three stores, as some speculate is the case, then the purchase model should be as streamlined for the different types of content. Forcing different workflows will only confuse users who can’t remember which type of content comes from where.

This uses the new <video> tag in the HTML5 spec, but only supports browsers which implement the H264 codec.

Two interesting bits. First, this is great news for anyone on a platform where Flash is either unavailable (like iPhone) or where performance is terrible (like Mac and Linux). Second, there is a debate about the HTML5 spec’s video codec, as it supports video in Ogg Theora or in H264. Safari and Chrome support the H264 codec, while Firefox and Opera support Ogg Theora. But this player only supports the H264 codec. I wonder if other sites will follow suit, and implement a de-facto standard based on H264.

Link

Just a few years ago, Greenpeace would regularly call out Apple for their environmental impact. In that time, they’ve managed to climb a few ranks, mostly on the back of their efforts eliminating toxic chemicals like PVC. Kudos to them.

Not to minimize the efforts of other companies high on that list. Nokia, Sony Ericsson, Toshiba, Phillips, LG, Sony, Motorola, and Samsung, you guys rock for not polluting all over the place.

Link

Ars Technica:

ChangeWave queried 4,068 current and potential smartphone consumers last month and noted that a full 21 percent said that they would prefer Android on their next smartphones—a jump of 15 percentage points from the year before. Comparatively, 28 percent of respondents said they would prefer iPhone OS; this makes the iPhone the leader in this category, though this number dropped four percentage points year over year.

Many iPhone developers and Apple enthusiasts are quick to shrug off the Android platform, for a variety of reasons ranging from aesthetic and design, to functionality and developer tools. Many of these criticisms are certainly valid. But iPhone has its own share of problems, and certainly is deficient in many ways to the Android.

With Google’s press conference tomorrow, and CES for the remainder of the week, there will be a lot of focus on the Android platform. It will become a much stronger platform in 2010. It will be interesting to see how Apple responds with iPhone OS 4.0 (which history suggests they’ll probably talk about in March).

Link

The templates that ship with Xcode are not the greatest. Some of them are inconsistent and don’t enforce good coding standards (e.g. missing a dealloc method). Other templates which would be useful just flat out don’t exist (e.g. an NSOperation subclass, or a protocol header file). This project aims to supplement or replace the built-in templates for Xcode to speed up coding and improve the quality of code.

Coding Standard

All files will be processed by Xcode. The generated source files must produce consistent, readable, commented code. The code must have these characteristics:

  • Each file must have a comment block at the top describing the file.
  • Each class must implement its superclass’ designated initializer and dealloc.
  • Stub methods must be organized by their purpose, class or protocol. — Each group must be organized by their class hierarchy, with protocol stubs following. — Each group must be prefaced by a pragma mark naming the class or protocol the methods were implementing. — Clusters of methods (such as relating to KVO) should be organized along the lines above, with a pragma mark.
  • All method implementations should contain a method call to their super implementation if needed.
  • All method implementations should contain a commented out stub line that will signify where to insert their code.
  • All comments must be in the form of two slashes //, and none using the /* */ form. This will allow developers to comment out large blocks of code as needed.

Wish List

  • Different people want different things in their template. For instance, someone may want to include an implementation of observeValue:… for every class. Would be nice to have a template generator application (yeah yeah, very meta) which would make templates customized to the developer.
Link

Richard Esguerra from the EFF:

Apple’s announcement comes nearly a year after Amazon.com’s DRM-free MP3 deals went live, demonstrating that the record labels were holding the DRM card until they could wring business concessions from Apple (in the form of variable pricing). This just underscores that DRM is not really about stopping piracy, but rather about leverage over authorized distributors.

DRM-free music is certainly phenomenal news, but Apple has a long way to go before they can brag about being DRM-free entirely.

Link

I don’t know how true or untrue this rumor is; personally, I’m leaning against it. However, it’s prudent to mention some recent events around SproutCore, the JavaScript framework Apple uses to write the MobileMe web apps and the MobileMe Web Gallery.

Apple has recently made a large commit to the SproutCore project on GitHub, which was stuck getting clearance from Apple’s higher-ups for awhile. Here’s a post from Charles Jolley briefly talking about their 1.0 march, dated 12/5/2008.

I haven’t personally taken a look at what the changes in Bitburger (the name of Apple’s branch) entail, so I’m not sure they’re more geared towards developing an application like anything in iWork. Just food for thought.