At the last App.net hackathon, I unveiled Apparchy, a proxy server that converts your App.net timelines into data that looks like the Twitter API, which you could then connect to via Twitter’s official iOS apps to post and view to and from App.net. This was a really cool hack, but it suffered from many problems. It relied on a proxy server, which had issues relating to security and privacy, as well as being a single point of failure. If the proxy server went down, everyone’s app broke. Apparchy itself was built to work with Twitter’s official apps, which use a LOT of private methods on Twitter’s server API, and those private APIs changed often from release to release, meaning the app would break if you updated it. It was a big pain to set up, taking many steps that were easy to get wrong. But perhaps the most important and the most philosophical problem that plagued Apparchy was that the early adopter audience of App.net were not the kind of people who embraced Twitter’s official apps. They used Tweetbot or Twitterrific or some other app because it was better suited for their needs or looked better or some other reason.

Even with all these problems, it was fun and fascinating to make. Going into this weekend’s App.net hackathon, I wanted to top it. How? By building something that was just as mindblowingly cool, that also fixed all of those problems.

Since the last hackathon four or five months ago, App.net has been hustling on getting new APIs out, having added the Messaging API and the Files API, among other things. I’ve been dreaming for years of a better chat app with first-class, bulletproof file transfer support, and App.net has all the ingredients for a killer implementation. Similarly, for over a year, I’ve been sitting on the knowledge of the existence of the IMServicePlugIn framework, waiting for an opportunity to use it for something.

And thus, #ProjectAmy was born. App.net messaging integrated natively into Messages for the Mac.

Read More

My Giant Hard Drive: Building a Storage Box with FreeNAS


UPDATE 2/20/2015: This build failed after about 15 months, due to extensive drive failure. By extensive, I mean there were a total of 9 drive replacements, before three drives gave out over a weekend. This correlates closely to data recently published by Backblaze, which suggested 3 TB Seagate drives are exceptionally prone to failure. I’ve replaced these with 6 HGST Deskstar NAS 4TB drives, which were rated highly, and are better suited for NAS environments.

For many years, I’ve had a lot of hard drives being used for data storage. Movies, TV shows, music, apps, games, backups, documents, and other data have been moved between hard drives and stored in inconsistent places. This has always been the cheap and easy approach, but it has never been really satisfying. And with little to no redundancy, I’ve suffered a non-trivial amount of data loss as drives die and files get lost. Now, I’m not alone to have this problem, and others have figured out ways of solving it. One of the most interesting has been in the form of a computer dedicated to one thing: storing data, and lots of it. These computers are called network-attached storage, or NAS, computers. A NAS is a specialized computer that has lots of hard drives, a fast connection to the local network, and…that’s about it. It doesn’t need a high-end graphics card, or a 20-inch monitor, or other things we typically associate with computers. It just sits on the network and quietly serves and stores files. There are off-the-shelf boxes you can buy to do this, such as machines made by Synology or Drobo, and you can assemble one yourself for the job.

I’ve been considering making a NAS for myself for over a year, but kept putting it off due to expense and difficulty. But a short time ago, I finally pulled the trigger on a custom assembled machine for storing data. Lots of it; almost 11 terabytes of storage, in fact. This machine is made up of 6 hard drives, and is capable of withstanding a failure on two of them without losing a single file. If any drives do fail, I can replace them and keep on working. And these 11 terabytes act as one giant hard drive, not as 6 independent ones that have to be organized separately. It’s an investment in my storage needs that should grow as I need it to, and last several years.

Building a NAS took a lot of research, and other people have been equally interested in building their own NAS storage system, so I have condensed what I learned and built into this post. Doing this yourself is not for the faint of heart; it took at least 12 hours of work to assemble and setup the NAS to my needs, and required knowledge of how UNIX worked in order to make what I wanted. This post walks through a lot of that, but still requires skill in system administration (and no, I probably won’t be able to help you figure out why your system is not working). If you’ve never run your own server before, you may find this to be too overwhelming, and would be better suited with an off-the-shelf NAS solution. However, building the machine yourself is far more flexible and powerful, and offers some really useful automation and service-level tools that turn it from a dumb hard drive to an integral part of your data and media workflows.

Read More