Late last night, a tweet was spread far and wide showing that a DMCA notice had blocked a file from being shared on a Dropbox user’s account.

https://twitter.com/darrellwhitelaw/status/450096476060794880

As of this afternoon, it’s seen just shy of 3 thousand retweets.

What was going on? Was Dropbox suddenly doing something sketchy? Were they suddenly lurking around their users’ folders, digging for copyrighted material hiding amongst personal files?

Nope. The system is neither new, nor sketchy. It’s been in place for years, and it’s about as unsketchy as an anti-copyright infringement system can get. It allows Dropbox to block pre-selected files from being shared from person-to-person (thus keeping Dropbox from getting raided by the Feds), without their anti-infringement system having any idea what most of your files actually are.

Before we dive in, a few things to clear up:

• Some interpreted the original tweet to mean that a file just sitting there in a user’s private dropbox had been DMCA’d and blocked. This wasn’t the case. Only when a file is shared from user-to-user (or with the Internet at large) does the DMCA check system come into play. In this case, a share link was generated to be sent over IM.
• Some thought the original file was deleted from the user’s Dropbox — that’s not the case, either. Dropbox just blocks the file from being shared.
• The original author of the tweet has followed up to clarify that he doesn’t think Dropbox is doing anything evil here, and that he just found it interesting — he didn’t intend for it to spread the way it has.

If you know what “file hashing against a blacklist” means, feel free to skip the rest of this post. Dropbox checks the hash of a shared file against a banned list, and blocks the share if there’s a match.

If those words sound like voodoo to you, read on.

How It Works:

In computer science, there’s an incredibly common concept called “hashing”.

It’s used just about everywhere — from allowing web services to check your password without having to actually store your original password, to confirming that a file wasn’t somehow changed as it traveled from user to user.

A hash function, in this case, is just an algorithm which spits out a unique identifier based on what you feed into it.

Hashes are usually just strings of characters. The hash for File A might be “4f2900f2fdfaf”, while the hash for File B might be “dba7b12a19fe9”. Dropbox’s hashes are probably waaaay longer than that (to allow for a higher number of unique hashes), but you get the idea.

With a properly implemented hash function, running the same exact file through the algorithm twice will return the same identifier both times — but changing a file even slightly completely changes the hash. So changing File A by even a few bits should change its hash to something entirely different, like “e3c277c771c8e”.

(This image is a modified version of a public domain image shared through Wikimedia Commons)

This identifier can be used to tell you if a file is exactly the same as another file — but it’s a one way street. The hash couldn’t tell you what that original file is, without you already knowing or having a copy of the file to compare it to.

It might help to think of a hash like a fingerprint. Everyone’s fingerprint is unique, but it can’t be used to identify a person unless you already have a record of that person’s fingerprint to compare it to. Likewise, a hash-based DMCA compliance system can’t tell what a file is, unless it’s exactly the same as a file that has received a takedown request.

When you upload a file to Dropbox, two things happen to it: a hash is generated, and then the file gets encrypted to keep any unauthorized user (be it a hacker or a Dropbox employee) who somehow stumbles it sitting on Dropbox’s servers from easily being able to open it up.

(Note on encryption: Dropbox handles the encryption keys, so they could look at your files if they were legally required to. Their system has checks in place, both physical and technical, to keep employees from poking about your stuff on a whim.)

After a DMCA complaint is verified by Dropbox’s legal team, Dropbox adds that file’s hash to a big blacklist of hashes known to be those corresponding to files they can’t legally allow to be shared. When you share a link to a file, it checks that file’s hash against the blacklist.

If the file you’re sharing is the exact same file that a copyright holder complained about, it’s blocked from being shared with others. If it’s something else — a new file, or even a modified version of the same file — a hash-based anti-infringement system shouldn’t have any idea what it’s looking at.

In other words: at least based on what they’ve stated publicly, Dropbox isn’t actively scanning through your crap on a hunt for copyrighted materials. There’s no human (or even a robot) listening to your MP3s to try and find hot leaked Fergie tracks, or reading through your Harry Potter fanfic collection. They’ve just got a big list of files that they can’t let be shared, and they identify these files in a way that is deliberately blind to what any non-blacklisted files actually are.

Now, none of this is to say the hash-based system is without its security concerns. If required to by the government, for example, Dropbox theoretically could identify any user who had a certain file stored on their account. But the same would hold true for pretty much any cloud-based storage system where the user isn’t handling all of the encryption themselves.

For the record, here’s Dropbox’s official comment on the tweet:

There have been some questions around how we handle copyright notices. We sometimes receive DMCA notices to remove links on copyright grounds. When we receive these, we process them according to the law and disable the identified link. We have an automated system that then prevents other users from sharing the identical material using another Dropbox link. This is done by comparing file hashes. We don’t look at the files in your private folders and are committed to keeping your stuff safe.”