Compromising Twitter's client for Android
In order to evaluate the viability of Twitter's "best-effort security" strategy, I decided to see how difficult it would be to obtain the OAuth consumer secret key from Twitter's own official client application for Android. As I expected, it was trivially easy. I used the Astro application on Android to back up the Twitter application to an SD card and then copied it from the SD card to my computer.
The next step was to extract the contents of the Twitter APK package and attempt to figure out which one contained the relevant value. After briefly poking around the contents of the package, I settled on the classes.dex file as the most likely place. I used the strings command at the command line to extract all of the contiguous textual strings from the binary dex file. I used the grep command to filter the output and identify strings of sufficient length. After glancing over the list of potential candidates that I had extracted from the dex file, I was quickly able to find the OAuth consumer key and consumer secret:
Consumer key: 3nVuSoBZnx6U4vXXXXXX Consumer secret: Bcs59EFbbsdF6Sl9Ng71smgStWEGwXXKSjYvPXXXXXX
These keys are particularly significant because Twitter has configured them to enable access to special APIs which aren't generally available yet that can be used to exchange login credentials for an access token—an OAuth flow that is intended for mobile applications but could also be used to bulk-authenticate accounts. As a courtesy to Twitter, I have replaced the last six characters of each key with the letter X so that spammers can't simply copy and paste it out of this article. My decision to not disclose the entire keys is not going to help Twitter much, however, because practically anybody with basic knowledge of command line tools, Android development, and OAuth will be able to access the keys handily. If this is the extent of Twitter's "best-effort" security, we should all be appalled.
After I obtained the keys, I was able to put them in my own client application and use them to authenticate my user account and post a message. The keys come from version 1.0.3 of the official Twitter client for Android, which was published in the Android Market this week. It's very likely that the administrators at Twitter will respond to this article by invalidating the key that I've partially published above and issuing an updated version of the program with a new key. One can only hope that they will at least try to take steps to obscure the key better next time.
Referring to the standard
Twitter's approach to OAuth is obviously misguided, but it gets even crazier when you compare the company's implementation against the actual standard. The OAuth specification itself describes the secret key security issue and says explicitly that implementors should not do what Twitter is trying to do:
"In many applications, the Consumer application will be under the control of potentially untrusted parties. For example, if the Consumer is a freely available desktop application, an attacker may be able to download a copy for analysis. In such cases, attackers will be able to recover the Consumer Secret used to authenticate the Consumer to the Service Provider. Accordingly, Service Providers should not use the Consumer Secret alone to verify the identity of the Consumer."
Part of the problem is that the specification doesn't provide much guidance about what implementors should do instead, which has forced them to improvise. Facebook and Google Buzz have both come up with reasonable solutions and offer desktop-appropriate OAuth authentication flows that do not require a secret key or require the end user to go through a complicated copy/paste process.
Google's relatively pragmatic solution is to allow client applications to supply a bogus placeholder instead of the actual consumer secret key. In every API call where a consumer key and consumer secret are required, the developer simply uses the text string "anonymous" as a stand-in. Google Buzz supports an xoauth_displayname parameter that the application can optionally supply to identify itself, but this is used solely to advertise the program in the user's messages.
Facebook, which has a clean OAuth implementation based on the OAuth 2.0 specification draft, goes a step further than Google and simply allows desktop applications to omit the consumer secret entirely. Getting an application up and running on Facebook tends to be much easier than on many other OAuth-enabled services.
As Google and Facebook have demonstrated, there are obviously reasonable solutions to the key secrecy issue. Twitter continues to stubbornly ignore those solutions despite the serious problems with its own approach.
Twitter's OAuth implementation and open source clients
Requiring third-party developers to embed a consumer secret key in the source code of their Twitter client applications potentially puts free and open source (FOSS) client software at greater risk of key exposure than closed-source client software. The key would be visible as plain text in the source code, where anybody could find it and use it for their own purposes. Indeed, one can already easily find dozens of OAuth consumer secret keys by using Google's code search engine.
Twitter felt that allowing FOSS Twitter clients to use OAuth posed an unacceptable risk. The company warned that it would invalidate any OAuth keys that it found published in the source code of FOSS client applications. This was deeply troubling to the developers who maintain such software, including me. I am the developer behind Gwibber, a GPL-licensed microblogging client that is used in Ubuntu and other Linux distributions.
Twitter initially said that the only real solution for FOSS Twitter client developers is to have each individual end user register their own application key to copy and paste into the program. The process of registering Twitter application keys is somewhat unintuitive because it is intended for application developers. It's simply not reasonable to expect regular end users to walk through those steps. Several prominent FOSS Twitter developers objected to Twitter's position on this issue, including TTYter developer Cameron Kaiser and Spaz developer Ed Finkler.
In response to the concerns raised by the FOSS community, Twitter committed to implementing an alternate OAuth authentication mechanism specifically for FOSS applications. The alternate authentication flow would allow users to register a sub-key that they could paste into the application. It would still involve an extra copy-and-paste step, but it would offer a simpler user interface than the standard key registration system.
It was really a bad idea, one that only became necessary in the first place because of Twitter's misguided requirement that desktop applications use secret keys. Despite promising to have it ready for FOSS client applications, Twitter still has not completed this system. It made it available experimentally to a small handful of developers, but it's not production-ready or intended for widespread use.
By turning off Basic authentication without offering a suitable alternative for FOSS clients, Twitter effectively made it impossible for FOSS client applications to continue functioning normally. This is especially troubling for Linux users, because Adobe AIR (which is used by virtually all cross-platform closed-source Twitter clients) does not always work well on the Linux platform.
Linux users aren't the only ones negatively affected, however. Twitter clients that are developed as browser add-ons are written in JavaScript and are necessarily distributed with their source code available as plain text. This includes some extremely popular Twitter clients, such as ChromedBird.
Most FOSS client developers have simply chosen to embed their keys in their source code with the hope that Twitter won't notice. I was about to give up on Gwibber, but Canonical intervened on my behalf (special thanks to Ken VanDine) and negotiated a compromise with Twitter that will allow Gwibber to continue using the service.
Despite claiming to love open source and using an awful lot of it on the backend, Twitter doesn't seem to care very much at all about FOSS Twitter clients or their users and developers. Finkler expressed frustration yesterday about the ongoing absence of the FOSS OAuth system that Twitter had promised.
It's unclear when or if Twitter will change its OAuth implementation to make it less hostile to FOSS clients. If Twitter does the right thing and eliminates the requirement for desktop applications to use secret keys, it would effectively resolve the problem for FOSS clients.
Bugs and other technical problems
Aside from handling the consumer secret issue poorly, Twitter's OAuth implementation has a number of bugs, defects, and inconsistencies that pose challenges for users and developers.
Third-party developers are finding that it is maddeningly difficult to debug client-side support for Twitter's OAuth implementation because Twitter tends to spit out very generic 401 errors for practically every kind of authentication failure. It doesn't provide enough specific feedback to make it possible for the developer to easily troubleshoot or isolate the cause when authentication is unsuccessful.
This is especially frustrating in situations where authentication is failing because of a bug or defect in Twitter's implementation. For example, authentication will sometimes fail if the system clock on the end user's computer is running slightly fast. This issue has to do with the timestamp that is embedded in the requests, but it's not entirely obvious what causes it to occur.
On the matter of timestamps, the specification itself only says that each API request must have a higher timestamp value than the previous request—a requirement that could obviously wreak havoc if the user ever changed their system clock while using the software, but wouldn't necessarily cause the clock-skew authentication failure that is commonly experienced. Developers can't see exactly how this stuff works on Twitter's side, and it's not adequately documented, so they are left to guess.
Another similar problem is that Twitter's authentication servers will report an authentication failure in cases where the service is simply overburdened and doesn't have sufficient capacity to address the request. Twitter has acknowledged this problem and wants to find a solution, but isn't sure how to fix it and don't know when a fix will be made available.
reader comments
62