Time for Security!


Starting next Monday I'll begin my new job as a Cloud Security Engineer here at Mozilla! I'm really excited about this new role. I get to work with some super talented guys, many of whom I'm already good friends with. Julien, Guillaume, Michal, Tinfoil, Joe and some newer folks on the team that I'm getting to know. The job will involve bringing my development and operations background to bear and learning the ins and outs of security engineering.

Tomorrow will be my two year anniversary at Mozilla. I've worked in the operations team of our Cloud Services organization for most of that time (except for a short stint in the Identity Engineering group). I'll still get to see and work with the guys on my old team all the time so there's not much bitter in the bittersweetness of this change.

On to new challenges. Looking forward to it.

Marine Technology


This morning was very foggy. In my daily ferry trip to work I learned a little about marine technology. I had assumed that, here in the 21st century, ships, like the ferry I ride to work, used some combination of high resolution GPS, radar, and maybe radio telemetry triangulation to identify where the ship was and where everything else in the bay was (other ships, the bridge pylons, etc.) I found out this morning that this must not be the case. As we trucked along, at a pretty good clip, across the bay with zero visibility of any kind, all of a sudden, the engines go from about 3/4 of full thrust to zero thrust. The whole boat lunges forward as we rapidly decelerate (due to water resistance) and out of the pea soup appears the massive bay bridge center anchorage dead ahead. The captain turns the boat sharply to the left and we swing around it and accelerate again. I'm glad our captain has good reflexes.

Later, upon landing and walking to work, the entire embarcadero was echoing with different ships fog horns being blasted.

It turns out that the two methods employed these days to deal with fog are slamming on the brakes if you see something ahead of you and making a ton of noise in hopes that someone else out there hears you and doesn't crash into you. I thought we were living in the future, what happened?

Why Lastpass is a great piece of software that sucks

Update : I've added comments to sections where the release of Lastpass 3.0 and subsequent versions have affected the issue that I've identified. Look for those updates in bold. I've moved all issues that have been resolved into a section at the end for historical purposes.

I am a big proponent and supporter of Lastpass. I've used for about 5 years now and have many hundreds of credentials in my Lastpass vault. At the moment I think it's the best software out there for password management. Unfortunately it has a lot of problems. Here are some of them


  • Submitting bug reports that are subsequently confirmed to indeed be bugs, almost always fails to result in a bugfix. Of the 12 bugs I've reported, all have been ignored but 2. Below you'll see some of the problems with Lastpass with the date I originally reported the bug and the fact that it continues to be broken up to now.
  • Lastpass has a status page hosted at https://lastpass.com/status.php Understandably, when they have an outage, the status page is inaccessible since their site is down. The last time they had an outage I opened a support request asking how customers should determine what's going on when the status page is inaccessible (always when there's an outage) and when they don't post anything to twitter. Instead of addressing the question about status notification, they tried to explain to me why I really shouldn't have experienced an outage "your browser should have automatically failed over to the next on the list with a relatively small timeout" and "Accessing stored passwords is always possible even if LastPass is offline if you utilize the extension which is what we intend". They totally missed the point and I, to this day, don't know how to determine the status of an outage when they happen.


  • Deleting multiple records from your vault causes weird behavior. Update : Lastpass 3.0 doesn't solve this problem but it appears that the speed improvements in the vault UI in Lastpass 3.0 has made the issue less pronounced. I reported this bug December 2011, got no resolution, followed up October 2012 and got the response of "Quite honestly, I am not sure when we will get to it"

    There is a weird behavior when deleting multiple records from lastpass. Here's how to reproduce it :

    1. Create a handful of records. For me this usually results from trying to come up with a password that conforms to some sites unspecified password requirements. This results in a handful (let's say 5) "Generated Password for foo" records.
    2. Go to your lastpass vault inside firefox : chrome://lastpass/content/home.xul
    3. Click "Delete" on the first record you want to delete. When prompted in the modal window that pops up, click "Yes". You'll see the record disappear as expected from the vault.
    4. Immediately click "Delete" on the next record and again click "Yes"
    5. You'll now see the second record disappear as expected, then re-appear for a second, and then disappear for a second time.

    The expected behavior here is for the record, once deleted, to disappear permanently. I'm assuming this is being caused by some kind of race condition between the vault checking back with the lastpass servers for updates and the "delete" action propagating to the lastpass servers.

    If you have trouble reproducing this just do it with 2 or 3 records in a row.

    I encounter this pretty frequently since usually when I'm deleting records, I'm deleting multiple ones as I clean cruft out of lastpass.

  • The monospace font that Lastpass uses for the password field makes the lowercase letter "L" and the number "1" indistinguishable. Update : Lastpass 3.2.20 claims to address this issue. I'll upgrade and check it out Here's a screen shot of it. To work around this whenever I'm viewing a password, for example to type into my phone or some flash interface that Lastpass can't deal with, I have to copy the password into a text editor to be able to distinguish between the characters. I reported this bug October 2012 and got "I will pass your feedback to the development team for their review for future releases." which I'm confident means that it will be ignored.

  • Intermittently when creating new credential sets, after creating them, they fail to appear in the vault. This is not only annoying but also really scary since the impossible-to-remember password you just set on whatever website you were creating an account on is now apparently gone. Logging fully out of lastpass and back in or waiting some period of time for whatever server client interaction to happen sometimes fixes it. I reported this in September 2011. They said to upgrade my Lastpass browser plugin which I did. This still occurs to this day intermittently.

  • I got tripped up by their instructions on installing Lastpass on Linux and reported my confusion with their documentation. They responded with "I mentioned this to developers, but they felt that this step was common sense if you're a linux user. (ie, it'd be like telling a windows user to double-click on a program to launch it)" and left the instructions in their ambiguous state.

  • When you log into Lastpass in your browser plugin it starts by downloading your encrypted blob of data. It then decrypts the blob locally on your machine in javascript. While this whole process happens, which when you have 600 credentials and secure notes can take some amount of time, the page used to display text as if your vault is empty and encourages you to add credentials into it. Now it merely displays an empty vault. As a user, this is pretty scary to see a page where you expect to see your credentials and instead you see an empty vault. They should be displaying a progress bar or a please wait while they download and decrypt the blob.

  • In your browser, when you go to a site that has a password field, and you're not yet logged into lastpass, it pops up a notification bar that says "Simplify your life, use Lastpass to capture this site's login" and a button that says "Save Passwords with Lastpass". This language makes sense if you're on a page for which you don't already have a stored credential. The much more likely case of coming to a site with a login when you haven't yet logged into lastpass is that you're going to the site to login with your existing credentials stored in lastpass. This language should reflect that fact that it's possible and likely that you're not there to create a new credential set but instead to login into Lastpass so you can login to the site you're on.

Browser Plugin Interaction

  • There's no way to tell Lastpass that a given field is a CAPTCHA. Lastpass has some heuristic to detect a CAPTCHA in a login page and display a useful UI behavior instructing you to fill it out. If that heuristic fails and it doesn't understand that a field is a CAPTCHA, there's no way to tell it to treat that field as one. Upon asking how Lastpass determines that a field is a CAPTCHA, they report that "We determine this via field name." and helpfully offer to solve the problem with "If you let us know what this is on the site, and it's likely to come up again, we can look into adding it as a possible field name that would pop the security notification." The way to solve this problem is not to wait for users to report sites that have unexpected field names for their CAPTCHAs and add them into some list stored in Lastpass. This is a totally unscalable, poor user experience approach. Of course after giving them the details of the web page that had a CAPTCHA with a field name they hadn't predicted (I mean how could they have?) they responded with "I will put this on the list of suggestions and hopefully our dev team can add it in." Obviously this never happened.
  • When you encounter a site that uses basic or digest HTTP authentication (where the login window pops up in front of the site and you log into your browser instead of typing your password into a field on a web page), Lastpass has a checkbox that you can check to save the credentials that you're typing in. This makes sense. The checkbox state however is sticky. Once you save a credential set for a site that uses basic or digest auth, the next time you come to some other site that has basic or digest auth, and Lastpass fills in your username and password for you, the checkbox is still checked and so Lastpass saves a second copy of the credentials set. As long as this checkbox is checked Lastpass continues saving duplicate credential sets over and over again. This UI is absurd. Lastpass knows when you're going to a site for which it has credentials and when you're going to a new site. Why present a checkbox in a dialog on a site where you already have credentials indicating that you want to save a duplicate copy of those credentials?!
  • Lastpass uses a combination of settings called "Equivalent Domains" and "URL rules" along with it's internal rules to decide what credentials go to what sites. These are confusing and difficult to use. I frequently get into a situation (in fact I'm stuck in one right now) where I can't get Lastpass to offer the credential set for the site I'm on. I have to pull it up in my vault, and copy and paste it into the form. This is especially bad when it happens in the basic/digest HTTP auth pop up window. In that case the plugin will indicate that you've got multiple matching credential sets and when you click the drop down to pick one, it displays only one.


  • Lastpass charges $12 a year for premium service which gives a user access to using a second factor of authentication. In any given year the likelihood that one will have to change one's credit card number due to fraud, it being lost, changing cards, etc. is pretty high. When the annual auto charge comes up, and fails due to the card number having changed, Lastpass decides that the appropriate way to deal with this is not to notify the customer, wait some window of time and then disable their two-factor auth. Instead they silently disable two-factor auth, opening your account up to attack and then later notify you that they're having trouble charging your credit card. I reported this problem first in December 2011 when it happened the first time then again in April 2012. I'm sure the next annual renewal if my credit card has changed it will fail again and disable two-factor on my account.

    Recently my wife's lastpass premium account came up for renewal. It was configured for yearly auto-renewal against our credit card. A few months back we had to change our credit card number because the bank sent us a new one due to some security compromise. We forgot to put this new number in google checkout/lastpass so that the auto-renewal would work.
    Problem is, what happened when the renewal date came up and the charge didn't work (because the old card number was invalid), is that the 2-factor yubikey authentication was just disabled. No email warning of this fact to the user, just disabled.
    To have a billing issue open up an account to attack (by having only one factor) without notifying the user, is a risky security practice.
    Some better solutions would be :

    • Locking the account down until the user either logs in and disables 2-factor auth, or re-enrolls in lastpass premium with a new credit card
    • An email to the security email address of the users account warning them that 2-factor has been disabled
    • An email to the security email address stating that you tried to charge their card with no luck and 2-factor auth will be disabled in 48 hours, 5 days, etc. if the billing issue isn't corrected. Really anything other than silently disabling 2-factor auth.


  • The Lastpass mobile applications for Android, IOS and others are unusable from a security standpoint, here's why :
    • To have a secure Lastpass vault, you must have a strong master password
    • If you were required to type this strong password on your phone each time you wanted to access a password from your vault to log into a site, it would be too onerous and people wouldn't use the app.
    • Consequently, when you log into the Lastpass app on your phone, it decrypts your hosted blob of passwords and stores them on the phone. These cleartext passwords stay on the phone until you log out.
    • This means that if at any time, your phone gets lost or stolen, all of your credentials are accessible in plaintext on the phone.
    • Lastpass provides a function on your account that says "Kill other sessions on login". I asked if the mobile apps do some kind of polling to know to kill their session if you log in at another machine and it turns out they don't (understandably because of the battery hit for doing so).
    • Sadly this means there is no way to safely and conveniently access your lastpass vault from your phone.
    • I asked them to enable an option to poll and got back "We may enable some sort of option in a future version to set a polling time for the app or at all." and as you can imagine, no such option has appeared.


  • Lastpass provides a sharing feature that lets you share credentials with other Lastpass users. I use this a lot and it's very helpful. I routinely get notifications that shared credential sets have been changed by the other user. Here's how that interaction goes :
    • I get an email that says "LastPass Shared Item Update Notification" and says "LastPass items you have shared have been updated by other users. Please click the below link to login to your LastPass Vault. Login to your LastPass Vault. Any items in your vault that have a icon next to them indicate that they have outstanding updates. Click the icon to view, accept, or delete the updates"
    • I click the link and it opens up my Vault
    • I scroll down pages and pages of credentials (600+) searching for the one that has a little icon next to it indicating it has pending changes from someone who I've shared it with
    • I find it and click the icon and it brings up a dialog showing something like 3 or 9 changes.
    • I click on one of the changes and it says that there is no change and the credential is the same.
    • I continue and click on all of them to find that the credential hasn't changed?
    • I'm not sure how to identify all the ways in which this sucks

Other stuff

Issues that have been resolved and are no longer an issue

  • There are 2 Lastpass Vault interfaces. Update : Lastpass 3.0 does a good job at unifying the look and feel of the locally hosted vault and the lastpass.com website vault. One hosted locally in your browser and one on the Lastpass website. Unfortunately the "Settings" interface for Lastpass only works in the Vault interface on the website, not in the browser. Their solution? Put the "Settings" interface inside an iframe in your localy hosted browser Vault interface. Why do we need two different Vault interfaces, each with slightly different UIs? Why does the user need to use one Vault for some actions and the other vault for other actions?

  • Update : Lastpass cleaned up their website at the end of October 2013 and it looks great now. Previously the Lastpass website was a hilarious explosion of confusing imagery. Go check it out, I'll wait here : https://web.archive.org/web/20131024014857/https://lastpass.com/.

  • Update : With the lastpass October 2013 website redesign these ads for premium no longer display. While logged into Lastpass, when you go the lastpass.com website, it's aware that you're logged in and displays at the top a button that says "Jon Doe's Vault ->". So the website understands that you're logged in and already a Lastpass user. It also knows, based on your username and the fact that you're logged in, if you're an existing lastpass premium (paying) customer. Why then would you display ads asking people to "Get Lastpass Premium! ($12 a year, that's $1 a month)"? Why not display a useful page to customers since you know that they're already customers, instead of a page of ads and attempts to convince the user to use Lastpass. They already use Lastpass, leave them alone.

  • Update : This message has been fixed in Lastpass 3.0 The 2-factor yubikey dialog box that prompts you for your yubikey one time password says to "touch-button for 2 seconds". Unfortunately doing so will never work because with the yubikey you have to touch it for less than 2 seconds. More than that and it types out your static password, not your one time password. I reported this UI bug in December 2011, checked back in October 2012 and got "It's still on our list to review and update". It would take 2 minutes to fix. Maddening.

    When logging into lastpass with 2-factor authentication enabled using a Yubikey, a window pops up with the field to enter the Yubikey one-time-password into. In that window you'll find the following text :

    1. Insert your YubiKey in the USB-port with the USB-contact facing upward
    2. Wait until your YubiKey touch-button shines with a steady light
    3. Hold your fingertip on the touch-button for 2 seconds"

    The yubikey doesn't use a 2 second button touch though. If you go to the yubikey manual : http://static.yubico.com/var/uploads/pdfs/YubiKey_Manual_2010-09-16.pdf On page 20, you'll see that for most yubikeys (mine included) that have the OTP configuration used with lastpass set in the primary position, need a button touch of between 0.3 and 1.5 seconds. Even keys that are configured with the OTP used for lastpass in the secondary position also wouldn't use a 2 second press, they take a 2.5 to 5 second press.

    I'd recommend you change the text to something like "Hold your fingertip on the touch-button for a half second"

    Here's the text from the manual explaining the default behavior :

    Ensure that the cursor is placed in a valid input field and touch the button with a finger tip and hold steady
    for approx 0.5 seconds and the OTP string is emitted. The indicator will then be turned off for approx 2 seconds
    where the touch button is disabled to prevent multiple triggers.

    And here's the text from the manual talking about yubikeys with a second configuration enabled :

    Yubikey 2 supports an optional second configuration. This allows the Yubikey to be used for multiple services where
    both configurations are completely separated from each other. A typical usage is that one configuration is used for
    a remote service and one for a local service in static mode. If both configurations are set, the trigger behavior is
    slightly different as the user must select which OTP configuration that is desired:
    Short press 0.3 - 1.5 seconds) and release - OTP from configuration

    1 is yielded Long press (2.5 - 5 seconds) and release - OTP from configuration

    2 is yielded


Sometimes I'm briefly shocked by the change in technology that has occurred in such a short time. I'm watching a video of an intro to git (a software version control system) and in it the guy, while describing how git is different than other version control systems, says "You can work when you're off the internet entirely; so if you're working on an airplane or something." and I think, ya that makes perfect sense. Then I think, wow, the only time I don't have the ability to be digitally connected to billions of devices is while I'm on a plane. Pretty much all other places and times I'm connected. Head 'asplode.