Six Years of Lastpass Support Requests : What have I learned

I went through the 38 support tickets I’ve opened with LastPass over the last 6 years to see if it’s really as bad as it feels. Here are some themes that are revealed

  • Lastpass has a persistent and serious problem with product quality and quality assurance. I’ve never used a service or software before with this many problems over this long of a timespan. The breadth of areas where they have user facing bugs is staggering. This is not one product in their suite of products with a bunch of problems. Everything from their browser plugins to pocket, to their website, to their credit monitoring service have all been critically broken for long periods of time over the last 6 years.
  • Lastpass support personnel has never followed up once in 6 years. With every bug that I’ve reported that they’ve confirmed they’ve never once notified me when the bug was fixed unless the fix came within the following day of the report, which only happened in the early years of Lastpass. This causes every user that reports a problem which is confirmed to be a real problem feel as if their report has gone into a black hole and their effort to report the problem was wasted
  • Lastpass takes years (literally years) to fix known bugs. They are either so short on resources or their priorities of favoring new features over a functioning product are so out of whack that known bugs that break core functionality can be acknowledged and remain unfixed for years.

Here is a short summary of the last 6 years of support tickets. I’ve summarized the multi-paragraph bug reports that I’ve sent in (with screenshots and screen capture videos) into a few sentences each to give an idea of the problem. I’ve also summarized their response. Finally I’ve added a subjective rating of how badly they handled the support ticket from 1 to 5, 1 being fine and 5 being as bad as it gets.

Date Type Problem Response How bad
2017-10-25 Pocket bug Pocket segfaults in Ubuntu 16.04 No reply yet 1
2017-10-04 Site bug Received email saying to signup for lastpass families. Clicking the link in email gives page with error “The page you are trying to access is currently closed” We sent an email and then close signups for the feature we invited you to 3
2017-09-13 Browser plugin bug Search bar is no longer focused when vault is launched requiring mousing to click before searching Over a month now and still broken 5
2017-09-10 Site bug Signing up for lastpass credit monitoring results in “ERROR: An unexpected error has occurred” Known issue 4
2017-09-09 Browser plugin bug Export feature doesn’t work Devs notified 4
2017-05-02 Browser plugin bug New saved credentials are saved with a URL of “http://” Devs notified and bug persisted for months 5
2017-04-20 Site bug Changelog is many revisions behind Devs notified 3
2017-04-07 Site bug Downloading the firefox plugin downloads 4.1.44 which the immediately autoupdates to 4.1.45. Please put the current version on the download page Devs notified 4
2017-04-07 Browser plugin bug Browser plugin auto upgrade logs the user out After 13 days of back and forth : It’s because you use yubikey and we can’t fix it 4
2017-04-07 Browser plugin bug Trying to delete a local “equivalent domain” fails and the vault UI becomes unclickable Known issue, please use Chrome 5
2017-04-07 Site bug Lastpass 4 announcement page video is broken Ya, it’s broken 3
2016-08-02 Pocket bug Pocket 2.0.2 crashes with “terminate called after throwing an instance of ‘char const*'” It’s because you use yubikey, can you disable MFA? 4
2016-08-02 Site bug Lastpass pocket product page has a download button which isn’t a button but an image of what the button looks like when you go to the download page. Baffling Still that way to this day 3
2016-06-09 Site bug Download page says version 4.1.7 but link points to 4.1.15a No fix, incorrect 4.1.7 version left on site 3
2016-03-18 Browser plugin bug New in firefox plugin v4 when a plugin autoupdate occurs I’m logged out or I’m prompted with a yubikey prompt mid-session Disable and re-enable MFA and see if that fixes it 3
2016-02-29 Share feature bug Email notification about share updates says to check for checkerboard icon to see change. There is no checkerboard icon in vault Reported to product team 2
2016-02-09 Browser plugin bug Firefox plugin v4 drops characters in vault search field when loading and causes full browser freeze for 3 seconds on vault load You have too many sites in your vault no our problem 5
2015-11-04 Pocket bug Pocket fails on Ubuntu 14.04 with libssl.so.0.9.8: cannot open shared object file Workaround provided to install libssl while bug is fixed 1
2015-08-10 Site bug Status page redirects to blog but should redirect to twitter status account Fixed after 1 day 1
2015-08-10 Site bug Release notes for 3.2.20 say firefox is supported, but there is no firefox download of that version Passed onto devs 2
2015-01-18 Question How can I update my credit card information Answered after 2 days 1
2014-11-03 Support system problem Adding comment to existing support ticket gets Request Denied Said to try a different browser, I gave up 3
2014-10-22 Site problem Adblock Plus added rule which breaks Lastpass “Share” function Devs notified, don’t know if it was fixed 3
2014-08-26 Pocket bug Deleting credential in pocket fails due to API 302 response I commented 2 months later that it was still broken, they fixed it the next day 4
2012-10-25 Browser Plugin Bug Change monospace font used to show passwords as the number 1 and letter l look the same Bug unfixed for ages, at some point it was fixed with no notification 5
2012-04-25 Security question Does malware running on your client result in full lastpass vault compromise despite having MFA? Answered yes in 1 day 1
2012-04-20 Question How can I update my credit card information Answered after 2 days 1
2012-01-04 Security risk How can I use lastpass on my phone and not risk compromise of my data if I lose my phone via a “log me out of all devices” feature on the website? Thanks for the suggestion 2
2012-01-04 Browser Plugin Feature How to I configure a lastpass credential set to identify a given field as a captcha and not automatically submit? You can’t lastpass just has a list of common names for captcha fields, if the name isn’t in their list they can’t tell 3
2011-12-29 Site Bug During outage like today, how can users determine status of lastpass if status page is hosted on lastpass.com? Didn’t understand the problem, then suggested using pocket during outages 4
2011-12-17 Security risk When credit card info changes and annual renewal comes up, billing fails and MFA is disabled. Feedback passed onto devs 3
2011-12-05 Browser Plugin Bug Yubikey prompt text is wrong saying to press yubikey for 2 seconds Bug wasn’t fixed after 11 months, gave up waiting, bug was fixed some years later with no notification 3
2011-12-05 Browser Plugin Bug Entering URL in tab of vault goes to last site in vault, not what’s entered in url bar Fixed and released in 1 day 1
2011-12-05 Browser Plugin Bug Deleting multiple records from vault causes UI glitch where they are re-added due to race condition After 10 months bug was still present, gave up waiting 4
2011-09-24 Browser Plugin Bug Saved credentials were lost Said to try upgrading browser plugin 3
2011-05-10 Site Bug Locked account email states wrong lock duration Bug fixed in 1 day 1
2011-05-05 Outage Lastpass inaccessible because of breach and offline access doesn’t work for yubikey users No response 1
2011-05-05 Documentation Bug Pocket instructions for linux users are confusing After 4 months they say devs disagree, no fix 2

Hose clamps for homebrew beer lines

Typical homebrew beer liquid hose has an outer diameter (OD) of 7/16″ or 0.4375″ or 11mm

Typical homebrew beer gas hose has an outer diameter (OD) of 9/16″ or 0.5625″ or 14mm

A worm gear drive hose clamp size 04 has a diameter

  • when fully open of 5/8″ or 0.62″ or 16mm
  • when close of 1/4″ or 0.25″ or 6mm

Ideally the host clamp is larger than the OD of the hose you’re clamping but not significantly larger so that the shape of clamp is circular and not pinched next to the gear when it’s cinched down.

This aliexpress seller sells hose clamps for cheap. The two sizes that may be ideal here are

  • Type 1: 8mm-12mm
  • Type 2: 10mm-16mm

You could also buy them in bulk from this seller though they look more expensive

Workaround for “No new people were added to the group One person is already a member of the group. The provided email address might be a primary, secondary, or alternate email address of this person.”

If you encounter the Google Groups error

No new people were added to the group
One person is already a member of the group. The provided email address might be a primary, secondary, or alternate email address of this person.

when trying to Direct Add or Invite a user to a Google Group, but there are no users in the Group which could possibly have alternate, primary or secondary email addresses that conflict with the address you’re trying to add, here’s a workaround to this apparent bug in Google Groups.

Change the permissions on the Google Group manifesting this problem to allow people to apply to join the group :

"Manage"... "Permissions"... "Basic Permissions"... "Join the group" : "Anyone can ask"

Then have the user who you couldn’t add or invite, go to the Google Group and click the “Apply for membership” link. Then approve their Join Request and they’re in.

Afterwards, set the “Join the group” permissions back to “Only invited users” if that’s what you had before. Here’s where I reported this bug

I experienced this problem with two different Google Groups and three different users and was able to get them all added by this method.

Mozilla GitHub repository naming convention

I wanted to create a new GitHub repo in the Mozilla GitHub org and went searching for a document defining a naming convention for repository names. I didn’t find anything so I thought I’d look at what names people used and deduce a convention.

  • 928 (100%) Mozilla GitHub Repositories
  • 878 (95%) have no uppercase characters
  • 909 (98%) have no underscore character
  • 517 (56%) have a dash character

Looks like the convention is :

  • Use all lower case
  • Separate words with a dash character if needed

I used PyGitHub to query the GitHub API.

Chef, cookbook versioning and the development cycle

The problem

A common development paradigm is as follows :

  • Joe developer commits code into a feature branch
  • Joe developer deploys that code on his own machine to see if it works
  • Joe developer either continues committing and deploying or he finds code that he likes and merges it into a dev branch
  • At the same time Alice developer is committing code in her feature branch, deploying on her machine and eventually merging into a dev branch
  • The current live production code lives in yet another branch and is deployed on different machines than the one’s Alice and Joe deploy to

Alice and Joe’s code is segregated from each other with the VCS branch model. Their deployed instances of their code are segregated from each other because they’re deploying to different machines. The same is true for production.

If your organization runs a single Chef server for all environments (production, staging, development) and not one Chef server for each environment, there exists a namespacing issue with cookbooks. Whereas above in traditional development each developer has their own deployment environment, here there is a common shared Chef server. The result is that as Joe and Alice develop their code, in order to see it work on their respective machines, they need to upload it to the same Chef server. To retain the fully asynchronous development model traditionally available to developers using VCS and that have their own development machine, a separate Chef server would need to be run for each feature branch and environment. A promotion model would need to be created to enable promoting cookbooks and other Chef data (databags, environments, run_lists) from on Chef server to another (e.g. from a feature branch to the dev branch, or from the staging branch to the production branch)

Chef does provide one variable to differentiate different branches of a cookbook from each other, the cookbook version. Each cookbook has a metadata file containing information on the cookbook including the version of the cookbook in the 1.2.3 form. One complexity to the way Chef handles cookbook versions is that, by default, nothing prevents you from uploading a modified cookbook with the same version as the existing cookbook on the Chef server, thereby changing the functionality of that Cookbook at the version without a change in version number.

More information

Freeze

Chef provides the ability to “freeze” cookbooks which flags them such that the Chef server won’t allow you to overwrite an existing version of that cookbook (unless you pass it a “force” option)

Environment Based Cookbook Version Constraints

Chef also allows you to specify what version or versions of a given cookbook are allowed to be used in a given environment.

Solutions

Multi chef server

You could make the Chef server provisioning method very streamlined and enable anyone to spine up their own ephemeral Chef server easily, synchronize it’s datastore with a parent version, develop and deploy to your own Chef server, and destroy it when the feature is complete and merged.

This would be complex and hard to implement.

Chef solo

You could develop on Chef Solo and not deal with the constrained cookbook namespace until you merge your feature branch into the dev branch

This prevents you from using any of the Chef Server functionality (the ability to search across your server base, databags, etc) since it wouldn’t be present in Chef Solo.

Namespacing with version number : Mapping the version fields

You could dedicate each of the 3 version fields (major, minor, patch) to a different environment to avoid collision.

This constrains you to 3 environments and doesn’t allow for feature branches. It also requires that, in development, every time a developer wants to test his change to see if it works, the cookbook version needs to be incremented. This results in a large proliferation of different versions of a given cookbook which doesn’t fit well with the Chef model. It also violates semantic versioning standards

Namespacing with version number : Assigning version ranges

You could designate ranges of versions for each environment. For example production gets 9000.0.0 – 9999.0.0, staging gets 8000.0.0 – 8999.0.0, dev gets 7000.0.0 – 7999.0.0 and each developer gets their own range for feature branches.

This method results in foreign looking version numbers and again violates semantic versioning standards. It also bounds the maximum number of feature branches that you have.

Namespacing with version number : Protect only what you need to

You could

  • modify the Chef server, removing the REST interface for the cookbook upload function
  • create a redirect on the Chef server that reveals a new REST interface which points to the cookbook upload function
  • create a knife plugin that extends the environment feature to add a “set cookbook version”
    • The new knife plugin would
    1. See if the environment being affected is production or not production
    2. If it’s not production, set the cookbook version for the environment
    3. If it is production, append the tuple of the cookbook and cookbook version being set to a list in a databag indicating that this cookbook at this version has ever run in production. Next, set the production environment cookbook version to the one indicated. We’ll use this later to know if a given cookbook at a given version has ever run in production and if so to protect it from being changed.
  • create a knife plugin that extends the cookbook upload functionality and talks to this new API interface
    • The new knife plugin would
    1. Query the Chef server to see if it has a cookbook of the same version as the one being uploaded already
    2. If it doesn’t, do a couple git queries of the local git repo to determine the name of the branch that it’s on and the committer name of the most recent commit and add those into the cookbook metadata file. Then upload the cookbook and revert the local metadata file changes.
    3. If there is a conflicting version, check the databag mentioned above to see if cookbook at the give version, for which there’s a copy already on the Chef server, is now running or has ever been run in production.
    4. If it has, refuse to make the environment cookbook version change. This prevents anyone from accidentally changing production
    5. If it hasn’t, upload the cookbook to the Chef server replacing it’s existing cookbook version with the new contents. Read the branch name and last committer from the cookbook metadata, check to see if it’s different than the current users branch and last committer name and if so report to the user that they’ve just overwritten that persons cookbook at that version.

Upsides :

  • Nobody can accidentally change production behavior by uploading a cookbook. The only way to change production behavior is to change the production cookbook version constraint which should be obvious as to it’s impact.
  • Rapid development iteration is possible without concern over impacting other developers or environments. The developer can iterate changes over and over without incrementing cookbook version, testing each time he commits and uploads. The upload function could be linked to the commit as a commit hook.
  • The complexity of enabling this model is low (server side redirects, 2 simple knife plugin extensions).
  • Cookbook versioning can conform to semantic versioning standards because the versions are chosen by humans.

Downsides :

  • There is nothing to prevent any developer, QA or devops engineer from changing production behavior by changing the production cookbook version constraint. Depending on how collaborative or combative these groups are with each other, mere policy constraints saying that developers shouldn’t change production behavior may not be enough.
  • If a developer created their own knife plugin to interact with the new cookbook upload REST interface that didn’t block overwriting an existing cookbook of a version being used in production, they could change production behavior by overwriting the contents of a cookbook running in production. Again, this comes down to the place on the collaborative combative spectrum that your organization lies.
  • Developers can unintentionally change the behavior of other development or staging environments if they commit to the same cookbook version as one that somebody else is developing on. This would happen if two developers begin working on feature branches of the same cookbook around the same time and choose the same version number. The developer will be notified that this has happened but not prevented from doing it. In this case, keep in mind, no code is lost since everything is in VCS, it merely means that one developer could momentarily step on another developers feet.

Summary

These are some of the solutions that I’ve come up with. I don’t feel that any of them are very elegant. I also could be misunderstanding the problem in which case there may be a whole other class of solutions available that I haven’t imagined. What do you think?