How to increase the session expiration time in Subsonic

Here’s how you can increase the length of a session in Subsonic from the default of 30 minutes to something longer. In this example I’m increasing it to 2 weeks (20160 minutes).

mkdir workspace
cd workspace
sudo cp /usr/share/subsonic/subsonic-booter-jar-with-dependencies.jar /usr/share/subsonic/subsonic-booter-jar-with-dependencies.jar.orig
unzip /usr/share/subsonic/subsonic-booter-jar-with-dependencies.jar
cat > webdefault.patch <<End-of-message
--- org/mortbay/jetty/webapp/webdefault.xml.orig  2014-06-11 10:06:12.000000000 -0700
+++ org/mortbay/jetty/webapp/webdefault.xml 2014-06-11 10:06:39.000000000 -0700
@@ -323,7 +323,7 @@

   <!-- ==================================================================== -->
   <session-config>
-    <session-timeout>30</session-timeout>
+    <session-timeout>20160</session-timeout>
   </session-config>

   <!-- ==================================================================== -->
End-of-message
patch -p0 webdefault.patch && rm webdefault.patch
sudo service subsonic stop
sudo zip -r /usr/share/subsonic/subsonic-booter-jar-with-dependencies.jar ./
sudo service subsonic start

Time for Security!

Locked

Starting next Monday I’ll begin my new job as a Cloud Security Engineer here at Mozilla! I’m really excited about this new role. I get to work with some super talented guys, many of whom I’m already good friends with. Julien, Guillaume, Michal, Tinfoil, Joe and some newer folks on the team that I’m getting to know. The job will involve bringing my development and operations background to bear and learning the ins and outs of security engineering.

Tomorrow will be my two year anniversary at Mozilla. I’ve worked in the operations team of our Cloud Services organization for most of that time (except for a short stint in the Identity Engineering group). I’ll still get to see and work with the guys on my old team all the time so there’s not much bitter in the bittersweetness of this change.

On to new challenges. Looking forward to it.

Marine Technology

San_Francisco_Oakland_Bay_Bridge_Zan_810

This morning was very foggy. In my daily ferry trip to work I learned a little about marine technology. I had assumed that, here in the 21st century, ships, like the ferry I ride to work, used some combination of high resolution GPS, radar, and maybe radio telemetry triangulation to identify where the ship was and where everything else in the bay was (other ships, the bridge pylons, etc.) I found out this morning that this must not be the case. As we trucked along, at a pretty good clip, across the bay with zero visibility of any kind, all of a sudden, the engines go from about 3/4 of full thrust to zero thrust. The whole boat lunges forward as we rapidly decelerate (due to water resistance) and out of the pea soup appears the massive bay bridge center anchorage dead ahead. The captain turns the boat sharply to the left and we swing around it and accelerate again. I’m glad our captain has good reflexes.

Later, upon landing and walking to work, the entire embarcadero was echoing with different ships fog horns being blasted.

It turns out that the two methods employed these days to deal with fog are slamming on the brakes if you see something ahead of you and making a ton of noise in hopes that someone else out there hears you and doesn’t crash into you. I thought we were living in the future, what happened?

Chef, cookbook versioning and the development cycle

The problem

A common development paradigm is as follows :

  • Joe developer commits code into a feature branch
  • Joe developer deploys that code on his own machine to see if it works
  • Joe developer either continues committing and deploying or he finds code that he likes and merges it into a dev branch
  • At the same time Alice developer is committing code in her feature branch, deploying on her machine and eventually merging into a dev branch
  • The current live production code lives in yet another branch and is deployed on different machines than the one’s Alice and Joe deploy to

Alice and Joe’s code is segregated from each other with the VCS branch model. Their deployed instances of their code are segregated from each other because they’re deploying to different machines. The same is true for production.

If your organization runs a single Chef server for all environments (production, staging, development) and not one Chef server for each environment, there exists a namespacing issue with cookbooks. Whereas above in traditional development each developer has their own deployment environment, here there is a common shared Chef server. The result is that as Joe and Alice develop their code, in order to see it work on their respective machines, they need to upload it to the same Chef server. To retain the fully asynchronous development model traditionally available to developers using VCS and that have their own development machine, a separate Chef server would need to be run for each feature branch and environment. A promotion model would need to be created to enable promoting cookbooks and other Chef data (databags, environments, run_lists) from on Chef server to another (e.g. from a feature branch to the dev branch, or from the staging branch to the production branch)

Chef does provide one variable to differentiate different branches of a cookbook from each other, the cookbook version. Each cookbook has a metadata file containing information on the cookbook including the version of the cookbook in the 1.2.3 form. One complexity to the way Chef handles cookbook versions is that, by default, nothing prevents you from uploading a modified cookbook with the same version as the existing cookbook on the Chef server, thereby changing the functionality of that Cookbook at the version without a change in version number.

More information

Freeze

Chef provides the ability to “freeze” cookbooks which flags them such that the Chef server won’t allow you to overwrite an existing version of that cookbook (unless you pass it a “force” option)

Environment Based Cookbook Version Constraints

Chef also allows you to specify what version or versions of a given cookbook are allowed to be used in a given environment.

Solutions

Multi chef server

You could make the Chef server provisioning method very streamlined and enable anyone to spine up their own ephemeral Chef server easily, synchronize it’s datastore with a parent version, develop and deploy to your own Chef server, and destroy it when the feature is complete and merged.

This would be complex and hard to implement.

Chef solo

You could develop on Chef Solo and not deal with the constrained cookbook namespace until you merge your feature branch into the dev branch

This prevents you from using any of the Chef Server functionality (the ability to search across your server base, databags, etc) since it wouldn’t be present in Chef Solo.

Namespacing with version number : Mapping the version fields

You could dedicate each of the 3 version fields (major, minor, patch) to a different environment to avoid collision.

This constrains you to 3 environments and doesn’t allow for feature branches. It also requires that, in development, every time a developer wants to test his change to see if it works, the cookbook version needs to be incremented. This results in a large proliferation of different versions of a given cookbook which doesn’t fit well with the Chef model. It also violates semantic versioning standards

Namespacing with version number : Assigning version ranges

You could designate ranges of versions for each environment. For example production gets 9000.0.0 – 9999.0.0, staging gets 8000.0.0 – 8999.0.0, dev gets 7000.0.0 – 7999.0.0 and each developer gets their own range for feature branches.

This method results in foreign looking version numbers and again violates semantic versioning standards. It also bounds the maximum number of feature branches that you have.

Namespacing with version number : Protect only what you need to

You could

  • modify the Chef server, removing the REST interface for the cookbook upload function
  • create a redirect on the Chef server that reveals a new REST interface which points to the cookbook upload function
  • create a knife plugin that extends the environment feature to add a “set cookbook version”
    • The new knife plugin would
    1. See if the environment being affected is production or not production
    2. If it’s not production, set the cookbook version for the environment
    3. If it is production, append the tuple of the cookbook and cookbook version being set to a list in a databag indicating that this cookbook at this version has ever run in production. Next, set the production environment cookbook version to the one indicated. We’ll use this later to know if a given cookbook at a given version has ever run in production and if so to protect it from being changed.
  • create a knife plugin that extends the cookbook upload functionality and talks to this new API interface
    • The new knife plugin would
    1. Query the Chef server to see if it has a cookbook of the same version as the one being uploaded already
    2. If it doesn’t, do a couple git queries of the local git repo to determine the name of the branch that it’s on and the committer name of the most recent commit and add those into the cookbook metadata file. Then upload the cookbook and revert the local metadata file changes.
    3. If there is a conflicting version, check the databag mentioned above to see if cookbook at the give version, for which there’s a copy already on the Chef server, is now running or has ever been run in production.
    4. If it has, refuse to make the environment cookbook version change. This prevents anyone from accidentally changing production
    5. If it hasn’t, upload the cookbook to the Chef server replacing it’s existing cookbook version with the new contents. Read the branch name and last committer from the cookbook metadata, check to see if it’s different than the current users branch and last committer name and if so report to the user that they’ve just overwritten that persons cookbook at that version.

Upsides :

  • Nobody can accidentally change production behavior by uploading a cookbook. The only way to change production behavior is to change the production cookbook version constraint which should be obvious as to it’s impact.
  • Rapid development iteration is possible without concern over impacting other developers or environments. The developer can iterate changes over and over without incrementing cookbook version, testing each time he commits and uploads. The upload function could be linked to the commit as a commit hook.
  • The complexity of enabling this model is low (server side redirects, 2 simple knife plugin extensions).
  • Cookbook versioning can conform to semantic versioning standards because the versions are chosen by humans.

Downsides :

  • There is nothing to prevent any developer, QA or devops engineer from changing production behavior by changing the production cookbook version constraint. Depending on how collaborative or combative these groups are with each other, mere policy constraints saying that developers shouldn’t change production behavior may not be enough.
  • If a developer created their own knife plugin to interact with the new cookbook upload REST interface that didn’t block overwriting an existing cookbook of a version being used in production, they could change production behavior by overwriting the contents of a cookbook running in production. Again, this comes down to the place on the collaborative combative spectrum that your organization lies.
  • Developers can unintentionally change the behavior of other development or staging environments if they commit to the same cookbook version as one that somebody else is developing on. This would happen if two developers begin working on feature branches of the same cookbook around the same time and choose the same version number. The developer will be notified that this has happened but not prevented from doing it. In this case, keep in mind, no code is lost since everything is in VCS, it merely means that one developer could momentarily step on another developers feet.

Summary

These are some of the solutions that I’ve come up with. I don’t feel that any of them are very elegant. I also could be misunderstanding the problem in which case there may be a whole other class of solutions available that I haven’t imagined. What do you think?

Why Lastpass is a great piece of software that sucks

Update : I’ve added comments to sections where the release of Lastpass 3.0 has affected the issue that I’ve identified. Look for those updates in bold. I’ve moved all issues that have been resolved into a section at the end for historical purposes.

I am a big proponent and supporter of Lastpass. I’ve used for about 3 years now and have many hundreds of credentials in my Lastpass vault. At the moment I think it’s the best software out there for password management. Unfortunately it has a lot of problems. Here are some of them

Support

  • Submitting bug reports that are subsequently confirmed to indeed be bugs, almost always fails to result in a bugfix. Of the 12 bugs I’ve reported, all have been ignored but 2. Below you’ll see some of the problems with Lastpass with the date I originally reported the bug and the fact that it continues to be broken up to now.
  • Lastpass has a status page hosted at https://lastpass.com/status.php Understandably, when they have an outage, the status page is inaccessible since their site is down. The last time they had an outage I opened a support request asking how customers should determine what’s going on when the status page is inaccessible (always when there’s an outage) and when they don’t post anything to twitter. Instead of addressing the question about status notification, they tried to explain to me why I really shouldn’t have experienced an outage “your browser should have automatically failed over to the next on the list with a relatively small timeout” and “Accessing stored passwords is always possible even if LastPass is offline if you utilize the extension which is what we intend”. They totally missed the point and I, to this day, don’t know how to determine the status of an outage when they happen.

UI

  • Deleting multiple records from your vault causes weird behavior. Update : Lastpass 3.0 doesn’t solve this problem but it appears that the speed improvements in the vault UI in Lastpass 3.0 has made the issue less pronounced. I reported this bug December 2011, got no resolution, followed up October 2012 and got the response of “Quite honestly, I am not sure when we will get to it”

    There is a weird behavior when deleting multiple records from lastpass. Here’s how to reproduce it :

    1. Create a handful of records. For me this usually results from trying to come up with a password that conforms to some sites unspecified password requirements. This results in a handful (let’s say 5) “Generated Password for foo” records.
    2. Go to your lastpass vault inside firefox : chrome://lastpass/content/home.xul
    3. Click “Delete” on the first record you want to delete. When prompted in the modal window that pops up, click “Yes”. You’ll see the record disappear as expected from the vault.
    4. Immediately click “Delete” on the next record and again click “Yes”
    5. You’ll now see the second record disappear as expected, then re-appear for a second, and then disappear for a second time.

    The expected behavior here is for the record, once deleted, to disappear permanently. I’m assuming this is being caused by some kind of race condition between the vault checking back with the lastpass servers for updates and the “delete” action propagating to the lastpass servers.

    If you have trouble reproducing this just do it with 2 or 3 records in a row.

    I encounter this pretty frequently since usually when I’m deleting records, I’m deleting multiple ones as I clean cruft out of lastpass.

  • The monospace font that Lastpass uses for the password field makes the lowercase letter “L” and the number “1″ indistinguishable. Update : Lastpass 3.0 does not solve this issue Here’s a screen shot of it. To work around this whenever I’m viewing a password, for example to type into my phone or some flash interface that Lastpass can’t deal with, I have to copy the password into a text editor to be able to distinguish between the characters. I reported this bug October 2012 and got “I will pass your feedback to the development team for their review for future releases.” which I’m confident means that it will be ignored.

  • Intermittently when creating new credential sets, after creating them, they fail to appear in the vault. This is not only annoying but also really scary since the impossible-to-remember password you just set on whatever website you were creating an account on is now apparently gone. Logging fully out of lastpass and back in or waiting some period of time for whatever server client interaction to happen sometimes fixes it. I reported this in September 2011. They said to upgrade my Lastpass browser plugin which I did. This still occurs to this day intermittently.

  • I got tripped up by their instructions on installing Lastpass on Linux and reported my confusion with their documentation. They responded with “I mentioned this to developers, but they felt that this step was common sense if you’re a linux user. (ie, it’d be like telling a windows user to double-click on a program to launch it)” and left the instructions in their ambiguous state.

  • When you log into Lastpass in your browser plugin it starts by downloading your encrypted blob of data. It then decrypts the blob locally on your machine in javascript. While this whole process happens, which when you have 600 credentials and secure notes can take some amount of time, the page used to display text as if your vault is empty and encourages you to add credentials into it. Now it merely displays an empty vault. As a user, this is pretty scary to see a page where you expect to see your credentials and instead you see an empty vault. They should be displaying a progress bar or a please wait while they download and decrypt the blob.

  • In your browser, when you go to a site that has a password field, and you’re not yet logged into lastpass, it pops up a notification bar that says “Simplify your life, use Lastpass to capture this site’s login” and a button that says “Save Passwords with Lastpass”. This language makes sense if you’re on a page for which you don’t already have a stored credential. The much more likely case of coming to a site with a login when you haven’t yet logged into lastpass is that you’re going to the site to login with your existing credentials stored in lastpass. This language should reflect that fact that it’s possible and likely that you’re not there to create a new credential set but instead to login into Lastpass so you can login to the site you’re on.

Browser Plugin Interaction

  • There’s no way to tell Lastpass that a given field is a CAPTCHA. Lastpass has some heuristic to detect a CAPTCHA in a login page and display a useful UI behavior instructing you to fill it out. If that heuristic fails and it doesn’t understand that a field is a CAPTCHA, there’s no way to tell it to treat that field as one. Upon asking how Lastpass determines that a field is a CAPTCHA, they report that “We determine this via field name.” and helpfully offer to solve the problem with “If you let us know what this is on the site, and it’s likely to come up again, we can look into adding it as a possible field name that would pop the security notification.” The way to solve this problem is not to wait for users to report sites that have unexpected field names for their CAPTCHAs and add them into some list stored in Lastpass. This is a totally unscalable, poor user experience approach. Of course after giving them the details of the web page that had a CAPTCHA with a field name they hadn’t predicted (I mean how could they have?) they responded with “I will put this on the list of suggestions and hopefully our dev team can add it in.” Obviously this never happened.
  • When you encounter a site that uses basic or digest HTTP authentication (where the login window pops up in front of the site and you log into your browser instead of typing your password into a field on a web page), Lastpass has a checkbox that you can check to save the credentials that you’re typing in. This makes sense. The checkbox state however is sticky. Once you save a credential set for a site that uses basic or digest auth, the next time you come to some other site that has basic or digest auth, and Lastpass fills in your username and password for you, the checkbox is still checked and so Lastpass saves a second copy of the credentials set. As long as this checkbox is checked Lastpass continues saving duplicate credential sets over and over again. This UI is absurd. Lastpass knows when you’re going to a site for which it has credentials and when you’re going to a new site. Why present a checkbox in a dialog on a site where you already have credentials indicating that you want to save a duplicate copy of those credentials?!
  • Lastpass uses a combination of settings called “Equivalent Domains” and “URL rules” along with it’s internal rules to decide what credentials go to what sites. These are confusing and difficult to use. I frequently get into a situation (in fact I’m stuck in one right now) where I can’t get Lastpass to offer the credential set for the site I’m on. I have to pull it up in my vault, and copy and paste it into the form. This is especially bad when it happens in the basic/digest HTTP auth pop up window. In that case the plugin will indicate that you’ve got multiple matching credential sets and when you click the drop down to pick one, it displays only one.

Payment

  • Lastpass charges $12 a year for premium service which gives a user access to using a second factor of authentication. In any given year the likelihood that one will have to change one’s credit card number due to fraud, it being lost, changing cards, etc. is pretty high. When the annual auto charge comes up, and fails due to the card number having changed, Lastpass decides that the appropriate way to deal with this is not to notify the customer, wait some window of time and then disable their two-factor auth. Instead they silently disable two-factor auth, opening your account up to attack and then later notify you that they’re having trouble charging your credit card. I reported this problem first in December 2011 when it happened the first time then again in April 2012. I’m sure the next annual renewal if my credit card has changed it will fail again and disable two-factor on my account.

    Recently my wife’s lastpass premium account came up for renewal. It was configured for yearly auto-renewal against our credit card. A few months back we had to change our credit card number because the bank sent us a new one due to some security compromise. We forgot to put this new number in google checkout/lastpass so that the auto-renewal would work.
    Problem is, what happened when the renewal date came up and the charge didn’t work (because the old card number was invalid), is that the 2-factor yubikey authentication was just disabled. No email warning of this fact to the user, just disabled.
    To have a billing issue open up an account to attack (by having only one factor) without notifying the user, is a risky security practice.
    Some better solutions would be :

    • Locking the account down until the user either logs in and disables 2-factor auth, or re-enrolls in lastpass premium with a new credit card
    • An email to the security email address of the users account warning them that 2-factor has been disabled
    • An email to the security email address stating that you tried to charge their card with no luck and 2-factor auth will be disabled in 48 hours, 5 days, etc. if the billing issue isn’t corrected. Really anything other than silently disabling 2-factor auth.

Security

  • The Lastpass mobile applications for Android, IOS and others are unusable from a security standpoint, here’s why :
    • To have a secure Lastpass vault, you must have a strong master password
    • If you were required to type this strong password on your phone each time you wanted to access a password from your vault to log into a site, it would be too onerous and people wouldn’t use the app.
    • Consequently, when you log into the Lastpass app on your phone, it decrypts your hosted blob of passwords and stores them on the phone. These cleartext passwords stay on the phone until you log out.
    • This means that if at any time, your phone gets lost or stolen, all of your credentials are accessible in plaintext on the phone.
    • Lastpass provides a function on your account that says “Kill other sessions on login”. I asked if the mobile apps do some kind of polling to know to kill their session if you log in at another machine and it turns out they don’t (understandably because of the battery hit for doing so).
    • Sadly this means there is no way to safely and conveniently access your lastpass vault from your phone.
    • I asked them to enable an option to poll and got back “We may enable some sort of option in a future version to set a polling time for the app or at all.” and as you can imagine, no such option has appeared.

Sharing

  • Lastpass provides a sharing feature that lets you share credentials with other Lastpass users. I use this alot and it’s very helpful. I routinely get notifications that shared credential sets have been changed by the other user. Here’s how that interaction goes :
    • I get an email that says “LastPass Shared Item Update Notification” and says “LastPass items you have shared have been updated by other users. Please click the below link to login to your LastPass Vault. Login to your LastPass Vault. Any items in your vault that have a icon next to them indicate that they have outstanding updates. Click the icon to view, accept, or delete the updates”
    • I click the link and it opens up my Vault
    • I scroll down pages and pages of credentials (600+) searching for the one that has a little icon next to it indicating it has pending changes from someone who I’ve shared it with
    • I find it and click the icon and it brings up a dialog showing something like 3 or 9 changes.
    • I click on one of the changes and it says that there is no change and the credential is the same.
    • I continue and click on all of them to find that the credential hasn’t changed?
    • I’m not sure how to identify all the ways in which this sucks

Other stuff

Issues that have been resolved in Lastpass 3.0 and are no longer an issue

  • There are 2 Lastpass Vault interfaces. Update : Lastpass 3.0 does a good job at unifying the look and feel of the locally hosted vault and the lastpass.com website vault. One hosted locally in your browser and one on the Lastpass website. Unfortunately the “Settings” interface for Lastpass only works in the Vault interface on the website, not in the browser. Their solution? Put the “Settings” interface inside an iframe in your localy hosted browser Vault interface. Why do we need two different Vault interfaces, each with slightly different UIs? Why does the user need to use one Vault for some actions and the other vault for other actions?

  • Update : Lastpass cleaned up their website at the end of October 2013 and it looks great now. Previously the Lastpass website was a hilarious explosion of confusing imagery. Go check it out, I’ll wait here : https://web.archive.org/web/20131024014857/https://lastpass.com/.

  • Update : With the lastpass October 2013 website redesign these ads for premium no longer display. While logged into Lastpass, when you go the lastpass.com website, it’s aware that you’re logged in and displays at the top a button that says “Jon Doe’s Vault ->”. So the website understands that you’re logged in and already a Lastpass user. It also knows, based on your username and the fact that you’re logged in, if you’re an existing lastpass premium (paying) customer. Why then would you display ads asking people to “Get Lastpass Premium! ($12 a year, that’s $1 a month)”? Why not display a useful page to customers since you know that they’re already customers, instead of a page of ads and attempts to convince the user to use Lastpass. They already use Lastpass, leave them alone.

  • Update : This message has been fixed in Lastpass 3.0 The 2-factor yubikey dialog box that prompts you for your yubikey one time password says to “touch-button for 2 seconds”. Unfortunately doing so will never work because with the yubikey you have to touch it for less than 2 seconds. More than that and it types out your static password, not your one time password. I reported this UI bug in December 2011, checked back in October 2012 and got “It’s still on our list to review and update”. It would take 2 minutes to fix. Maddening.

    When logging into lastpass with 2-factor authentication enabled using a Yubikey, a window pops up with the field to enter the Yubikey one-time-password into. In that window you’ll find the following text :

    1. Insert your YubiKey in the USB-port with the USB-contact facing upward
    2. Wait until your YubiKey touch-button shines with a steady light
    3. Hold your fingertip on the touch-button for 2 seconds”

    The yubikey doesn’t use a 2 second button touch though. If you go to the yubikey manual : http://static.yubico.com/var/uploads/pdfs/YubiKey_Manual_2010-09-16.pdf On page 20, you’ll see that for most yubikeys (mine included) that have the OTP configuration used with lastpass set in the primary position, need a button touch of between 0.3 and 1.5 seconds. Even keys that are configured with the OTP used for lastpass in the secondary position also wouldn’t use a 2 second press, they take a 2.5 to 5 second press.

    I’d recommend you change the text to something like “Hold your fingertip on the touch-button for a half second”

    Here’s the text from the manual explaining the default behavior :

    Ensure that the cursor is placed in a valid input field and touch the button with a finger tip and hold steady
    for approx 0.5 seconds and the OTP string is emitted. The indicator will then be turned off for approx 2 seconds
    where the touch button is disabled to prevent multiple triggers.

    And here’s the text from the manual talking about yubikeys with a second configuration enabled :

    Yubikey 2 supports an optional second configuration. This allows the Yubikey to be used for multiple services where
    both configurations are completely separated from each other. A typical usage is that one configuration is used for
    a remote service and one for a local service in static mode. If both configurations are set, the trigger behavior is
    slightly different as the user must select which OTP configuration that is desired:
    Short press 0.3 – 1.5 seconds) and release – OTP from configuration

    1 is yielded Long press (2.5 – 5 seconds) and release – OTP from configuration

    2 is yielded