Jira, Kanban & Physical Boards: An Agile Life at TweetDeck

This article talks about our working process at TweetDeck and how it has evolved over the last few years, from barely SCRUM to an efficient production machine. The reason I am talking about this now is that I have been running a small project team for TweetDeck over the last few weeks, separate from the core TweetDeck team, giving us an exciting chance to develop new ways of managing our workflow.

So, let’s take a trip back through time, to see where we were and how things have changed.

3 years ago

I’ve only been with the team for two and a half years, so I can’t say for sure, but when I joined the team was just emerging from the dark in to the world of Agile. As far as I know, at the time they had daily stand-ups and had recently started using Pivotal to manage their workflow.

2.5 years ago

  • SCRUM
  • Weekly retrospectives on Friday
  • Planning & estimation session on Monday mornings
  • Daily stand-ups
  • Pivotal used to keep track of tickets
  • Irregular release cycle – once every few weeks.
  • Points based estimation, 1,2,3,5,8,13,21
  • Planning poker

Our early retrospectives were largely whining sessions and generally failed to result in any lasting changes. We used a whiteboard with three columns, “Good”, “Bad” and “Questions”, under which we placed post-its. We’d then review all the post-its (there were lots to get through), which were usually about annoying bugs, broken systems, poor comms with SF and the office lighting.

Despite that, the retrospectives did result in a few good things. There were some small improvements around daily stand-ups, and we started getting regular updates from the project management team regarding progress and upcoming work. There were also some minor improvements around working practice – reviews, code guidelines, etc. It was definitely a step forward, but nothing major.

At some point, we introduced pre-planning, where the team leads got together on a weekly basis to prune and prioritise the backlog.

Point based estimation was used. Pivotal used the points to estimate how much work would fit in to the Sprint, and was reasonably accurate.

2 years ago

  • Regular release cycle – once a week on Thursday
  • Timed planning poker

Much the same as before. The team was growing larger and more unwieldy. Retrospectives had pretty much stagnated and very little good was coming out of them. Action points weren’t being reviewed and often weren’t acted upon.

One that was acted upon was the idea of “investigation” stories. If a story was poorly defined, or we needed to figure out how to repro a bug, or we needed to research a new technology, we marked it as “investigate”. These were always marked as a 3 on the points scale.

We also built a timer for Pivotal – we got a hard limit of 3 minutes to discuss a story. If estimation didn’t get finished, it got marked as an investigate and put back a week.

Retrospectives continued to have an effect (the changes to this process over time are largely a result of retrospective actions), though still didn’t feel wonderfully helpful.

1.5 years ago

  • SCRUM/Kanban
  • Weekly retrospectives on Friday
  • Planning & estimation session on Monday mornings
  • Daily stand-ups
  • Jira
  • Regular release cycle – once every week
  • Points-based estimation: 1,3,5,8

To everyone’s horror, we were forced to abandon Pivotal and switch to Jira. There was much complaining. The reason for the switch was to provide visibility in to our work for upper management, but creating tickets took longer, we lost our planning timer, the system was harder to use and the board was less readable. It felt like a step backwards, but one we had to learn to live with.

Not long after the switch, we got some advice from an agile consultant who introduced us to Kanban. We decided it was worth a try and switched to the Kanban Jira board, including soft Work In Progress (WIP) limits.

Our Kanban board had four columns and a backlog (not on the board):

Backlog
Pruned and prioritised based on input from product management & the team leads. However many tickets we think we can do each week are moved in to the To Do column in the Friday planning session
To Do
WIP: 4 tickets per developer. Tickets have to be moved to “In progress” manually.
In progress
WIP: One ticket per developer. Tickets automatically move to “In Review” when a patch is submitted.
In Review
WIP: Two tickets per developer. Tickets automatically move to “Done” when a patch is merged.
Done
Merged, in master. Tickets have to be manually removed from this column when they are shipped.

WIP limits

WIP limits varied according to how many developers we had. Unfortunately JIRA doesn’t support hard WIP limits and so they were perhaps not as effective as they could have been, but they definitely helped us improve throughput.

Restricting the number of tickets in the To Do column made us feel like we were achieving more, because we could see it almost empty by the end of the week, and restricting the number of tickets In Review helped us to get reviews done in a timely manner.

A few months ago

  • Kanban/SCRUM
  • Regular release cycle: twice-weekly
  • Jira
  • Pre-planning with project leads on Friday mornings
  • Planning & estimation session on Monday mornings
  • Retrospective on Friday afternoon
  • Daily standups
  • Points based estimation: 1,3,5,8

We changed our retrospective format for the better. We added a few minutes at the beginning to review what we had done the week before, which was well received – a chance to pat each-other on the back for a job well done. This largely replaced the “Good” column.

Perhaps most significantly, we changed the column headers to be action-focussed: “Actions” & “Questions”. In this way, we managed to remove all the whining about day-to-day issues which no one could do anything about. This resulted in the number of post-its dropping significantly.

People stopped complaining about the temperature and bugs and comms and instead began to focus on what they could do to make things better. I can’t stress enough the positive impact this had, both on the quality of the actions we take and the overall mood of the team.

Now

  • Kanban
  • Jira
  • Physical board
  • Irregular release cycle: as required, at least twice-weekly. Moving toward continuous deployment.
  • Ad-hock planning sessions
  • Weekly retrospective on Friday morning
  • No estimation

Our little project team has (as of this week) taken our Kanban process to the next level, dropping the weekly planning session and adding two new columns to the board. For the small team, we now have the following columns:

Upcoming

WIP: min 5 tickets. No max.

Feature-level stories.

  • May or may not have design
  • Has an associated Jira ticket
  • Needs some work to make ready for development

When there are not enough tickets in To Do to satisfy the WIP limits, we take one from Upcoming and move it to In Analysis.

In Analysis
WIP: max 2 tickets. Involves talking to Product Management, Design and the Team Lead to produce designs & requirements and, if necessary, creating smaller tickets and tasks. Once the story has been analysed, it (and any tickets & tasks created) are moved in to To Do.
To Do
WIP: min 3 tickets. No max. Stories/tasks that are ready to be developed. They have a complete set of requirements and design. They may or may not have a Jira ticket.
In Review
WIP: max 2 tickets per developer. Patches in review, as before
Done
WIP: 10 tickets. We’re not quite there yet, but in a few weeks any patch that gets in to Done will automatically be shipped to production. Exciting & scary!

The Physical Board

We also have a physical board in our room – a big whiteboard with the above columns and lots of index cards. If an issue is so small that adding a ticket for it seems overkill, we don’t bother and just write it up on a card. These cards usually drop straight in the To Do column.

Having a physical board reduces the amount of time we all spend in Jira and increases visibility within the team – we can instantly see what’s happening without any context switching. As it changes, it gives us a good impression on how much work we’re doing, a nice positive-reinforcement technique. “Hey, look at all the stuff we did today!”. Also, when there’s a bunch of stuff piled up in the “Done” column we know it’s time to ship.

Maintaining “To Do”

We’re loving the flexibility of our To Do column. It allows us to introduce new tasks without going through the planning rigmarole. As we develop stories, we always find new things that need doing that are not directly related to the task in hand. These can be written up and added to To Do in a few seconds. This helps us keep our tech-debt in check, and ensures there’s always something useful for someone to do.

To Do is constantly rearranged in terms of priority, with new cards being added regularly. It’s constantly reviewed by the Team Lead (myself). If something goes in the list that doesn’t need doing immediately, or needs analysis, I’ll take it off the board, create a Jira and prioritise it in Upcoming.

When we start to run out of things in the To Do column, it’s time to pull something out of Upcoming and develop the story.

There are no hard-and-fast rules about what goes in the To Do column, but the important part is that it is managed by the developers, rather than by design or product management. From the pov of Product Management and upper management, so long as those feature-level stories keep moving along and are kept up to date in Jira, it really doesn’t matter what happens between Upcoming and Done.

Estimation

You might have noticed that we dropped estimation entirely. We just couldn’t see what the point of it was. We do high-level estimation up-front, months in advance of work being started. Day to day, the points weren’t being used for anything. So… why bother? In SCRUM you might want them for a burndown chart, but Kanban doesn’t have such a thing.

This also means we can drop our weekly planning session. It is largely replaced by the In Analysis column. One less meeting per week! Oh yeah! Though, actually, we’ve replaced it with “Monday morning tea and biscuits”, in which we chat about upcoming work. Sounds much more pleasant.

What’s next?

The process is evolving week-on-week (or even day-on-day) and I’m hoping that by the end of the project we’ll have locked down a system that requires minimum maintenance and maximum flexibility while providing good visibility to management, and good visibility of upcoming work to developers.

We’re going to have a review of this process in a few weeks and see whether we can apply it to the larger team. I’ll let you know how it goes.

TweetDeck: New Direct Message API

Last week, we launched two new Direct Message features on TweetDeck. The one that got picked up by the press was the ability to share tweets via Direct Messages. Although useful, this feature is just the tip of the iceberg. Underlying it is an entirely new API, support for which Twitter has been working on for a few months across all its clients (web, iOS, Android, Mac & TweetDeck).

In TweetDeck, there are three noticeable changes that result from the switch to the new API.

Historic Messages

Perhaps the most obvious is that you can now scroll back through your entire history of Direct Messages, back to when you first signed up for Twitter. For me, this was a trip down memory lane. For others, not so pleasant.

Unread Count

The unread count for your messages is now per conversation, rather than per message.

A Slight Delay

This one’s not really a positive change, but then perfection is rarely achieved. There’s a delay of up to 10 seconds between a message being sent and you seeing it in TweetDeck. This is because the new Direct Messages are not yet supported by the Streaming API, which TweetDeck relies on to receive Tweets instantly. There’s a team working hard to allow TweetDeck to provide instant Message notifications.

Clients which do not rely on the Streaming API, such as iOS and Android, will display new Messages instantly.

Shh!

There’s one other notable change which I’m not really supposed to talk about. It’s a subtle fix for an issue that’s been around since 2013, which has taken a huge amount of work to resolve back at Twitter HQ. If/when it becomes public, I’ll say more :)

FlightJS: with-child-components

Tom Ashworth (@phuunet) has recently released withChildComponents (github,
bower: flight-with-child-components), a mixin for Flight which allows you to construct component trees.

When the parent component is torn down, all child components instantiated with attachChild (rather than attach) will also be torn down.

TweetDeck has been using withChildComponents for some time. We find it very useful, enabling us to define complex component hierarchies without worrying about unwanted dependencies or detached components (which may cause unforeseen behaviour and memory leaks).

TweetDeck Login

Have all your columns disappeared? Have all your settings been lost? You’ve probably logged in with your Twitter account.

Don’t worry. There is a simple fix. Let me explain.

You have been signing in to TweetDeck with an email address & password

Up until a few months ago, to use TweetDeck you would have had to have created a TweetDeck account using an email address and a password. You might have used the same email address and password you use for Twitter, or something completely different. You might not even have realised you were creating a TweetDeck account and just entered your Twitter email address and password. It wasn’t very clear what was going on.

Once signed in to TweetDeck with this account you would have added a Twitter account to TweetDeck. Some people may have added many Twitter accounts to TweetDeck, allowing TweetDeck to access all those accounts.

Confused? Well, yes. Obviously.

Logging in with your Twitter account

In the hope of simplifying your life, TweetDeck started supporting “login with Twitter” a few months back. This allows you to log in with your Twitter username & password instead of an email address (as above).

When you try to log in to TweetDeck, the login form you’ll see first is the “login with Twitter” one.

New TweetDeck users will use this form, logging in with their Twitter account. Simple. For them, anyway.

Old TweetDeck users, ones who created a TweetDeck account with an email address (that’s you!), can’t use that form. Doing so would create a new TweetDeck account using your Twitter credentials, and thus you will not see any of the columns you created under your original TweetDeck account.

Here’s how to fix it

Log out of TweetDeck. Click the “Sign in with your TweetDeck account” link above the login form. Log in with your TweetDeck account. This is an email address, rather than your Twitter username. It’s the address you first signed up to TweetDeck with.

And here’s how to make sure it never happens again

You can switch to using your Twitter login permanently. Once logged in with your old TweetDeck account, select “Start using Twitter sign-in” from the settings menu (cog, bottom left corner) and follow the instructions.

Note that if you are running a shared TweetDeck (sharing username/passwords), you’ll need to follow the instructions very carefully!

Managing multiple Twitter accounts in TweetDeck

Making Use of Custom Timelines

Custom Timelines are a new feature for Twitter, allowing you to create a timeline containing specific Tweets. As with all Timelines, it is possible to embed Custom Timelines in a web page, or link to the Timeline directly on Twitter.com.

See the TweetDeck blog post on how to create Custom Timelines.

So, what are they good for? Well…

Live news

Add Tweets to a Custom Timeline to curate an evolving news story.

Stories

Custom Timelines allow you to add Tweets to a Timeline in a specific order. This means you can create a story by selecting Tweets from a variety of sources.

Here’s @passy’s take on Custom Timelines

Q&A

It can be quite hard to follow a Q&A session, as these normally rely on hashtags and can become polluted by spam, and it may be hard to tell which replies are regarding which questions. Custom Timelines allow a user to see the full Q&A without all the noise.

Also, as Custom Timelines are fixed over time, it is possible to preserve the results of a Q&A for all time.

Here’s a @GuardianUS Q&A session on the NSA

Reading List

When you see a Tweet referring to an article, you can now add that Tweet to a Custom Timeline, creating a “to-read” list for your later perusal.

Here’s my growing to-read list (mostly UK news) Postliminary Perusal

Specific Subject

Interested in JavaScript, politics or green energy? You can create a Custom Timeline for that subject, picking the best tweets for sharing with other users. A bit like Tumblr, perhaps?

Here’s the Guardian again with New York travel tips

Media

You can create a Custom Timeline containing your favorite images & Vines. Everyone likes pictures :)

Getting Started with Flight

I’ve written a book! It’s a getting-started guide to Twitter’s JavaScript framework, Flight. If you’re creating JavaScript applications of any sort, I’d really recommend taking a look. There’s lots of lessons-learned in there from my experience working on TweetDeck, as well as basic how-tos for constructing components & mixins, all presented in what I hope is a clear and helpful manner.

It’s available now on Amazon:

0957OS_cov

What is a Flight component?

Twitter’s Flight framework is component-based. So, what is a component?

A component is an atomic piece of functionality. It is atomic because it is self-contained: it does not depend on other components.

A component can be thought of as comprising three distinct parts:

  • Component definition
  • Component factory
  • Component instances

Component definition

A named function which defines the component’s behavior. This includes event listeners & triggers, default configuration and private methods. A component does not have any public methods or members – all interactions are managed through events.

E.g.:

Component factory

Flight’s defineComponent method accepts a component definition and returns a component factory.

  • A component factory is used to create component instances and attach them to the DOM.
  • A component factory can create many component instances, but only one instance may be attached to any one node.
  • A component’s default attributes can be overridden at instantiation

E.g.

Component instance

  • A component instance is created using a component factory’s attachTo method.
  • attachTo does not return a reference to the component instance.
  • A component instance can interact with the DOM node it is attached to, and its subtree.
  • A component instance can interact with the rest of the application via events

E.g.

Test the interface, not the implementation

There’s a common misconception when it comes to writing unit tests that how something works is important. It is not.

Imagine you are testing a simple amplifier. An amplifier’s interface has two inputs and one output: audio in, the volume knob, audio out.

To test whether the amplifier works correctly, I could disassemble it and test every internal component, making sure that all the named parts are present and work as expected. I could check that the colour banding on the resistors is correct. I could run a current through each part of the circuit to check the connections. In fact, while I was building the amplifier, I would do exactly this.

However, to test whether the amp works all that needs to be done is to turn the volume knob and check that the volume changes accordingly.

When writing unit tests it is important to understand that an amplifier is a unit.

Before you start writing tests, first define the interface. Then test that and only that.

RIP TweetDeck! Long Live TweetDeck!

The state of TweetDeck has in recent months been rather mis-represented in the press. As a result many people seem to think TweetDeck is being shut down, though this is definitely not the case.

TweetDeck is shutting down some old applications and concentrating instead on the latest web & desktop offerings.

This was first announced in a post on TweetDeck’s blog: An update on TweetDeck.

What’s staying

The following versions of TweetDeck will continue to be available and actively developed:

It looks like this:
Screen Shot 2013-06-14 at 11.35.46 AM

What’s going

  • TweetDeck AIR (v0.38)
  • Android
  • iOS
  • Facebook integration

So, if you were using AIR (the old Yellow bird version), you can replace it with the Chrome, Windows, Mac or web version.

For mobile users the obvious replacement is Twitter’s official client, available for Android and iOS.

I hope that’s cleared things up.

TweetDeck: Link shortening

TweetDeck automatically shortens any urls you Tweet using the t.co service (or bit.ly – see below). Shortened urls are 22 characters long.

Before you send your Tweet, TweetDeck will show you the full URL you typed in. The short link is not created until it is sent.

When TweetDeck displays Tweets it looks up the original url for any t.co url and displays that instead of the t.co so it becomes clear exactly where a link you are thinking of clicking on is going to take you.

You can test that link shortening is working by entering a very long link. E.g.

http://this.link.is.very.very.very.long.and.can.not.possibly.fit.into.a.single.tweet.but.thatis.okay.because.TweetDeck.will.shortenit.automatically.com

If you paste that link in to the compose box, it’ll tell you you’ve got 118 characters left.

bit.ly

Currently TweetDeck only offers one alternative link shortening service, bit.ly.

To switch to bit.ly, go to Settings > Services and select bit.ly from the dropdown. You will then need to enter your bit.ly username and API key. You can get these on the bit.ly advanced settings page. At the bottom of that page, click “Show legacy API key” and copy them across to TweetDeck.