Thinking before Typing

As I get older, I spend less time typing than I used to.  It has increased over time, but I used to dive right in and code a solution. 

This struck me today as I struggled with solving a problem that seemed simple enough.  Here’s the setup:

We’ve got a system that continually ingests tweets.  That’s not quite right, it is supposed to continually ingest.  Nothing ever goes as planned, though.  It crashes. We stop it for upgrades.  In either case, when it isn’t running we’re losing content.  Thankfully, we have the ability to “replay” a period of time.  

To date this is done manually, by noting the times when it stopped and started, and then punching those times into the replay.  It’s tedious, error prone, and might lose content without notice.

A long time ago I worked on an eBay style marketplace for industrial equipment in Europe.  I had to implement the billing generator.  The bill rate was broken into time chunks, the first day was most expensive, the next 2 days less, and so on.  The listing could be stopped and started, but pausing the billing in a similar manner.  It was complex, and I shudder to think of what my solution might have been.  I know that dealing with dates and times is tricky. 

In a nutshell, I’ve got a continuous stream of data that I want to capture completely.  I can replay holes in the timeline by specifying a start and end timestamp.  

So how do you go about solving this robustly?  The stream of data isn’t very identifiable, so overlapping time periods is hazardous.

I spent a couple hours noodling on it, not typing a thing.  In the end I was happy with the solution I came up with.

Of course, it’s also possible that I just think slower now.

The Crotator: A Custom Selection Tool

As a developer, I have tremendous appreciation for usability and respect for those that can conjure it.  Like many developers, though, it’s challenging for me to do on my own.  I’d like to quickly detail the fruits of a small usability example that I came up with recently for a customer’s project.

The Requirements

The customer wanted to take an image and crop out individual pictures within the image. To complicate things, the pictures would not be perpendicular but would need to be cropped and rotated to be straight. The easy thing to do would be to build a rectangular selection tool like you might find in PhotoShop. It would satisfy the requirement of cropping/rotating the pictures, but would require many steps to initiate the initial drag, rotate the whole thing, move it, resize corners, and so on. It was tedious.

What we needed was this:

  • Must crop the image
  • Must rotate the image so it is straight
  • Fast
  • Easy to learn
  • Relatively accurate

A Possible Solution

We talked through a few options, including the previously mentioned PhotoShop style selection tool. Some ideas were good but impractical. Slowly I started to imagine dragging an initial rectangle from corner to corner, and then dragging one of the other corners to the right position with the opposing corner reflecting the movement.

Doesn’t make sense? That was the trouble I had explaining it to others. Ultimately I built the prototype, and once you could actually see what was happening, it made sense.

The Crotator

It sounds like something you need a chiropractor for, but because it both crops and rotates it was named the Crotator. Here is an animation that hopefully makes clear how it works.

In the animation above, the initial selection box is created by dragging from the top left to lower right corner. Once complete, the top right corner of the selection box is dragged to the top right of the image and the bottom left point mirrors the movement to maintain rectangularity.

A Success

The good news is that the Crotator was a success. It fit the requirements and the customer loved the simplicity. In practice, you could show someone once or twice how to use it and they got it. In many cases, you can crop and rotate an image with two drags.

Thankfully the team wasn’t willing to accept the obvious and poor answer to the problem. We were able to focus on finding a great solution to our specific problem that was key to the functionality we were adding.

Agile Hurts

It was about 5 years ago that I first got slapped by agile. In that time, I’ve been in several purported agile environments, from pure high velocity environments to full blown faux-gile environments.

It takes me time to really internalize things such as agile. At this point in time I feel like I get it (don’t we always feel that way?). And here is my big lesson: agile is painful, and it’s supposed to be. A quick followup to that: agile isn’t a solution, it’s a mindset that will lead you to a solution.

This may be stating the obvious, but the processes falling under agile such as scrum and kanban drive out the pain points into the open. Extending that idea, you can’t homogenize a process across the board because each team’s problems are different. Perhaps more importantly, to successfully introduce agile into an organization requires everyone having the courage to face the problems that surface.

The biggest failure I see is ignoring these problems and not allowing the team to fix them. If you can’t trust and empower your team members, maybe you hired the wrong people.

Embracing Agile on a Fixed Price Project

Agile and Fixed Price

I’ll be honest, I was nervous. I’ve been developing in agile environments for several years now, and the thought of a fixed price project was frightening. It doesn’t seem like the principles of agile are conducive to a fixed price project. Customer collaboration? Continuous feedback? Responding to change? These concepts sound like they were designed to scuttle a fixed price project. But this is what the client wanted, and I was asked to lead the effort. Sure we had a change control process in place, but that was just going to break momentum in the long run.

Not only was it fixed price, but it was a new team, tackling a new-to-us codebase, and a new technology with a steep learning curve (WPF in this case). Did I mention that nobody on the team was involved in the time estimate? What we had was a list of stories to complete, and an incentive for finishing early. Likewise, there was a dis-incentive for finishing late.

The End Result

I take a lot of pride in the work I do, and I love delivering a product the customer loves just as much as I love a beautiful code solution. Delivering on time seemed impossible, much less delivering a product the customer loved.

It turns out we delivered a four month project three weeks early, and delivered a great solution that the customer loves. I was as surprised as everyone and have been reflecting on how we achieved it.

The Keys to Success

When it comes to agile, there are many principles to live by. The following items are what I tried to focus on.

  • The simplest thing that could possibly work. Our UX guy probably lost several years off his life after all the great ideas that had to be left behind. Build the simplest thing, minimum viable product, maximize the work not done, whatever you want to call it.
  • Continuous feedback from the customer is great. But you’ve got to realize you cannot deliver everything they want, or even everything you want. When you are under a fixed price, you must deliver what is spelled out. There is nothing wrong with telling the customer what is possible, and in fact we would do this all the time. But know that without change control you just can’t do it.
  • Be ruthless. Hopefully your contract has spelled out stories and not detailed specs. With stories, you have more latitude to deliver the simplest thing possible, and ever better, deliver the right solution to the problem. But you must be ruthless. Anything out of scope is out of scope. It’s best if you do this as nicely as possible.
  • We never had any change requests. To be honest I’m not sure if that is good or bad. Maybe we over-delivered. I think if we HAD gotten change requests, I would have liked to tackle them after the initial delivery. I think the momentum loss from the change request would outweigh tackling a change while things are fresh. That may seem a little backwards, but I feel that in a fixed price project you must stay focused on delivering.

Final Thoughts

I won’t lie. The key points above may have had nothing to do with our success. It may have had everything to do with having a great small team (3 devs and 1 UX) focused on a common goal. But if there is a take away, it’s that it is possible, and agile and fixed priced are not necessarily at odds.

The Beauty of the Open/Closed Principle

It’s good to be SOLID. Of the principles that make up SOLID, I’ve found the Open/Closed Principle (OCP for short) to be one of the more difficult. There are plenty of explanations around the web, but while identifying the presence or lack open/closed is easy, it’s tougher to achieve.

I also use OCP from the micro to macro scale. It can be applied to classes or it can be applied at a module/library level. Most material on the web covers the easier class level OCP. I’d like to cover the latter using some changes made recently to the Moe state machine library.

The Problem

On my current project, I encountered a great place to use a state machine to simplify a bunch of code. The usage of the state machine itself was interesting and I hope to post about it soon. From a Moe standpoint, I found that the way state actions were implemented didn’t suit my needs. First, I didn’t want to centralize enter/exit actions, particularly at the time of construction. Beforehand, you would add entry and exit actions when you built your state machine. This would mean they were all in a single chunk of code, or you would have to pass the builder around. Not ideal. Second, I also wanted to swap out these actions on demand. When the state machine wasn’t in use, I wanted to disconnect state actions so windows didn’t appear based on wayward events (ie, a late arriving error event from a webcam we were using).

The Solution

I revisited Moe to add the functionality I wanted. What I envisioned was a convention for state actions. Any class could have methods following the form “OnEnter” and “OnExit”. These methods would auto-magically be invoked upon entering and exiting states. These classes could also be added or removed as listeners at any time. I decided to call this functionality state delegates.

While working on Moe previously, I had done a lot of work to support the OCP and this can be seen in the Moe.StateMachine.Extensions library. In that library you can find several helpful additions for Moe, such as logging, state watchers, timers, and asynchronous state machine execution. I specifically wanted that functionality to exist outside of the core state machine. Would the hard work of that refactoring pay off and allow me to add state delegates?

The Changes

You can see the changes that were made in this commit. The solution was to add a traffic cop that would wire up to every state and intercept state enter/exits. When one of those happened, the StateDelegatePlugIn would run through the current listeners and delegate to any methods matching the state name.

This huge addition to functionality (in value, that is) was achieved without touching the state machine at all. This is the OCP on a larger scale. We’ve added valuable functionality to a library without modifying the library itself.

How to move towards OCP

It’s tough, if not impossible, to produce code supporting the OCP from the beginning. Before we can abstract, we need to SEE at least a few examples of usage first. If you were to go back and look at past commits, you’d find several that are focused on refactoring to the OCP after several functional additions were already in place. To begin with, the functional additions were intertwined with the state machine itself. By unwinding those dependencies, I was achieving OCP through actual usage. It would have been too much for my brain to try and see the resulting patterns and needs without already having multiple examples.

When I do something like this, I’m looking to make core classes as simple as possible, keeping as close to the Single Responsibility Principle as I can. Something that I should have done earlier was introduce the Extensions library. By forcing myself to move functional additions to a separate assembly, I would have no choice but to support OCP. When thinking of simplifying or SRP, think of the grand Unix commands. Grep, sed, sort, and so on. Have your components do one thing and do it well.

Is MVC the best we can do on the web?

Recently I’ve been thrust back into web development after a 5 year hiatus.  Sadly I’ve discovered that not much has changed.  We’re still forcing an inappropriate pattern onto web development.

Why not MVC?

The pattern we all know and love has been great for many years, and certainly in a thick client environment, it and its brethren seem appropriate. In these cases, the UI’s are often being developed by the same folks that are developing the M and the C. With web development, this is often not the case.

On my first significant web project back in 1999, the workflow was a designer delivering HTML files, and then a developer hacking that into a JSP/Servlet combo. This results in your controller and views being tightly coupled together. What happens if the designer needs to deliver completely new screens and workflows? My guess is you’d have to throw out the views and the controllers and start again. How diligent is your team at keeping the controllers thin and free of business logic? Most of us know that’s a good practice, but how often have you seen that violated?

Introducing a new TLA: SVF

Service, View, Flow. The last one is Workflow, but W just has too many syllables. Let’s partition a new way.


A service takes data in, and returns data out. A service might fetch a customer or a customer’s orders, or it might create a new order for a customer. This has been common in software development for some time now, and it continues to make perfect sense in web development. Nice, simple, reusable, beautiful.


A view should only require some data, and be able to render that data. Imagine a view template that specifies the data it needs. A Customer. A list of orders for that customer. The template specifies it needs that info be specifying the services needed. An incoming web request/get doesn’t go to a controller, instead it goes straight for the view. The view will specify what it needs. It will work to render itself.

What about performing operations? Let the form or the link specify a target service and bind data to be passed. Now the designer can compose the views however they want, without impacting code much, if at all.


Flow, or workflow if you like, is the traffic cop. The designer can specify much of the workflow through the views, but occasionally some more complex logic may be necessary. In this case, your workflow can be a simple codified state machine that can determine the next view based on some logic.

What does it mean?

Developers can focus on building small simple services, designers can not only build the views, but actively maintain them over time (as opposed to painful markup merges or outright rewrites). We are minimizing the development of “throwaway” code, and in this case I’m referring to throwaway as code developed for a single purpose of gluing/coupling components together. In the realm of MVC, Controllers are essentially throwaway. To some extent the Views are too, since they are so tightly coupled to their controllers.

To visualize, imagine web services, with HTML documents that interact with those web services directly. All we need is a framework to tie them together.

An example

It seems that Twitter style micro-blogging is the Hello World of web apps, so let’s see what this might look like. We’ll take a look at a user’s home page, which will provide a form for a new micropost, as well as a list of our own posts. Such a page will need two primary services, one for the User, and one for the user’s Posts.

The View

@Fulfill User = User.Current
@Fulfill Posts = Posts.ByUser(User.Id)
    <title>Hello Microblogger World</title>

Here, we see a couple template markups in our invented template language that are binding variables to services. First, the service identified by User.Current is bound to the variable User. Likewise, the service Posts.ByUser is passed a parameter (the current users ID), and the results are bound to the Posts variable. These are pre-requisites for displaying this view.

Next, we’ll have a nav bar that includes links to various other views (not controllers, not actions, but the actual views).

<nav class="round" >
    <li><a href="@View()" >Home</a></li>
    <li><a href="@View('Users')" >Users</a></li>
    <li><a href="@View('Users', User.Id)" >Profile</a></li>
    <li><a href="@View('Users', User.Id, 'Edit')" >Settings</a></li>
    <li><a href="@View('Help')" >Help</a></li>
    <li><a href="@Service('User.Logout', 'Login')">Sign out</a></li>

Most of our links are pointing straight at other views. We are maintaining RESTy-ness with the links that would be generated by @View. The final link for signing out actually calls a User.Logout service, and specifies Login as the followup view. By doing this, the designer can maintain control of workflow instead of the developer coding this into a controller.

Now let’s see how we would handle posting a new microblog message.

<h1>Tell me what you're doing:</h1>
@ServiceForm('Posts.Create') {
  <textarea cols="40" id="newpost" name="@Bind('message')" rows="20"/>
  <input id="post_submit" name="commit" type="submit" value="Post" />

We’re going to bind the HTML form to a service called Posts.Create. The textarea element is bound to the input parameter on the service, “message.”

Finally, we’ll iterate the user’s own messages

@ForEach('Posts') {
  <td class="gravatar">
    <a href="@View('Users', User.Id)" >
      <img class="gravatar" src="...."/></a>
  <td class="post" >
    <span class="content" >@this.Message</span>
    <span class="timestamp" >@this.Timestamp</span>

We build a link much like before, and we iterate over the posts with the @ForEach tag.

The Services

We can cleanly build our services now. In this case, we need Users.Current (which would handle the authenticated user), User.Logout, Posts.ByUser, and Posts.Create. These services are very general, and once built the views can be endlessly changed and redesigned and reused without further code necessary.

class PostsService < Service
  def by_user
    user_id = request[:user_id]
    ... fetch return posts for user_id

  def create
    user_id = context[:user].user_id
    message = request[:message]
    ... create new post for user

It’s interesting to note that these services are practically web services if the inputs and outputs were transformed from/to JSON or XML.

Getting with the Flow

For the sake of discussion, we’ll say that our Help views can be initiated from anywhere in the application. The help contains numerous individual views to assist the user, but when the user clicks a Done button, they should return to the screen they were on when they first clicked Help.

class HelpFlow < Flow
  attr_accessor :view_stack

  on_viewing(:help) do

  on_viewing(:done) do

Changing roles

We are changing the roles and responsibilities now. Developers are focusing on developing code to process and return data. Designers are focusing on building the user experience. While offloading some responsibilities from developers, we are adding some for the designers. Communication will be necessary to work out what services will be needed, what data needs to go in and what data is coming out. In other words, the developers and designers will be coming up with an API between them.

Just today the value of this became apparent. The team lead and the designer were trying to figure out how to refactor the partitioning of the views. It turns out that partial views and layouts had been pulled out by some developer along the way to suit his purposes. Unfortunately, how he had partitioned them was damaging and resulted in considerable duplication. If the designer had been wholly responsible for views, they could have properly partitioned the different headers, footers, and so on in a logical manner that would minimize duplication.

Crunch time and Theory of Constraints

I’m continually amazed at the wastefulness and lack of focus I often see in software development. Worst of all, the problems seem to intensify as one gets closer to the deadline. It doesn’t have to be this way if you are focused on continual improvement.

A colleague of mine that I have tremendous respect for turned me on to Eli Goldratt and The Goal: A Process of Ongoing Improvement. While The Goal focuses on manufacturing, it is applicable to other industries as well (see Critical Chain). The basic idea is to identify bottlenecks (or constraints) to throughput, and systematically make improvements to reach the goal of maximizing throughput for the system as a whole. Of course, the theory of constraints was the inspiration for the name of this blog.

Crunch Time

Anyone who has spent time in software development has experienced crunch time. You’re up against a tight deadline, and there is more work than time. The closer you get to the deadline, the more chaotic things get. You have X number of features that must go out, and they are all top priority.  If a feature has to be done, it is of COURSE top priority.  Along with all the other required functionality.  How can we maximize chances at success?

Step One: Identify the constraint

This can be tricky, but very often it is development that is constraining the system. Coding is a time consuming process. For this post, we are going to assume it is the development that is the bottleneck.

Step Two: Exploit the constraint

No, this doesn’t mean treating your developers unfairly. It means utilizing your resource to the fullest. Remove any functionality not absolutely necessary. Re-evaluate targeted functionality for the simplest solution that could possibly work. Prioritize work, even if priorities are arbitrary. When the human brain encounters 10 different #1 priority tasks, nothing is going to get done. By prioritizing, you let development focus on individual tasks.

Step Three: Subordinate every other decision to the constraint

Have you ever been in crunch time and found yourself doing something that is unrelated to your goal and is wasting your time? As an example, if I’m a developer under the gun to get functionality developed, preparing release notes and responding to bug reports via some cumbersome bug tracking system is wasting my time. Find another resource that can handle these things for me so that I (the bottleneck) can focus on my goal. Don’t hand incomplete requirements to your developers that will require them to stop working to clarify.

Step Four: Elevate the constraint

Elevating the constraint is tough during crunch time. In this case, elevation could involve adding people or faster machines for the developers. Neither of these is a good idea when up against a deadline. But outside of that scenario, the above should be considered. Particularly computers. It absolutely baffles me the salaries organizations will pay developers and leave them with 2 or 3 year old machines. Spending $3k on a faster computer is a no-brainer.

Step Five: Repeat

Chances are that if you repeat the above steps, at some point in time your constraint is going to shift. Most likely, it will shift to your testing crew. Repeat the focusing steps to continually improve what you do.