Taking Advantage of Vim

Once you have mastered using Vim to replace Notepad.exe, it is time to starting taking advantage of what Vim has to offer.  Doing so can increase your productivity.  Below are most of the commands I use most frequently.  It is important to note that Vim has a veritable cornucopia of commands and thus supports many different usage styles.  What is presented here are just a fraction of the available commands.

Many commands in Vim have a similar format.  This format is [repetitions][command][movement].  For instance, 2dw will delete the next 2 words.

Useful commands that take this format include:

Command	Action
c	Change.  Delete the text moved over and enter insert mode.
d	Delete.  Delete the text moved over.
y	Yank.  Copy the text moved over.

No number implies 1.  The combination cw will delete the rest of the word under the cursor and enter insert mode.  This is probably the most commonly used combination for me.

There are two more variants on these commands.  A capital letter will execute the command to the end of the line.  D will thus delete the rest of the line.  Two repeated letters will execute the command on the entire line.  Thus cc changes the whole line and 2yy yanks the next 2 lines.

Common movement commands include:

Movement	Action
w	word.  A single word.
b	word back.  A single word, backward.
)	sentence.  A single sentence.
h	left.  One character to the left.
j	down.  One line below.
k	up.  One line up.
l	right.  One character to the right.

Movements can be issued without a command in front of them.  2w will move the cursor 2 words to the right.  You can maneuver all over the document using hjkl instead of the cursor keys if you like.  Personally, I use the cursor keys and only use l when I want to change some number of characters.  5cl will delete 5 characters and enter insert mode.

Some commands operate without a movement.  These include:

Command	Action
x	Delete one character
~	Change the case of one character.
r	Change one letter and return to normal mode.

These commands can accept a number.  Thus 5x will delete the next 5 characters.

Other useful commands:

:<line number>	Jump to that line number
%	If on top of a ( or {, jump to the matching one.  Very useful in coding.
m<letter>	Sets a bookmark labeled <letter> at the cursor.
'<letter>	Returns to <letter> bookmark.  Useful for jumping around a file.
u	Undo the last change.  Useful if you forgot you weren't in insert mode.
gg	Go to the top of the file.
G	Go to the bottom of the file.
>>	Increase indent.
<<	Decrease indent.
o	Insert a new line below and enter insert mode.

These commands, in addition to the search and replace command introduced in the notepad post are the ones I find myself using most frequently.  Once you become accustomed to them, editing in something without them will feel strange.  I want to hit 2cw to change the next 2 words in Word all the time.  I find the /search very useful also and miss it whenever editing in something that does not have Vim keybindings.  My repertoire is fairly limited presently.  As I discover new ways of doing things in Vim, I'll post them as Vim tips.  If there are commands you use frequently that I don't discuss above, please mention them in the comments.

We Need A Better Way To Test

Testing started simply.  Developers would run their code after they wrote it to make sure it worked.  When teams became larger and code more complex, it became apparent that developers could spend more time coding if they left much of the testing to someone else.  People could specialize on developing or testing.  Most testers in the early stages of the profession were manual testers.  They played with the user interface and made sure the right things happened.

This works fine for the first release but after several releases, it becomes very expensive.  Each release has to test not only the new features in the product but also all of the features put into every other version of the product.  What took 5 testers for version 1 takes 20 testers for version 4.  The situation just gets worse as the product ages.  The solution is test automation.  Take the work people do over and over again and let the computer do that work.  There is a limit to the utility of this, but I've spoken of that elsewhere and it doesn't need to be repeated here.  With sufficiently skilled testers, most products can be tested in an automated fashion.  Once a test is automated, the cost of running it every week or even every day becomes negligible. 

As computer programs became more complex over time, the old testing paradigm didn't scale and a new paradigm--automated testing--had to be found.  There is, I think, a new paradigm shift coming.  Most test automation today is the work of skilled artisans.  Programmers examine the interfaces of the product they are testing and craft code to exercise it in interesting and meaningful ways.  Depending on the type of code being worked on, a workforce of 1:1 testers to devs can usually keep up.  This was true at one point.  Today it is only somewhat true and tomorrow it will be even less true.  Some day, it will be false.  What has changed?  Developers are leveraging better programming models such as object-oriented code, larger code libraries, greater code re-use, and more efficient languages to get more done with less code.  Unfortunately, this merely increases the surface area for testers to have to cover.  Imagine, if you will, a circle.  When a developer is able to create 1 unit of code (r=1), the perimeter which a tester must cover is only 3.14.  When the developer uses tools to increase his work and the radius stretches to 2, the tester must now cover a perimeter of 12.56.  The area needing to be tested increases much faster than the productivity increase.  Using the same programming models as the developers will not allow test to keep up.  In the circle example, a 2x boost in tester performance would only cover 1/2 of the circle.

Is test doomed?  Is there any way to keep up or are we destined to be outpaced by development and to need larger and larger teams of test developers just to keep pace.  The solution to the problem has the same roots as the solution to manual testing problem.  That is, it is time to leverage the computer to do more work on behalf of the tester.  It will soon be too expensive to hand-craft test cases for each function call and the set of parameters it entails.  Writing code one test case at a time just doesn't scale--even with newer tools.  In the near future, it will be important to leverage the computer to write test cases by itself.  Can this be done?  Work is already beginning, but it is just in its infancy.  The tools and practices that will make this a widespread practice likely do not exist today.  Certainly not in a readily consumed form.

This coming paradigm shift makes testing a very interesting place to be working today.  On the one hand, it can be easy for testers to become overwhelmed with the amount of work asked of them.  On the other hand, the solutions to the problem of how to leverage the computer to test itself are just now being created.  Being in on the ground floor of a new paradigm means the ability to have a tremendous impact on how things will be done for years to come.

 

Update:  There are a lot of people responding to this post who are unfamiliar with my other writing.  Without some context, it may seem that I'm saying that test automation is the solution to all testing problems and that if we're smart we can automate all of the generation.  That's not what I'm saying.  What I advocate in this post is only a powerful tool to be used along with all of the others in our toolbox.  If you want some context for my views, check out:

• When to manually test and when to automate


• The dangers of test automation


• Too much test automation?

 

GVim As Notepad

When I first encountered Unix in the early 1990s, I needed a text editor.  I tried Emacs but the meta key and double control keys struck me as wrong.  I tried Vi but couldn't figure out how to type anything.  I came across Pico and for years used that as my editor.  Why?  I wasn't doing much other than modifying some shell scripts or dot files and it was easy.  All the commands I needed were listed at the bottom of the screen.  Pico is the Notepad.exe of the Unix world.  Years later I was forced to do some programming on a Linux system via an ssh connection.  Needless to say, Pico isn't really a great programming editor.  I had a friend who was really excited about Vim (Vi Improved) at the time so I gave that a try.  Emacs still didn't sound inviting. 

I began by reading tutorials on the web.  They all intended to get me up to speed on all the nifty functions in Vim very quickly.  For instance, most tutorials start by teaching that hjkl are the cursor keys.  They are, but you don't really need to know that for basic functionality.  The ones with arrows on them will work just fine for most purposes.  Options like this serve just to confuse at the beginning.  The trouble is, I can only learn so much at once.  Before I could learn how to run, I had to learn how to walk.  I needed to be comfortable doing the basics before I could grok the extra stuff.  I needed to be able to use Vim like notepad functionality before I could move on to various visual modes, regular expressions, and about 80 billions ways to get around the screen. 

This then is the tutorial I wish I had found.  It is how to use GVim as Notepad.  GVim is the graphical version of Vim.  It's the same as the shell-based version, but you get to use your mouse and there are menus.  What I am about to present doesn't show any of the power of Vim, but it will help you become comfortable enough that the myriad other tutorials on the web can be of assistance.

Modes

The first thing to understand about Vim is that it is modal.  Most editors are modeless.  Letters are entered by typing them.  Control keys do special things.  This is true all the time.  Control-S will invoke Save in notepad no matter what you are doing.  Not so in Vim.  Vim has different modes and different commands will have a different effect depending on which mode the editor is in.  It is important to keep in mind which mode Vim is in.  For now there are 2 modes you need to know about. 

First there is "normal" mode.  Vim always opens in normal mode.  This is the mode where you enter commands.  Whether you want to save a document or change the case of a letter, normal mode is where you want to be.  The confusing part is that in normal mode, you can't actually add text to your document.  If you start typing in a blank document, not much will happen.  If you start typing in a full document, all sorts of strangeness will.  In Vim's normal mode, nearly every key on the keyboard is a command.  See this graphical cheat sheet if you want to see them all.

Insert mode is probably what you are familiar with.  It's just like being in Notepad or Pico or even Word.  Insert mode works just like you expect an editor to.  The letters you type show up in the document as text.  Cursor keys or the mouse can be used to move the cursor around.  To enter insert mode, just type i.  The cursor will change from an XOR cursor to a bar cursor.  The word -- INSERT -- appears at the bottom of the screen when in insert mode.  If you are new to Vim, this is the mode you'll spend the most time in.

Esc will exit insert mode and return to normal mode.

Basic Editing

If you are using GVim, editing is simple.  Just enter insert mode and it works the ways Notepad works.  You can select text with the mouse, cut with <ctrl-c>, paste with <ctrl-v>, cut with <ctrl-x>, and delete with the del key.

To save your work, hit esc to go to normal mode.  The combination :w (write) will save the document.  Many commands in Vim are prepended with a colon ':'.  There are good reasons for this, but we won't go into them now.  Just accept that they are needed to increase the number of available commands.

Exiting Vim can be done via the menus or by issuing the :q command from normal mode.  If you want to quit without saving, :q! will accomplish that.

To load a new file, I recommend using the menus at this point in your Vim career.  If you must, :e filename will load filename.

Search & Replace

Often times you'll need to find something in your document.  To do so, enter normal mode (hit esc).  Type / following by the word you are looking for.  For example, /Steve will find the next time Steve appears in the document.  As you type, you'll notice two things.  First, the cursor immediately jumps to the first S following the cursor, then to the first St, etc.  It's doing an incremental search as you type.  In this way, you can stop when you have reached the desired location.  No need to type the whole word.  To stop, just press <enter>.  The other thing you'll notice is that all instances of the word Steve have been highlighted.  This makes it easy to see all the places in your document where the word appears.  To go to the next instance, press n while still in normal mode.

To replace some text, say the word "from" with other text, the word "to", do this in normal mode:  :%s/from/to/g.  :%s means substitute on every line of the document.  The /'s separate the parts of the command ala regular expressions.  The trailing g says to replace all instances on each line.

If you already search for the word you want to replace, the syntax can be simplified by leaving out from.  Thus :%s//to/g will replace whatever was last searched for with "to".

Conclusion

This should be enough to get you up and running in Vim.  You won't notice any advantage from Vim yet, but you will be able to edit.  Once you are comfortable with GVim as a Notepad alternative, you'll be ready to learn the parts that make Vim such a great editor.  I have a post to help with that here.

For the Non-Visual User

If you happen to be using Vim from a command-line instead of GVim, a few more commands will be necessary for basic operation.

To select text, hit <ctrl-v> to enter visual mode and use the cursor keys* to select the text you want to operate on.  If <ctrl-v> does a paste, you can use <ctrl-q> instead.

To cut or delete the selection, press x.

To copy the selection, press y (yank).

To paste a previously cut or copied selection, go to where you want to insert it and then (from normal mode) press p.

----

*This won't work in GVim.  If you know how to make it work, let me know.

Are Audiophiles Going Extinct?

This article says that they are.  More importantly, it says that people do not care as much about their music or the quality of their music-listening experience as much as they once did.  That's very interesting if true.  I wonder if this is a permanent trend or if it is temporary.  Beyond a certain point, the average person definitely does not care about the quality of their sound.  How else can we explain the explosive growth of low-bitrate MP3s?  They are moving to higher bitrates now, but when the Diamond Rio had 32 megs of RAM, people weren't listening to 192kbps tracks.  The list of high-quality audio formats built to replace the CD is long.  The list of failed products is equally long.  DVD-Audio?  SACD?  DCC?  DTS CDs?  None made more than a blip on the radar.

Of course, quality audio for music may be declining, but people are paying more attention to the audio in their home theaters now.

Get Rid Of Your Security Blankets

A while ago I took  a class on Scrum and Agile Project Management.  During the discussion on Scrum, it became apparent to me that there are several unchallenged assumptions in many peoples' minds that make accepting Scrum difficult.  People assume that Scrum/Agile takes away something they have, but in reality they don't have it.  People assume they have the assurance of a fixed schedule and proper documentation.  In reality, they have neither.  They are like security blankets.  The thought of them makes people feel safe, but the reality isn't that they help.

The Agile Manifesto prefers working software over comprehensive documentation.  That doesn't mean no documentation, but it does mean limiting the output of documentation.  Some projects create reams of paper before writing any code.  Their managers gain comfort from the idea that everything is planned well in advance.  Unfortunately, that comfort is usually founded on hope, not reality.  Projects that do a lot of documenting up front run into one of two problems.  Either the project is in a straight-jacket and can't react when the plans are wrong or it does react and the documentation gets out of date.  I have seen many a project with a lot of documentation that is completely useless because it hasn't been updated.  The trick to getting good documentation is to use it.  As with many things in life, if it isn't being measured, it won't be accurate.  Moving to an agile project model may reduce the overall amount of documentation, but it shouldn't reduce the amount of useful documentation.

But how will the team know what to do?  Won't there be miscommunication without documentation?  No.  Not if things are done right.  If "customers" are close enough to give feedback on iterations and teams work to agreed-upon interfaces and integrate often, these problems can be dealt with. 

The Manifesto also calls for responding to change over following a plan.  Agile projects work on iterations.  At the end of each iteration, the project will be in a working state and closer to the final goal.  The difficulty many managers have is that Agile projects won't commit to getting all N features done by M date.  With a more traditional waterfall model, marketing knows when the product will ship and what features will be present.  It makes life a lot easier.  Except that it doesn't.  The dates are not realistic.  With an Agile project, the team is just admitting that they don't know how long everything will take.  This means everyone can react to the reality of the project rather than making plans based on unrealistic expectations that will be shattered later.  The expected ship date is just a security blanket.  Agile makes it clear you are giving this up (or giving up control of what features will be ready), but doesn't actually make the project any less predictable.  It's merely exposing the unpredictability that is innate in software development.

Many of the objections to an agile or lean software project are based on perceived loss of control.  The trouble is, that control is not real.  Losing something you don't really have isn't actually a loss. 

Intel Atom's CISC Legacy

Interesting article over at Ars Technica discussing the CISC nature of the Atom processor and how that affects its performance compared to other embedded processors.  Does CISC have more liabilities in the mobile space than it did in the desktop space?

Vim Tip: Finding a Variable Declaration

For most programming, I use Vim as my text editor of choice.  It has a steep learning curve, but is very efficient once you get accustomed to it.  I'm far from an expert on Vim and I'm always learning new things.  From time to time I will endeavor to post about these tips so others can benefit from my learning.

 

Often times when editing code in a function, one runs across a variable that's type isn't immediately obvious.  Here's a few navigation techniques I just discovered to help find out what the variable is:

gd - go to declaration.  This starts at the top of the function/method and works down until it finds the declaration of the variable the cursor is on.  Note that this will also highlight all instances of the variable, making it easy to pick it out in a crowd.

g, - go to last edit.  When using something like gd, this helps you get back to where you were.  g; will go forward to the "next" edit if you need to bounce between locations.

Design Principle: Don't Repeat Yourself

There's a design principle I neglected to mention in my initial list but which certainly merits attention.  That principle is this:  whenever possible, don't repeat yourself (DRY).  Put another way, do things one time, in one place rather than having the same or similar code scattered throughout your code base.

There are two main reasons to follow the DRY principle.  The first is that it makes change easier.  The second is that it helps substantially when it comes time for maintenance.

I was once told by an instructor in a design patterns class that the Y2K problem wasn't caused by using 2 digits to represent 4.  Instead, it was caused by doing so all over the place.  While the reality of the situation is a little more complicated than that, the principle is true.  If the date handling code had been all in one place, a single change would have fixed the whole codebase rather than having to pull tons of Cobol coders out of retirement to fix all the business applications.

When I first started working on DVD, I inherited several applications that had been written to test display cards.  They had started life as one large application which switched behavior based on a command-line switch to test various aspects of DirectDraw that DVD decoders relied upon.  Some enterprising young coder had decided we should have separate executables for each of these tests so he made 6 copies of the code and then modified the command-line parser to hard-code the test to be run.  The difficulty here is that we now had 6 copies of the code.  Every time we found a bug in one application, we would have to go make the same fix in the other 5.  It wasn't uncommon for a bug to be fixed in only 4 places.

Shalloway's Law:  “When N things need to change and N>1, Shalloway will find at most N-1 of these things.”

This principle applies to everything you write, not just to copying entire applications.  When you find yourself writing the same code (or substantially the same code) in 2 or more places, it is time to refactor.  Extract a method and put the duplicated code in that method.  When the code is used by more than one application, extract the code into a function call that you put into a shared library.  This way, whenever you want to change something, a change in one place will enhance all callers.  Also, when something is broken, the fix will automatically affect all callers.

Note that this is a principle and not a law.  There are times when substantially similar code is just different enough that it needs to be duplicated.  Consider the alternatives first though.  Can templates solve the problem?  Would a Template Method work?  If the answer is no to everything, then duplicate the code.  You'll pay the price in increased maintenance, but at least you'll be aware of what you are getting yourself into.  It might not be a bad idea to put a comment in the code to let future maintainers know that there's similar code elsewhere that they should fix.

Keep Your Eyes on the True Goal

We recently went through mid-year reviews and I found myself repeating similar advice several times.  Here it is:  Be very clear about what your goal is and continuously reassess whether you are on track to get there.

To make this more concrete, consider a plane trip from Seattle, WA to to Austin, TX.  If you look at a map, you can see that the heading to Austin is something like 135 degrees from Seattle and the flying time might be 7 hours.  Does the pilot take off, set the compass to 135 degrees and set an alarm for 7 hours later?  No.  If he does, he could be in Little Rock by the time he wakes up.  Why is that?  Wind will push the plane around.  If the pilot doesn't correct for the wind, he won't end up where he expected.  Instead, the pilot constantly monitors his location and recalculates the necessary heading and speed to get to Austin on time.

There are two tendencies I've noticed among developers which can be rectified by following the example of the pilot.  The first is the tendency to fail to monitor progress.  The second is to focus on the trip, not the destination.

There are three aspects to the tasks most developers are asked to accomplish.  They are called upon to implement some functionality by a specified time and in a quality fashion.  When a developer decides up front what work needs to be done and then plows through it without reassessing along the way, he runs a high risk of failing to deliver.  There are too many unknowns (and unkunks) for a plan created at the onset to be successful.  Without reassessing on a regular basis, the developer won't recognize that he is behind and thus will not be able to correct.  Like the pilot who lands in Little Rock instead of Austin, the developer who does not evaluate progress along the way will get to the end of the project and realize he still has a month of work to do.  This is one of the secrets of Scrum.  It forces reassessment on regular intervals.  Another technique I have found to work is to break up your tasks over the project (or milestone) into granular work items.  Each item should be a few days at most.  Then, at least weekly, mark off progress against them.  Keep track not of how much time you've spent so far but rather how much time is left and compare that to the amount of work left.  When work remaining exceeds time left, it is time for a course correction.  Because quality cannot be sacrificed, either move out the delivery date (if possible) or cut features.

A different problem is when a developer mistakes the assigned tasks for the goals.  An example comes from the test development side of the world.  Let's say the goal is to test a new video manipulation API.  This Core Video or DirectX Video Acceleration or something.  To test whether this works correctly, it is determined that it makes sense to write a simulator.  The test developer starts writing this simulator.  Along the way, he confuses his real goal (test the API) with his task (write the simulator).  If, at the end of the milestone he has a great simulator but hasn't had time to actually use it to test the API, has he succeeded?  In his mind, he has.  Unfortunately, he is like the pilot who confused "Fly at 135 degrees for 7 hours" as his goal instead of "Land in Austin."  What is the solution?  Always keep your eyes on the real goal.  If the simulator is taking too long to complete, consider alternatives.  Is there a way to cut a feature and still test most of the API?  Would it be better to cut the losses and approach from another direction? 

Constant course correction can only work if the pilot knows his true goals.  Having the wrong goal in mind or not recognizing when he has flown off course is disastrous for a pilot.  Likewise, not noticing you are behind or that you aren't going to accomplish the real goals of your project is disastrous for a developer.  Work with your manager (or program manager) to understand what the reason for your project is and always keep that in mind.  This way, you'll be able to correct course before it becomes a problem.  Don't mistake your tasks for your goal.

 

Note:  I'm not a pilot.  Some of what I say about planes is inevitably incorrect.