Except the codec wasn’t really a codec.

The link wasn’t really to YouTube.

The email wasn’t really from her friend.

Here we are in 2010, 65 years after the advent of "program instructions as data" (von Neumann architecture) and the brilliant innovations that propelled computing forward now haunt us.  The power and complexity of computing is amplified by the connectivity of the Internet, creating an ecosystem ripe for exploitation through clever social engineering combined with some technical trickery.

And it’s not fair.  It’s not fair to the users who suffer, who lose data, whose systems are held hostage.  It’s not fair and the blame for calamities cannot be laid at the feet of careless or naive users.  Daily computing tasks should not require constant vigilance, let alone deep technical knowledge.

Many of us schooled in the last century hold on to notions of software that are simply unsafe for many of our users.  We insist on building software that must run in environments that are not inherently safe.  But there are trends afoot: the closed ecosystem of the iPhone and the iPad, the coming Windows Phone 7 closed ecosystem, and the increasing ubiquity of Internet applications.  Slowly but surely we are moving toward paradigms in which the average user will have much less to worry about because software is vetted and sandboxed.

As a developer I don’t want to be sandboxed in every situation.  I want to be able to make educated decisions about software and run it on my devices, but the default should be safe and secure.  It’s shameful that things are still so treacherous.

I’ve long thought that Donald Knuth had it right when he titled his books The Art of Computer Programming.

Creating software bears some resemblance to art as in the work of an artist.  It bears an even stronger resemblance to art as in the work of an artisan.  Software artisans use the tools and techniques of modern software development to create the wide variety of software that entertains us and runs our businesses.

Eric Sink had it right when he put Software Craftsman on his business card.

Sadly some of the most enthusiastic artisans in our field are disregarded out of hand as geeks and nerds when in fact they have creative and curious minds more commonly associated with artists like poets and sculptors.

I love the way Kate describes the art of software:

The number one response by my aunts/uncles/friends parents etc was “you’re programming? But you were always so creative as a child…” – people need to be told that this is a very creative field. I make business solutions out of ones and zeroes. I change people’s working lives, the entire 8 hours they spend at the office every working day, forever – using nothing but the skin on my fingertips.

- Kate Gregory

I worry that software development as a discipline is stunted by its false reputation of being science-y when in fact it appeals to people who are creative and innovative.  I’ve noticed that a disproportionate number of my colleagues in software development are musicians and/or fans or even creators of comic books, science fiction, fantasy fiction, and role-playing games.  Those types of pastimes are qualitatively different than watching reality TV or playing sports. *

The dirty secret of comp sci is that most us who trained as "computer scientists" do precious little that could ever be considered true computer science and certainly not science at all by many definitions of science.  We are artisans abiding by the platforms and boundaries defined by computer science to create truly amazing and useful things.  We are the glassmakers, sculptors, and blacksmiths of our time.

—-

Recommended reading: Knuth: Computer Programming as an Art

* Relax, I’m generalizing. Lots of programmers watch reality TV (though I judge them for that) and/or play sports.  Lots of programmers are not musicians (like, say, me). The set of all RPG players (A) overlaps the set of all computer programmers (B).  B is not a subset of A.

(Take a moment to let it sink in that Future Coder might very well be Future You.)

OK, take a deep breath and exhale slowly.  Future Coder can’t hurt you unless your code is still in use after someone invents time travel.  And if Future Coder comes back to stop you from writing that code, Future Coder won’t have a job fixing your code in the future… (ah, the paradoxes of time travel).

Code has a past and a future.  When you inherit code or parachute into a project, you must embrace the legacy of the code.  Before you mentally skewer the coders who came before you, remember that you probably lack insight into the context in which previous coding decisions were made.  It is, of course, quite possible that the programmer before you was a complete idiot.  More likely, though, that programmer was working under some constraints that drove him/her toward the solution you now curse.

Although it is conceivable that you are the last ever maintainer of said code, you will likely pass on the responsibility to someone else eventually.  You have a moral obligation to your fellow developers to leave behind something maintainable.  Consistency and predictability in your code is a major part of maintainability.  That means if 30 data entry forms use strongly typed DataSet objects but you prefer NHibernate, you write the 31st data entry form using a strongly typed DataSet.  It is irresponsible to create Frankencode systems.  Your personal whims do not trump your responsibility to your fellow developer.

Maintenance developers do not get enough credit.  It is frustrating and often thankless work trying to wrap your brain around systems that have been tweaked by countless programmers to adapt to changing business needs.

The next time you curse someone else’s code, remember that somewhere out there another developer is cursing something you once coded.

<endRant />

There is lots of data out there.  My first professional gig after university was consuming text files dumped out by a COBOL app.  (They were delicious.)  You can parse XML, munge HTML, manipulate images, or decipher networking protocols.  But despite the variety of possible data sources, we end up sticking a lot of our data in relational database systems.

And then we write a lot of code for moving data in and out of the database.  And then we complain about how much data access code we have to write and maintain.  In Survival Skills for Developers, the first item for your basic survival pack is a data access toolkit (homegrown or open source or commercial).  The reason is not to be trendy or to sound up-to-date at developer conferences.  The point is to make better use of your time by relying on frameworks and libraries to do some of the heavy lifting for you.

Do Not Be Afraid

In the .NET programming world, ADO.NET is the underlying data access technology.  Many data access patterns and frameworks have been built on top of ADO.NET and yet scores of developers still write ADO.NET data access code the way they learned nearly a decade ago.  What a waste.

There is no reason to be afraid of modern data access techniques.  You do not have to rewrite your existing codebase.  .NET programmers can still use classic ADO.NET when it’s expedient.  You can mix and match tools and frameworks.  I have a small project that I’m transitioning from SubSonic (used during prototyping) to NHibernate.  Currently the code features a mix of SubSonic and NHibernate as I transition and it works just fine.

Frankly I think developers who refuse to explore alternatives are being irresponsible.  I’m not holding up any specific frameworks, toolkits, or approaches as best.  I’m simply saying that a failure to be well informed about options is intellectual laziness and we cheat our employers, clients, and stakeholders when we insist on writing everything by hand every time we need a record from the database.  We would question the judgment of a house builder relying on all manual tools, yet we mindlessly churn out the same data access code over and over and over again.

Stop Freaking Out About Inconsequential Performance

A mental roadblock for many folks is a worry about performance going down the toilet by relying on something like an ORM (object-relational mapping) tool.  If there is one serious mental shortcoming amongst programmers, it is our obsessive need to prematurely optimize everything even when there are ample computing resources to deal with our less-than-completely-efficient code.

Stop freaking out.

The JVM and the .NET runtime have proven that letting the computer handle some tedious work is totally worth it even if it’s less efficient than what you could (theoretically) write by hand.

CPU time is cheap.  Programmer time ain’t.  Spend time optimizing code only when it becomes necessary.

Do You Really Need a Database?

Simple lists of objects can be similar to tables of records.  The relationships between objects can be similar to the relationships between tables.  Nevertheless, there is a mismatch between object oriented programming and relational data (perhaps you’ve heard of the object-relational impedance mismatch).  Depending on your situation, you might not really need a relational database.  Check out some of the technologies in the NoSQL side of the world for alternatives to relational database storage engines.

What’s Available?

There are a lot of options out there for working with relational data.  Check out Barry Gervin’s article All I Wanted was My Data for some options.

What are your suggestions?  Drop them in the comment box.

An eerily similar phenomenon occurs in software development as requirements flow from domain experts to business analysts to system designers (architects) to developers.  Each communication channel in the process introduces new opportunities for information to be distorted.

I was digging through some old papers this week and found a sketch I made nearly a decade ago on the back of a Dilbert comic.  I’ve prettied it up here as a reminder to all of us to always be watchful for potential distortions as requirements flow from users and stakeholders to the people who build and test the software:

The importance of good communication in the software development process cannot be overstated.

If you are passionate about building software and want to thrive in your career, hone your communication skills.  Ask questions.  Write things down.  Repeat back what you’ve heard to validate it.  Ask more questions if things don’t make sense.  Communicate, communicate, communicate.

In 2009 I had lunch with an executive with a lot of experience managing developers.  When I asked for his biggest piece of advice for professional developers outside of learning technology, his answer was to learn to communicate effectively.

(Image credits: malias, freedigitalphotos.net, Salvatore Vuono, Danilo Rizzuti, Maggie Smith)

(I bet you see where I’m going with this.)

Back at the office, where your mouse and keyboard fret over your safe return, there awaits an entirely different survival scenario:

Survivor: Cubicle.  Outcode.  Outbuild.  Outlast.

Surviving as a software developer is more than stringing together some lines of code that read and write from a database.  Sure, those are basic skills.  To survive in the woods you obviously need to walk and breathe, but you also need to start a fire and build a shelter.

The following 8 items form a basic survival pack that can get you through most modern software development forests:

1. Data access toolkit (homegrown or open source or commercial)

If reading and writing from a database is a basic skill like walking, then a good data access framework is your walking stick.  I’m a .NET guy so I’ve worked with NHibernate, SubSonic, Entity Framework, and a couple homegrown solutions.  An ORM tool is not necessary.  Use the core ADO.NET classes directly if that works for you.  What you absolutely should avoid is writing all your data access code from scratch every time you need to work with data.

Personally I recommend you learn the basics of several modern data access toolkits and learn about some of the code generation solutions available on the market so you can make informed choices in your projects.

2. Regex

OK, let’s crank the controversy meter up one notch.  Regular expressions are like waterproof matches.  Sure you don’t need them, but they can make things a heck of a lot easier in some circumstances.  When it comes to text processing, there are lots of approaches that can work, but a regular expression can turn an arduous coding exercise into a single line of code.

If you use Visual Studio, I highly recommend you explore the regular expression syntax available for the Find dialog box.  It is amazingly useful once you learn the wonky syntax.

3. Unit testing

Let’s try one more notch higher on the controversy meter.  You must be able to write and execute unit tests for the language and platform you are developing on.  Unit testing is like a compass.  You don’t always need it and you won’t need it constantly, but it can be priceless in many situations.

Unit testing has its fair share of zealous proponents and vehement haters.  I like to think I have a nice moderate middle-of-the-road perspective.  Unit testing is incredibly important in some circumstances.  I am not an adherent to the Church of Test Driven Development.  I do not usually attempt to achieve complete code coverage in unit tests.  But without the ability to write and execute unit tests, you risk wasting precious time walking in circles as you test the same things over and over again manually.

4. Basic printing, reporting, and charting

Not every system has to print or produce charts and reports, but lots do.  Even web site developers have to consider how their pages will look when printed and perhaps generate PDF documents for printing.  Regardless of whether you write desktop apps, web apps, or middleware, you should be able to print and generate some basic reports.

If you are a .NET developer like me, check out some of the vendor solutions like Telerik Reporting, ActiveReports, LogiReport, and XtraReports.

Printing, reporting, and charting are about user needs.  Users need to print invoices and put charts of projected vs. actuals into the shareholder report.  Sometimes you need paper to get your business done.

5. Internet: sending email and downloading HTTP content

Back in the last century, we invented this thing called the Internet.  It’s nice.  It lets software programs on different systems communicate with each other.  And the Internet has this way of creating new opportunities so you never know when your software will have to become Internet-aware.

Sending email and accessing HTTP content are pretty easy tasks to accomplish in most languages given the plethora of libraries available (e.g., classes built into the .NET framework).  And even if your current coding project does not require Internet access, you might see new opportunities once you’ve learned how easy it is.  For example, you might decide to automatically email crash reports to your help desk.

If you pass other hikers in the woods, you might not need to talk to them, but then again, maybe you will.  Better to be prepared.

6. HTML and basic CSS

Regardless of the type of software you build, I firmly believe you should know a little bit about HTML, XHTML, and CSS (and some XML would be good, too).  If you don’t build web applications, you don’t need to master these technologies, but a grasp on the basics will come in handy.  Examples:

• creating HTML formatted emails to send to users
• updating a development team web site
• customizing a product wiki
• writing end user documentation
• updating the company web site (happens in small companies)
• setting up a blog for the marketing department

(X)HTML + CSS is the lingua franca of the web so just learn it already – no fancy survival metaphor required.

7. Scripting or command line development

Many software components need to work with standard input / output streams or simply do not require a graphical user interface.  If you only know how to build Windows or web applications, you are severely limiting your ability to efficiently handle the many small tasks that often accompanying development work like parsing a file or deploying software updates.

If you work in Windows, learn to use the command line along with some basic VBScript and batch (.bat) file "programming."

If you work in a Linux / UNIX environment, you probably already know the things you need to learn.  In case you don’t, I suggest digging into some shell scripting, sed, awk, and grep after you’ve mastered the command line fundamentals.

If you work in an OS X environment, you just need Photoshop.  No, I’m kidding.  Learn the UNIX utilities.

Decent command line and scripting skills are your ferrocerium – more hardcore than a match and much more durable.

8. Services, daemons, and cron

Sometimes software just needs to run on its own.  If you’re a Windows developer, learn how to build a Windows Service and how to run programs with the Task Scheduler.  If you’re in a Linux or UNIX environment, learn how to write a daemon and schedule cron jobs.  Even if you don’t specifically need a Windows Service or a daemon process in the foreseeable future, understanding the concepts will make you a better programmer.  Unlike desktop apps that can be easily restarted or web apps in which code executes in short bursts, daemons and services are long-running processes that require careful creation.

Thinking about responsible resource management in a long-running process will make you a better programmer, just like building a campfire in the wilderness from collected wood will make you that much better at building your next campground sing-along-and-make-some-s’mores campfire.

Mmmm…  s’mores…

Let the Comments Flow

Agree?  Disagree?  Think I missed something?  Leave a comment and let’s discuss.

(photo credits: Alaskan Dude)

This site (ardentdev.com) will now be updated every Wednesday, so stay tuned via the RSS feed or subscribe by email.

New Design

I decided to simplify the design, opting for a clean white background, more whitespace, and less colour.  Hope you find it easier to read.

Page Size Optimization

One of my goals in redesigning the site was to reduce the page size a bit.  I like to include images in my posts so going overboard on optimization wasn’t going to make total load time much better, but small improvements are always welcome.

Thanks, YSlow

To measure my optimization progress, I used YSlow.  YSlow is a Firefox add-on that integrates with Firebug.  Firebug is a web development add-on for Firefox.  I highly recommend you hook yourself up with the Firefox + Firebug + YSlow triad of awesome if you do any web development.

My quest for page size reduction began with cleaning out the old WordPress theme.  For example, by removing Recent Posts and Recent Comments from the footer, I reduced the amount of HTML and gained the happy side effect of reducing the number of database hits required to build a page.

ArdentDev.com serves its HTML content compressed.  At the time of the design switch, the old theme was sending down 13.7K of compressed HTML for the home page.  The theme changes reduced that to 12.4K.  Dropping the number of posts shown on the home page from 10 to 5 brought things down to 7.7K.  That’s 6K less than the original (not including images from posts no longer visible on the home page).

I did a clean up of the CSS files for the site as well.  The site has a CSS file for web browsers and a CSS file for printing.  With some editing and some whitespace removal, the CSS files went from 4.6K down to 3.2K.  It was not a super thorough scrub of the CSS, but I trimmed another 1.4K.

The previous theme referenced nine images via the CSS file.  I dumped all those extraneous little images, which knocked  29.8K off the page size.  I then introduced a new logo that added back 2K.  That left a 27.8K reduction, not to mention 8 fewer HTTP requests.

I also dropped a banner ad that removed 0.4K of JavaScript and an additional HTTP request (plus another subsequent HTTP request to download the ad itself).

Net Gain (Well, Net Loss Really…)

The total reduction in page size came out to 35.6K as well as 9 fewer HTTP requests (not counting the request for the banner ad).  For a broadband Internet user, the optimizations should make the home page load 1/10 to 1/4 of a second faster than before.  Not a massive savings but a rewarding exercise as I searched for ways to reduce the bulk of the home page.

Everyone’s mileage will vary but please leave a comment about your own page optimization experiences.

Previous YSlow Stats

New YSlow Stats

I tried deleting the Temporary ASP.NET Files (%userprofile%\AppData\Local\Temp\Temporary ASP.NET Files) but that did not solve the problem.

I tried deleting the obj folder.  No luck.

I tried waiting it out one evening.  I eventually feel asleep and when I awoke sometime in the middle of the night, my web app was happily waiting for input and Visual Studio debugging was completely responsive.  That proved to be a temporary salve.

The problem resurfaced again a few days later and I finally figured out the problem with my uncooperative debugger.  I have a folder containing approximately 20,000 images that are not included in the Visual Studio web project but are sitting in a directory in the web site.  I had turned on "Show All Files" in Solution Explorer to add some script files into the project.  When "Show All Files" is off, F5 (Start Debugging) works like  champ.  When "Show All Files" is on, Visual Studio becomes unresponsive.  In reality it is not "frozen" but simply taking a very long time to process those 20,000 image files.

Mystery solved.

How does a software developer change a car tire in the dark?  By the light of his laptop screen of course:

Nice!

(BTW, Jason is currently job hunting if you know someone looking to tap into that creative thinking.  You can find Jason on LinkedIn or contact him via his blog.)