Technical Background

A team needs members with a technical background. These guys are the ones that ensure everything works. They can write the hard bits of code, keep everything performing and make sure the code is maintainable. Without these guys, you will have problems. On the other hand, sometimes those with a technical background view problems solving as a puzzle, where the solution is the end in itself.
In my experience, team members with a less technical background are more empathic toward the user community. They view problem solving as a means to a goal. A customer focus is what makes a project successful. Software can be successful despite technical problems, but it cannot be successful if doesn’t meet the customer’s needs. Having some non-technical people helps prevent your resident technocrats from building a system that will service 10,000 concurrent users, but doesn’t meet the needs of the 329 users you have today.
Perhaps at the extreme end of this spectrum is the extreme programming practice of including a customer on the development team.

Experience

In expert versus novice programmers, I make the distinction between skill level and experience. Age, skill and experience often correlate, but not always. Your young apprentice programmers are important to have on a team. They can react faster. Sometimes not knowing better is a bonus. They’ll do things more senior team members don’t want to do. They do things they’re not supposed to do, sometimes for the benefit of the team.
I think people underestimate how long it takes to actually become an expert, as I mention in my previous post. But, then they also underestimate how similar all software projects are. They also overestimate the impact of their toolset. Generally the technologies in the laundry lists that form most job listings are not the critical success factors for that project.
People want a programmer with who can hit the ground running, who is familiar with all of the teams tools, languages and libraries. But, this is a monoculture as well. Sometimes you need people who aren’t from your toolset culture to point out where you have swallowed the hook too deeply or to introduce new tools and techniques from their alternate background. Sometimes their learning process can teach you something.
While some experience is valued in programmers, age sometimes isn’t. However, you need the older, more stable people as well. They keep the young ones from repeating the mistakes of the past. They keep the project on an even keel. It helps to have an older perspective on the team. I think they help keep attention focused on what matters. The tension between the older and younger team members helps keep the team on track.

Traditional Diversity Factors

I have no opinion on the impact of race. I have an opinion about women on software teams. I wish there were more of them. I have worked on software development teams with diverse nationalities. I regard that experience as positive. Different cultures have different tolerances for risk, different respect for authority or tradition. I think cultural diversity on a team is a good thing.

Managing Diversity

Over the years, there have been many methods proposed of matching people with the jobs they are best at. The solution that works best is to ask them. People gravitate toward the tasks that require their special skill sets. Good team management means hiring diversely and allowing that migration. On a well managed, diverse team, people will self select the jobs they are best at. As a result, the diverse team is more resilient and I believe potentially more successful.

First there was CORBA. Then insane complexity of CORBA was supplanted by the intolerable complexity of EJB. Influenced by an agile mindset and the power of Unit testing, a group of java programmers began to construct simpler alternatives to EJB. Thus, the inversion of control frameworks were born. Martin Fowler came along, clarified and renamed the pattern dependency injection. This activity has originated in the Java world, but the pattern applies in PHP as well.

It is heartening to see an industry solve a problem over the course of a decade, moving from complex vendor driven middle-ware to simple patterns. The thing I like most about DI is how dead simple it really is.

Fowler’s article is a must read on the topic. However, I have two problems with most of the introductions to dependency injection. One is the use of irrelevant girl kisses boy style examples. The other is the over-emphasis on the container. The whole point of dependency injection is to move away from invasive component architectures, such as EJB. From my point of view, it is far more interesting to explore what impact dependency injection has on your design than what features your container has.

With that in mind, I tried to write an introduction to dependency injection that avoiding talking about DI containers and that tried to use real, relevant examples. For an example, I started with a typical, run of the mill dependency, torn from a popular PHP library that shall remain nameless, but easy to guess. Then I build on that with a plug-able backend in a common PHP style and then again using dependency injection.

For me dependency injection is relatively new territory, but one that I feel is an important technique. I’d like to see this technique become more widespread in PHP, especially in the current crop of frameworks.

I have a significant bias, but I think you should read the article. I hope you find it useful.

I know I would like to see an update to this groundbreaking work with more modern interpretations of what have become the canonical patterns.

Rebecca talks about how the interface type has come to replace the C++ abstract class. One area that I am interested in is how Duck typing may replace the interface construct in many of the classic patterns. I’m not sure that patterns intended for statically typed languages such as C++ and java necessarily have the same implementation in dynamically typed languages, such as smalltalk, Ruby and of course, PHP.

For the last few weeks, I’ve been doing an interesting educational exercise. I’ve been simultaneously reading Design Patterns, Refactoring, Refactoring to Patterns, and Agile Software Development. I’m finding the cross-referencing between each of these highly complementary books to be be educational. I’m hoping to add Design Patterns Smalltalk Companion and PHP|Architect’s Guide to PHP Design Patterns (Jason Sweat’s book) to get a more dynamic perspective in the mix. This is my reading project for the next few months.

That 10 years after first reading the Design Patterns book, i am still studying from it shows how much of a classic this book has become.

• The departure of the hyper-enthusiasts – “The Java hyper-enthusiasts have left the building” (along the lines of this.)
• The New Methodology – Martin Fowler describes Agile methodologies — recently updated.
• PHP on Caucho – PHP on the JVM.
• XML 2.0 – some thoughts on XML 2.0.
• Web Patterns – Under construction — check back later.
• Web Design Patterns.
• Carnival of the Agilists.

I’m currently adding UTF-8 support to and generally improving WACT’s “liberal” xml/html parser. A few sources of tests cases and information:

• Who knows a title from a hole in the ground?
• FeedBurner feeds give heartburn to PHP XML parsers?
• Web SGML and HTML 4.0 explained
• CDATA confusion
• Comment syntax in SGML and HTML
• Empty elements in SGML, HTML, XML, and XHTML
• Comparison of SGML and XML
• Extensible Markup Language (XML) Conformance Test Suites
• Ian Hixie’s HTML parsing test cases
• Conformance Testing for XML and Related Technologies
• Universal Feed Parser liberal feed parser with many test cases.
• XHTML test cases

• UTF-8 and Unicode FAQ for Unix/Linux
• bad UTF-8 test files
• The W3C Markup Validation Service: Tests

Agile software development methods including extreme programming (XP) have been growing in popularity and maturity alongside an increase in the popularity of open source projects. One of the fundamental principles of XP is early and often feedback. The value of feedback is obvious to anyone who has used a strange shower. The shorter the time between turning the knob and feeling the temperature change, the faster you can arrive at the perfect temperature. Nobody likes flailing between too hot and too cold in a lazy shower. In the same way, immediate feedback on a development project can avoid wasted effort and produce a better result faster.

In XP and other Agile programming methods, an automated test suite provides important early and often feedback. It is far better to catch a bug in the test suite than for the customer to find it. But, how do you know if your tests are testing all of the features of your program?

The XP practice of test driven development (TDD) demands that you only add features for which you have a failing test case. Therefore, you write the test first, and then write the code to satisfy it. For a theoretical perspective, this should give your code 100% code coverage. Every line of code should be covered by a test. Unfortunately, developers are human and any technique that relies of the perfection of people is doomed to fail. Unfortunately, the converse of the TDD coverage assertion is simply not helpful: If you don’t have 100% test coverage, you aren’t correctly doing TDD. If you are not doing TDD, how do you gauge the quality of your test suite?

Open source projects present a special challenge for the concept of TDD. No project wants to reject useful contributions, yet many contributors may not write tests for their contributions. Even if tests are required, there is no way to tell if they were written first, as in TDD, or after the fact. XP would have the code author writing the unit tests. In an open source project, this might not be possible or likely, although still desirable.

While certainly not perfect, code coverage tools do provide an indicator of how well tested a section of code is. Code coverage reports are a feedback loop to help test writers find weak areas and to better target tests to specific lines of code. Just picking a file at random from PHP reveals low-hanging testing fruit: zend_operators.c.

I feel that a coverage report can be an important tool in an open source project. More so than in other kinds of projects. Test writing based on a coverage report inherently lends itself to the distributed nature of open source development. Many people have a stake in the stability, robustness and reputation of PHP. Tests are easy enough to write, easier perhaps than many other ways to contribute. Hopefully easily available coverage reporting would help coordinate testing activities and draw people in that might not otherwise participate.

Ideally, contributions would be accompanied by test code contributed by the same author. I also don’t think it is out of line to look at untested contributions with increased skepticism. Coverage reports can help identify contributions that might cause future problems. One can hope the existence of coverage reports might inspire contributors to include better tests by exposing their code as untested. My (unverified) theory and hope is that the feedback of coverage reports can help inspire a culture of testing on an open source project.

While there are areas (such as GUI apps) where unit testing is difficult at best (thread safety may be another), PHP overall is very testable. The stability and reputation of PHP can only be improved by increasing the coverage of its unit tests.

I am looking forward to seeing the availability of an automated http://cov.php.net/, if indeed that is planned.

Ruby on Rails today looks poised to eat Java’s mindshare on the web tier. If not Rails, then something else.

It does seem like Java’s mindshare is on the decline, along with sun’s fortunes. There is some evidence that rails is drawing mindshare from PHP as well. (Mind share is the grown-up way of saying “what the cool kids are doing.”)

So why isn’t PHP that something else?

i think its the unit testing, “You aren’t gonna need it,” “Do the simplest thing that could possibly work” folk that are dismantling Java’s mindshare. As they gain it, Java loses it. Some of that is showing up in the java community itself in concepts like POJOs and dependency injection. Some of it shows up in an exodus from Java to dynamic languages.

So why are the agile leaders like Martin Fowler and Dave Thomas working with Ruby instead of PHP? David Heinemeier Hansson directly credits these guys with influencing his decision to stop programming rails in PHP and move to ruby. I suspect that its the dynamic, smalltalk, like features of Ruby that appeal the the agile methods leaders.

Rails could have been a PHP framework, but wasn’t. Although, I’m not sure that it would get the attention it is if it were a PHP framework. The rails, ruby combination may be greater than the sum of the parts.

I can’t speak for anyone else, but I have some ideas about why PHP might not appeal to the agile method and java is too complicated folk:

• Too many globals – globals make unit testing more difficult and prevent integrating work from different developers, let alone different organizations.
• No culture of objects – PHP has made great strides in this area, but I don’t get the impression that many of the folk that work on and with PHP really buy into object oriented programming the way that the agile methods folk do.
• Crowded namespaces – How many functions is PHP up to? Constants?
• incomplete component model – This is another integration thing. PHP has too many islands and not enough continents. (I’m talking beans here, but definitely not EJB.)
• No mature frameworks – Ruby has been around a long time, but didn’t really start gulping mindshare until Rails came along. See The Value of MVC.
• No closures – Its a dynamic thing.
• Expert level gotchas – hopping on the learning curve is easy. Reaching the top, not so much.

I’ll elaborate on each of these points in a series of upcoming posts.

I’ll close by saying that life is not a zero sum game. The success of Ruby and Rails does not diminish the success of PHP. The cool kids were doing Java and now some of them are doing Ruby. Oh, well. Its disappointing that more of them aren’t doing PHP, but PHP has many things going for it. That, however, is a different series of posts.

Cognitive science research on problem solving tries to examine the difference between experts and novices in a domain. Rather than distinguish between developers and architects, I think it is better to distinguish between experts and novices at programming. The Dreyfus model of skill acquisition details five skill levels to help in this task. Here is a summary from Coding Horror:

Level 1: Beginner

• Little or no previous experience
• Doesn’t want to learn: wants to accomplish a goal
• No discretionary judgement
• Rigid adherence to rules

Level 2: Advanced Beginner

• Starts trying tasks on their own
• Has difficulty troubleshooting
• Wants information fast
• Can place some advice in context required
• Uses guidelines, but without holisitic understanding

Level 3: Competent

• Develops conceptual models
• Troubleshoots on their own
• Seeks out expert advice
• Sees actions at least partially in terms of long-term plans and goals

Level 4: Proficient

• Guided by maxims applied to the current situation
• Sees situations holistically
• Will self-correct based on previous performance
• Learns from the experience of others
• Frustrated by oversimplified information

Level 5: Expert

• No longer relies on rules, guidelines, or maxims
• Works primarily from intuition
• Analytic approaches only used in novel situations or when problems occur
• When forced to follow set rules, performance is degraded

The Java World article laments about companies that advertise for experts, but don’t interview for it. I have to say this struck a nerve with me. Multiple choice tests like the Zend Certification disappoint me. Tests like this don’t measure skill on this scale at all, they measure exposure.

In the past, I’ve used a coding sample as an interview question. The code, about 250 lines, was distilled from an existing system and is horribly bad in so very many ways, but functional. By showing the code to an interviewee and asking them what they would do to improve it, I found I could get a good idea of their skill level. Novices simply had no idea what to do with it and would just move code around. Sometimes they would insert comments, trying to make the code “better.” Some candidates only found and fixed one problem (there were several major ones.) No one fixed all the problems. The one person that I interviewed and didn’t have look at the code (for time reasons), we had to let go. He simply fooled us at his interview about his skill level. The thing is that I am certain he could have passed a Delphi certification test. (like the one I took at Brain Bench.)

Moving from novice to expert programmer takes a very long time. From Teach Yourself Programming in Ten Years:

Researchers (Hayes, Bloom) have shown it takes about ten years to develop expertise in any of a wide variety of areas, including chess playing, music composition, painting, piano playing, swimming, tennis, and research in neuropsychology and topology. There appear to be no real shortcuts: even Mozart, who was a musical prodigy at age 4, took 13 more years before he began to produce world-class music. In another genre, the Beatles seemed to burst onto the scene with a string of #1 hits and an appearance on the Ed Sullivan show in 1964. But they had been playing small clubs in Liverpool and Hamburg since 1957, and while they had mass appeal early on, their first great critical success, Sgt. Peppers, was released in 1967. Samuel Johnson thought it took longer than ten years: “Excellence in any department can be attained only by the labor of a lifetime; it is not to be purchased at a lesser price.” And Chaucer complained “the lyf so short, the craft so long to lerne.”

So you can’t become an expert without experience. However, you can have experience without becoming an expert. Some people just put in their time and never develop themselves. So these people may have the answers to the certification trivia question for their specific environment and be able to get past the HR resume screeners with their buzzword detectors, but they will not have the impact that a true expert would have in the same situation. Just like our Delphi Dud.

They have released a coverage reporting tool, Spike PHP Coverage, for PHP that works with XDebug coverage data. It works with Simple Test and it seems to be able aggregate the results of remote test runs, such as for web based tests. I have wanted something exactly like this. I can’t wait to get the chance to generate a consolidated coverage report for WACT.

It seems that they have the capability for generating and aggregating code coverage reports for PHP itself. I haven’t had a chance to check this out, but its something that I have always wondered about.

Additionally, it looks like they accept contributions of Test Suites. If I understand this correctly, it means that I can upload the WACT test suite and that it can be used to test and generate coverage reports for PHP 4 and for PHP 5 as well as the framework. If this is true, this is a big deal. As there are more and more applications with automated test suites, it only makes sense to aggregate them to test PHP itself.

It also seems that they have embraced Simple Test for php testing. Congratulations, Marcus. Simple Test is a fine piece of software.

I think code coverage measurements are important for gauging the quality of a test suite on open source projects, where there can be less formal development practices and a large number of contributers of varying skill levels and motivation.

I’ve been following an iterative development cycle where I deploy to my client’s site every Friday. (My old ERP colleagues would be mortified — install on Friday? My response: automated testing) Thankfully, last Friday’s upload went through, although it was the last internet connection my internal modem would ever make. It took me a couple days to rule out line quality and ISP problems and determine that the problem was hardware failure.

I have a PC with windows XP available. Its not mine and I really don’t use it except for checking web sites in internet explorer and playing an occasional game. So Sunday night I tried using it for some serious browsing for the first time. After about five minutes, I realized that I can’t stand Internet Explorer. The lack of a tabs felt like a missing finger. I had no idea how much I used that feature in both Safari and in Firefox. Also, the fonts under XP just plain looked bad.

Tiring with IE, I installed Firefox. I’m impressed with Firefox on the PC. I prefer Safari on the Mac, but I do use Firefox on the Mac for web development because of its ability to control and inspect headers, cookies, the DOM, etc. I don’t use Firefox on the Mac for day to day browsing because its interface feels a little bit off. No such feeling for Firefox on Windows. Win Firefox felt very comfortable. I like it.

When I upgraded this blog to WordPress 1.5, I integrated this blog with del.icio.us. I have been using del.icio.us for my “to blog about later links.” del.icio.us is very helpful when you have to browse in a foreign environment.

Sunday afternoon I had to start contemplating the idea that I might have do next Friday’s deployment from the XP machine or even temporarily move my development environment to the Windows machine. It didn’t take long to rule out this option. OS X comes pre-installed with Apache, PHP, CVS, SSH, and a bunch of other things that I probably take for granted. Windows has none of that. I decided that setting up a development environment under windows would probably take too much time.

I still remember the trauma from the last time I tried to ftp from a stock Windows machine. Its probably better now, but I still decided that I didn’t want to try that route. Besides, I just got my automated deployment scripts working on the Mac.

So Monday morning, I drove 60 miles to go to pick up a USB Modem at Best Buy. Now here is the problem with owning a Mac. Best Buy has two models in stock. Neither has OS X drivers. I bought one anyway in case it might “just work.” It doesn’t. So I thought I was stuck until the USB modem I ordered from online arrives.

But then I realized that I had enough ethernet stuff in the basement to hookup a network between the PC and the Mac. I have never done this before because I simply don’t use the windows machine enough to care. Hooking up the network was a breeze. A couple wizards, a few property dialogs and a half hours worth of reading was all it took to set up internet connection sharing on the PC side. The Mac recognized the new network immediately and without configuration. Sweet!

So, I am once again connected and now just waiting for my Mac compatible USB modem to come in the mail.

There is an added benefit. Over the last year or so, I have been gradually switching from table based layout for my web pages to a CSS based layout. I’ve quite a bit of trouble maintaining the same layout of older pages in CSS due to browser variances. Some of my table based layout components are somewhat ambitious for CSS. I never used to do much testing on Win IE because I was pretty familiar with how the table based layouts would look. Win IE is very stable on these types of layouts and they almost always looked exactly on IE how I had developed them on the Mac. Additionally, I had long ago worked out the cross browser kinks and was pretty much recycling the same debugged layout components.

This is definitely not the case for CSS. I don’t yet have a full library of debugged layout components and I have to say my understanding of cross browser kinks is incomplete, although growing. If you browse around on this blog in IE and in Firefox, you can see that while not terrible, it doesn’t render some pages correctly on IE.

Testing under IE was a pain for me. Editing CSS files on Windows kept bringing up FrontPage instead of a plain text editor, which is what I wanted. Now that the machines are networked, I can now just point windows to the test environment that is already running on my Mac and preview right from there. I also have occasional access to a Windows Laptop, which will make this type of testing a lot easier.

I’m too far behind this week, but maybe next week I will finally figure out what is wrong with the style sheet for this blog under Win IE.

I have to agree with the sentiment. I quit my last job (in 2000) to work in web development largely for this reason. I worked on a custom ERP package that was deployed in a few dozen manufacturing plants. Deployment for developed code was a nightmare. It wouldn’t surprise me if the effort involved with deploying a change was an order of magnitude greater than making it. Testing, writing installation scripts, testing the installation scripts, sending a person to the plant to perform the installation and train the users, repeating for every plant. Pray that there were no database changes.

The thing that attracted me to web development was that it felt much more ‘light weight,’ and frankly, just more fun.