Fast and Loose Innovation

I've been writing code for 30 years now, a lot has changed in software development in coding syntax, design-code-test cycles, and of course the amount of data that can be processed by systems and the speed at which advanced algorithms can be processed. However, one thing that works to my advantage with my longevity in the business is my degree of pattern recognition.. 

I see essentially the same patterns repeating themselves again and again. Often we think of these as solution patterns, like the bubble sort and other basic methods for ordering a collection of items.  But patterns can also be errors- systems making essentially the same errors that were made when addressing similar problems 10 or 20 years ago.

The patterns do improve incrementally, and  of course they are executed with more speed and precision. But analytical transactions for decades have shown the essential characteristics of:
1. modularity, or tight cohesion of operations that should be bound closely due to similarity of function or order of execution
2. symbolic substitution, or the abstraction of variables that are not fixed elements in determining the solution.
3. successive improvement in accuracy through iteration,
4. loose coupling via polymorphism, or standard syntax and operations hiding inner complexity in processing data
 
This all makes sense up to this point- forward progress builds on the four foundations I identified and is accelerated by Moore's Law, but about 15 years ago, two major currents were taking place:  first, wages for technically proficient IT personnel were skyrocketing and the cost of systems development was greatly increasing as a proportion of corporate spending.  Second, there was an increasing realization that time to market dictated winners and losers, and increasingly the speed of IT development dictated time to market timing.

So Management looked for ways to cut corners, they lobbied congress for increased VISA's for IT workers in the U.S., they outsourced significant portions of software development without well honed processes for handling the handoffs between on-shore and off-shore personnel, and they cut corners on the traditional, well-documented "waterfall" methodology for software development and over time morphed it into something called Agile development.  While some organizations were able to adapt and win with these transformations, many organizations are still fighting an uphill battle with cost overruns, delays, lost customers, and employee burnout.