I was in a discussion the other day about automation, which in itself is an extremely broad term, but we were talking around quality assurance and the importance of having some kind of automated process when the “product” was ready. Whether this be spreadsheets, reports, commercial products, applications, system builds, etc. At some level you should have automation to verify that at least at a high level the product is consumable.
Test driven development and agile development have played a huge role in the quality of software over the past 10 years and it shocks me to know companies are simply not doing this today.
My side of the discussion was around the work I did when I joined Iris Associates – which changed to Lotus, then ultimately to IBM Collaboration Solutions. While this was around software quality, the story I was in discussion about was data integrity in spreadsheets. We had several layers of automation following a build. Once the build completed (on five different platforms) a series of automated tests were run for build verification. Once the automated tests ran there were some manual tests run if the automated tests had passed. We then had what we called “automated system tests”, also known as DNA (Domino Notes Automation). These tests were much more functional in nature and not at a single API level. So you can imagine these tests using many (if not hundreds or thousands of API’s) to complete what would normally be done by a human. I remember talking to a QA person about automating Event Admin (a Domino Server feature) and he said the hundred or so tests run by DNA would take me three months to do manually – and DNA did it in about 20 minutes – every day, on every build, on every platform. At the end of the day, and I am not sure of the exact number, but there were literally 10′s of thousands of tests run every single day on almost every single supported platform for both client and server. The results of the tests were of course published to a Lotus Notes database and notification emails were generated.
Why is this important?
So on any given day you could see exactly what and who broke the build. This was a huge leap forward in terms of quality because you no longer introduce a problem into the build and find out two months later during feature testing or even worse after the code is shipped. Resolution was often fast because we could isolate the exact submitted code from the ClearCase source control system.
Having a “clean” build meant any developer could synchronize the latest binaries and start work on “the next thing”. This allowed over 1200 developers to not be dependent on a single build model. A single build model is, we do builds on Fridays. That would mean if you wanted to start working on “the latest code” you would have to wait until Friday or use last Friday’s build. Over time this could kill a development cycle, especially if Friday’s build was bad. And remember, if you do builds once a week that’s 1200 developers checking code in every day – unchecked. By the time Friday roles around you may have merge conflicts and even worse – run time conflicts.
Today, much of this is done in the form of Unit Tests – like JUnit or our custom internal unit test suite called DLLTest for C/C++. Many members on my team had patents around this process and some of the small utilities we wrote in order to accomplish this. See below for the two patents I received for my automation work.
Patent References: (patent profile)
| 2006/0070,034 | 2006 | System and method for creating and restoring a test environment | |
| 2005/0289,517 | 2005 | System and methods for client and template validation |
I now see the attraction of the Microsoft Surface. While it has been documented, written about, reviewed, etc by many and in a sense does not get the recognition some of the other tablets get I think it is clearly getting short changed. And the reason is…
I have been playing with Worklight over the past several weeks and its definitely a very cool platform to do your development on for web and devices. I am really interested to hear some real world experiences around creating applications based on Worklight. If you care to share them here I would appreciate your feedback. Or if you have a review on your own blog or know of one please share!
Customers are getting smarter. Smarter about products, brands, services, and most of all the way social works. The younger generation catches on quickly and often thinks outside of the box to get what they want. They call you out when they learn things, often using social media to tout their findings. In this post I outline two approaches “kids” use to figure out about a brand or get that special discount. I say kids but the reality is many internet savvy people are doing these things every day and you as a brand need to understand what happens when you are called out.
