Verify the World

Background

In my company, there’s a good focus on self-improvement, and that focus often operates at the team level.  To this end, the QA group spends an hour a week reading a book on a topic of interest.  When it came time to choose our last book, I opted for a tutorial book on Java.  There were a few options that made the short list, including:

• Kathy Sierra & Bert Bates’ Head First Java, 2nd Edition
• Bruce Eckel’s Thinking in Java (4th Edition)
  
• Barry Burd’s Java For Dummies
  

Though my inner geek wailed and moaned, the consensus was reached to next read Java for Dummies.

Now, maybe I got into this with the wrong mindset.  I have read the first two books in the list.  I liked them (especially Eckel’s Thinking in Java).  Head First Java is more of a beginner’s book, but it’s very interactive and spastic, and even expects the reader to scrawl and doodle on the page.  That’s not conducive to a reading group book.  Thinking in Java is very cerebral, very comprehensive, and sometimes difficult to follow.  It’s not the sort of book you read out load, because that would be far too slow for a nigh-1500 page tome.  Since I couldn’t rightly fight for the devils I know, it was with a heavy heart that I gave in to the devil I didn’t.

The Voicing

The author wastes little time buddying up to the reader.  The casual language, frequent breaks for anecdotes or *shudder* puns… these can be distracting to a group focused on learning.  There are so many elements of the book that are orthogonal to the topic at hand that, were they trimmed from the final copy, we could wind up with less of a book and more of a pamphlet.

The Layout

Scarcely a page goes by wherein we do not find reference to another chapter, ahead or behind.  The need to explicitly refer the reader to previous or future chapters is a bad writing smell, and indicates a need to restructure your book, as well as a lack of faith in the reader to retain the information earlier provided.

The Information

Actual, actionable information is thin.  Examples are incomplete, often insufficiently explained, and include paradigms not yet (or not EVER) explained to the reader.  There are no exercises provided for the “learn by doing” reader.  Much of the book is less an exploration of the Java language and its capabilities than a tutorial for operating the included IDE software, JCreator.  I was particularly offended by an inset block of text titled: “All about generics”.  In Eckel’s book, Java generics are given nearly 200 pages of exploration, and even then with the caveat that the information was not exhaustive, with references to even more complete sources of information at the end of the chapter.

Inheritance is only glanced over, polymorphism is all but ignored, first-class elements of the language like access modifiers and other keywords are completely left out.  There is no description of any of the tools included with the JDK.  JavaDoc and annotations are avoided like the plague.  No mention is made of techniques to test one’s code, nor advice on how to break large problems down into small, reusable units of code.

In Conclusion

I can’t recommend this book to anyone.  If you’re learning on your own, I recommend Head First Java.  If you’re looking to fill in gaps in your knowledge, I recommend Thinking in Java.  If you’re leading a class or a reading group, I’d love to hear what book you choose.

OK, “complete” in the sense that it is operational. One thing I’ve learned in my efforts is that, once you go Maven, you go a lot of other directions, as well.

For instance, I elected not only to let Maven build my project, but also to modularize it. My functional test automation framework is now split into four modules:

1. Core
   • Page definition file interpreters
   • Interfaces and abstract classes to represent pages, business logic classes
   • Abstract data validation classes
   • UI object models
   • All the logic and models that make the framework go.
2. Test
   • Page layout definitions
   • Page action definitions
   • Test manifests
   • Data-driven test classes (Data providers)
   • Application-level model classes
   • Concrete data validation classes
   • Concrete page interaction classes
   • Application-level configuration
   • Functional tests
3. Report
   • Test failure screenshot logic
   • Test result recording
     • HTML reports
     • Database records
4. Tools
   • Server access scripts
   • Supporting utilities

That’s the conceptual split between these modules.  In practice, there’s still a lot of work to make each as independent as I’d like.

Maven is already building and modularizing my project.  Why not let it help me to integrate new dependencies, as well?  Since converting to Maven, I’ve added a connection pool manager called BoneCP.  I didn’t have to download a JAR (and all the JARs that JAR depends on).  I just added the dependency to the POM file, and Maven did the rest automatically, behind the scenes.  Now, when I build my project, that dependency is automatically downloaded (if necessary) and linked into my classpath.  I did the same with Apache Commons DbUtils.  Just a quick reference in the POM in the module that requires these dependencies, and it’s done.  When I release a version of my framework to my company’s repository manager (Artifactory), those dependencies will be automatically acquired and linked by any other project that depends on mine.

I have a few more near-term features and goals for which this transition to Maven is going to be very helpful, if not essential:

1. Automate the execution of my functional tests whenever a build of our web application is done.
2. Extract any configuration of my application out of custom XML files and Java code, and into Maven profiles.
3. Replace BoneCP with a more mature (read: more complex) connection pool manager.
4. Incorporate object/relational mapping into the data validation code.

I’ll be posting on the more interesting of these goals as they become reality.

Image via Wikipedia

Image via Wikipedia

My company is moving our build process from an Ant script to Maven.  This is going to make everything so much easier!  Whereas our Ant script required a bunch of projects to be checked out, configured just so, and deployed to attachable JARs, now all these projects can be stored in our repository manager and just be referred to by the parent POM as dependencies!

The challenge herein is to adapt my automation framework to use Maven instead of Ant.  This is essential for developer adoption, and also heightens my awareness of a tool that we intend to use to ensure the quality of our product.  Either of these are noble goals, but taken together, are too much impetus to ignore.

This transition requires quite a shift in perspective, and unfortunately, some additional planning I had avoided in true hacker style.  See, I was using Ant not only to build my application, but also to execute it (via the TestNG Ant task) and to produce test reports (via ReportNG).  I chose to go this route because: a) I was lazy, and b) I was inexperienced in software development.

Now, Maven may, with some tooling, allow me to preserve this approach.  But at what cost?  My super-custom build structure would undo much of the benefit of Maven; that of  ”convention-over-configuration“.  If my process is so specialized as to require a new developer on the project, or a new user of the artifacts, to read and understand the build process, then Maven will have cost me much and bought me little.

Rather than to port my build process to Maven, I’ll need to re-evaluate the structure of my application as a whole.  I’m considering splitting the project into the core framework and the application-specific implementation.  Instead of a task in the build tool, I’ll write an entry point class that kicks off the TestNG tests with my desired configuration.  Wrap the whole shebang into a JAR and upload it to the local Maven repository manager, and I’ve got a dependency that can be used during the integration-test phase.  Voila!

Java Power Tools

This book was an impulse purchase.  About a year ago, I was wandering through my local book mega-store with my wife, looking for guides on how to raise a puppy.  As I tend to do, I gravitated toward the technical section of the bookstore.  I had taken a few classes on Java at my local community college, and wanted to eventually make a living at it, so I scanned the available books on Java.  That was when Java Power Tools caught my eye.  This book is certainly focused on Java, but it is not a programming manual, not a language specification, not an exam study guide.  It is a reference that leads the reader through the extensive world of open-source tools written to make Java software development easier and more fun.

Any organization worth its salt uses at least some of the tools described herein, and many of the larger-scale enterprises will use quite a few, often together in interlocking ways.  If you’re serious about Java development, you need to know how to use the available build tools, unit testing frameworks, issue trackers, source control management, continuous integration, code coverage, and profilers.  This book answers the vital questions of:  ”what is it?”, “where can I find it?”, “how do I use it?”, and “why do I need it?”.

As a young tester peeking my head out of the black box for the first time, I found answers to all the questions that I had stored away in my head about these mysterious terms “Ant”, “CruiseControl“, “JUnit“, and “CVS”.  Most importantly, I had finally internalized that there is far more to Java software development than just the programming language chosen.  In order to develop and release quality software, an organization must rely upon many tools (power tools) to work with the greatest possible efficiency and focus.  In the past year, I’ve used many of these tools myself, either to improve my organization’s build and release process, or to power my own Java-based automation framework.  I don’t know that I could’ve progressed with as much confidence and persistence had it not been for this book.  I’d recommend it to any blossoming Java developer, quality assurance professional in a Java shop, or project manager looking to improve the release process.

Hello world, with <angular/>!

I really wanted to attend GTAC this year. For reasons not thoroughly explained, my application for attendance was denied. Am I bitter? Not really. I’m not at all one of the big minds in test automation. If anything, I’m a fly on the wall, gathering and applying as much from the stream of information that I can. I’ve been doing this for less than a year, all told. While I’m proud to the point of bursting for what I have accomplished in that time, I know better than to consider myself a prodigy or among the elite. That doesn’t mean I wouldn’t like to get there, and this blog is a part of that process.

So, now that GTAC is behind us, and the speakers have published their slides, I’ve taken the time to read through them, to see what new information or techniques I can apply to my own work. I’ll link to each and post a brief review, as applies to web test automation using Selenium & TestNG.

More »

Much more empirical than my own.

Give it a read here!

I am a self-taught automation engineer for a major privately-held web company.  When I joined the organization, it was as a manual tester.  We had an automation engineer with one foot in the development department.  His automation framework was a custom mish-mash of C# and Java.  Code was duplicated everywhere.  Many of the tests (for a web application, mind you) were reliant on the specific [X,Y] coordinates of a link, button, etc.   Only one test would run at a time, and all input data was hard-coded.   Put simply, it was a mess.   When our lone automation engineer left for greener pastures, with no one who could maintain, read, or even execute his framework, we found ourselves releasing software riddled with issues that just didn’t come out in testing..

QA had very close and amicable ties to the developers, but often found ourselves speaking the language of the users, unable to translate to the language of the engineers (Java), so I went ahead and learned Java and design patterns.  They’d written unit tests using JUnit that were apparently run with every build, so I learned JUnit.  All that was fine, but QA’s testing was still being done manually, and often ad-hoc!

While hiring for automation and performance test engineers, I noticed a pattern in the resumes I received: the vast majority of candidates out there work almost exclusively in drag-and-drop tools like Quick Test Pro and LoadRunner.  These tools are prohibitively expensive, inflexible, and, frankly, serve mostly to separate the engineer from the code under test!  Frustrated by the lack of passion and quality in the resumes coming in, I pulled the postings from the job boards, and began an effort to bring this QA department up to modern using exclusively open-source tools.

It seemed like a good idea at the time.  It certainly didn’t seem like I’d be tripling my technical library and dedicating my nights and weekends to all the base topics I’d need even to get started! I’ve been endeavoring to build the entire system utilizing the same technologies as our development team, so that it can integrate most seamlessly with our continuous integration (CruiseControl) and deployment tools (Ant & Maven).  I’ve taught myself Perl, JDBC, XHTML & CSS, JavaScript, XML, SAX, DOM, TestNG(parallelism!), Selenium (IDE, RC, and Grid), JMeter, CVS, SVN, and git.

My new test automation framework is cranking out hundreds of tests in parallel and really helping the development cycle.   The tests are flexible to changes in the UI or in the activity flow.  Input data is validated against the database… it’s a beautiful thing.  The framework, though operable, is still far from complete.  Do I know everything?  Am I finished learning?  Are my opinions as good as law?  No!  Not by a long shot!  However, the true value of a project like this is that it never has to be finished.  You can always keep learning.

I urge my readers (both of you) to go open-source, become active in user communities, contribute where you can; be it in the reporting of bugs or the fixing of them, and to never ever let good knowledge go un-spread.

Reblog this post [with Zemanta]

In the world of unit testing frameworks for Java, there are really only two that truly stand out: JUnit and TestNG.

While JUnit is the de-facto standard, and rightly so (it’s been imitated in every language from C# to Perl).  JUnit provides a comprehensive assertion library, integrated support for build management tools like Ant and Maven, as well as just about every continuous integration tool worth mentioning, and even IDE support for the most popular Java IDEs.  However, there are a few areas where JUnit just doesn’t meet every developer’s needs.

Enter TestNG.  TestNG (the “NG” stands for “Next Generation”) was developed as a replacement for JUnit 3.x, which, at the time was found to be very restrictive.  JUnit test classes had to extend org.junit.TestCase, setUp() and tearDown() methods had to be named just that and contain empty-argument signatures, to name just a few of the limitations.  TestNG, on the other hand, used the whiz-bang new J2SE 5.0 Annotations paradigm to bless any method you please as a unit test, set-up method, or otherwise.  JUnit was quick to follow suit in 4.x.  What JUnit still can’t buy you these days is parallel execution support.  Much of the value of unit tests is that they are fast.  However, in a large application, you could have thousands of unit tests, and the amount of time spent running each in turn could negate the benefit of having them in the first place.

Re-enter TestNG in billowing cape and tights.  With TestNG, all your unit tests can run in parallel using a configurable thread pool, with fail triggers that watch execution time or run the same test multiple times in a row, reporting a failure only if the test fails more than a certain percentage of executions.

Another really cool feature of TestNG is the “data provider” method.  Unit tests are great, because they can exercise a method in your class-under-test and expect a particular result.  However, that method would probably produce a deterministic result for any number of different data.  Using a data provider, you can call your test method over and over again with different arguments, exercising your method-under-test with a wide range of inputs!  Just change your test method signature to take in whatever input values you’d like to apply to your method-under-test.  Then, add the dataProvider=”someString” attribute to your @Test annotation.  Now, create a new method with the @DataProvider annotation and a matching name=”someString” attribute.  A data provider method takes no arguments and returns either an Object[][] or an Iterator<Object[]>.  (I find the latter to be more useful, as it conveniently allows you to use a Collection<Object[]> within the method, and then call iterator() in the return statement.) The first dimension of the array is the list of argument sets.  The second dimension of the array the actual list of arguments.

You’re all set!  When your test method is executed, it will be run once for each entry into the data provider’s return object!  Now that’s some good exercise for your method-under-test!

Like what you’re reading, but already using JUnit 3 or 4?  The official distribution of TestNG comes with a conversion mechanism that will run through your code and update all your JUnit tests to TestNG tests!  How cool is that??

There are more cool features of TestNG than I can get into, here.  I’d recommend reading the official page thoroughly.  For my purposes, I give TestNG two thumbs up!

Reblog this post [with Zemanta]

Raise me upon your shoulders and rejoice!  For I am on the intersphere!  My blogoscope is longer than yours, and I can confirm this with mathematics!

Poppycock aside, it’s beyond obvious to any puny-brained sociologist that an ego like mine, once digitized, could not be confined to the meek spaces of IRC, twitter, and lolcats, but must burst freely from the dusty vial in which it was once contained to spill into this dark, sticky vertex of the web in which you now flutter.

From this space most cybery, I shall drip mercilessly onto the white slacks of convention!  I shall drink from the cup of my betters, and I shall backwash mightily. Beware, clean things, “known” things, things that are accepted without test or interrogation.  By the fing-longering grace of the internet, I (we!) can confirm or deny where others once bobbled their heads mindlessly to your ever-repeating muzak.  There are no questions but those we seek to answer.  There are no answers unsought.

My stars and garters, but that was cathartic.

Reblog this post [with Zemanta]