Last week I published an article, An Intro into Test Driven Development with Junit4, which got somewhat mixed reviews. One particular commenter on reddit suggested I didn't understand that unit tests shouldn't talk to its data source. I decided to do some investigating into his claims, you can check out my findings here. I was however intrigued none the less and started looking for a tool that would allow me to easily mock a JDBC connection. I found Mockrunner to be the easiest to use of the ones I have found. Below is a brief tutorial on how to use Mockrunner. (Keep in mind that I am still fairly new to using Mockrunner)

In the class I am testing I am explicitly calling the columns I am retrieving by name. So I need to add these column names to my mock ResultSet.

result.addColumn("Date");
result.addColumn("High");
result.addColumn("Low");
result.addColumn("Condition");
result.addColumn("ID");

Obviously I also retrieve data from a result set so I must also insert data into the mock ResultSet. There are several ways to do this, but the easiest I found was using a list. You must add elements to the list in the same order as you added columns to the ResultSet.

List<Object> rowItems = new ArrayList<Object>();
rowItems.add(new Date(0));
rowItems.add(72);
rowItems.add(56);
rowItems.add("Cloudy");
rowItems.add(1);

In my actual test case I can verify that I am actually gathering the contents of the result set correctly and that the correct SQL statement was run.

prepareRS();
ForecastDAO dao = new ForecastDaoImpl(getConnection());
List<Forecast> forecasts = dao.getAllForecasts();

assertTrue(!forecasts.isEmpty());
verifySQLStatementExecuted("SELECT * FROM forecast");
assertEquals(73, forecasts.get(1).getHigh());

This is but scratching the surface of MockRunner's potential and using mocks at large. I hope this brief tutorial was helpful in getting you started using MockRunner. You can checkout the entire test class here and the class that I am testing here. If you want to get an entire working version of the project be sure to read the technical guide. Let me know if you have any questions or want to add your own comments.

Technorati Tags: Mock, Programming, Testing, Unit testing

Last week I published an article on test driven development. One person who read my article (briefly) suggested I did not know that unit tests shouldn't talk to their data source. I plan on covering how to mock JDBC connections later this week, however I wanted to do some research to see if what my critic says is an industry standard or a philosophical choice.

Surprisingly there seems to be relatively little information on this subject, or I have been incapable of finding information. From what I have gathered though, the theory that data sources should be mocked is sound, but its may not always be practical to implement it. Below is two reasons why data source connections should be mocked and three reasons why they should not.

Why you should mock connections

Speed – Talking to a data source is one the primary operations that slows an application's performance. Most data sources require some sort of network connection and retrieving data is often a processor intensive task. Obviously performing these tasks take time. Unit tests are most useful when they are run automatically and frequently to ensure changes being made are not breaking the application. Unit tests that are run automatically should be somewhat speedy, by mocking a data source connection, this can reduce considerably the amount of time it takes to run unit tests.

Remove the reliance on the data source logic – Unit tests should be testing relatively small parts of code, individual methods and/or small collections of methods. By connecting to a data source you are depending upon the logic of not only how you are connecting, but how the data source operates. This goes beyond the scope of what a unit test should be testing.

Why you shouldn't mock connections

It's time consuming – Writing mocks can be a quite laborious task. I often avoid writing mocks like the plague, because even relatively simple mocks can require a considerable amount of time to get running and running correctly. Even my very simple example of running a single query against a five column table took several hours to setup. Granted this was my first time creating a JDBC mock and future mocks would be easier and quicker to write, but it would still take longer than simply connecting to my database.

It's ugly – It takes a lot of typing to get even relatively simple mocks working. A lot of the rules that apply to how real business code should be written need not be followed when writing unit tests, that said looking at the code of a unit test shouldn't make your eyes bleed. Unit tests that cannot be easily maintained often become ignored when they break and broken unit test have no value (if ignored).

It's not real – Mocks don't really care what you put into or take out of them. Real data sources are often not so forgiving. If I misspelled a column or table name in a query, a mock would not pick up on this (you can verify the sql statements run, but again that could be misspelled), where as a real database would. By actually connecting to the real data source you can be more confident that the application will perform its intended tasks.

Why you should use your head

Even within the same project there will be instances where mocking a connection to a data source is the right choice and instances when you should actually connect to a real data source. In the early stages of a project using mocks can be more practical as the structure of the data source is more abstract and subject to change. Rewriting a mock is often quicker and easier than restructuring a data source. However as a project matures and the unit tests become more complex, connecting to the real data source may be more practical as it is not only (more) well defined, but writing the mocks begins to require more time. As with all practices, you need decide which one is best to follow (or not) based upon your requirements and restrictions. Though I think it is worth noting standards are standards for a reason. Feel free to weigh in with your own thoughts on the subject.

Additional reading:
http://www.javaranch.com/journal/2003/12/UnitTestingDatabaseCode.html
http://www.buunguyen.net/blog/unit-testing-the-data-access-layer.html

Technorati Tags: Best practices, Mock, Programming, Testing, Unit testing

http://www.developerart.com – A new blog like my own. While not a whole lot yet, the content that is on his site is of high quality.

Kind of a smallish link dump I know. I plan on getting some new articles posted this week, namely an update to my TDD article I posted last Monday. So check back soon.

Technorati Tags: Best practices, Programming

Please read the technical guide before starting this tutorial.

This article will mark the first of a long-term series covering professional software development. For the lowdown on this project check out this article. Be sure to give me your feedback as it will be vital in helping me develop better tutorials in the future.

Test driven development seemed like a natural choice as a lead off to my series of tutorials as I had to explain why I am writing all these tests. It is also a very good development methodology that will actually save a lot of time by reducing the amount of time spent debugging. For this tutorial and the entire project, I will be using Junit4. For a synopsis on test driven development you can check out the wikipedia article here. If you need a brief refresher on JUnit you can read my tutorial here (written in Junit3).

The goal of this tutorial, from a project stand point, is to get the connection to our (MySQL) database working. To do this we are going to need several things: a MySQL server, a client to access the server, and a JDBC driver to allow our application to access the server.

The MySQL server:
I will be using MySQL server 5.0 for this project, you can download it here. The tutorial instructions will be based upon the user being “root,” the password “turnleaf,” and the url “http://localhost:3306.”

The MySQL client:
I will be using HeidiSQL to query and manipulate the database. Once the client is installed import this sql file to setup the database and table.

The JDBC Driver:
You can download the JDBC driver here. Please include it on your projects build path.

To begin this project, please checkout revision 7 from the code repository.

Once you have downloaded the project go ahead and navigate around it a little bit. You will notice I already have several source files; Forecast, ForecastDao, and ForecastDaoImpl. Forecast is a bean for holding all weather information for a specific date (this bean will likely be modified in the future). We also have ForecastDao which is an interface and ForecastDaoImpl which implements ForecastDao. I plan on covering interfaces in more detail in the future, but if you are unfamiliar with the concept you can read a brief description here.

Update to revision 8 and you will see I added in my first unit test. The unit test is checking to see if I get any returns when I run getAllForecasts(). Here is the code:

@Test

public void testGetAllForecasts(){

List<Forecast> forecasts = dao.getAllForecasts();

Assert.assertTrue(!forecasts.isEmpty());

}

If you attempt to run the test you should get a null point error when you attempt to check if forecasts is empty. If you look at the implementation of getAllForecasts() in ForecastDaoImpl it is obvious why, getAllForecasts() is returning null. This is one of the tenants of TDD, write fail first tests.

Go ahead and update to revision 9. You will see I have added connection information to getAllForecasts(). Don't worry about it being messy we will refactor it later. Go ahead and run the unit test again. You will still fail, but in the console shows we are successfully connecting to the database.

Update to revision 10. I have made several more changes, most noticeably if you run the test it should now pass! (If not check to make sure your database is setup correctly) Before we start celebrating too much, lets actually check the contents of the list to make sure we are getting the right data. Update to revision 11, I have added a few more checks, and we are in fact getting the correct data, awesome!

Author note: For some dumb reason when you create a new Junit4 test class in Eclipse it does not automatically inherit TestClass. Anyways I inherit that class now so you can just do assertEquals(expected, actual) instead of Assert.assertEquals(expected, actual).
If you checked the implementation of getAllForecasts() before updating to revision 10, you will noticed I am only setting the date value of my forecast bean. My new tests initially failed (I originally thought I was setting all fields), thus the importance of not only writing unit tests, but writing good unit tests.

So now it is time to start refactoring. With our unit test we now have a good baseline of how the system should behave. This way when we are making changes we can be confident we are not breaking the system because we will be getting the same output for the same input.

Update to revision 13. You will see that I have created the getConnection() method and that contains the database connection logic.

private Connection getConnection() {

Connection conn;

try {

String userName = "root";

String password = "turnleaf";

String url = "jdbc:mysql://localhost/weather";

Class.forName("com.mysql.jdbc.Driver").newInstance();

conn = (Connection) DriverManager.getConnection(url, userName,

password);

} catch (Exception e) {

throw new DataAccessException(e);

}

return conn;

}

I pretty much just copied and pasted (one of the VERY few times copying and pasting is ok) the connection logic into this method. The most noticeable change I made is in the catch clause. There is a bunch of different exceptions that could be thrown when attempting to connect to the database, sine they all end in the same scenario, unable to connect to the database, I kept the same generic catch(Exception). However you should never just throw exception so I created my own custom exception called DataAccessException. Inside the super(Throwable) constructor I set it up to add a message stating “Unable to connect to data source,” I also have it extend RunTimeException turning it into an unchecked exception.

***Philosophical warning***
This is philosophical decision. As projects become more complex it can become difficult to catch and/or throw an exception at every level. However, you should catch the exception at some point to give the user a reasonable error message and let you know when the application is throwing exceptions. There is another school of thought that all exceptions should be checked as it lets a developer know what exceptions and method could throw, and other various reasons.

Between the message and the name of the exception, it should be pretty clear that when this exception is thrown it means the application failed to connect to the database. Running the unit test we will see everything is still passing so we can be confidant that the refactoring did not break the system.

Update to revision 14 and you will see I refactored out the closing of the connection. Exceptions should never be ignored, personally I would say being unable to close a database connection could be a major issue. So instead of catching Exception I changed it to only catch SQLException and throw the DataAccessException except I will have my own message in their stating the connection could not be closed. Here is what the closeConnection() method looks like:

private void closeConnection(Connection conn) {

if (conn != null) {

try {

conn.close();

} catch (SQLException e) {

throw new DataAccessException("Could not close connection", e);

}

}

}

From what it originally looked like, the getAllForecast() method is starting to look a lot better as well as the ForecastDaoImpl class in general. Adding new methods that retrieve data from the database will be easier as all I have to do to get a connection is call getConnection() and to close it closeConnection(Connection). On top of that if I need to change my database connection information I only have to do it in one area.

Despite this refactoring, this is hardly an ideal data access layer. With the test unit already created go ahead and continue to refactor getAllForecast(), a good place to start would be how I create a new Forecast object. I would also suggest adding some new functionality yourself a couple of examples might be; getForecastByDate(Date) or getForecastByCondition(String condition) (Remember create unit tests!). I will continue to refactor and update this work myself. So you can run updates and compare your work to my own.

Addendum and thanks:
Because I am an idiot I couldn't remember how to write a JDBC connection. My initial connection method is predominantly based off of the code in this article: http://www.kitebird.com/articles/jdbc.html
For being able to go to a specific revision with a url:
http://www.perhammer.com/2008/07/subversion-in-url-revision-browsing.html

Technorati Tags: Best practices, Java, Programming, Unit

A couple of weeks ago I posted about my ideas for the future of this site. Well today I start making good on those lofty promises. I will be posting my first tutorial, which will cover test driven development, shortly. Before I do that I want to give a bit of an explanation of this project, why I am doing it, what I hope to accomplish, a few other thoughts, and finally some technical notes.

The what

As I said in my previous post, this project's main goal is to check to see how accurate meteorologists really are with their predictions. NOAA offers a web service for checking forecast information and I will be using that to gather my data. Just following the exact requirements of those project will give plenty of opportunities to learn new things, but I also intend to meander around a bit so we can learn a lot more. So when I start adding user sign in and other seemingly odd/unnecessary features, now you know why.

The why

I intend for this project to serve a duo purpose, to teach others and to learn a thing or two myself. One of my main goals with this site is to help teach other developers. There where many important subjects that I did not fully grasp in my early career and it really held me back. Luckily I was given the opportunity to learn from my mistakes as well as those around me. I hope to repay some of this debt by helping others.

There is a lot to software development, a lot. Sometimes it is hard to know where to begin, but it is said necessity is the mother of invention and I hope both intentionally and unintentionally this project forces me to learn things as it is the only way to (effectively) solve a problem.

A few words on practices

Occasionally there will be bad practices in this project. It will be a mixture of both intentional and unintentional. I plan on making example of why those practices are so bad, and sometimes the only effective way to do so is by actually showing the technical debt taken on by those practices. When I do intentionally make a bad design choice, I plan on explicitly stating so and I will also show how to fix the issues resulting from those practices.

That said, I am both human and a junior developer. I am going to make mistakes, lots of them. You don't need to burn me at the stake. If you see a problem in my code, please leave a comment and if possible offer a solution. If I think you are correct I will make amendments to my articles to correct any problems.

There will be philosophical differences. When I am cognizant of differing views points on a subject, I will state my design choice is philosophical. If you are of an opposing view please speak up and say why your choice is better, but it has to be something more substantive than “you are wrong.”

Lastly all practices are guidelines. For every good, great, or best practice, there is going to be a few examples out there that contradict the practice. Thus they are practices and not rules. Understand that there is no possible way I could be familiar with every person's business domain or systems the you work on or use. I have to write with a certain degree of generality.

I need you

I can only fix what I know to be broken. If you see a problem, want me to cover something in more detail, or anything else, please let me know. Your feedback is absolutely vital in the process of making this site and the content I publish here better. I will do the best I can to self correct, but I can only do so much alone.

Technical notes

All coding work will be done utilizing Eclipse 3.5 (Galileo), you can download it here. I will be using subclipse as my primary version control client. If you need help using subclipse check out my subclipse primer. I will be utilizing several other technologies as this project because more advanced. As I do I'll let you know what I am using and where to get it. Hopefully in the future I will be able to do this all automatically, though network/financial issues may prevent this.

I'm still contemplating on how to get a live version of this project hosted. I think I may be able to use Google's app engine, but I am still unsure if it will fully mesh with the goals of this project. If you know of free or cheap Java hosting let me know.

This project is published under the MIT license. Which as far as my understanding goes allows you to do pretty much anything with the source code. Rather that be the case or not you do have my permission to use the source code in this project, at least the code I personally write, to whatever purposes you see fit (I'd prefer good). Attribute me if you want, I don't really care.

Well anyways I hope you like this series of tutorials. It's going to be a long, but I hope a very fun and enlightening, journey.

In my recent article, 8 signs your code sucks, one of my signs of bad code is: “You need to use comments to explain the code.” I have since taken a lot of flak for suggesting this and I want to clarify my point and why I am not backing down.

Not javadoc - I am NOT referring to Javadoc or its equivalents in other languages. I absolutely do agree that a method or class should have accompanying documentation at its declaration stating its purpose and its inputs and/or outputs.

Comments are not bad, the code is – I'm not suggesting to forgo the use of comments, but that they are only band-aids. When you don't have time to properly write a piece of code; do it the “messy” way, write a comment and give it a TODO as a reminder to go back and fix it later.

Comments make code accessible - Code should not be written to the level of the dumbest developer, the dumbest developer should be brought to the level of the code (though I want to reiterate good code should be easily readable). If a developer is unwilling or unable to meet or at least understand the rest of the team's coding standards, then questions should be asked rather you want that developer to understand the code at all.

They are more like guidelines – When it comes to design patterns there are no rules. From day one we are taught goto statements are evil and should never be used, but there is even a time and place for them. There may be rare occasions when you have to write a comment to explain a piece of code, just like you may need to use a goto to simplify the code, but both occasions are rare and when encountered in code should be scrutinized.

Technorati Tags: Best practices, Programming

My blog passed a major milestone today, I finally eclipsed 1,000 visitors in a single day! I want to say thanks to everybody who has visited this site and I hope you find it useful. I have a lot of big ideas for the future so be sure to check back again sometime. Help support Turnleaf Design by leaving comments, or sharing posts with friends, colleague, or on social sites. Thanks again!

As a wide eyed junior developer when I first began working on large projects I simply accepted that it is difficult to fix bugs or find where an action is being executed. If only I knew then what I know now, I would had saved myself hours of frustration. The first step to writing good code is accepting the code you write (or work on) is crap, but sometimes you need to know what to look for. Here are some signs that your code sucks.

A method is larger than the screen – A method should only perform one specific task. A method should not contain the logic code to determine if the username field contains data, is valid, and that user exists. If a method is too large to fit within a single screen, that is a (very) good sign it is doing too much.

You are reusing variables – Unless you are working on embedded devices, memory is cheap. Don't be a memory scrooge and knee cap code maintainability, which in nearly all instances trumps performance, by reusing the same variable for multiple uses.

You are directly accessing the request/session – It not only makes writing unit tests (much) harder, but it is also difficult to know what data the application has access to. All data should be taken out of the session/request and stored in a bean. A bean, through its getters and setters, creates a “contract” of what data the application has access to, which greatly helps with code maintainability.

You need to use comments to explain the code – Code should be able to explain itself and should be in a format that is easily readable. If you find yourself needing to explain what your code is doing then you may want to look into rewriting that code.
EDIT: This is not referring to using comments (e.g. javadoc) to explain the purpose of a method/class and its inputs and outputs.

An exception's stack trace doesn't return the original problem -  You should never “eat” an exception, that is catch an exception, but not print its stack trace. How can a bug be fixed if you don't even know where the bug is occurring?

Your code is a mud ball – Just the name sounds ugly. A “mud ball” is when there is little separation between the layers or concerns of an application. Code should be modular allowing for ease of reusability and modifiability. Anything concerning the user interface happens in the view, program flow and usually data validation is the domain of the controller, handling business logic is the model's job, and only the model should be interacting with the data access layer.

It is hard to write a unit test – If you find a bug or write a new piece of code and it takes you more than a few minutes to write a unit test then that portion of code is handling too complex of a task.

The author is Billy Korando – Wait what?!

Add your own experiences or signs for dealing with sucky code.

UPDATE: There are differing opinions on my signs, since I am but a mere "grasshopper" I maybe wrong. Anyways check it out and make up your own mind.

Technorati Tags: Best practices, Programming

Integrated development environments make developing application fair easier. They highlight syntax, let you know if you have a compilation error, and allow you to step through your code among so many other things. Like all IDEs Eclipse has a bunch of little shortcuts and tools that can make your life a lot easier, I've compiled a list of several that I use on a daily basis:

1. Auto-complete – Eclipse has an auto-complete feature that can be accessed with ctrl + space. When clicked a small pop-up box is displayed with a list of context sensitive suggestions. If there is only one possibility then Eclipse completes it for you.

2. Quickly format your code – Code that is being heavily modified can quickly become an ugly sight. Without proper indention it can become extremely difficult to determine what is happening in code. Code can be quickly formatted using the shortcut: Ctrl + shift + F. You can even setup your own style rules by going to: Project>Preferences>Java Code Style>Formatter, check “enable project specific settings” edit the profile and then save it under a new name.

3. Get rid of unnecessary code – Heavily worked on projects that aren't maintained well can quickly build up a bunch of unnecessary imports, casts, among other common easily detectable coding mistakes. Quickly remove these mistakes by running a code clean up. Right click on your project folder>source>clean up. Like the code formatter you can also customize your code clean up.

4. Go to declaration – Want to know where that method, variable, or class is declared? This can easily be done by holding down ctrl and then clicking on the reference.

5. Find all references – If you need to find all the references for a method, variable, or class highlight the desired reference right click>reference and select the desired search scope.

6. Quickly select groups of characters – Need to delete an entire string, or everything within a method's argument declaration? Select the enclosing character (e.g. the “ or "(" respectively), this will select everything within the enclosing characters. This even works with the bodies of classes and methods. You can also select an entire line by triple clicking.

7. Rename all instances – If you need to rename a method, variable, or class you can rename ever instance by highlighting a reference and pressing alt + shift + r an box will highlight the reference and you can rename it. Upon pressing enter all instance will be renamed in your project.

8. Change a method signature – If you need to change any part of a methods signature; a methods name, its arguments, or return type, you can quickly make a change to all references of the method in the project by highlighting the method and pressing alt + shift + c a pop-up box will appear giving you options to change the method's signature.

9. Automatically generate getters and setters – Writing getters and setters is for suckers. After you write out all the members of a class right click anywhere in the code screen>source>generate getters and setters. Select the variables you want to have getters and setters made for.

10. Javadoc is a cinch – Eclipse makes writing Javadoc easy, just type “/**” and press eneter above a declaration. Eclipse automatically creates context sensitive Javadoc annotations (i.e. a methods parameters, or the author of a class). After that you only have to write out what the method actually does.

11. Run a unit test – To run a unit test highlight the unit test and press alt + shift + x followed by t. You can also run a unit test in the debugger by pressing alt + shift + d followed by t.

12. Comment out code – If you need to quickly comment out a chunk of code, highlight the corresponding lines and press ctrl + /, you can uncomment code the say way.

Chime in with your own Eclipse shortcuts.

Technorati Tags: Programming

In the third and final part to my SVN primer I will finish up with my day to day tasks and I will go over how to add SVN properties to your project. The first two parts can be found here: 1,  2.

More Terminology:

BASE – Refers to the most recent revision currently on your local workspace.

SVN Commands:

Comparing revisions – To see what changes have been made between two revisions a comparison can be run. The changes will be highlighted in gray, along with a box showing where in the older revision the change fits in. To compare revisions right click>compare with and then select the option that best suits your needs.

Merging – One of my least favorite of SVN commands, not because there is anything wrong with the merge feature (it's actually a great feature), but often when I run merges I have to deal with several conflicts, which I never find an enjoyable experience.  Anyways merging is a little trickier than you might think. Initially you might try merging the head of your branch (or whatever you are merging) with the head of your trunk, however this is incorrect. You actually need to merge the head of your branch with the revision of your trunk when you made the branch. This needs to be done because Subversion needs to know what actually has changed since the branch was made.

Add – If there is a file that is not under version control, but you would like it to be include this can be done by right clicking on the file team>add to version control

Ignore – On the flip side, if  you do not want a file to be under version control right clicking>team>add to svn:ignore

SVN properties – One really useful feature is svn properties. There are a wide assortment of svn properties that allow you to do things like add meta information to a file (author, revision number, last modified, etc.), include a tracking number and link to a bug tracking system, among many other options. The easiest way to setup properties using subclipse is through the “Show Properties” view. Right click on the SVNProject root folder and go to Team>Show Properties. To add a property right click any where in the spreadsheet looking area and select “add property.” Select the desired property and setup the property's value. To setup the ability to use a bug tracking through svn we will need the following three properties added:

Bugtraq:label - Displays a message next to the box where a developer would put the (comma separated) issue numbers.
Bugtraq:url – This is the url to your bug tracking system. The “%BUGID%” appends the tracking numbered entered to its location in the url.
Bugtraq:message - Displays a message in the history view. “%BUGID%” creates a link to the bug entered during the commit.

The end product, when you select a specific revision in the view history view in the bottom left you should get a message like “Tracker #s: 123” With "123" being a link tracker 123 in your bug tracking system. Eclipse has a built in browser, so when this link is clicked a browser window should open within Eclipse taking you to the issue.

This concludes my SVN primer. I hope anybody who reads this finds it useful. If you have any questions let me know. For more information on SVN you can check out these links:
http://markphip.blogspot.com/ - The blog run by the director of subversion engineering at Collbanet
http://svnbook.red-bean.com/en/1.5/index.html – A huge guide on how to use Subversion

Technorati Tags: Subversion, SVN