Last week I published an article on test driven development. One person who read my article (briefly) suggested I did not know that unit tests shouldn't talk to their data source. I plan on covering how to mock JDBC connections later this week, however I wanted to do some research to see if what my critic says is an industry standard or a philosophical choice.

Surprisingly there seems to be relatively little information on this subject, or I have been incapable of finding information. From what I have gathered though, the theory that data sources should be mocked is sound, but its may not always be practical to implement it. Below is two reasons why data source connections should be mocked and three reasons why they should not.

Why you should mock connections

Speed – Talking to a data source is one the primary operations that slows an application's performance. Most data sources require some sort of network connection and retrieving data is often a processor intensive task. Obviously performing these tasks take time. Unit tests are most useful when they are run automatically and frequently to ensure changes being made are not breaking the application. Unit tests that are run automatically should be somewhat speedy, by mocking a data source connection, this can reduce considerably the amount of time it takes to run unit tests.

Remove the reliance on the data source logic – Unit tests should be testing relatively small parts of code, individual methods and/or small collections of methods. By connecting to a data source you are depending upon the logic of not only how you are connecting, but how the data source operates. This goes beyond the scope of what a unit test should be testing.

Why you shouldn't mock connections

It's time consuming – Writing mocks can be a quite laborious task. I often avoid writing mocks like the plague, because even relatively simple mocks can require a considerable amount of time to get running and running correctly. Even my very simple example of running a single query against a five column table took several hours to setup. Granted this was my first time creating a JDBC mock and future mocks would be easier and quicker to write, but it would still take longer than simply connecting to my database.

It's ugly – It takes a lot of typing to get even relatively simple mocks working. A lot of the rules that apply to how real business code should be written need not be followed when writing unit tests, that said looking at the code of a unit test shouldn't make your eyes bleed. Unit tests that cannot be easily maintained often become ignored when they break and broken unit test have no value (if ignored).

It's not real – Mocks don't really care what you put into or take out of them. Real data sources are often not so forgiving. If I misspelled a column or table name in a query, a mock would not pick up on this (you can verify the sql statements run, but again that could be misspelled), where as a real database would. By actually connecting to the real data source you can be more confident that the application will perform its intended tasks.

Why you should use your head

Even within the same project there will be instances where mocking a connection to a data source is the right choice and instances when you should actually connect to a real data source. In the early stages of a project using mocks can be more practical as the structure of the data source is more abstract and subject to change. Rewriting a mock is often quicker and easier than restructuring a data source. However as a project matures and the unit tests become more complex, connecting to the real data source may be more practical as it is not only (more) well defined, but writing the mocks begins to require more time. As with all practices, you need decide which one is best to follow (or not) based upon your requirements and restrictions. Though I think it is worth noting standards are standards for a reason. Feel free to weigh in with your own thoughts on the subject.

Additional reading:
http://www.javaranch.com/journal/2003/12/UnitTestingDatabaseCode.html
http://www.buunguyen.net/blog/unit-testing-the-data-access-layer.html

Technorati Tags: Best practices, Mock, Programming, Testing, Unit testing

http://www.developerart.com – A new blog like my own. While not a whole lot yet, the content that is on his site is of high quality.

Kind of a smallish link dump I know. I plan on getting some new articles posted this week, namely an update to my TDD article I posted last Monday. So check back soon.

Technorati Tags: Best practices, Programming

Please read the technical guide before starting this tutorial.

This article will mark the first of a long-term series covering professional software development. For the lowdown on this project check out this article. Be sure to give me your feedback as it will be vital in helping me develop better tutorials in the future.

Test driven development seemed like a natural choice as a lead off to my series of tutorials as I had to explain why I am writing all these tests. It is also a very good development methodology that will actually save a lot of time by reducing the amount of time spent debugging. For this tutorial and the entire project, I will be using Junit4. For a synopsis on test driven development you can check out the wikipedia article here. If you need a brief refresher on JUnit you can read my tutorial here (written in Junit3).

The goal of this tutorial, from a project stand point, is to get the connection to our (MySQL) database working. To do this we are going to need several things: a MySQL server, a client to access the server, and a JDBC driver to allow our application to access the server.

The MySQL server:
I will be using MySQL server 5.0 for this project, you can download it here. The tutorial instructions will be based upon the user being “root,” the password “turnleaf,” and the url “http://localhost:3306.”

The MySQL client:
I will be using HeidiSQL to query and manipulate the database. Once the client is installed import this sql file to setup the database and table.

The JDBC Driver:
You can download the JDBC driver here. Please include it on your projects build path.

To begin this project, please checkout revision 7 from the code repository.

Once you have downloaded the project go ahead and navigate around it a little bit. You will notice I already have several source files; Forecast, ForecastDao, and ForecastDaoImpl. Forecast is a bean for holding all weather information for a specific date (this bean will likely be modified in the future). We also have ForecastDao which is an interface and ForecastDaoImpl which implements ForecastDao. I plan on covering interfaces in more detail in the future, but if you are unfamiliar with the concept you can read a brief description here.

Update to revision 8 and you will see I added in my first unit test. The unit test is checking to see if I get any returns when I run getAllForecasts(). Here is the code:

@Test

public void testGetAllForecasts(){

List<Forecast> forecasts = dao.getAllForecasts();

Assert.assertTrue(!forecasts.isEmpty());

}

If you attempt to run the test you should get a null point error when you attempt to check if forecasts is empty. If you look at the implementation of getAllForecasts() in ForecastDaoImpl it is obvious why, getAllForecasts() is returning null. This is one of the tenants of TDD, write fail first tests.

Go ahead and update to revision 9. You will see I have added connection information to getAllForecasts(). Don't worry about it being messy we will refactor it later. Go ahead and run the unit test again. You will still fail, but in the console shows we are successfully connecting to the database.

Update to revision 10. I have made several more changes, most noticeably if you run the test it should now pass! (If not check to make sure your database is setup correctly) Before we start celebrating too much, lets actually check the contents of the list to make sure we are getting the right data. Update to revision 11, I have added a few more checks, and we are in fact getting the correct data, awesome!

Author note: For some dumb reason when you create a new Junit4 test class in Eclipse it does not automatically inherit TestClass. Anyways I inherit that class now so you can just do assertEquals(expected, actual) instead of Assert.assertEquals(expected, actual).
If you checked the implementation of getAllForecasts() before updating to revision 10, you will noticed I am only setting the date value of my forecast bean. My new tests initially failed (I originally thought I was setting all fields), thus the importance of not only writing unit tests, but writing good unit tests.

So now it is time to start refactoring. With our unit test we now have a good baseline of how the system should behave. This way when we are making changes we can be confident we are not breaking the system because we will be getting the same output for the same input.

Update to revision 13. You will see that I have created the getConnection() method and that contains the database connection logic.

private Connection getConnection() {

Connection conn;

try {

String userName = "root";

String password = "turnleaf";

String url = "jdbc:mysql://localhost/weather";

Class.forName("com.mysql.jdbc.Driver").newInstance();

conn = (Connection) DriverManager.getConnection(url, userName,

password);

} catch (Exception e) {

throw new DataAccessException(e);

}

return conn;

}

I pretty much just copied and pasted (one of the VERY few times copying and pasting is ok) the connection logic into this method. The most noticeable change I made is in the catch clause. There is a bunch of different exceptions that could be thrown when attempting to connect to the database, sine they all end in the same scenario, unable to connect to the database, I kept the same generic catch(Exception). However you should never just throw exception so I created my own custom exception called DataAccessException. Inside the super(Throwable) constructor I set it up to add a message stating “Unable to connect to data source,” I also have it extend RunTimeException turning it into an unchecked exception.

***Philosophical warning***
This is philosophical decision. As projects become more complex it can become difficult to catch and/or throw an exception at every level. However, you should catch the exception at some point to give the user a reasonable error message and let you know when the application is throwing exceptions. There is another school of thought that all exceptions should be checked as it lets a developer know what exceptions and method could throw, and other various reasons.

Between the message and the name of the exception, it should be pretty clear that when this exception is thrown it means the application failed to connect to the database. Running the unit test we will see everything is still passing so we can be confidant that the refactoring did not break the system.

Update to revision 14 and you will see I refactored out the closing of the connection. Exceptions should never be ignored, personally I would say being unable to close a database connection could be a major issue. So instead of catching Exception I changed it to only catch SQLException and throw the DataAccessException except I will have my own message in their stating the connection could not be closed. Here is what the closeConnection() method looks like:

private void closeConnection(Connection conn) {

if (conn != null) {

try {

conn.close();

} catch (SQLException e) {

throw new DataAccessException("Could not close connection", e);

}

}

}

From what it originally looked like, the getAllForecast() method is starting to look a lot better as well as the ForecastDaoImpl class in general. Adding new methods that retrieve data from the database will be easier as all I have to do to get a connection is call getConnection() and to close it closeConnection(Connection). On top of that if I need to change my database connection information I only have to do it in one area.

Despite this refactoring, this is hardly an ideal data access layer. With the test unit already created go ahead and continue to refactor getAllForecast(), a good place to start would be how I create a new Forecast object. I would also suggest adding some new functionality yourself a couple of examples might be; getForecastByDate(Date) or getForecastByCondition(String condition) (Remember create unit tests!). I will continue to refactor and update this work myself. So you can run updates and compare your work to my own.

Addendum and thanks:
Because I am an idiot I couldn't remember how to write a JDBC connection. My initial connection method is predominantly based off of the code in this article: http://www.kitebird.com/articles/jdbc.html
For being able to go to a specific revision with a url:
http://www.perhammer.com/2008/07/subversion-in-url-revision-browsing.html

Technorati Tags: Best practices, Java, Programming, Unit

In my recent article, 8 signs your code sucks, one of my signs of bad code is: “You need to use comments to explain the code.” I have since taken a lot of flak for suggesting this and I want to clarify my point and why I am not backing down.

Not javadoc - I am NOT referring to Javadoc or its equivalents in other languages. I absolutely do agree that a method or class should have accompanying documentation at its declaration stating its purpose and its inputs and/or outputs.

Comments are not bad, the code is – I'm not suggesting to forgo the use of comments, but that they are only band-aids. When you don't have time to properly write a piece of code; do it the “messy” way, write a comment and give it a TODO as a reminder to go back and fix it later.

Comments make code accessible - Code should not be written to the level of the dumbest developer, the dumbest developer should be brought to the level of the code (though I want to reiterate good code should be easily readable). If a developer is unwilling or unable to meet or at least understand the rest of the team's coding standards, then questions should be asked rather you want that developer to understand the code at all.

They are more like guidelines – When it comes to design patterns there are no rules. From day one we are taught goto statements are evil and should never be used, but there is even a time and place for them. There may be rare occasions when you have to write a comment to explain a piece of code, just like you may need to use a goto to simplify the code, but both occasions are rare and when encountered in code should be scrutinized.

Technorati Tags: Best practices, Programming

As a wide eyed junior developer when I first began working on large projects I simply accepted that it is difficult to fix bugs or find where an action is being executed. If only I knew then what I know now, I would had saved myself hours of frustration. The first step to writing good code is accepting the code you write (or work on) is crap, but sometimes you need to know what to look for. Here are some signs that your code sucks.

A method is larger than the screen – A method should only perform one specific task. A method should not contain the logic code to determine if the username field contains data, is valid, and that user exists. If a method is too large to fit within a single screen, that is a (very) good sign it is doing too much.

You are reusing variables – Unless you are working on embedded devices, memory is cheap. Don't be a memory scrooge and knee cap code maintainability, which in nearly all instances trumps performance, by reusing the same variable for multiple uses.

You are directly accessing the request/session – It not only makes writing unit tests (much) harder, but it is also difficult to know what data the application has access to. All data should be taken out of the session/request and stored in a bean. A bean, through its getters and setters, creates a “contract” of what data the application has access to, which greatly helps with code maintainability.

You need to use comments to explain the code – Code should be able to explain itself and should be in a format that is easily readable. If you find yourself needing to explain what your code is doing then you may want to look into rewriting that code.
EDIT: This is not referring to using comments (e.g. javadoc) to explain the purpose of a method/class and its inputs and outputs.

An exception's stack trace doesn't return the original problem -  You should never “eat” an exception, that is catch an exception, but not print its stack trace. How can a bug be fixed if you don't even know where the bug is occurring?

Your code is a mud ball – Just the name sounds ugly. A “mud ball” is when there is little separation between the layers or concerns of an application. Code should be modular allowing for ease of reusability and modifiability. Anything concerning the user interface happens in the view, program flow and usually data validation is the domain of the controller, handling business logic is the model's job, and only the model should be interacting with the data access layer.

It is hard to write a unit test – If you find a bug or write a new piece of code and it takes you more than a few minutes to write a unit test then that portion of code is handling too complex of a task.

The author is Billy Korando – Wait what?!

Add your own experiences or signs for dealing with sucky code.

UPDATE: There are differing opinions on my signs, since I am but a mere "grasshopper" I maybe wrong. Anyways check it out and make up your own mind.

Technorati Tags: Best practices, Programming

In part one I went over how to get SVN setup and configured on your system. In Part two I will get into the day to day activities of using SVN. In this part of my SVN primer I will cover; Checking out, committing, updating, synchronizing, branchings, and tagging. I will also go over some terminology to help make things a little more clear. I will be using subclipse as my SVN client in this part of the primer. I find using SVN from within an IDE to be more natural and including directions for both subclipse and tortoiseSVN to be confusing and there is very little difference between the two anyways.

Terminology:

Head Revision – Refers to the most recent revision in the repository.

Trunk – The main working directory of a project.

SVN Commands:

Checking out – To retrieve a project from the repository open up the “SVN Repositories” view and navigate to the project you wish to check out. Following with the example from part one it would be SVNProject>Trunk>SVNProject click finish when the next screen pops up. I find this method to be the easiest and quickest way to checkout a project.

Committing – When you have modified a file (or created a new file) you will need to “commit” it for the file to be included in version control. Subclipse keeps track of the files you have modified and when they are out of sync with the repository they display an icon next to that file stating so, subclipse shows a brown box with a white asterisk (new files are a question mark). To commit your file right click on the file(s) you wish to commit and go to team>commit . You will be presented with a pop up box, you here you will put a message of why you are committing a file, ALWAYS put a message. Messages will server as a quick reference when looking over a files history (explained below).

Updating – To get the latest file(s) from the repository you will need to run an update. To perform an update select the file/folder(s) you want to have updated and right click team>update to HEAD.

Conflicts – A conflict arises when you have a file that is out of sync with version control and you have edited line(s) of code that have been checked in since the last time you have run an update. Because SVN cannot itself determine which piece of code is correct, you must manually look at the code and determine which is correct. Here is an example of a conflict:

1.This is the change you made
2.This is the change that has been made in subversion since you last updated
3.This is several versions of the file that is in conflict
1.XXXX.java.mine is the file on your local machine
2.XXXX.java.r(lowest number) is the most recent file you retrieved from the repository
3.XXXX.java.r(highest number) is the newest file in the repository

In order to resolve a conflict you need to determine which change is correct. In this case I will keep my change, so I will delete the SVN markup (<<< .mine, ====, etc.) the change I do not like. So my file will like this:

You will also notice a weird little symbol that looks like a X with a square over it, this is the “conflict” symbol. You cannot commit a file to the repository that is in conflict. To remove the conflict you must delete the .java.mine, java.r(lowest number), etc. files. Once this is done you can commit your file to the repository. To resolve the conflict right click on the file and select Mark as Resolved, then select the action you wish to take. The "conflicts have been resolved option" uses the current file in your code view, "my file" will use the ".mine" file, incoming file will use the .r(highest number), base will use the .r(lowest number) file. (Thanks Mark!)

Branching – Most work on a SVN project is done in the “trunk” folder. There maybe times though when a change needs to be made to a project, but that change cannot affect the trunk, or a change need to be made, but it cannot affect code that will be going to production. In these scenarios a “branch” is crated. A branch is a copy of the current head revision of a file(s) that is placed in a separate folder (typically under “branches”). To create a branch right clicks on the file/folder you wish to branch and go to team>branch/tag>select your destination ({project name}/branches>name the new branch

Tagging – A tag is like a branch, however it represents snapshot of a project instead of a working directory. Typically a tag represents a project version or code that has been pushed to a production environment. When you attempt to commit a file to a tag a warning message is displayed stating a file is being committed to a tag. To create a tag team>branch/tag>select your destination ({project name}/tags>name the new tag.

REMINDER: When you branch or tag a project, the branch/tag is created off the Head revision of what is in the repository NOT what is stored locally on your computer.

View history - Subclipse allows you the the view the history of a file(s) stored in the SVN repository. The history view will show when the file has been modified (checked into SVN), the revision number, and the comments when the file was committed. To view the history of a file right click on it go to team>show history.

Reverting – Reverting allows you to retrieve a previous version of a file from the repository. Reverting is typically done to get rid of unwanted changes or for debugging reasons (i.e. a bug is known to exist in version X of a project). A neat thing SVN allows you to do is revert an entire project to a specific revision. When this is done all files are not the same revision number, but the version number they where at the time that revision was made. To perform a revert right click>team>update to revision>select the revision you wish to revert to.

This will conclude part two of the SVN primer. In part three I will go over a few more day to day task of using SVN. I will also go into a few advanced topics like setting up SVN properties which can be quite useful. Check back later this week for part three! If you have any questions please let me know.

Technorati Tags: Best practices, Subversion, SVN

http://www.codeigniter.com – An excellent and lightweight framework for developing PHP applications

http://sites.google.com/site/yacoset/ - I may not agree with everything the author says and frankly he probably doesn't care. But there is plenty of good information on his site none the less.

http://solitarygeek.com/ - If you like my site then you will probably love this one. Similar in writing style and purpose, just a more mature blog written by a more experienced developer.

Technorati Tags: Best practices, Java, Programming

Similar to link dumps, as I come across tips and ideas, that alone may not justify a blog post, but I think may be useful, I will bundle them into these posts. Some of these tips may be obvious, others esoteric, and some representing my personal preference, but I think there should be something in here for everybody. Let me know if you have any others you want me to add.

Use interfaces instead of implementations – This increases code portability and modifiability by not tying a method to a specific implementation of a type.

Delete unused code – Unless you know the code will be used in the future remove it from your code base. Code that isn't there can't be broken and does not need to be debugged. If you do need the code, you can always retrieve it from your source repository.

Don't use maps in place of beans – If you are using a map to hold different types of objects, then you should create a bean to hold that information. Beans have a contract, so you and every other developer can easily figure out what data that bean contains.

Only have one return statement in a method – Control from a method should only leave in one area. By having multiple return statements it becomes more difficult to understand the flow of a program. Having multiple returns may also be suggest of a method that should be further broken down.

Technorati Tags: Best practices, Programming

http://sourcemaking.com/ - Co-authored by Martin Fowler, a bunch of useful information and best practices on this site.

http://www.pbell.com/index.cfm/design-patterns -A wealth of information covering many different areas of programming. Definitely something here for everybody.

http://marxsoftware.blogspot.com/ - A well established blog with hundreds of articles covering mostly Java.

Technorati Tags: Best practices, Java, Programming

A successor in spirt to my 7 tips for every person wishing to be a programmer, here are 8 tips for when you finally land your first development job. These tips will hopefully help prepare you for your new job, or help get you up to speed with the rest of your team.

1.You don't know anything – So you finally have your degree and after four years of intensive studying you think you know everything there is to know about programming. Like every other developer you will soon learn (or already have) that you know nothing. Be humble, put in your hard work, and know that over time you will learn.

2.Ask questions – As you come across bugs, or areas of code that you don't understand, ask a developer on your team for help or an explanation. Make these questions opportunities to learn, don't just ask what a section of code is doing, but why it was written the way it was.

3.Study up – If you have free time at work or home, use it to learn about a concept you are having trouble with. Talk to other developers, look at code examples, or Google it/read a book. (Author note: maybe start a blog and let everybody know what you have learned)

4.Don't break the build – Everybody is guilty of it, but whenever you check your code into the repository make sure it is in a compilable state, there is nothing more frustrating to a developer then to get the latest code from the repository only to find out it doesn't work. A reputation of breaking the build is not one you want to gain.

5.Learn shop standards – Every shop has a different way writing code and/or fixing problems. Take time to learn how your shop behaves. While it may had been ok to constantly restart the server when working on a project in school, it probably won't fly with your coworkers.

6.Comment when you check in code – When you check code into the repository make sure you say why you are checking it in. This can greatly help when attempting a fix a bug that has recently been introduced into the code base. The more descriptive the better, but even a simple “Fixing issue #123” or “Fixed login bug” is far better than nothing.

7.It has probably already been done – Every business has industry specific issues to deal with. For example data may need to be in a certain format before it can be inserted into the database. So check with other developers or look through the code base to see if a solution already exists to a seemingly common business requirement.

8.Learn about the industry you are in – Understanding the needs of the users that will be using the application(s) you are working on will help save time by catching poorly written or counter-intuitive business requirements. Just like developers have QA to catch the mistakes we make, developers must occasionally catch the mistakes of business rule writers. If you think there are any other tips I missed please feel free to add them in the comments section.

Technorati Tags: Best practices, Programming