Depending on your development methodology, unit tests can serve a number of different functions. But if used purely to detect bugs then they are a very high cost approach for low return.Unit tests dont test the overall program, only isolated units. This creates many holes integration, timing, re-entrancy, inter-module communication that need to be tested another way. These limitations are not unique to application development so Ill focus instead on something that is: the difficulty of isolating your units for testing.Advocates of unit testing claim that you can follow good program design and reduce coupling and integration, making tests easy to write.Unfortunately, in application development, thats not exactly possible.
Some notes I wrote sometime ago while working on Performance testing using JMeter for a client:
Definition of Terms
Performance : The application’s ability to meet a certain goal, such as a desired response time & latency, level of throughput, concurrency, etc.
Capacity: The total load the application can handle – Quantity of Data, Number of Users, User Activity, Size of related Datasets.
Scalability: The application’s ability to maintain performance as it becomes larger by some metric (more servers, for example). Performance in broad sense is capacity and scalability together.
Availability: The percentage of time the application is able to respond to requests. This is usually measured in “nines”; for example, “five nines” means the application is available 99.999% of the time, which translates to roughly five minutes of downtime per year.
Fault tolerance : The application and overall system’s ability to handle failures gracefully. Even a system designed for high availability can fail. When it does, a fault-tolerant application can continue to provide as much functionality as possible, rather than being entirely unavailable.
We can use Performance Testing for measuring Performance (duh!), Capacity, and Scalability. The Stress test measures the Availability and the Fault Tolerance of the application.
Performance Testing
Performance Testing should occur after the functional testing is mostly complete and the software has become quite stable. Functional defects may be revealed during the performance testing, but that is not considered as the objective for Performance Testing. The objective of the performance testing is to validate the software “Speed” against the business need for the “speed” as documented in the software requirements. Software speed is generally defined as some combination of response time and workload during peak load times.
To make the application reveal its true production speed, the tests must be executed in a testing environment that approximates the intended production environment as closely as possible. In essence, the performance test provides data that allows us to meet the requirements in Scalability of the application. Performance testing is achieved by a series of tests that introduces increasingly more workload of increasingly more complex business transaction mixes.
Stress Testing
The objective of the stress testing is to measure the fault tolerance and recoverability of the software application. The stress testing provides valuable insights in achieving the availability requirements. The stress test needs to be done on various components that are depicted, preferably in a, deployment diagram.
The test plan for stress testing includes shutting down the various components of the application deployment such as web server, load balancer, nodes, database, etc. while the system is at peak workload.
Recoverability and Fault Tolerance
To test whether the recoverability of the system, the application must be under peak workload and we reboot/restart one of the component and observe the system’s behavior. For example, while the system is under peak load using the JMeter script, the database can be shutdown forcefully, and then need to be restarted. We need to check how the system behaves in such case. Ideally no data should be lost and the user should be notified and the application should try to restore its state gracefully. Same can be done by restarting one of the tomcat nodes in the cluster as well as the load balancer.
Using Maven sometimes becomes a pain for a large project. Compiling sometimes takes forever. Recently I heard a tip on our office floor that if I use a RamDisk for maven repository it may be faster. So I tried to find out easy way to setup the RamDisk on Mac OS X (Snowleopard). Here’s how I did it.
Though you can use commands to create the memory based file system, there’s an excellent free preference pane from here and install it. It is pretty straight forward from there, make sure you check Save in disk image and Self auto restore. In this way your RamDisk will be saved to the disk and restored automatically when you start the session. Now move your maven repository which defaults to ~/.m2 in Mac OS X to the newly created RamDisk. Make sure you update the location of repository in your maven’s configuration settings.xml.
If your project is small enough, you can even move the source code to the RamDisk and start up from there. I found the speed to be pretty good since the disk I/O is now minimal.
After reading the Bruce Eckel’s blog after a long time, I found an interesting and entertaining link to a book on ruby: Why’s (poignant) guide to Ruby . Being an extremely influence by Bruce, I got some conviction that there are only four REAL computer languages in the world other than Perl: C++, Java and Python. But this book seems to be re-orienting my views. I am going to download Ruby and let me check how far I can understand the real philosophy of the language.
Talking about learning a language, I feel that one should understand what the language is good for? Learning a new computer language gives me a new perspective to look at the problems. Besides learning the syntax, which is also essential part of the learning, one should understand the philosophy behind it. The way the language provides the abstraction of the solution space so that problems in the real world can be solved. The essential feature of any programmer is to map and abstract the problem space to the solution space. The problem space is the subset of the real world. It mightbe a domain (vertical) or any particular set of problems in one field. The solution space in the programmer’s perspective is composed of computers (Workstations, Servers, Network, and Memory). The basic purpose of the computer language is to provide tools for abstracting the solution space so that the real world or the entities of the problem space can be represented. And that’s why we have the Object Oriented programming.
Consider a problem space of manufacturing company’s one particular department: Accounts. In the problem space (Accounts department) we might have the following entities: Account, Accountant, Accounts Manager etc. But the solution space (software for accounts department) we are developing for the Accounts department will contain the computers/networks and memory. So the language we are going to use should be able to give us the facility to create the entities of account, accountant and manager. And it should take care of the essential conversion of the accountant (properties and behavior) into the computer’s memory. This is what essentially all the OO programming languages do.
I am going to start learn Ruby, will post more code here.
Coming to work place: I am working on design of three new prototypes to test feasibility. One of which is the protocol switching. I am currently writing some proof of concept coding.
Perl is one of my favorite languages, especially because of its versatility. Perl is everywhere: CGI, GUI, Application Development, XML Processing, Database Accesss and of course System administration tasks.
The main challenge in developing a web-application is proper design. The decoupling of the static content from the dynamic content inside the individual pages is the trickiest part. I’ve used HTML::Template and Templete Toolkit modules for providing the templates. And the templates are filled in with real data using a Perl script. The HTML::Template module provides good flexibility of separating the dynamic parts of the page from the static content. But as far as the application is small, and there’s a harmony between the application-developer and the UI-designer of the web-app, everything seems to be perfect. But as far as the UI designer is concerned he/she must know how exactly the dynamic parts are being replaced by the application logic. Otherwise, it’s going to be a big mess.Particularly for large web-applications this doesn’t work. I guess same is the case with other template mechanisms. Unless the webpage designer knows how the templates and Perl work, it’s really hard to produce good web-application.
Perl is supported by major web-servers: Apache, IIS etc. Apache has also mod_perl (kind of embedded Perl interpreter) which provides fast, in fact very fast execution of Perl Script. The main idea of using mod_perl is to get access to the Apache API itself, and extend the web-server to provide our we-app functionality, that means: essentially turning your web-servers into web-app-server. The mod_perl can do wonders for large web-sites or web-applications. With all the power of Perl and the super stable Apache, completely reliable applications can be built.
Where Am I?
You are currently browsing entries tagged with development at
VijayKiran.com.
