[Tutorial] Plotly graph drawing in Java webapp - Part 2

Plotly Javascript library supports generating various charts. Java Servlet & JSP based web applications can use it to display graphical representations of data.

In this tutorial, you will learn to include graphs into a simple Java web application. Image above shows the graph generated at the end of the tutorial. Complete project is already moved to github for your references.

This tutorial consists of two parts.

Part 1

First step is to create a simple web application with index.jsp that can submit a user entered parameter to a Servlet which is redirecting the user back to original index.jsp.

Part 2

Here the servlet is modified to return some data (collection of Customer objects) and index.jsp is modified to draw a chart using that data.

Note: If you are not that familiar with Java web applications, I recommend you to start with previous Part 1 of this tutorial before moving further.

Part 2

Java web application project structure in Part 1 must be modified as shown in below image.

Servlet Changes

Servlet class must return a list of Customers to the index.jsp. For that, a simple Customer class and a modification to Servlet is needed.

1. Create a Customer class

This class is used to represent the data send to index.jsp from CustomerServlet. It has an id as well as age & salesCount.

package com.digizol.webapp.plotly;

public class Customer {

    private String id;
    private int age;
    private int salesCount;

    public Customer(String id, int age, int salesCount) {
        this.id = id;
        this.age = age;
        this.salesCount = salesCount;
    }

    public String getId() {
        return id;
    }
    public int getAge() {
        return age;
    }
    public int getSalesCount() {
        return salesCount;
    }
}

2. CustomerServlet returning Customer object list

doGet() method is modified to return a list of Customer objects. This class has 10 Customer objects. Size of returned list will be based on the 'size' request parameter submitted via index.jsp. This Servlet will return that list as "customersList" to index.jsp.

package com.digizol.webapp.plotly;

import javax.servlet.ServletException;
import javax.servlet.http.*;
import java.io.IOException;
import java.util.*;

public class CustomerServlet extends HttpServlet {

    @Override
    protected void doGet(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {

        System.out.println("Parameter [size] value = " + request.getParameter("size"));
        // return a customers list to index.jsp
        request.setAttribute("customersList",
                getCustomers(resultSize(request.getParameter("size"))));
        request.getRequestDispatcher("/index.jsp").forward(request, response);
    }

    private List<Customer> customers = new ArrayList<Customer>();

    public CustomerServlet() {
        initCustomersList();
    }

    private int resultSize(String sizeParam) {
        return sizeParam==null?customers.size():Integer.parseInt(sizeParam);
    }

    private List<Customer> getCustomers(int size) {
        return customers.subList(0, size);
    }

    private void initCustomersList() {
        customers.add(new Customer("cust-1", 25, 17));
        customers.add(new Customer("cust-2", 36, 99));
        customers.add(new Customer("cust-3", 17, 0));
        customers.add(new Customer("cust-4", 58, 10));
        customers.add(new Customer("cust-5", 49, 32));
        customers.add(new Customer("cust-6", 80, 14));
        customers.add(new Customer("cust-7", 31, 78));
        customers.add(new Customer("cust-8", 22, 89));
        customers.add(new Customer("cust-9", 43, 21));
        customers.add(new Customer("cust-10", 74, 45));
    }
}

3. Get plotly javascript

Download the plotly-latest.min.js Javascript file from here. Place this file under src/main/webapp/plotly/js directory as shown in image.

4. index.js Changes

index.js file needs below changes.

1. Add plotly javascript file to index.jsp
2. Javascript function to draw chart
3. Add a DIV for chart
4. Update index.jsp to organize data for Javascript
5. Update index.jsp to draw chart

Fully completed index.jsp page code looks as below, do not worry. Each change is explained in details after the code.

<%@ taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core" %>

<html>

<head>
    <title>Customer Information Center</title>
    <script src="plotly/js/plotly-latest.min.js" type="text/javascript"></script>

    <script type="text/javascript">
        function plotChart(elementId, data, layout) {
            Plotly.newPlot(document.getElementById(elementId), 
                           data, layout, {displayModeBar: false});
        }
    </script>
</head>

<body>

<div style="background:#ffffee; text-align:center; padding-bottom:2px">
<h1>Customer Information Center</h1>

Draw customer information graph.<p/>

<form action="customers" method="get">

    Results size:
    <select name="size">
      <option value="5">5</option>
      <option value="10">10</option>
    </select>
    <p/>
    <button style="padding:5px">Draw Chart</button>
</form>
</div>

<c:if test="${not empty customersList}">

<h2>Age and Sales Count Chart</h2>

<div id="customersChart" ></div>

<script>

    var customerAges = {
          name: 'Age',
          type: 'lines+markers',
          line: { width: 6},
          marker: { size: 8}
        };
    var customerSalesCount = {
          name: 'Sales Count',
          type: 'lines+markers',
          line: { width: 3},
          marker: { size: 4}
        };
    var age_X = new Array();
    var age_Y = new Array();
    var sales_count_X = new Array();
    var sales_count_Y = new Array();

    <c:forEach items="${customersList}" var="customer" varStatus="i">
        age_X[${i.index}] = "${customer.id}";
        age_Y[${i.index}] = "${customer.age}";

        sales_count_X[${i.index}] = "${customer.id}";
        sales_count_Y[${i.index}] = "${customer.salesCount}";
    </c:forEach>

    customerAges.x = age_X;
    customerAges.y = age_Y;

    customerSalesCount.x = sales_count_X;
    customerSalesCount.y = sales_count_Y;

    var data = [customerAges, customerSalesCount];
    var layout = {
        xaxis: {
            title: 'ID',
            showgrid: true,
            zeroline: true,
        },
        yaxis: {
            showgrid: true,
            showline: true,
            zeroline: true,
        }
    };

plotChart("customersChart", data, layout);
</script>
</c:if>

</body>
</html>

Each change has a specific reason.

1. Add plotly javascript file to index.jsp

A reference to plotly Javascript file is placed within <head> tags of index.jsp as below.

<head>
    <title>Customer Information Center</title>
    <script src="plotly/js/plotly-latest.min.js" type="text/javascript"></script>
</head>

2. Javascript function to draw chart
Write a simple function named "plotChart" to draw graphs using plotly as follows inside <head> tag of index.jsp

<head>
    <title>Customer Information Center</title>
    <script src="plotly/js/plotly-latest.min.js" type="text/javascript"></script>

    <script type="text/javascript">
        function plotChart(elementId, data, layout) {
            Plotly.newPlot(document.getElementById(elementId), 
                           data, layout, {displayModeBar: false});
        }
    </script>
</head>

3. Add a DIV for chart

It is required to provide an empty DIV element for plotly to generate the chart.

<div id="customersChart" ></div>

4. Update index.jsp to organize data for Javascript
index.jsp page must iterate through the list of Customer objects returns from CustomerServlet as "customersList". Then, Javascript objects must be populated using that data in order to generate the graphs.

As Customer has two attributes age & salesCount, it is possible to draw two lines in one chart. As shown below, x & y properties of customerAges and customerSalesCount must be populated correctly.

    var customerAges = {
          name: 'Age',
          type: 'lines+markers'
        };
    var customerSalesCount = {
          name: 'Sales Count',
          type: 'lines+markers',
        };
    var age_X = new Array();
    var age_Y = new Array();
    var sales_count_X = new Array();
    var sales_count_Y = new Array();

    
        age_X[${i.index}] = "${customer.id}";
        age_Y[${i.index}] = "${customer.age}";

        sales_count_X[${i.index}] = "${customer.id}";
        sales_count_Y[${i.index}] = "${customer.salesCount}";
    

    customerAges.x = age_X;
    customerAges.y = age_Y;

    customerSalesCount.x = sales_count_X;
    customerSalesCount.y = sales_count_Y;

5. Update index.jsp to draw chart

Using customerAges & customerSalesCount Javascript objects, invoke plotChart() method to generate a chart inside "customersChart" div.

    var data = [customerAges, customerSalesCount];
    var layout = {
        xaxis: {
            title: 'ID'
        },
        yaxis: {
            showgrid: true
        }
    };

    plotChart("customersChart", data, layout);

Final Result

Below is the final outcome of the application after building & deploying. It shows two line graphs, for age and sales count.



Hope this helps.

Draw graphs in Java webapp with Plotly - Tutorial

Plotly Javascript library supports generating various charts. Java Servlet & JSP based web applications can use it to display graphical representations of data.

In this tutorial, you will learn to include graphs into a simple Java web application. Image above shows the graph generated at the end of the tutorial. Complete project is already moved to github for your references.

This tutorial consists of two parts.

Part 1

First step is to create a simple web application with index.jsp that can submit a user entered parameter to a Servlet which is redirecting the user back to original index.jsp.

Part 2

Here the above servlet is modified to return some data (collection of Customer objects) and index.jsp is modified to draw a chart using those data.

Note: If you are familiar with Java web applications, you can jump to Part 2 directly.

Part 1

1. Build a simple web application

This project contains only one JSP page and a Servlet. JSP is used to display a form & a chart while the Servlet is used to generate raw data for the chart.

First create the Java web application project structure as shown in image.

An index.jsp and a CustomerServlet with web.xml for a simple web application. It also has a pom.xml for the build.

CustomerServlet

doGet() method prints the size parameter in the request; then forwards the request to index.jsp page.

package com.digizol.webapp.plotly;

import javax.servlet.ServletException;
import javax.servlet.http.*;
import java.io.IOException;

public class CustomerServlet extends HttpServlet {

    @Override
    protected void doGet(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {

        System.out.println("Parameter [size] value = " + request.getParameter("size"));
        request.getRequestDispatcher("/index.jsp").forward(request, response);
    }
}

web.xml

This is the file that configures Servlet related information. Here URL "/customers" is mapped to CustomerServlet.

<web-app xmlns="http://xmlns.jcp.org/xml/ns/javaee"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee
         http://xmlns.jcp.org/xml/ns/javaee/web-app_3_1.xsd"
         version="3.1">

    <servlet>
        <servlet-name>customers</servlet-name>
        <servlet-class>com.digizol.webapp.plotly.CustomerServlet</servlet-class>
    </servlet>

    <servlet-mapping>
        <servlet-name>customers</servlet-name>
        <url-pattern>/customers</url-pattern>
    </servlet-mapping>

</web-app>

index.jsp

This has the ability to submit the results size to "/customers" URL which actually is received by CustomerServlet.

<html>
<body>
    <div>
        <h1>Customer Information Center</h1>
        Draw customer information graph.<p/>
        <form action="customers" method="get">
            Results size:
            <select name="size">
              <option value="5">5</option>
              <option value="10">10</option>
            </select>
            <p/>
            <button style="padding:5px">Draw Chart</button>
        </form>
    </div>
</body>
</html>

Clicking on "Draw Chart" button caused the doGet() method to be invoked on CustomerServlet.

pom.xml

Maven build file is used to generate the WAR file including the libraries.

<project xmlns="http://maven.apache.org/POM/4.0.0" 
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
  http://maven.apache.org/maven-v4_0_0.xsd">

  <modelVersion>4.0.0</modelVersion>
  <groupId>com.digizol.webapp.plotly</groupId>
  <artifactId>webapp-with-plotly</artifactId>
  <packaging>war</packaging>
  <version>1.0-SNAPSHOT</version>
  <name>webapp-with-plotly maven webapp</name>

  <dependencies>
    <dependency>
      <groupId>javax.servlet</groupId>
      <artifactId>javax.servlet-api</artifactId>
      <version>3.1.0</version>
    </dependency>
    <dependency>
        <groupId>javax.servlet</groupId>
        <artifactId>jstl</artifactId>
        <version>1.2</version>
    </dependency>
  </dependencies>

  <build>
    <finalName>customers-plotly</finalName>
  </build>

</project>

2. Build & Deploy

With Maven 3.*, you can build the project using "mvn clean package" command. Deployable customers-plotly.war is generated inside target folder.

Copy customers-plotly.war into webapps directory of a Tomcat deployment and start Tomcat server.

Use below URL to see the web page & try clicking "Draw Chart" button.

http://localhost:8080/customers-plotly



Adding charts to this webpage is covered in Part 2.

[Eclipse] How to use local DTDs to validate XMLs

By default Eclipse does not support local DTD files. Because of this, many developers loose a privilege of Eclipse; content assistance for parameter/attribute is missing while working with DTD based XML files. However if you are connected to Internet, you will not face this issue.

Here is a tip to show you how to point your Eclipse instance to local DTD files.

1. Add XML Catalogs

First you need to open up the "XML Catalog" dialog by:
Windows -> Preferences -> XML -> XML Catalog

Then click on "Add..." and select "Catalog Entry". For a single local DTD file you need to add one Catalog Entry.

For the location: you need to find DTD file from your local disk.
For the Key: you need to enter the relevant key as per the <!DOCTYPE entry of your XML files.

2. Hibernate Examples Demo

This tip is demoed using Hibernate as an example. Hibernate Configuration DTD and Hibernate Mappings DTD is added locally into Eclipse below. First of all, you need to download and store hibernate jar file somewhere in your local disk; we used hibernate-core-3.6.10.Final version (available here as well as here).

2.1 Hibernate Configuration DTD

DTD file referred by hibernate.cfg.xml file is "hibernate-configuration-3.0.dtd". To point eclipse to that DTD, you need to enter below values and select OK in above "Catalog Entry" screen.

Key: -//Hibernate/Hibernate Configuration DTD 3.0//EN
Location: jar:file:/<path/to/jar>/hibernate-core.jar!/org/hibernate/hibernate-configuration-3.0.dtd

Then Eclipse will create a new Catalog and save with below details.

Entry element:  Public
Location:   org/hibernate/hibernate-configuration-3.0.dtd in jar file opt/libs/hibernate-core.jar
URI:       jar:file:/opt/libs/hibernate-core.jar!/org/hibernate/hibernate-configuration-3.0.dtd
Key type:   Public ID
Key:    -//Hibernate/Hibernate Configuration DTD 3.0//EN

2.2 Hibernate Mappings DTD

DTD file referred by <entity>.hbm.xml file is "hibernate-mapping-3.0.dtd". Same as above step, in "Catalog Entry" screen you need to enter below values and select OK.

Eclipse will create a new Catalog and save with below details.

Entry element:  Public
Location:   org/hibernate/hibernate-mapping-3.0.dtd in jar file opt/libs/hibernate-core.jar
URI:       jar:file:/opt/libs/hibernate-core.jar!/org/hibernate/hibernate-mapping-3.0.dtd
Key type:   Public ID
Key:    -//Hibernate/Hibernate Mapping DTD 3.0//EN

2.3 Results

After creating the Catalogs, you can check this my opening your hibernate.cfg.xml file and trying to open up the DTD file link in the top of the file by "Ctrl + Mouse Over". You will notice that Eclipse prompts two links and one is the local DTD.

Similarly, Eclipse will provide content assistance for XML file editing even when internet connectivity is gone.

3. Conclusion

Using this method, you can create catalogs for any DTD and let Eclipse point to your local DTD files. Hope this helps you.

[Eclipse] Exporting & Importing Launch configurations

Many Eclipse "Launch Configuration" created has so much details; OSGi configurations for instance. So there arises a requirement of exporting the launch configurations and later importing them into other Eclipse workspaces. Interestingly, Eclipse has an import/export feature for "Launch Configuration"; which is quite simple but handy. I believe some software developers are not taking the advantage of this.

Here are the steps to follow.

1. Exporting Configurations

First you need to select:
  File -> Export...
Then select:
  Run/Debug -> Launch Configurations

Then you will see a view with all available launch configurations as shown below. You can select the configurations you need to export. You need a provide a location to store the exported configurations.

Each configuration will be saved into separate XML files, but with the extension as ".launch".

2. Importing Configurations

Similar to export, first you need to select:
  File -> Import...
Then select:
  Run/Debug -> Launch Configurations

Then you will see a view where you can select a directory with ".launch" files. When that folder is selected, available launch configurations will be displayed in the UI; so that you can select which ones to be imported as shown below.

3. Sample .launch file

Below is the exporter file for the OSGi launch configuration we created for Eclipse Kepler 4.3.1.

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<launchConfiguration type="org.eclipse.pde.ui.EquinoxLauncher">
    <booleanAttribute key="append.args" value="true"/>
    <booleanAttribute key="automaticAdd" value="true"/>
    <booleanAttribute key="automaticValidate" value="false"/>
    <stringAttribute key="bootstrap" value=""/>
    <stringAttribute key="checked" value="[NONE]"/>
    <booleanAttribute key="clearConfig" value="false"/>
    <stringAttribute key="configLocation" value="${workspace_loc}/.metadata/.plugins/org.eclipse.pde.core/Multi-Management-App"/>
    <booleanAttribute key="default" value="true"/>
    <booleanAttribute key="default_auto_start" value="true"/>
    <intAttribute key="default_start_level" value="4"/>
    <booleanAttribute key="includeOptional" value="true"/>
    <booleanAttribute key="org.eclipse.jdt.launching.ATTR_USE_START_ON_FIRST_THREAD" value="true"/>
    <stringAttribute key="org.eclipse.jdt.launching.JRE_CONTAINER" value="org.eclipse.jdt.launching.JRE_CONTAINER/org.eclipse.jdt.internal.debug.ui.launcher.StandardVMType/JavaSE-1.6"/>
    <stringAttribute key="org.eclipse.jdt.launching.PROGRAM_ARGUMENTS" value="-os ${target.os} -ws ${target.ws} -arch ${target.arch} -nl ${target.nl} -consoleLog -console"/>
    <stringAttribute key="org.eclipse.jdt.launching.SOURCE_PATH_PROVIDER" value="org.eclipse.pde.ui.workbenchClasspathProvider"/>
    <stringAttribute key="org.eclipse.jdt.launching.VM_ARGUMENTS" value="-Declipse.ignoreApp=true -Dosgi.noShutdown=true"/>
    <stringAttribute key="pde.version" value="3.3"/>
    <booleanAttribute key="show_selected_only" value="true"/>
    <stringAttribute key="target_bundles" value="org.apache.felix.gogo.command@default:default,org.apache.felix.gogo.runtime@default:default,org.apache.felix.gogo.shell@default:default,org.eclipse.equinox.console@default:default,org.eclipse.osgi@-1:true"/>
    <booleanAttribute key="tracing" value="false"/>
    <booleanAttribute key="useCustomFeatures" value="false"/>
    <booleanAttribute key="useDefaultConfigArea" value="true"/>
    <stringAttribute key="workspace_bundles" value="com.digizol.osgi.multiservice.english@default:default,com.digizol.osgi.multiservice.sinhala@default:default,com.digizol.osgi.multiservice@default:default,com.digizol.osgi.baseservice@default:default"/>
</launchConfiguration>

Hope this helps you.

OSGi Console of Eclipse Kepler 4.3.1

Eclipse IDE is developed based on OSGi plugin architecture, but I had not seen the list of OSGi bundles that are used in my Eclipse installation. Today I noticed that there is a way to find the list of bundled used, and it is quite simple. So I thought of sharing it with you.

I am running Eclipse version 4.3.1 (JEE Kepler). First we need to open the OSGi console of the running Eclipse instance. For that we need to open up the 'Console view' first via:

Window -> Show view -> Console

Then you need to select "Host OSGi Console" on the sub-toolbar in this Console view as shown in the image below.

As soon as you go to this console view, you will see below WARNING, which is quite OK since our intention is also to connect to the running instance of Eclipse.

WARNING: This console is connected to the current running instance of Eclipse!
osgi>

If you are already familiar with OSGi console in OSGi software development, then most likely you know how to find details about the plugins. However even if you are new to OSGi, no need to worry. You simple have to enter "ss" command in this console to see the list of bundles with id and state. I got a list of 822 plugins and below is a trimmed down version of the output I received.

WARNING: This console is connected to the current running instance of Eclipse!
osgi> ss
"Framework is launched."


id State       Bundle
0 ACTIVE      org.eclipse.osgi_3.9.1.v20130814-1242
1 ACTIVE      org.eclipse.equinox.simpleconfigurator_1.0.400.v20130327-2119
2 RESOLVED    ch.qos.logback.classic_1.0.7.v20121108-1250
             Fragments=525
3 RESOLVED    ch.qos.logback.core_1.0.7.v20121108-1250
4 RESOLVED    ch.qos.logback.slf4j_1.0.7.v20121108-1250
             Master=817
5 ACTIVE      com.ibm.icu_50.1.1.v201304230130
6 RESOLVED    com.jcraft.jsch_0.1.46.v201205102330
7 RESOLVED    com.ning.async-http-client_1.6.5.20130531-2315
8 RESOLVED    com.sun.el_2.2.0.v201303151357
9 RESOLVED    com.sun.syndication_0.9.0.v200803061811
10 RESOLVED    java_cup.runtime_0.10.0.v201005080400
11 RESOLVED    javaewah_0.5.6.v201307211000
12 RESOLVED    javax.activation_1.1.0.v201211130549
13 RESOLVED    javax.annotation_1.1.0.v201209060031
14 RESOLVED    javax.el_2.2.0.v201303151357
15 RESOLVED    javax.inject_1.0.0.v20091030
16 RESOLVED    javax.jws_2.0.0.v201005080400
17 RESOLVED    javax.mail_1.4.0.v201005080615
18 RESOLVED    javax.persistence_2.1.0.v201304241213
19 RESOLVED    javax.servlet_3.0.0.v201112011016
20 RESOLVED    javax.servlet.jsp_2.2.0.v201112011158
21 RESOLVED    javax.wsdl_1.6.2.v201012040545
22 RESOLVED    javax.wsdl_1.5.1.v201012040544
23 RESOLVED    javax.xml_1.3.4.v201005080400
24 RESOLVED    javax.xml.bind_2.1.9.v201005080401
25 RESOLVED    javax.xml.rpc_1.1.0.v201209140446
26 RESOLVED    javax.xml.soap_1.2.0.v201005080501
27 RESOLVED    javax.xml.stream_1.0.1.v201004272200
28 RESOLVED    javax.xml.ws_2.1.0.v200902101523
29 RESOLVED    net.sourceforge.lpg.lpgjavaruntime_1.1.0.v201004271650
30 RESOLVED    org.apache.ant_1.8.4.v201303080030
31 RESOLVED    org.apache.axis_1.4.0.v201005080400
32 RESOLVED    org.apache.batik.css_1.6.0.v201011041432
33 RESOLVED    org.apache.batik.util_1.6.0.v201011041432
34 RESOLVED    org.apache.batik.util.gui_1.6.0.v201011041432
35 RESOLVED    org.apache.bcel_5.2.0.v201005080400
36 RESOLVED    org.apache.commons.codec_1.4.0.v201209201156
37 RESOLVED    org.apache.commons.collections_3.2.0.v2013030210310
38 RESOLVED    org.apache.commons.discovery_0.2.0.v201004190315
39 RESOLVED    org.apache.commons.httpclient_3.1.0.v201012070820
40 RESOLVED    org.apache.commons.io_2.0.1.v201105210651
41 RESOLVED    org.apache.commons.lang_2.6.0.v201205030909
42 RESOLVED    org.apache.commons.logging_1.1.1.v201101211721
43 RESOLVED    org.apache.commons.logging_1.0.4.v201101211617
44 RESOLVED    org.apache.commons.net_3.2.0.v201305141515
45 ACTIVE      org.apache.felix.gogo.command_0.10.0.v201209301215
46 ACTIVE      org.apache.felix.gogo.runtime_0.10.0.v201209301036
47 ACTIVE      org.apache.felix.gogo.shell_0.10.0.v201212101605
48 RESOLVED    org.apache.httpcomponents.httpclient_4.1.3.v201209201135
49 RESOLVED    org.apache.httpcomponents.httpcore_4.1.4.v201203221030
50 RESOLVED    org.apache.jasper.glassfish_2.2.2.v201205150955
51 RESOLVED    org.apache.log4j_1.2.15.v201012070815
52 RESOLVED    org.apache.lucene.analysis_3.5.0.v20120725-1805
53 RESOLVED    org.apache.lucene.core_3.5.0.v20120725-1805
54 RESOLVED    org.apache.velocity_1.5.0.v200905192330
55 RESOLVED    org.apache.ws.commons.util_1.0.1.v20100518-1140
56 RESOLVED    org.apache.wsil4j_1.0.0.v200901211807
57 RESOLVED    org.apache.xalan_2.7.1.v201005080400
58 RESOLVED    org.apache.xerces_2.9.0.v201101211617
59 RESOLVED    org.apache.xml.resolver_1.2.0.v201005080400
60 RESOLVED    org.apache.xml.serializer_2.7.1.v201005080400
61 RESOLVED    org.apache.xmlrpc_3.0.0.v20100427-1100
62 STARTING    org.eclipse.ant.core_3.2.500.v20130402-1746
63 STARTING    org.eclipse.ant.launching_1.0.300.v20130514-1341
64 STARTING    org.eclipse.ant.ui_3.5.400.v20130514-1341
65 STARTING    org.eclipse.compare_3.5.401.v20130709-1308
66 STARTING    org.eclipse.compare.core_3.5.300.v20130514-1224
67 RESOLVED    org.eclipse.core.commands_3.6.100.v20130515-1857

...

818 RESOLVED    org.sonatype.m2e.mavenarchiver_0.15.0.201207090125-signed-20130612210623
819 RESOLVED    org.uddi4j_2.0.5.v200805270300
820 RESOLVED    org.w3c.css.sac_1.3.1.v200903091627
821 RESOLVED    org.w3c.dom.smil_1.0.0.v200806040011
822 RESOLVED    org.w3c.dom.svg_1.1.0.v201011041433
osgi> 

If you are interested in finding further details about these OSGi bundles, you can enter 'help' in the OSGi console and find the list of commands to try out.

[Java] String format against opening & closing characters

How to validate the format of a given string according to a given set of opening-closing character pairs? This has been raised as an interview question, so I thought of writing a simple program to solve this.

Let me elaborate the question. There are characters that are considered as opening characters and closing characters; when '(' is an opening character, the relevant closing character is ')' For example; ([]) is correctly formatted, but ([) is not. So your task is to find out whether all the starting characters are properly ended with a ending character.

Here, I am using a Stack (java.util.Stack) to solve the problem.

GroupingPair class

A simple class named GroupingPair is written to hold pair of opening and closing characters. For instance, '[' for starting and ']' for ending.

package com.digizol.interviews.string.format;

public class GroupingPair {

    private Character start;
    private Character end;

    public GroupingPair(Character start, Character end) {
        this.start = start;
        this.end = end;
    }

    public Character getStart() {
        return start;
    }

    public Character getEnd() {
        return end;
    }
}

FormatValidator class

This class has implemented the logic using a stack. This can be initialized with a list of GroupingPair objects. Then a given strings each character is compared against the GroupingPair objects to find out whether there is an opening or closing characters. If an opening character is found, related GroupingPair instance is pushed to the stack. Similarly, if a closing tag is found, the top GroupingPair is popped and compared with the character to see whether it matches with the expected.

package com.digizol.interviews.string.format;

import java.util.HashMap;
import java.util.Map;
import java.util.Stack;

public class FormatValidator {

    private Map<Character, GroupingPair> openingChars;
    private Map<Character, GroupingPair> closingChars;

    public FormatValidator(GroupingPair[] pairs) {
        initOpeningCharacters(pairs);
        initClosingCharacters(pairs);
    }

    private void initClosingCharacters(GroupingPair[] pairs) {
        closingChars = new HashMap<Character, GroupingPair>();
        for (GroupingPair pair: pairs) {
            closingChars.put(pair.getEnd(), pair);
        }
    }

    private void initOpeningCharacters(GroupingPair[] pairs) {
        openingChars = new HashMap<Character, GroupingPair>();
        for (GroupingPair pair: pairs) {
            openingChars.put(pair.getStart(), pair);
        }
    }

    public boolean validate(String string) {
        return validate(string, false);
    }
    
    public boolean validate(String string, boolean validateOtherCharacters) {

        Stack<GroupingPair> stack = new Stack<GroupingPair>();

        char[] characterArray = string.toCharArray();

        for (Character c: characterArray) {

            if (openingChars.containsKey(c)) {
                stack.push(openingChars.get(c));
            } else if (closingChars.containsKey(c)) {
                if (!c.equals(stack.pop().getEnd())) {
                    return false;
                }
            } else if (validateOtherCharacters) {
                System.out.println("Unexpected character '" + c + "' found in string: " + string);
                return false;
            }
        }
        return stack.isEmpty();
    }
}

Test class

This is a simple class with the main method to test few examples. Character pairs (), '<'>' and [] are given for initialization.

package com.digizol.interviews.string.format;

public class Test {

    public static void main(String[] args) {

        FormatValidator validator = new FormatValidator(createPairs());

        String[] toTest = { "[(])", 
                            "([<>])", 
                            "([)", 
                            "()[]<>", 
                            "(()[]<>)",
                            "(mal [ formatted )",
                            "(this [ is < well > formatted ] text)"
                            };

        for (String string : toTest) {

            boolean valid = validator.validate(string, false);

            if (valid) {
                System.out.println(string + " \t-> well formatted");
            } else {
                System.out.println(string + " \t-> malformed");
            }
        }
    }

    private static GroupingPair[] createPairs() {
        GroupingPair[] pairs = new GroupingPair[] {
                                new GroupingPair('(', ')'), 
                                new GroupingPair('<', '>'), 
                                new GroupingPair('[', ']') 
        };
        return pairs;
    }
}

Following is the output you will get when this Test class is executed.

[(]) -> malformed
([<>]) -> well formatted
([) -> malformed
()[]<> -> well formatted
(()[]<>) -> well formatted
(mal [ formatted ) -> malformed
(this [ is < well > formatted ] text) -> well formatted

As you can see, the program has identified whether a string is well formatted or not. This eclipse project is available for download.

[Jenkins] Automatically retry a failed build

Are your Jenkins builds failing due to unavoidable reasons like unavailability of external databases, file systems etc? The only solution you might be having right now is to reschedule the build after fixing that external issue. In this post, I will be discussing on how you can automatically rerun a failed build.

Setup

For this, we are going to use a plugin named Naginator, version 1.8 is available here for download. As the first step, please install this plugin into your Jenkins.

When it is installed, there will be a new action named "Retry build after failure" added to the post-build action list in job configuration page (as shown).

Configuration

First create a new job. If you need to retry an existing job, please open the job configuration page. Then click on the "Add post-build action" button and select "Retry build after failure". This will add a new configuration section as shown in the below image.

There are 3 configuration settings (as numbered in above image). Let's discuss each in details below.

Setting 1 -> Rerun build for unstable builds as well as failures

This option is there to indicate whether you should be retrying the UNSTABLE state builds as well in addition to FAILURE state builds. Let's set it to true.

Setting 2 -> Delay before schedule

This allows you to define how long to delay a retrying build start-time from the time it was failed. There are two options to choose from; one is to provide a "fixed period" and the other is to provide an "increasing period". Let's use fixed period; and set 300 there (meaning 5 mins).

Setting 3 -> Max number of successive failed build

This is used to control the number of continuous failure builds. Jenkins will not automatically retry more than this maximum number of consecutive failures. So let's set 2 there so that it is retried only two times.

Conclusion

As expected, this is working without any issues and stops retrying as soon as the build is successful. Hope this will help you in avoiding manual retries.

[Log4j] How to integrate with your Java project

This is a quick guide on how to use log4j in your Java project. This task requires only five simple steps listed below. Each of these step is explained in details below.

1. Set up the project with log4j
2. Create log4j.properties file
3. Write a class to record log messages
4. Compile and run program
5. Check the log messages

1. Set up the project with log4j

As the first step, let's download the log4j 1.2.17 latest archive (tar or zip). When you extract the archive, log4j-1.2.17.jar file is available in the root of the directory.

Let's create a new project named say log4j-helloworld and copy the log4j-1.2.17.jar into a folder named lib inside the project.

2. Create log4j.properties file

Next task is to create a log4j.properties file (shown below) inside the project folder. This file is the configuration file that provides how, where, which etc instructions on generating log messages.

# root level configurations
log4j.rootLogger=INFO,console,file

# configuration for console outputs
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.layout=org.apache.log4j.PatternLayout

# configuration for file output (into a file named messages.log)
log4j.appender.file=org.apache.log4j.RollingFileAppender
log4j.appender.file.File=messages.log
log4j.appender.file.layout=org.apache.log4j.PatternLayout

# threshold for file output
log4j.appender.file.Threshold=ERROR

3. Write a class to record log messages

Let's write a simple class named Main (in src folder) with a main method to generate some log statements. Logger is added to the class as an attribute and used to write logs of different levels like debug, info and error.

package com.digizol.log4j.helloworld;

import org.apache.log4j.Level;
import org.apache.log4j.Logger;

public class Main {

    private static Logger log = Logger.getLogger(Main.class);

    public static void main(String[] args) {

        log.debug("This is debug method");
        log.info("This is info method");
        log.error("This is error method");

        log.log(Level.DEBUG, "This is debug from Level.DEBUG");
        log.log(Level.INFO, "This is info from Level.INFO");
        log.log(Level.ERROR, "This is error from Level.ERROR");

    }
}

4. Compile and run program

If you compile and run from command line, followings are the commands. Make sure to create classes folder inside project for generated class files before executing the commands.

# compile
javac -cp lib/log4j-1.2.17.jar -d classes -sourcepath src src/com/digizol/log4j/helloworld/Main.java

# run
java -cp lib/log4j-1.2.17.jar:./classes:./ com.digizol.log4j.helloworld.Main

If you are using an IDE like Eclipse, you can simply add log4j jar file into the build path before compiling. When running the Main class, you need to give the path to log4j.properties file in run configuration.

5. Check the log messages

In the Main class, we have added two debug level messages. However the rootLogger is configured in INFO level in log4j.properties file, so none of the debug level log messages are expected as outputs.

Console output as as below.

This is info method
This is error method
This is info from Level.INFO
This is error from Level.ERROR

Console has received both INFO and ERROR log messages, but not DEBUG.

However messages.log file has received only the ERROR level log messages as below.

This is error method
This is error from Level.ERROR

This is because the Threshold level of the file logging appender is set to ERROR which overrides the rootLogger configuration.

Hope this helps you to start using log4j in your next project. I have added this project here for your references.

Minimum difference in two sorted arrays

Given two sorted arrays of same length, find pairs of numbers (one from each array) which has the minimum distance between those two numbers. This is the last weeks Thursday Coding Puzzle. Since both arrays are sorted, binary search is going be really useful in searching the values in arrays.
Even though this problem is not related to finding the exact matches as in binary search, it is easy to solve this problem with the same approach used in binary search. As a preparation for this puzzle, it is really useful if you can try and write binary search yourself without referring to internet for help.

This problem can be simplified into two steps and implemented incrementally. Initially, we can simplify this problem to use only one array out of the two; then use one value from the other array rather than the second array itself. Then enhance that solution to use all values in second array against the first array to complete the answer for the problem.

One Array against One Value problem

In this problem, it is required to find a value within the array such that it is the closest to the given value. There can be one of multiple values that are the closest to a given value; so that has to be supported.

Test Cases

As usual, test cases are written first before writing any production code. Followings are the four test cases.

1. shouldReturnEmptyForAnEmptyArray
     > If the array is empty, result has to be empty

2. shouldReturnSameValueIfFoundAnywhereInGivenArray
     > If the given array contains the given value, the distance is zero (0). So it has to be returned from the array other than any value.
     e.g.: For an array with 1, 4, 7, 11 against value 4, then number pair has to be (4,4)

3. shouldReturnCorrectIfOneClosestValueIsAnywhereInGivenArray
     > If the given array contains one closest value, then that is the minimum distance. So it has to be returned with the given value.
     e.g.: For an array with 1, 4, 7, 11 against value 5, then minimum distance is 1 for the number pair (4,5) which has to be returned.

4. shouldReturnCorrectIfMultipleValuesAreTheClosest
     > If the given array contains multiple values closest to the given value, then those has to be returned with the given value.
     e.g.: For an array with 1, 4, 8, 11 against value 6, then minimum distance is 2 for the number pairs (4,6) and (6,8) which has to be returned.

Above test cases are implemented in the following manner with JUnit4.

package com.digizol.puzzle;

import static org.junit.Assert.assertEquals;
import java.util.*;
import org.junit.Test;

public class MinimumDifferencesTest {

    private MinimumDifferences sut;

    public MinimumDifferencesTest() {
        sut = new MinimumDifferences();
    }

    @Test
    public void shouldReturnEmptyForAnEmptyArray() {
        assertEquals(0, sut.getMinPair(new int[] {}, 1).length);
    }

    @Test
    public void shouldReturnSameValueIfFoundAnywhereInGivenArray() {
        int[] sorted = { 1, 4, 7, 11, 15, 19, 26, 30, 33, 40 };
        for (int i = 0; i < sorted.length; i++) {
            int[] result = sut.getMinPair(sorted, sorted[i]);
            assertEquals(1, result.length);
            assertEquals(sorted[i], result[0]);
        }
    }
    
    @Test
    public void shouldReturnCorrectIfOneClosestValueIsAnywhereInGivenArray() {
        int[] sorted = { 1, 4, 7, 11, 15, 19, 26, 30, 33, 40 };
        int[] result = sut.getMinPair(sorted, sorted[0] - 1);
        assertEquals(1, result.length);
        assertEquals(sorted[0], result[0]);

        for (int i = 0; i < sorted.length; i++) {
            result = sut.getMinPair(sorted, sorted[i] + 1);
            assertEquals(1, result.length);
            assertEquals(sorted[i], result[0]);
        }
    }

    @Test
    public void shouldReturnCorrectIfMultipleValuesAreTheClosest() {
        int[] sorted = { 1, 4, 7, 11, 15, 19, 26, 30, 33, 40 };
        int[] result = sut.getMinPair(sorted, 28);
        assertEquals(2, result.length);
        List<Integer> expected = new ArrayList<Integer>();
        expected.add(26);
        expected.add(30);

        for (int i = 0; i < result.length; i++) {
            expected.remove(Integer.valueOf(result[i]));
        }
        assertEquals(0, expected.size());
    }
}

Production Code

By simply tweaking the binary search code, it is really easy to implement this expected functionality. You can compare the below implementation code with the binary search code; specially look at the three comments shown in both .

package com.digizol.puzzle;

public class MinimumDifferences {

    public int[] getMinPair(int[] sorted, int value) {
        if (sorted.length == 0) {
            return new int[] {};
        }
        return getMinPair(sorted, value, 0, sorted.length - 1, new int[]{});
    }

    private int[] getMinPair(int[] sorted, int value, 
                             int leftIndex, int rightIndex, int[] minValues) {

        // 1. index check
        if (leftIndex > rightIndex) {
            return minValues;
        }

        // 2. middle index
        int middleIndex = (leftIndex + rightIndex) / 2;

        if (minValues.length == 0) {
            minValues = new int[] { sorted[middleIndex] };
        } else {
            int oldDiff = diff(value, minValues[0]);
            int newDiff = diff(value, sorted[middleIndex]);
            
            if (newDiff < oldDiff) {
                minValues = new int[] { sorted[middleIndex] };
            } else if (newDiff == oldDiff)  {
                int[] temp = minValues;
                minValues = new int[temp.length + 1];
                for (int i = 0; i < temp.length; i++) {
                    minValues[i] = temp[i];
                }
                minValues[minValues.length - 1] = sorted[middleIndex];
            }
        }

        // 3. recursive invoke
        if (sorted[middleIndex] > value) {
            return getMinPair(sorted, value, leftIndex, middleIndex - 1, minValues);
        } else if (sorted[middleIndex] < value) {
            return getMinPair(sorted, value, middleIndex + 1, rightIndex, minValues);
        } else {
            return minValues;
        }
    }

    private int diff(int value, int currentValue) {
        
        int diff = currentValue - value;
        if (diff < 0) {
            diff = -diff;
        }
        return diff;
    }
}

Two Arrays Problem

Now we have got the minimum distance number pairs against a given value. Now the enhancement is quite simple, just need to invoke the above "getMinPair(int[] sorted, int value)" method with each value in the second array of the problem and find the smallest minimum of those.

Test Cases

Following is one of the test cases to check this two array based implementation.

    @Test
    public void shouldReturnMultiplePairsIfFoundInGivenArrays() {
        int[] first = { 1, 10, 20, 30, 40};
        int[] second = { 6, 14, 25, 35, 45};
        int[][] result = sut.getMinPair(first, second);
        assertEquals(2, result.length);
        
        for (int i = 0; i < result.length; i++) {
            if (result[i][0] == 6) {
                assertEquals(10, result[i][1]);
            } else if (result[i][0] == 10) {
                 if (result[i][1] != 6 && result[i][1] != 14) {
                     assert false;
                 }
            } else if (result[i][0] == 14) {
                assertEquals(10, result[i][1]);
            }
        }
    }

You can insert the following method into the above implemented MinimumDifferences class to complete the implementation for the puzzle.

    public int[][] getMinPair(int[] first, int[] second) {
        int[][] pairs = new int[0][];
        
        int diff = Integer.MAX_VALUE;
         
        for (int i = 0; i < second.length; i++) {

            // invoke modified binary search method
            int[] mins = getMinPair(first, second[i]);
            
            if (mins.length != 0) {
                int localDiff = mins[0] - second[i];
                if (localDiff < 0) {
                    localDiff = -localDiff;
                }
                if (diff > localDiff) {
                    diff = localDiff;
                    pairs = new int[mins.length][2];
                    for (int j = 0; j < mins.length; j++) {
                        int[] value = new int[]{mins[j], second[i]};
                        pairs[j] = value;
                    }
                } else if (diff == localDiff) {
                    int[][] temp = pairs;
                    pairs = new int[temp.length + mins.length][2];
                    for (int j = 0; j < temp.length; j++) {
                        pairs[j] = temp[j];
                    }
                    for (int j = 0; j < mins.length; j++) {
                        int[] value = new int[]{mins[j], second[i]};
                        pairs[j + temp.length] = value;
                    }
                }
            }
        }
        return pairs;
    }

Hope this helps you to understand how binary search can be modified to solve two sorted arrays based problem.

Related Articles

• Java: Binary Search (recursive) & TestCases
• Java Sorting: Comparator vs Comparable Tutorial

Java: Binary Search (recursive) & TestCases

Test cases for Binary search might not be something you have already written, but the implementation must be an old exercise you may have done in your algorithm lessons. May be it is easy for you to write it yourself without referring to examples or helping materials? I think it's time for you to try it yourself if you have not done it for a long time.

Since I wanted to solve a puzzle with binary search, I decided to write binary search myself. If you can not remember what the binary search is; a given value must be searched in a sorted array with the aim of minimizing the number of operations by limiting the search into the left or right half of the array by dividing it into two parts.

Test Cases

Test cases for this is not that specific to binary search, but mostly for any search. However as this search is executed in half by half approach, I wrote a number of test cases to search for the values in first, last, middle indexes followed by an all index search. Also since the array division is changing based on whether array length is odd or even, two test cases are added for that.

Here are the names of the test cases.

1. shouldReturnFalseIfArrayIsEmpty()
2. shouldReturnFalseIfNotFoundInSortedOddArray()
3. shouldReturnFalseIfNotFoundInSortedEvenArray()
4. shouldReturnTrueIfFoundAsFirstInSortedArray()
5. shouldReturnTrueIfFoundAtEndInSortedArray()
6. shouldReturnTrueIfFoundInMiddleInSortedArray()
7. shouldReturnTrueIfFoundAnywhereInSortedArray()
8. shouldReturnFalseIfNotFoundInSortedArray()

Above names are a bit self-explanatory, please see the following code for details.

package com.digizol.algorithms;

import static org.junit.Assert.*;
import org.junit.*;

public class BinarySearchTest {

    private BinarySearch sut;

    @Before
    public void setUp() throws Exception {
        sut = new BinarySearch();
    }

    @Test
    public void shouldReturnFalseIfArrayIsEmpty() {
        assertEquals(false, sut.find(new int[] {}, 1));
    }

    @Test
    public void shouldReturnFalseIfNotFoundInSortedOddArray() {
        assertEquals(false, 
                sut.find(new int[] { 0, 2, 4, 6, 8, 10, 12, 14, 16 }, 9));
    }

    @Test
    public void shouldReturnFalseIfNotFoundInSortedEvenArray() {
        assertEquals(false,
                sut.find(new int[] { 0, 2, 4, 6, 8, 10, 12, 14, 16, 18 }, 9));
    }

    @Test
    public void shouldReturnTrueIfFoundAsFirstInSortedArray() {
        assertEquals(true,
                sut.find(new int[] { 0, 2, 4, 6, 8, 10, 12, 14, 16 }, 0));
    }

    @Test
    public void shouldReturnTrueIfFoundAtEndInSortedArray() {
        assertEquals(true,
                sut.find(new int[] { 0, 2, 4, 6, 8, 10, 12, 14, 16 }, 16));
    }

    @Test
    public void shouldReturnTrueIfFoundInMiddleInSortedArray() {
        assertEquals(true,
                sut.find(new int[] { 0, 2, 4, 6, 8, 10, 12, 14, 16 }, 8));
    }

    // covers the 'true' cases above
    @Test
    public void shouldReturnTrueIfFoundAnywhereInSortedArray() {
        int[] sorted = new int[] { 0, 2, 4, 6, 8, 10, 12, 14, 16 };

        for (int i = 0; i < sorted.length; i++) {
            assertEquals(true, sut.find(sorted, sorted[i]));
        }
    }

    // covers the 'false' cases above
    @Test
    public void shouldReturnFalseIfNotFoundInSortedArray() {
        int[] sorted = new int[] { 0, 2, 4, 6, 8, 10, 12, 14, 16 };

        assertEquals(false, sut.find(sorted, sorted[0] - 1));    
        for (int i = 0; i < sorted.length; i++) {
            assertEquals(false, sut.find(sorted, sorted[i] + 1));
        }        
    }
}

Production Code

I want to stress the importance of you trying yourself to complete the code yourself rather than directly going to the code below.

I wrote below recursion based production code which passes against above test cases. There are three comments in the code, which helps you to remember the logic easily.

package com.digizol.algorithms;

public class BinarySearch {

    public boolean find(int[] sortedValues, int value) {
        return search(sortedValues, value, 0, sortedValues.length - 1);
    }

    private boolean search(int[] sorted, int value, int leftIndex, int rightIndex) {

        // 1. index check
        if (leftIndex > rightIndex) {
            return false;
        }

        // 2. middle index
        int middle = (rightIndex + leftIndex) / 2;

        // 3. recursive invoke
        if (sorted[middle] > value) {
            return search(sorted, value, leftIndex, middle - 1);
        } else if (sorted[middle] < value) {
            return search(sorted, value, middle + 1, rightIndex);
        } else {
            return true;
        }
    }

}

If you reached here after writing the program yourself, you deserve a round of applause. Congratulation!!!

Related Articles

• Java Sorting: Comparator vs Comparable Tutorial
• Minimum difference in two sorted arrays

[Part 2] Finding Second Highest Frequent Characters using Java

Initial part of finding second highest frequency characters in a text was discussed previously and covered the scenarios where a single character is the most frequent. Since this is the continuation post of that, I strongly recommend you to read the first part before this. The intention of this second post is to cover the scenarios at which the previous program failed, specifically the scenarios where multiple characters are of the same frequencies.

Previous production codes can return only one character as the second most frequent. If there are multiple characters matching the criteria, the return type of the code most be modified to a character array. Along with that change, all the previous test cases must be modified to support the returned character array. When there are no matches, it is better to expect the code to return an empty array rather than null, so the test cases has to be modified to reflect that.

Similar to the previous post, new test cases are identified prior to writing any implementation code. While supporting these multiple character scenarios, both map based and sorting based programs are modified to satisfy the test cases. You can find those below in this post.

New Test Cases

1. shouldReturnCorrectWhenManyMostFrequentsAndOneSecondMostFrequent()
    > result should be this second most frequent character
2. shouldReturnCorrectWhenManyMostFrequentsAndManySecondMostFrequents()
    > result should be these second most frequent characters

Above two test cases along with the modified previous test cases are written as follows.

package com.digizol.puzzle;

import static java.lang.Character.valueOf;
import static org.junit.Assert.assertEquals;
import java.util.*;
import org.junit.*;

public class SecondFrequentCharactersTest {

    private MapBasedFrequentCharacters sut;

    @Before
    public void setup() {
        sut = new MapBasedFrequentCharacters();
        // sut = new SortBasedFrequentCharacters();
    }

    @Test
    public void shouldReturnCorrectWhenManyMostFrequentsAndOneSecondMostFrequent() {
        // both i & c are the most frequent, y is the second most frequent
        assertEquals(valueOf('y'), 
                                valueOf(sut.getSecondMostFrequent("iiixiiyycbcccc")[0]));
    }
    
    @Test
    public void shouldReturnCorrectWhenManyMostFrequentsAndManySecondMostFrequents() {
        // both i & c are the most frequent, x & y are the second most frequent
        char[] secondMostFrequents = sut.getSecondMostFrequent("iiixxiiyycbcccc");
        assertEquals(2, secondMostFrequents.length);
        
        List<Character> expected = new ArrayList<Character>();
        expected.add('x');
        expected.add('y');
        for (int i = 0; i < secondMostFrequents.length; i++) {
            expected.remove(Character.valueOf(secondMostFrequents[i]));
        }
        assertEquals(0, expected.size());
    }
    
    // previous test cases are modified as below
    
    @Test
    public void shouldReturnNullForEmptyText() {
        assertEquals(0, sut.getSecondMostFrequent("").length);
    }

    @Test
    public void shouldReturnNullForTextOfSameCharacter() {
        assertEquals(0, sut.getSecondMostFrequent("dddddddd").length);
    }

    @Test
    public void shouldReturnCorrectCharWhenTextHasTwoCharsOneBeingMostFrequent() {
        assertEquals(valueOf('y'), valueOf(sut.getSecondMostFrequent("iiiiiyiiiii")[0]));
    }

    @Test
    public void shouldReturnCorrectOneForATextWithOneBeingSecondMostFrequent() {
        // most frequent is 'i', second most is 'd'
        assertEquals(valueOf('d'), 
                                valueOf(sut.getSecondMostFrequent("iaibicidieidif")[0]));
    }
}

Production Code

Similar to previous post, map based solution is modified first followed by the sorting based one to satisfy all above test cases.

Map based solution

Previous map based solution had the character frequencies recorded in the map. At the 'second most frequent character' finding logic, two 'char' variables were used. With the new scenarios there are be multiple characters for both most frequents and second most frequents, so the char variables can be replaced with two lists as shown below.

package com.digizol.puzzle;

import java.util.*;
import java.util.Map.Entry;

public class MapBasedFrequentCharacters {

    public char[] getSecondMostFrequent(String text) {

        char[] charArray = text.toCharArray();

        // calculate char frequencies
        Map<Character, Integer> charFrequenciesMap = new HashMap<Character, Integer>();

        // loop1
        for (char c : charArray) {
            int frequency = 1;
            if (charFrequenciesMap.get(c) != null) {
                frequency = charFrequenciesMap.get(c) + 1;
            }
            charFrequenciesMap.put(c, frequency);
        }

        int currentMostFrequency = 0;
        int currentSecondMostFrequency = 0;
        List<Character> mostFrequentChars = new ArrayList<Character>();
        List<Character> secondMostChars = new ArrayList<Character>();

        // find second most frequent char
        Iterator<Entry<Character, Integer>> charFrequencies = charFrequenciesMap
                .entrySet().iterator();

        // loop2
        while (charFrequencies.hasNext()) {
            Entry<Character, Integer> entry = charFrequencies.next();

            char currentChar = entry.getKey();
            int frequency = entry.getValue();

            if (frequency > currentMostFrequency) {
                secondMostChars.clear();
                secondMostChars.addAll(mostFrequentChars);
                mostFrequentChars.clear();
                mostFrequentChars.add(currentChar);
                currentSecondMostFrequency = currentMostFrequency;
                currentMostFrequency = frequency;
            } else if (frequency == currentMostFrequency) {
                mostFrequentChars.add(currentChar);
            } else if (frequency > currentSecondMostFrequency) {
                secondMostChars.clear();
                secondMostChars.add(currentChar);
                currentSecondMostFrequency = frequency;
            } else if (frequency == currentSecondMostFrequency) {
                secondMostChars.add(currentChar);
            }
        }

        char[] result = new char[secondMostChars.size()];
        for (int i = 0; i < secondMostChars.size(); i++) {
            result[i] = secondMostChars.get(i);
        }

        return result;
    }
}

Sorting based solution

Previous sorting based solution had sorted the list of 'Frequency' in the descending order of frequencies; and returned the second element in the list for the second most frequent. However with the new scenarios, there can be multiple most frequent characters so the second element may contain another most frequent character. Similarly, there can be multiple 'second most frequent' elements in the list. So it is required to traverse through the list from the start and extract an array of elements with the second most frequency. Following is the modified program.

package com.digizol.puzzle;

import java.util.*;

public class SortBasedFrequentCharacters {

    public char[] getSecondMostFrequent(String text) {

        char[] charArray = text.toCharArray();
        Arrays.sort(charArray);

        List<Frequency> list = new ArrayList<Frequency>();
        char previous = '\u0000';
        Frequency f = null;

        for (int i = 0; i < charArray.length; i++) {
            char c = charArray[i];

            if (i == 0 || previous != c) {
                f = new Frequency(1, c);
                list.add(f);
                previous = c;
            } else {
                f.count += 1;
            }
        }

        Collections.sort(
                        list, 
                        new Comparator<Frequency>() {
                            public int compare(Frequency fr0, Frequency fr1) {
                                // sort in descending order
                                return fr1.count - fr0.count;
                            }
                        }
                    );

        // supporting multiple characters being most frequent

        int currentFrequency = 0;
        boolean secondMostFound = false;
        int start = -1;
        int end = -1;
        for (int i = 0; i < list.size(); i++) {
            Frequency frequency = list.get(i);
            if (i == 0) {
                currentFrequency = frequency.count;
            } else if (currentFrequency != frequency.count) {
                if (secondMostFound) {
                    end = i;
                    break;
                } else {
                    secondMostFound = true;
                    start = i;
                    end = i;
                }
            }
        }

        char values[] = null;
        if (secondMostFound) {
            List<Frequency> secondMostFrequencies = list
                    .subList(start, end + 1);
            values = new char[secondMostFrequencies.size()];
            for (int i = 0; i < secondMostFrequencies.size(); i++) {
                values[i] = secondMostFrequencies.get(i).character;
            }
        } else {
            values = new char[0];
        }
        return values;
    }

    private class Frequency {
        int count;
        char character;

        public Frequency(int count, char character) {
            this.count = count;
            this.character = character;
        }
    }
}

In summary, the puzzle can be solved in many ways as shown in this series of articles. One important point to note is the use of test cases to identify the scenarios before any production code. As you may have already noticed, the same set of test cases were used to test two different implementations. If you have noticed any scenarios that are not covered here, please let me know via comments section so that I can address those.

Finding Second Highest Frequent Character in a text with Java

Finding second highest frequency character in a text is last weeks Thursdays Dzone Code Puzzle. Intention of this post is to try and work out this interview oriented problem in Java with you by explaining the way I am approaching this task. In return, both you and myself can learn through a discussion. 

The requirement is to return the character that is the second most frequent. Before writing any code, it is important to break down this requirement into a simple set of test cases. As a TDD (Test Driven Development) oriented developer, I always try to approach a problem in that manner. The most correct way would be to start with one test case and proceed with the implementation code, but for the easiness in explanation, I will show four initial test cases and production code to support that requirement. Part 2 will cover additional scenarios which this program fails to support.

Initial Test Cases

1. shouldReturnNullForEmptyText()
    > result should be null character for empty string
2. shouldReturnNullForTextOfSameCharacter()
    > result should be null character for a text with the same character since there has to be at least two different characters to have a second most frequency
3. shouldReturnCorrectCharWhenTextHasTwoCharsOneBeingMostFrequent()
    > result should be the character that is not the most frequent character
4. shouldReturnCorrectOneForATextWithOneBeingSecondMostFrequent()
    > result should be the second most frequent character

Above test cases are implemented in the following manner with JUnit4. All test cases check the equality of expected values against actual.

package com.digizol.puzzle;

import static java.lang.Character.valueOf;
import static org.junit.Assert.assertEquals;
import org.junit.Before;
import org.junit.Test;

public class SecondFrequentCharacterInitialTest {

    private MapBasedFrequentCharacter sut;

    @Before
    public void setup() {
        sut = new MapBasedFrequentCharacter();
        // sut = new SortBasedFrequentCharacter();
    }

    @Test
    public void shouldReturnNullForEmptyText() {
        assertEquals(valueOf('\u0000'), valueOf(sut.getSecondMostFrequent("")));
    }

    @Test
    public void shouldReturnNullForTextOfSameCharacter() {
        assertEquals(valueOf('\u0000'), valueOf(sut.getSecondMostFrequent("dddddddd")));
    }

    @Test
    public void shouldReturnCorrectCharWhenTextHasTwoCharsOneBeingMostFrequent() {
        assertEquals(valueOf('y'), valueOf(sut.getSecondMostFrequent("iiiiiyiiiii")));
    }

    @Test
    public void shouldReturnCorrectOneForATextWithOneBeingSecondMostFrequent() {
        // most frequent is 'i', second most is 'd'
        assertEquals(valueOf('d'), valueOf(sut.getSecondMostFrequent("iaibicidieidif")));
    }
}

Production Code

As per the test cases, production code should have a method to take the text as an input and to return a character. One approach to solve this is to use a map to record the frequencies of each character and extract the second most frequent. Another approach is to record the character frequencies in a list, then sort by the frequency and extract the second most frequent. There is another approach to record the frequencies in an array where the array index is the integer representation of the character, but size of the character set must be known prior. In this post, both map based and sort based approaches are implemented, not the array based one.

Map based solution

In the following implementation, the characters in the text is added to a map and the frequency of those are incremented. Completed production code looks as follows.

package com.digizol.puzzle;

import java.util.*;
import java.util.Map.Entry;

public class MapBasedFrequentCharacter {

    public char getSecondMostFrequent(String text) {

        char[] charArray = text.toCharArray();
        
        // calculate char frequencies
        Map<Character, Integer> charFrequenciesMap = new HashMap<Character, Integer>();
        
        // loop1
        for (char c : charArray) {
            int frequency = 1;
            if (charFrequenciesMap.get(c) != null) {
                frequency = charFrequenciesMap.get(c) + 1;
            }
            charFrequenciesMap.put(c, frequency);
        }

        int currentMostFrequency = 0;
        int currentSecondMostFrequency = 0;
        char mostFrequentChar = '\u0000';
        char secondMostChar = '\u0000';

        // find second most frequent char
        Iterator<Entry<Character, Integer>> charFrequencies
                                                = charFrequenciesMap.entrySet().iterator();
        // loop2
        while (charFrequencies.hasNext()) {
            Entry<Character, Integer> entry = charFrequencies.next();

            char currentChar = entry.getKey();
            int frequency = entry.getValue();

            if (frequency > currentMostFrequency) {
                secondMostChar = mostFrequentChar;
                currentSecondMostFrequency = currentMostFrequency;
                currentMostFrequency = frequency;
                mostFrequentChar = currentChar;
            } else if (frequency > currentSecondMostFrequency) {
                currentSecondMostFrequency = frequency;
                secondMostChar = currentChar;
            }
        }
        return secondMostChar;
    }
}

Above solution uses two loops to complete the operation, and number of iterations in second loop is related to size of character set. This might be considerable if the character set is larger in size e.g: UNICODE. If that is a concern, number of iterations can be reduced by combining two loops; using the frequency recording loop itself to record the second most frequent character as well.

To check this, I did a simple test only to compare the time taken by each variation of the above program against a text with over 5 million ASCII characters (5,264,279 to be exact). As per the results, two loops based program performs slightly quicker than the other. This may be due to the small size of ASCII character set. Anyway one loop variation is not shown here since it is simple enough for you to work out.

Sorting based solution

In this approach, characters in text are chronologically ordered before starting to record the frequencies. A new type is introduced named 'Frequency' to hold the character and it's frequency. After recording the frequencies, a Comparator is used to sort the list of 'Frequency' in descending order of character frequencies to facilitate the extraction of second most frequent.

package com.digizol.puzzle;

import java.util.*;

public class SortBasedFrequentCharacter {

    public char getSecondMostFrequent(String text) {

        char[] charArray = text.toCharArray();
        Arrays.sort(charArray);

        List<Frequency> list = new ArrayList<Frequency>();
        char previous = '\u0000';
        Frequency f = null;

        for (int i = 0; i < charArray.length; i++) {
            char c = charArray[i];

            if (i == 0 || previous != c) {
                f = new Frequency(1, c);
                list.add(f);
                previous = c;
            } else {
                f.count = f.count + 1;
            }
        }

        Collections.sort(
                        list, 
                        new Comparator<Frequency>() {
                            public int compare(Frequency fr0, Frequency fr1) {
                                // sort in descending order
                                return fr1.count - fr0.count;
                            }
                        }
                    );

        char value = '\u0000';
        if (list.size() > 1) {
            value = list.get(1).character;
        }
        return value;
    }

    private class Frequency {
        int count;
        char character;

        public Frequency(int count, char character) {
            this.count = count;
            this.character = character;
        }
    }
}

Both of the above production codes are not re-factored and purposefully kept it as an exercise for you.

More Test Cases

After this initial implementation, we need to consider other test cases that belongs to the requirement. As you may have already noted, above implementation does not support the scenarios where multiple characters being of the same frequencies. As an example, following test case fails against both of the above programs.

@Test
public void shouldReturnCorrectWhenManyMostFrequentsAndOneSecondMostFrequent() {
    // both i & c are the most frequent, y is the second most frequent
    assertEquals(valueOf('y'), valueOf(sut.getSecondMostFrequent("iiiiiyccccc")));
}

As this article is growing in length, Part 2 covers these extra scenarios.

Encrypted vs Hashed Passwords - Which is better?

Topics like password strength, protection, encryption are almost everywhere these days. Password maintenance related subjects like password managers, recovery tools and crackers are also gaining attention. A user name and a password is a must in almost all software applications like email applications, web sites, mobile and desktop applications; mainly to provide user specific information or functionality.
Security of the password is so much important not because the data stored behind a user account is so much valuable to the owner, but might be to some other bad guy who is looking for personal information. To overcome the pain of memorizing multiple passwords, users might use one single much stronger password across multiple applications which is a bad practice considering the security aspect.

As anyone would guess, most of the application specific databases are having a table named user or users including two columns named user name and password; and interestingly the password in plain text! If your application database is storing passwords in plain text format, there is no hope for security in your application. People would argue that the application is well protected, HTTPS or TLS is in action; so the users are safe. What if someone get access to your database? That is the end of the security of all your users; and if those users were reusing their most secret and strongest password across multiple web sites, can you imagine what will be the situation? If your application stores password in plain text, it must be time to think at least about encrypted passwords.

Is encryption good?

However the intention of this article is not to discuss about plain text passwords, but about encrypted passwords stored in databases. Plain text passwords can be encrypted using symmetric encryption algorithms like DES, AES or with any other algorithms and be stored inside the database. At the authentication (confirming the identity with user name and password), application will decrypt the encrypted password stored in database and compare with user provided password for equality. In this type of an password handling approach, even if someone get access to database tables the passwords will not be simply reusable. However there is a bad news in this approach as well. If somehow someone obtain the cryptographic algorithm along with the key used by your application, he/she will be able to view all the user passwords stored in your database by decryption. "This is the best option I got", a software developer may scream, but is there a better way?

Yes there is, may be you have missed the point here. Did you notice that there is no requirement to decrypt and compare? If there is one-way-only conversion approach where the password can be converted into some converted-word, but the reverse operation (generation of password from converted-word) is impossible. Now even if someone gets access to the database, there is no way that the passwords be reproduced or extracted using the converted-words. In this approach, there will be hardly anyway that some could know your users' top secret passwords; and this will protect the users using the same password across multiple applications. What algorithms can be used for this approach?

Cryptographic hash function

Cryptographic hash functions can be used to achieve one-way-only conversion requirement. As there is no support to go back from converted text to original text, there is no risk involved in the safety of the valuable and secret password. There are many well known and publicly available algorithms for this task, and most popular ones are MD5 and SHA-1. There are freely available tools implementing these algorithms; so incorporating hashed approach into applications is not a pain. Even though these algorithms provide a far better security, both MD5 and SHA-1 are proven to be weak and vulnerable. It is recommended to go with SHA-2 considering the preciousness of the password. However at the moment, there is an open competition to created a replacement algorithm for SHA-2 which is called SHA-3 and this will be available in 2012.

In summary; when an application level security is discussed/designed make sure that passwords are never kept in plain text, but at least in encrypted form; but try to reach the hash function based password handling as much as possible.

Related: Data Encryption Decryption using AES Algorithm, Key and Salt with Java Cryptography Extension

[Tomcat] How to change default JSESSIONID cookie/parameter identifier

Changing default JSESSIONID name of cookie and/or parameter is the objective. Deployed J2EE web applications use browser cookie or parameter based session management technique. By default session cookie name is defined as “JSESSIONID” and session id parameter as “jsessionid” in Apache Tomcat servers. These names can be renamed by specifying required values for correct system properties.

Requirements

This system properties based feature is only available in releases newer than Tomcat 5.5.28 and Tomcat 6.0.20.

System properties

Related system properties are;

• org.apache.catalina.SESSION_COOKIE_NAME (for cookie name)
• org.apache.catalina.SESSION_PARAMETER_NAME (for parameter name)

Passing system properties

System property can be passed using standard methodology; use “-D” parameter of Java command similar to following.

java -D<key>=<value>

Modify catalina.sh to pass system properties

Following extract is from a modified bin/catalina.sh to pass these system properties; similarly bin/catalina.bat can be modified for Windows based installations.

#!/bin/sh
# // .....
# ------------------------------------------
# Start/Stop Script for the CATALINA Server
# // .....
# ------------------------------------------
JAVA_OPTS="$JAVA_OPTS 
             -Dorg.apache.catalina.SESSION_COOKIE_NAME=MYJSESSIONID 
             -Dorg.apache.catalina.SESSION_PARAMETER_NAME=myjsessionid"
# // .....

Note:
The decision on whether cookie based or parameter based session management is used depend on client browser settings.

Related Articles

• [Tomcat] validateJarFile(servlet-api.jar) - jar not loaded. Offending class: javax/servlet/Servlet.class
• Tomcat + Apache HTTP: Servers Connecting Guide