Creating a separate (guest) network using different SSIDs and VLANs

This post is not about Java, but about networks. The hardware I'm using is from Cisco, but this should work on other brands just as well.

I had installed a WAP371 access point and an SG-200 managed switch for a customer. The router I used is a simple home gateway provided by the ISP and is not VLAN-aware. The customer asked me to put a guest network in place. The Cisco WAP access points can broadcast multiple SSIDs to create multiple networks. They are differentiated using VLAN IDs: when you connect to a network, all the frames are tagged with a VLAN id. The default VLAN id is 1, but you can assign different VLAN id's to other SSIDs. I created a second SSID (named guest) and gave it the VLAN ID 2. Now when someone connects to the guest network, all their frames are tagged with VLAN id 2. When they connect to the main network, their frames are tagged with id 1.

Setting up the access points is the easy part. I then needed to configure the switches. The router is not VLAN aware, so I needed to add a second router for the guest network. The second router is connected to the main router and to the switch, like this:

So I connected the access point to port 1 of the managed switch, the ISP router/gateway to port 2 and the guest router to port 3. The WAN side of the guest router is connected to the LAN side of the ISP router. This will cause double NAT, but that's not a problem for a guest network.

Now I still had to configure the ports of the switch correctly to separate the guest network from the main network. I configured the ports as follows:

• All ports except port 1 and 3: set to ACCESS mode and VLAN 1
• Port 1: set to TRUNK mode, untagged VLAN 1 and tagged VLAN 2
• Port 3: set to ACCESS mode and VLAN 2

Now when a user connects to the guest network, his frames are tagged with VLAN ID 2. These frames will only be able to access port 3, to which the guest router is connected. The user will get an IP address from the guest router and will not be able to access devices connected to the other ports of the switch.

When a user connects to the main network, the frames will be tagged with VLAN ID 1. The user will be able to access all ports, except for the port to which the guest router is connected.

Starting a Web Start application locally

Java Web Start is a technology that allows you to load your application from a http server. This has some advantages: the user doesn't have to install your application, you're sure that your users are always using the latest version of your application etc.. You can find more information on this technology at http://www.oracle.com/technetwork/java/javase/tech/index-jsp-136112.html

To create a webstart application you need to create a JNLP descriptor. See http://docs.oracle.com/javase/1.5.0/docs/guide/javaws/developersguide/syntax.html for the syntax.
The codebase specifies the location where java can find your files (all resources in the jnlp file are relative to this path). When you deploy your Web Start application, this will be the location on the http server where you upload your application (eg: http://www.mysite.com/myapp).

When you're developing or testing your application, you don't want to upload it every time you make a change. Unfortunately, you can't leave the codebase blank. Leaving it blank will give you the following exception:
The field <jnlp>href has and invalid value:

So what should you put there? You can't put the path to your application (eg: c:/myapp), since that will leave you with the same error message.
Your next try might be file:///c:/myapp or file:/c:/myapp, which won't work either. The correct syntax is:
file://localhost/C:/myapp

Now you can test your Web Start application locally!

Tomcat shutdown fails after installing Solr

In this post I described how to add Solr to an existing web application and how to query the index using Sorlj. Everything seemed to work well, but after a while I started noticing that tomcat didn't shut down successfully anymore: it seemed to hang on shutdown.
The culprit seems to be the EmbeddedSolrServer I was using. Apparently you need to shutdown the CoreContainer that it's using for the server to successfully shutdown. What this means is we'll have to modify our search bean a bit. Add the following fields to the class:

private static final SolrServer solrServer = initSolrServer();
private static CoreContainer coreContainer;

The initSolrServer() method looks like this:

private static SolrServer initSolrServer() {
  try {
   CoreContainer.Initializer initializer = new CoreContainer.Initializer();
   coreContainer = initializer.initialize();
   EmbeddedSolrServer server = new EmbeddedSolrServer(coreContainer, "");
   return server;
  } catch (Exception ex) {
   logger.log(Level.SEVERE, "Error initializing SOLR server", ex);
   return null;
  }
 }

Finally we'll also add a shutdown method to the search bean:

 public static void shutdownSolr() {
  if (coreContainer != null) {
   coreContainer.shutdown();
  }
 }

Now all we need to do is call the shutdownSolr() method on our search bean when the servlet container is shut down. For this we'll need to add a ServletContextListener to our web application. Open your web.xml and add the following lines:

 <listener>
  <listener-class>mypackage.SolrListener</listener-class>
 </listener>

And this is how the SolrListener should look like:

import javax.servlet.ServletContextEvent;
import javax.servlet.ServletContextListener;

public class SolrListener implements ServletContextListener {

 @Override
 public void contextDestroyed(ServletContextEvent arg0) {
  SearchBean.shutdownSolr();
 }

 @Override
 public void contextInitialized(ServletContextEvent arg0) {

 }

}

And now Tomcat should shutdown without any problems!

Adding Solr to an existing web application

In this article I'm going to discuss how to add a search function to a Java Web Application. These are the requirements:
- You have an existing web application, so you're not starting from scratch
- The contents you want to search is stored in a database you can access
- The contents should be automatically indexed and updated when changes occur to the database

1. Why Solr?

Apache's Lucene is the de facto indexing standard for java. It's fast and has a lot of features (see http://lucene.apache.org/ for more information). Apache Solr (http://lucene.apache.org/solr/) can be seen as an extension to Lucene, made for web applications. It's actually a web application in it's own right and if you start it you get a fully working search application and even an administration application. It also allows you to easily query and update the index. It's a very impressive, but also very complex project. Fortunately we'll only need a few parts from this project. I'll tell you which parts in the next chapter.

2. Downloading and installing Solr

The first thing you'll need to do is download Solr. Go to http://www.apache.org/dyn/closer.cgi/lucene/solr/ and grab the latest release (this article is based on release 3.3.0). Download the correct file for your platform and extract the archive somewhere on your system.
Solrs distribution format is a bit unusual: it's a war (Web ARchive) file and the people at Apache seem to expect that you will just drop this war file into your servlet container (eg Tomcat) and be ready to go. Unfortunately this is not what we want: we want to add Solr to an existing web application. So the first thing you'll need to do is extract this war file somewhere on your system. I'll call this location SOLR_WAR. A war file is basically just a zip file, so you can use winrar or similar if you're on a Windows system. So go ahead and extract the apache-solr-3.3.0.war file (it's in apache-solr-3.3.0\dist).
Now you'll need to add the Solr jar files to your existing web application (like all jar files, they go in WEB-INF\lib). You'll need at lease the following files for Solr to start correctly:

• SOLR_WAR\WEB-INF\lib\apache-solr-core-3.3.0.jar
• SOLR_WAR\WEB-INF\lib\apache-solr-solrj-3.3.0.jar
• SOLR_WAR\WEB-INF\lib\commons-codec-1.4.jar
• SOLR_WAR\WEB-INF\lib\commons-fileupload-1.2.1.jar
• SOLR_WAR\WEB-INF\lib\commons-httpclient-3.1
• SOLR_WAR\WEB-INF\lib\commons-io-1.4.jar
• SOLR_WAR\WEB-INF\lib\lucene-analyzers-3.3.0.jar
• SOLR_WAR\WEB-INF\lib\lucene-core-3.3.0.jar
• SOLR_WAR\WEB-INF\lib\lucene-highlighter-3.3.0.jar
• SOLR_WAR\WEB-INF\lib\lucene-spatial-3.3.0.jar
• SOLR_WAR\WEB-INF\lib\lucene-spellchecker-3.3.0.jar
• SOLR_WAR\WEB-INF\lib\slf4j-api-1.6.1.jar
• SOLR_WAR\WEB-INF\lib\slf4j-jdk14-1.6.1.jar
• SOLR_WAR\WEB-INF\lib\velocity-1.6.1.jar

To use the DataImportHandler feature (which will feed the data in the database to the Lucene index), you'll also need the following jar file:

• apache-solr-3.3.0\dist\apache-solr-dataimporthandler-3.3.0.jar

The next step is to edit your web.xml file to add the Solr servlets and filter. Here are the sections you should add:

<filter>
  <filter-name>SolrRequestFilter</filter-name>
  <filter-class>org.apache.solr.servlet.SolrDispatchFilter</filter-class>  
 </filter>

 <filter-mapping>
  
  <filter-name>SolrRequestFilter</filter-name>
  <url-pattern>/dataimport</url-pattern>
 </filter-mapping>
 <servlet>
  <servlet-name>SolrServer</servlet-name>
  <servlet-class>org.apache.solr.servlet.SolrServlet</servlet-class>
  <load-on-startup>1</load-on-startup>
 </servlet>
 <servlet>
  <servlet-name>SolrUpdate</servlet-name>
  <servlet-class>org.apache.solr.servlet.SolrUpdateServlet</servlet-class>
  <load-on-startup>2</load-on-startup>
 </servlet>
 <servlet>
  <servlet-name>Logging</servlet-name>
  <servlet-class>org.apache.solr.servlet.LogLevelSelection</servlet-class>
 </servlet>
<servlet-mapping>
  <servlet-name>SolrUpdate</servlet-name>
  <url-pattern>/update/*</url-pattern>
 </servlet-mapping>
 <servlet-mapping>
  <servlet-name>Logging</servlet-name>
  <url-pattern>/admin/logging</url-pattern>
 </servlet-mapping>

3. Configuring Solr

Solr has it's own configuration mechanism. It's not just a file, but an entire folder. The easiest way to set it up is to copy the solr folder from the distribution (it's in apache-solr-3.3.0\example) to a new location on your file system. I'll call this location SOLR_PATH.
The first thing you'll need to do is point your servlet container (eg Tomcat) to the location of the Solr configuration folder. This is done by adding the following VM argument:

-Dsolr.solr.home=SOLR_PATH

(where you replace SOLR_PATH by the location on your file system that you copied the solr configuration folder to). Where and how to add this argument depends on the servlet container and/or IDE you're using. For tomcat you could modify the catalina.bat file and add the following line at the top:

set %JAVA_OPTS%=%JAVA_OPTS% -Dsolr.solr.home=SOLR_PATH

(where you replace SOLR_PATH by the location on your file system that you copied the solr configuration folder to)
If you want to you can try starting your web server now. If you get any errors, please make sure you copied all the jar files and that the VM argument is configured correctly.

We're not done yet though. First we'll need to edit our schema, so Solr knows which fields you want to index. This is done in the SOLR_PATH\conf\schema.xml file. Open this in your favorite editor. The first element is the schema element. You can change the name to anything you want, but this doesn't matter (the name is only for display purposes). The schema element is followed by a types element. This is similar to a database: there are different data types depending on the type of data you want to store. I highly recommend reading http://wiki.apache.org/solr/SchemaXml to understand how the data types work. You don't need to modify anything to the types though.
The next section in the schema.xml file is the fields element. This is where you declare the fields that will be indexed (ie made searchable). To keep the example simple, I'll use products. Products have a unique id, a name and a description. You could define the fields as follows:

<field name="id" type="int" indexed="true" stored="true" required="true" />    
<field name="name" type="text_general" indexed="true" stored="false"/>
<field name="description" type="text_en" indexed="true" stored="false"/>

We'll also define an additional "text" field, which contains all of the product information. This field will allow us to search all the fields at once. Define it as follows:

<field name="text" type="text_general" indexed="true" stored="false" multiValued="true"/>

After the closing fields element, define the following elements:

<uniqueKey>id</uniqueKey>

and

<defaultSearchField>text</defaultSearchField>

We'll also define the copyfields, which will copy all the data to the "text" field we defined above:

<copyField source="name" dest="text"/>
<copyField source="description" dest="text"/>

If you don't understand this and would like to know what's going on, please read http://wiki.apache.org/solr/SchemaXml.

4. Configuring the DataImportHandler

Now we need a way to index the contents of the database in the Solr index. Fortunately, Solr has a mechanism for this, called the DataImportHandler (see http://wiki.apache.org/solr/DataImportHandler). First we need to tell Solr we want to use the DataImportHandler. Open the SOLR_PATH\conf\solrconfig.xml file and add the following code:

<requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler">
  <lst name="defaults">
   <str name="config">data-config.xml</str>
  </lst>
 </requestHandler> 

Now create the file SOLR_PATH\conf\data-config.xml. The contents should look like this:

<dataConfig>
<dataSource
      jndiName="java:comp/env/jdbc/myDB"
      type="JdbcDataSource"/>      
 <document>
  <entity name="product" pk="id" 
   query="select id, name, description FROM product WHERE '${dataimporter.request.clean}' != 'false' OR last_modified > '${dataimporter.last_index_time}'">
  </entity>
    </document>
</dataConfig>

You should modify the jndiName to the jndi name of the database you are using. You can also provide a connection URL and username/password if you're not using jndi, as follows:

<dataSource type="JdbcDataSource" name="ds-1" driver="com.mysql.jdbc.Driver" url="jdbc:mysql://db1-host/dbname" user="db_username" password="db_password"/>

As you can see in the query, your product table will need a last_modified field, which is a timestamp that contains the last data at which this record was updated (or created). If you don't have this field, Solr can't know which records have been updated since the last import and you will be forced to perform a full import each time (which is a big performance hit on large tables).
Now when you go to the following url: http://localhost:8080/MY_APP/dataimport?command=full-import&clean=false the data will be imported from your database in the lucene index. Set the clean parameter to "true" to do a full import.
You can schedule a request to http://localhost:8080/MY_APP/dataimport?command=full-import&clean=false to automatically update the index each hour, day, week... On linux you could add a cron job that does a

wget http://localhost:8080/MY_APP/dataimport?command=full-import&clean=false

5. Querying the index

We're almost there. We have a working index, which gets updated with data from our database. Now all we need to do is query the index. In our backend bean we define a reference to the Solr server:

private static final SolrServer solrServer = initSolrServer();
private static SolrServer initSolrServer() {
  try {
    CoreContainer.Initializer initializer = new CoreContainer.Initializer();
    CoreContainer coreContainer = initializer.initialize();
    EmbeddedSolrServer server = new EmbeddedSolrServer(coreContainer, "");
    return server;
  } catch (Exception ex) {
   logger.log(Level.SEVERE, "Error initializing SOLR server", ex);
   return null;
  }
}

And when we want to query Solr, we do the following:

SolrQuery query = new SolrQuery(keyword);
QueryResponse response = solrServer.query(query);
SolrDocumentList documents = response.getResults();
for (SolrDocument document : documents) {
  Integer id = (Integer) document.get("id");
  //load this product from the database using its id
}

We're using Solrj to query Solr. For more info on how to construct queries using Solrj, please see http://wiki.apache.org/solr/Solrj

6. Conclusion

You should now have a working index, that gets updated with data from the database and that you can query directly from Java, all nicely integrated in our existing web application!

Generate a table in JSF by iterating over a List

The problem

If you want to create a table in JSF, there are two tags you can use: the <h:panelGrid> and the <h:dataTable> tags. In the first tag, you specify the number of columns (using the colums attribute), and JSF will automatically put each child component of the panelGrid tag in a new column. When the number of columns you specified is reached, a new row will start.
The dataTable component works differently: this component needs <h:column> child components for each column in your table.
Another major difference between the two components is that the dataTable component can iterate over a collection (using the value and var attributes), while the panelGrid cannot.
In my case, I had a List of products in my backing bean. I wanted to display them in a table, 3 products per row. The panelGrid component would have been perfect for this, except that it cannot iterate over a List. I couldn't use the dataTable component either, since this component expects column child elements.

The solution

Since I couldn't use the dataTable or the panelGrid components, I decided to create my own table using the <ui:repeat> tag. This tag allows you to iterate over a collection, specifying whatever code you want in the tag body.
The only thing that remained was to show 3 products per row. Fortunately the repeat tag has a varStatus attribute, which we can use to query the index we're at.
This is the final code:

<table>
  <tr>
    <ui:repeat var="product" value="#{productBean.products}" varStatus="status">
      <f:verbatim rendered="#{status.index mod 3 == 0}">
        &lt;/tr&gt;&lt;tr&gt;
      </f:verbatim>
      <td>#{product.name}</td>
    </ui:repeat>
  </tr>
</table>

Ignore missing properties with Json-Lib

Introduction

When you're writing your own client and server protocol, you have to exchange data between them. There are many formats available (csv, xml, serialized objects...) and you can always define your own protocol.
One format that has been getting more popular recently is JSON (JavaScript Object Notation, see http://en.wikipedia.org/wiki/JSON). It's a text format that allows objects to be serialized in a human-readable form. There's a library for Java called Json-lib (http://json-lib.sourceforge.net/) that performs this serialization for you. This allows you to exchange JSON between your server and client without having to write any JSON code.

The Json-lib library

You can find some examples on how to use Json-lib here: http://json-lib.sourceforge.net/snippets.html. I use it like this:
On the server (serialize to JSON):

  User user = ...; //get the user from database
  String json = JSONObject.fromObject(user).toString();
  //Send the json to the client

Then on the client:

   JSONObject jo = JSONObject.fromObject(response);
   User user = (User) JSONObject.toBean(jo, User.class);

As you can see it's pretty straight-forward and it works well, as long as the class definitions (in this case: the User class) are the same on the server and the client. As you can imagine this is not always the case: you will sometimes add new properties to the class while users are still using the old definition of your class on their client. If this is the case (you are sending properties from the server that don't exist on the client class), you get slapped with this exception:

net.sf.json.JSONException: java.lang.NoSuchMethodException: Unknown property on class

You can fix this exception by adding an exclude filter on the server that doesn't send certain properties (see http://json-lib.sourceforge.net/snippets.html for how to do this), but this isn't a solution in our case: we want to send the properties so people who have the newer version of the client can use them.

The solution

After searching for a way of telling Json-lib to ignore missing properties and not finding it, I decided the best approach would be to create my own PropertySetStrategy. A PropertySetStrategy defines how Json-lib sets the properties on your objects. The default strategy works pretty well, except that it throws an exception when a property is not found. So I decided to create a wrapper around this default PropertySetStrategy that ignores any exceptions happening when setting a property.
This is my PropertySetStrategyWrapper:

import net.sf.json.JSONException;
import net.sf.json.util.PropertySetStrategy;

public class PropertyStrategyWrapper extends PropertySetStrategy {

    private PropertySetStrategy original;

    public PropertyStrategyWrapper(PropertySetStrategy original) {
        this.original = original;
    }

    @Override
    public void setProperty(Object o, String string, Object o1) throws JSONException {
        try {
            original.setProperty(o, string, o1);
        } catch (Exception ex) {
            //ignore
        }
    }
}

As you can see I ignore any exceptions that happen on a set. You can always log the exceptions if you prefer.
Now all we need to do is tell Json-lib to use our wrapper. Our client code now looks like this:

JSONObject jo = JSONObject.fromObject(response);
JsonConfig cfg = new JsonConfig();
cfg.setPropertySetStrategy(new PropertyStrategyWrapper(PropertySetStrategy.DEFAULT));
cfg.setRootClass(User.class);
User user = (User) JSONObject.toBean(jo, cfg);

We don't need to modify any of the server code and our client will now happily ignore missing properties!

Switching between PayPal live and test environments

When you want to go live with your PayPal web application, you will have to modify your PayPal code to use the live PayPal sever (paypal.com) instead of the test server you've probably been using (sandbox.paypal.com). This is an easy task and doesn't take much time, but what if you want to still continue to use the PayPal test environment on your test server, while using the live PayPal environment on your production server?

I'm going to explain one approach to solving this problem here. The example I'm giving uses JSF and Facelets, but it can easily be adapted to any framework.

The first thing you'll need to do is add an entry to your web.xml and add the following entry:

<context-param>
<param-name>APPLICATION.MODE</param-name>
<param-value>$APPLICATION.MODE.VALUE$</param-value>
</context-param>

What this does is define an initialization parameter on the servlet context, which determines the environment we're in. The "param-value" will be "production" on your production machine, or anything else on your test machine.

The next step is to define a listener on the web application. This listener will take the initialization parameter we just defined and bind it in the application (=servlet) context, so it can be accessed in the entire web application. You'll need to edit your web.xml file again and add the following entry:

<listener>
<listener-class>com.myorg.MyListener</listener-class>
</listener>

The code for the listener is as follows:

public class MyListener implements ServletContextListener {

@Override
public void contextDestroyed(ServletContextEvent arg0) {
}

@Override
public void contextInitialized(ServletContextEvent event) {
String mode = event.getServletContext().getInitParameter("APPLICATION.MODE");
event.getServletContext().setAttribute("APPLICATION.MODE", mode);
}

}

As you can see, this class just takes the initialization parameter you defined and adds it as an attribute to the servlet context.

The next step is framework-specific. I'm using JSF with facelets, so I defined a facelet for a "buy now" PayPal button. The facelet is called "paypalButton.xhtml", and this is how it looks:

<html xmlns="http://www.w3.org/1999/xhtml"
xmlns:h="http://java.sun.com/jsf/html"
xmlns:c="http://java.sun.com/jsf/core"
xmlns:ui="http://java.sun.com/jsf/facelets">
<ui:composition>
<h:panelGroup
rendered="#{applicationScope['APPLICATION.MODE'] == 'production'}">
<form action="https://www.paypal.com/cgi-bin/webscr" method="post"><input
type="hidden" name="cmd" value="_s-xclick" /> <input type="hidden"
name="hosted_button_id" value="#{prodButtonId}" /> <input type="image"
src="https://www.paypalobjects.com/WEBSCR-640-20110429-1/en_US/i/btn/btn_buynowCC_LG.gif"
border="0" name="submit"
alt="PayPal - The safer, easier way to pay online!" /> <img alt=""
border="0"
src="https://www.paypalobjects.com/WEBSCR-640-20110429-1/en_US/i/scr/pixel.gif"
width="1" height="1" /></form>
</h:panelGroup>
<h:panelGroup
rendered="#{applicationScope['APPLICATION.MODE'] != 'production'}">
<form action="https://www.sandbox.paypal.com/cgi-bin/webscr"
method="post"><input type="hidden" name="cmd"
value="_s-xclick" /> <input type="hidden" name="hosted_button_id"
value="#{testButtonId}" /><input type="image"
src="https://www.sandbox.paypal.com/WEBSCR-640-20110401-1/en_US/i/btn/btn_buynowCC_LG.gif"
border="0" name="submit"
alt="PayPal - The safer, easier way to pay online!" /> <img alt=""
border="0"
src="https://www.sandbox.paypal.com/WEBSCR-640-20110401-1/en_US/i/scr/pixel.gif"
width="1" height="1" /></form>
</h:panelGroup>
</ui:composition>
</html>

As you can see, this facelet renders a live "buy now" button if the APPLICATION.MODE equals "production" or a sandbox "buy now" button otherwise. You can then call this facelet as follows (from another facelet):

<ui:include src="/paypalButton.xhtml">
<ui:param name="prodButtonId" value="XXX" />
<ui:param name="testButtonId" value="YYY" />
</ui:include>

In my case I'm using buttons that are stored with PayPal, so the only parameter I need is the button id. If you're creating your own buttons, you can easily define different parameters for the facelet.

After performing these steps, you have a system renders a live or sandbox button based on the value of the APPLICATION.MODE parameter that you defined in your web.xml. To prevent having to modify this parameter by hand, you can have your build system fill it out. What I did was create 2 ant files: buildTest.xml and buildProd.xml. buildTest sets this parameter to "test", while buildProd sets it to "production". This way all I have to do is execute the correct ant file and I have the correct PayPal buttons for my environment!