WebSphere Portal and Web Analytics?

Customers who want to perform web analytics on an on premise portal environment usually want the following types of information

• How effective are the new campaign initiatives?
• Which items have the highest clickthrough ratio?
• What is the clickthrough ratio?
• Which items have the highest view to purchase ratio?
• What is the ratio for one product over another?
• Which rules are resulting in the most number of downloads?
• During the month of December, which categories of pages were browsed most often e.g. my special Christmas pages, or the regular part of my site?

What is provided out of the box?

Out of the box WebSphere Portal records the following information;

• Page management (creating, reading, updating, deleting a page)
• Requests of a certain page by users (including contained portlets)
• Session activities (login, logout, timed out, login failed)
• User management actions (creating, reading, updating, deleting users and groups).

WebSphere Portal supports NCSA Combined standard which means that if you switch on this logging this information can be read by most reporting tools.

What are the options for Web Analytics tools?

There are two broad areas:

1. On-premise
2. Hosted

This blog post is considering on-premise options.

On-premise

There are two types of on-premise options

(i) Log Trawler

(ii) On-premise hosted

The log trawler will look at the logs that Portal produces automatically. The on-premise hosted solutions will require some javascript to be added to the theme and potentially some Portal API work depending upon the data required.

I have provided a short list of possible on-premise options. Customers would need to validate the best product to meet their web analytic needs. AWStat is probably a good place to start for those customers who are unsure exactly what they need. Once a customer has a better idea on the types of data they require then it may be time to look at one of the commercial products.

(i) Log trawler examples

AWStat

(ii) On-premise hosted

Urchin

WebTrends

Unica

SCL

netmagic

piwik

There will be many others but these are the ones that I come across the most.

Using Jython with WebSphere Portal

Most developers and administrators working with WebSphere Application Server (WAS) know that both JACL and Jython languages can be used for various WAS administration and configuration tasks. However, JACL has always been a preferred choice, simply because this is the default language used by the product's admin tool (wsadmin) and also because JACL examples and documentation are more complete.

Using JACL might have been a valid option just a few years back (when WAS just came out) given the uncertainty surrounding the Jython project. Today, however, jython is clearly alive and well; an alpha version supporting Python 2.5 was announced recently. Therefore there is really no point in using JACL any longer, except may be for shops with a large collection of existing JACL scripts. JACL syntax is quite arcane compared with Python and the language is clearly not as widely used.

IBM confirmed this view by releasing JACL to Jython converter a couple years back.

Unfortunately, up until recently, jython was not officially supported in another IBM product, WebSphere Portal, which comes with wpsript tool for managing pages, deployable modules and other portal artifacts.

But since portal scripting relies on wsadmin's shell, jython is in fact fully supported by the product, it's just not documented. All that you need to do to switch to jython is to invoke wsadmin with "-lang jython" and "-wsadmin_classpath " followed by the list of portal jars (you can copy the classpath from SCRPATH variable definition in wpscript.sh).

As an example, I put together a simple Jython script for cleaning up a page hierarchy. Removing pages before applying an XMLAccess script with page definitions allows to start portal configuration from a clean "known" state. Very often, especially in a development environment, an application's page hierarchy gets polluted with various "test" pages created by developers. The script gets rid of them.

In WebSphere Portal 6.1 Jython is finally made a first-class citizen. The product's documentation proclaims that JACL support will be phased out and that jython is the way to go. Surprisingly, though, all examples still use good old JACL. I assume it's just a matter of time before they convert.

Use simple code to Check for WebSphere Portal iFixes and Fixpacks

As the internet didn't seem to know how to programatically lookup the installed iFixes and Fixpacks for a PortalServer, I'm going to share my solution, it the hope that someone can point me to a better way.

What I was able to work out from trawling the net and poking around at the WebSphere portal installation is that the install history for iFixes and Fixpacks for Portal are stored in the directory %PORTAL_HOME%/version/history. This makes the process of finding the fix level a case of:

1. Lookup the Portal home.
2. List the files in the version history directory.

Lookup the Portal Home

To lookup the portal home, use the fact that there is a WPS_HOME environment variable set in the WebSphere environment. Unfortunately looking up values in the WebSphere environment is non-trivial. The WebSphere environment is not made directly available, to access it you need to go through the WebSphere Admin service, and the WebSphere management mbeans.

The code to do this is:

com.ibm.websphere.management.AdminService as =
com.ibm.websphere.management.AdminServiceFactory.getAdminService();
String server = as.getProcessName();
javax.management.ObjectName serverName =
new javax.management.ObjectName("*:*,type=AdminOperations,process=" + server);
Set objectNames = as.queryNames(servName, null);
javax.management.ObjectName objectName =
(javax.management.ObjectName) objectNames.iterator().next();
Object[] args = new Object[] {"${WPS_HOME}"};
String[] signature = new String[] {"java.lang.String"};
String wpsHome = as.invoke(objectName,"expandVariable",args, signature);

The wpsHome variable will contain the string path to the websphere portal home directory.
List the Files in the Version History Directory

After retrieving the portal home, it is possible to lookup the fixpacks in the ${WPS_HOME}/version/history directory.

The following code does this:

File f = new File(wpsHome + File.separatorChar +
"version" + File.separatorChar + "history");
String[] children = f.list();
for (int i=0;i out.println(children[i]);
}

In summary the full code snippet for this is:

com.ibm.websphere.management.AdminService as =
com.ibm.websphere.management.AdminServiceFactory.getAdminService();
String server = as.getProcessName();
javax.management.ObjectName serverName =
new javax.management.ObjectName("*:*,type=AdminOperations,process=" + server);
Set objectNames = as.queryNames(servName, null);
javax.management.ObjectName objectName =
(javax.management.ObjectName) objectNames.iterator().next();
Object[] args = new Object[] {"${WPS_HOME}"};
String[] signature = new String[] {"java.lang.String"};
String wpsHome = as.invoke(objectName,"expandVariable",args, signature);

File f = new File(wpsHome + File.separatorChar +
"version" + File.separatorChar + "history");
String[] children = f.list();
for (int i=0;i out.println(children[i]);
}

The work of mapping file names to the installed fix packs/iFixes is left as an exercise for the reader.

Does anyone know if there is a better way of dong this?

IE7 and Websphere Portal 5.1 are in a boat. Who fell in the water? Iframes!

At least not have turned on your computer the day, there is little chance that you passed near the exit IE7. This afternoon I installed in a virtual machine to test the compatibility of web applications for my client. Going to see the corporate portal based on IBM Websphere Portal 5.1, I was very surprised to come across the following message:

A clipping portlet with IE7
I thought a joke at Microsoft, such iframes deactivation in the default configuration for a muddy excuse of security (they have done well up overnight for identifiers in the URL 1). After an unsuccessful tour of the nearby options IE7, I finally went to take a look at the configuration of the portal. And that was that the solution lay.
Indeed, it appears that WPS render pages according to the capabilities of user agents, and that necessarily, IE7 is not stated in it is the most degraded, which has been proposed. A laudable intention, but we made a good fright anyway (especially my client, which was decomposed at a speed that I could not believe biologically possible). In short, a small change in the regexp detection of user agents and the problem was resolved.


With a priority rule now on every match
So you're ready for IE 8.

WebSphere Portal development: where to start?

You have probably noticed at this time, the portals are all the rage. Figures for some time, I am rather paid on WebSphere Portal, and it is not displeasing to me after 3 & a half years devoted almost exclusively to the Lotus Domino Development (with a bit of PHP between the two). However, it is a world wide and very different from the latter, and the beast is not necessarily easy to master or even to address. Here are some links that I collected throughout my adventures, which could be of interest to those entering the field:

• What is a portlet part I et part II . What is a portlet part I and part II. A good introduction to portals and portlets development.
• The JSR-168 specification. The standard describes how portlets standards. Lotus Component Designer ) Is essential reading for anyone who wants to develop portlets (unless you are using WebSphere Portlet Factory, and, if I followed the Lotusphere, Lotus Component Designer)
• Understanding and developing portlets in Java (PDF), a technology watch of the company SQLI. Very good.
• Guidelines for Writing JSR-168 Portlets , Guidelines for Writing JSR-168 Portlets, a compilation of small mistakes and helpful tips.
• Best Practices for Applying Ajax to JSR 168 Portlets. Best Practices for Applying Ajax to JSR 168 Portlets, or how to properly AJAX with portlets, pending spec JSR-286.

Not directly related to WPS, but an excellent resource for anyone wanting to understand the JSP tag libraries (and WPS, a full y'en).

• Introducing JSTL and expression language. Même remarque. Same.
• Info Center : Designing your portal a very detailed section on how the themes and skins WPS 5.1. Necessary to customize the interface and the browsing experience of your portal (note: it has changed a lot with WPS 6).
• XMLAccess 101 part I et part II . XMLAccess 101 part I and part II. An excellent introduction to XMLAccess, which is just what WPS DXL is Domino. It is an XML dialect that describes the overall structure of the portal (pages, labels, portlets, portlet applications, web modules, users, etc) and allows convenient operation of import / export. His mastery is essential if one wishes to avoid the admin interface unfriendly to any operation which requires a dozen clicks.
• Info Center: XMLAccess reference. The reference XMLAccess.
• WPS. Tech Spot, a very interesting blog that talks about everything related to WPS.
• WebSphere Portal Security. A presentation of the Lotusphere 2006 on security of WPS 5.1 and 6.
• Table access rights. A reference for managing rights to pages and portlets.
• Change the titles and labels by default. Making changes to NLS .properties containing translations denominated system.
• Authentication in WebSphere Portal
• Customizing the WebSphere Portal Login / Logout mechanism
• Exploiting the WebSphere Portal 5.1.0.1 programming model: part 1, part 2, part 3, part 4
• Caching techniques in WebSphere Portal

That's all for now, I probably will complete this list as & as my discoveries. In general, your first instinct should be to consult the Info Center (v5.1, v6.0) because it is very comprehensive and full of lots of info needed.