Sumedha's blog

How to use Apache Pig to analyze a log file?

2012-01-07T16:13:00.000+05:30

Apache Pig is a platform for analyzing large amount of data. It consists of a built-in language called 'Latin' for writing data analysis logic. When functionality provided the by 'Latin' is not enough, Pig allows you to write you own UDFs (User Define Functions) & make use of them in data analysis scripts. UDFs can be written using several languages including Java and JavaScript.

Simple tutorial bellow demonstrates how I have used basic functionality provided by Latin to do simple analysis of a Apache access log. Here I have not used any UDFs to make examples very simple. Let's walk through the samples starting from step 0.

Download Apache Pig from http://pig.apache.org/
Unzip it to a directory on your machine (eg: I used pig-0.9.1)
Start pig (inside ping-0.9.1 directory)

java -Xmx512m -Xmx1024m -cp pig-0.9.1.jar org.apache.pig.Main -x local

You need yo have a Apache HTTP Server access log for the samples to run. I used a similar file generated from WSO2 Application Server. You can find it attached.
Once Apache Pig is started, it will take you to Grunt shell as follows.

Now, let's use few simple scripts written in Pig's Latin language to analyze the log file

Contents of the log file is similar to following.

0:0:0:0:0:0:0:1%0 - - [07/Jul/2011:09:17:35 +0530] "GET /carbon/admin/jsp/session-validate.jsp HTTP/1.1" 200 3197 "https://localhost:9443/carbon/admin/index.jsp?loginStatus=true" "Mozil la/5.0 (Macintosh; Intel Mac OS X 10_6_3) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.41 Safari/535.1"
127.0.0.1 - - [07/Jul/2011:09:17:35 +0530] "POST /services/ServerAdmin HTTP/1.1" 200 2165 "-" "Axis2"

Just paste following script segment to Grunt prompt & hit enter.

Scenario : viewing list of IPs from where access is made

Script

A = load 'http_access_2011-07-07.log' using PigStorage('-') as (f0,f1,f2,f3,f4);
B = foreach A generate f0;
C = distinct B;
dump C;

You will be presented with an output similar to following.

Output

(127.0.0.1 )
(0:0:0:0:0:0:0:1%0 )

More examples:

Browser Agents
Script
A = load 'http_access_2011-07-07.log' using PigStorage('"') as (f0,f1,f2,f3,f4,f5);
B = foreach A generate f5;
C = distinct B;
dump C;

Output
(Axis2)
(Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_3) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.41 Safari/535.1)

URI Accessed
Script
A = load 'http_access_2011-07-07.log' using PigStorage('"') as (f0,f1,f2,f3,f4);
B = foreach A generate f1;
C = distinct B;
dump C;

Output
(GET / HTTP/1.1)
(GET /carbon HTTP/1.1)
(POST /services/ServerAdmin HTTP/1.1)
(GET /carbon/admin/index.jsp HTTP/1.1)
(POST /carbon/admin/login_action.jsp HTTP/1.1)
(POST /services/RegistryAdminService HTTP/1.1)
(GET /carbon/yui/build/yahoo-dom-event/yahoo-dom-event.js HTTP/1.1)
(GET /carbon/data_service/images/data-services-uploadWSDL.gif HTTP/1.1)
(GET /carbon/viewflows/extensions/core/images/handler_flow.gif HTTP/1.1)

Access time & source
Script
A = load 'http_access_2011-07-07.log' using PigStorage('"') as (f0,f1,f2,f3,f4);
B = foreach A generate f0;
C = distinct B;
dump C;

Output
(127.0.0.1 - - [07/Jul/2011:09:20:17 +0530] )
(127.0.0.1 - - [07/Jul/2011:09:20:23 +0530] )
(0:0:0:0:0:0:0:1%0 - - [07/Jul/2011:09:17:09 +0530] )
(0:0:0:0:0:0:0:1%0 - - [07/Jul/2011:09:17:17 +0530] )
(0:0:0:0:0:0:0:1%0 - - [07/Jul/2011:09:17:18 +0530] )

Solving Apache Pig : java.lang.OutOfMemoryError

2012-01-07T11:25:00.001+05:30

If you start Apache Pig using following,

java  -cp pig-0.9.1.jar org.apache.pig.Main -x local

you will encounter OutOfMemoryError for even relatively moderate loads. The complete error is something similar to,

java.lang.OutOfMemoryError: Java heap space
 at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.(MapTask.java:781)
 at org.apache.hadoop.mapred.MapTask$NewOutputCollector.(MapTask.java:524)
 at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)

As the solution start Pig by providing sufficient memory for process as follows.

java -Xmx512m -Xmx1024m -cp pig-0.9.1.jar org.apache.pig.Main -x local

Don't promote leopard rush in Yala, Sri Lanka

2012-01-05T16:47:00.004+05:30

Found this attachment in a mail. Original author is unknown.

"Yala is full of wild life. Leopards are only one of them. Enjoy your safari. Drive slowly"

protobuf: Ant BuildException....Cannot run program "../src/protoc"

2012-01-01T12:13:00.002+05:30

If your building probuf-2.4.1 (for Java) on non Windows environment, chances are very much for encountering following error.

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.3:run (generate-sources) on project protobuf-java: An Ant BuildException has occured: Execute failed: java.io.IOException: Cannot run program "../src/protoc": error=2, No such file or directory -> [Help 1]

The reason in following entry inside protobuf-2.4.1/java/pom.xmlAs a work-a-round to this problem, open up the above pom.xml and do following replacement.

In vim - %s/..\/src\/protoc/protoc/g

For this to work, you need to successfully install protoc & it should be available on your path.

eg:
~$ protoc --version
libprotoc 2.4.1

Monitoring : replace (top,htop)

2011-12-25T09:12:00.000+05:30

Tried using htop to monitor processes running.
Installation is simple.

Linux: sudo apt-get install htop
Mac : sudo port install htop

Just type 'htop' and you will have an output similar to following.

Some of the commands I found useful:

Sorting by output column – Press F6 or >
Tree View – Press F5 or t
Processes per user – Press u
lsof Output inside htop – Press l
Trace a Process from htop – Press s
Follow a Process – Press F

Update: htop for Snow Leopard - http://jeetworks.org/node/60

Maven tips: Fixing OutOfMemoryError

2011-12-24T23:07:00.006+05:30

Refer: https://cwiki.apache.org/confluence/display/MAVEN/OutOfMemoryError
Simple as setting MAVEN_OPTS variable on your environment.

MAVEN_OPTS="-Xmx512m -XX:MaxPermSize=128m"

A widely known problem & a solution. Just adding a reference to Maven documentation for my future reference & anyone else's.

What I did not know before is following part.
"When using the Java compiler in embedded mode, each compiled class will consume heap memory and depending on the JDK being used this memory is not subject to gargabe collection, i.e. the memory allocated for the compiled classes will not be freed"

If your using 'maven-compiler-plugin' this can be set using pom.xml of your project as well.
Refer: http://maven.apache.org/plugins/maven-compiler-plugin/examples/compile-with-memory-enhancements.html

Comprehensive introduction to GSoC

2011-12-19T14:24:00.002+05:30

Pradeeban has done a comprehensive introduction to Google Summer of Code. You can find it @
http://www.box.com/shared/hpcfi39yfgub83sq99hz

How to enable case sensitivity in MySQL String Searches?

2011-12-05T14:15:00.001+05:30

How to enable case sensitivity in MySQL String Searches?Based on the collation specified, MySQL string searches case produce case insensitive results.eg:

mysql> select REG_NAME from REG_RESOURCE where !(REG_NAME is null) and REG_NAME = 'AdminModule' limit 1,10 ;
+-------------+
| REG_NAME    |
+-------------+
| adminModule |
| adminModule |
| adminModule |
| adminModule |
+-------------+
4 rows in set (0.00 sec)

For more info read following:http://dev.mysql.com/doc/refman/5.0/en/case-sensitivity.htmlhttp://dev.mysql.com/doc/refman/5.0/en/charset-collation-implementations.htmlFollowing steps explain how to override this behavior **for a column in a MySQL database table**.Step 1: Let's see what are the default values for 'CHARACTER SET' & 'COLLATE' (this command is MySQL specific)

mysql> show variables like 'c%';
+--------------------------+--------------------------------------------------------+
| Variable_name            | Value                                                  |
+--------------------------+--------------------------------------------------------+
| character_set_client     | latin1                                                 |
| character_set_connection | latin1                                                 |
| character_set_database   | latin1                                                 |
| character_set_filesystem | binary                                                 |
| character_set_results    | latin1                                                 |
| character_set_server     | latin1                                                 |
| character_set_system     | utf8                                                   |
| character_sets_dir       | /usr/local/mysql-5.1.50-osx10.6-x86_64/share/charsets/ |
| collation_connection     | latin1_swedish_ci                                      |
| collation_database       | latin1_swedish_ci                                      |
| collation_server         | latin1_swedish_ci                                      |
+--------------------------+--------------------------------------------------------+

Step 2: The defaults usually do not support case sensitive searches. Hence following behavior. (i.e. looking up for 'AdminModule'. But getting values for 'adminModule')This is due to default collation setting for column REG_NAME not supporting case sensitive behavior.

mysql> select REG_NAME from REG_RESOURCE where !(REG_NAME is null) and REG_NAME = 'AdminModule' limit 1,10 ;
+-------------+
| REG_NAME    |
+-------------+
| adminModule |
| adminModule |
| adminModule |
| adminModule |
+-------------+
4 rows in set (0.00 sec)

Step 3: Let's change the collation to a value which supports case sensitivity

mysql> ALTER TABLE REG_RESOURCE MODIFY REG_NAME varchar(256) CHARACTER SET latin1 COLLATE latin1_bin;
Query OK, 497 rows affected (0.09 sec)
Records: 497  Duplicates: 0  Warnings: 0

Step 4: Now if we run the query again..

mysql> select REG_NAME from REG_RESOURCE where !(REG_NAME is null) and REG_NAME = 'AdminModule' limit 1,10 ;
Empty set (0.01 sec)

Step 5: Changing back to old values...

mysql> ALTER TABLE REG_RESOURCE MODIFY REG_NAME varchar(256) CHARACTER SET latin1 COLLATE latin1_swedish_ci;
Query OK, 497 rows affected (0.09 sec)
Records: 497  Duplicates: 0  Warnings: 0

Step 6: Same case in-sensitive behavior returns.

mysql> select REG_NAME from REG_RESOURCE where !(REG_NAME is null) and REG_NAME = 'AdminModule' limit 1,10 ;
+-------------+
| REG_NAME    |
+-------------+
| adminModule |
| adminModule |
| adminModule |
| adminModule |
+-------------+
4 rows in set (0.00 sec)

Step 7: Switch back to Step 3.

MacOSX:Shortcuts for moving cursor in command line

2011-12-02T07:23:00.001+05:30

'Crtl + a' - to move to beginning of line

'Ctrl + e' - to move to end of line

'Ctrl + d' - to delete text under cursor'

'Esc + f' - tab like movement within an existing command

Google Docs : How to reference cell value in another sheet

2011-12-02T01:12:00.001+05:30

Formula : '='+ '[SheetName]' + '!' + '[Cell]'
eg:
=Sheet1!E1

Update: Article '10 Useful Google Spreadsheet Formulas You Must Know' contains few more useful tricks.

TI Mama - no more

2011-10-16T12:08:00.000+05:30

Tribute to Titus Thotawatta, A veteran Sri Lankan film producer & a pioneer in Sri Lankan cartoon field. Most of the kids in my era enjoyed the cartoon 'Dosthara Hoda Hitha (a.k.a Doctor with a good heart)'.

source: http://en.wikipedia.org/wiki/File:Titus_Thotawatta.jpg

Dennis Ritchie (1941-2011)

2011-10-13T10:23:00.004+05:30

source 1 : http://www.amazon.com/gp/product/images/0131103628/ref=dp_otherviews_0?ie=UTF8&s=books&img=0

source 2: http://www.muppetlabs.com/~breadbox/rip-dmr.html

update: Great Summary on Dennis Ritchie's contributions

Battle : multi-tenancy vs single-tenancy

2011-10-12T00:55:00.001+05:30

source: http://www.silicon.com/technology/software/2011/10/06/oracles-public-cloud-unveiled-39748052/?s_cid=115

"We think [multi-tenancy] is a really bad idea, unless you want to look at your competitors' leads. It's a very bad security model," according to Ellison.

Not everyone agrees, however. Benioff of Salesforce - which uses a multi-tenancy model - said: "Multi-tenancy is the foundation of the industry. You see it in the technology we use every day - in Twitter, in Google, in Larry's company Netsuite [Ellison has an investment in Netsuite] and our company."

"When you open a bank account, do they open a new server for you?" he said. "No, they use a shared database."

Good for Oracle as they do not need to pay licenses for using their own database engine!!!

WSO2 bags 6 awards @ NBQSA 2011 to become overall winner

2011-10-08T00:32:00.001+05:30

The pictures say it all...

https://twitter.com/#!/bcssrilanka

Last year performance

http://sumedha.blogspot.com/2010/10/wso2-comes-out-with-flying-colors-nbqsa.html

All winners

http://twitpic.com/6wisny

Team work!!

2011-09-28T11:38:00.001+05:30

Came across this image on team work. This is simply great & "sooo" true. Reminded me of this post (http://sumedha.blogspot.com/2008/11/team-work.html) I did sometime ago as well.

Don't know who the original author/painter is. But full credit goes to him/her.

WSO2 among InfoWorld's Bossie Awards for 2011

2011-09-09T06:40:00.000+05:30

"WSO2 remains one of the richest -- and best managed -- open source projects today..."

Pro Surf 2011 & WSO2Con 2011 - two world class events in September

2011-08-31T15:53:00.001+05:30

August 30th - September 4th

The Sri Lankan Airlines Pro Surfing Contest 2011, one of the world’s top-rated surfing events,got underway yesterday (August 30th) at Arugam Bay with a participation of nearly 100 professional surfers. This year the event will host two major categories. the Long Board Men’s Championship and the Short Board Women’s World Qualifying Series events. Read More..

September 12th - 16th

WSO2 Conference, a primer conference on middleware will be held in Sri Lanka featuring 30+ speakers from more than 10 countries. In addition to three day conference there will be two days of tutorial sessions. Some of the speakers include:

Dr C. Mohan - IBM Fellow and Former IBM India Chief Scientist
Sastry Malladi - Distinguished Architect, eBay
Gregor Hohpe - Author, Enterprise Integration Patterns and Google Engineer
Narendra Nathmal - Chief Architect, Cognizant Technologies
Andreas Wichmann, Ph. D - Systems Architect, T-Systems
Sanjiva Weerawarana, Ph. D - Chairman, Founder & CEO, WSO2 Inc
Paul Fremantle - Founder & CTO, WSO2 Inc

WSO2Con2011 : Tutorial : Hosting Web Applications and Web Services with WSO2 Application Server

2011-08-31T14:03:00.000+05:30

Abstract:
Web applications are a key component in presenting your business to end users & partners. Web Services provide a mechanism for integrating between disparate software systems & getting latest data into your system. Combination of Web Applications & Web Services provide a killer combination to your business.

WSO2 Application Server supports hosting both web applications & web services. It has a lightweight model that makes it easier to develop, deploy, manage and monitor Web applications and Web services. The completely open source, enterprise-ready server is based on Apache Tomcat and Apache Axis2, the world’s most popular Web application server and Web services engine, and it offers features to enable clustering and high availability.

This tutorial session will cover,
1. Writing, deploying, debugging Web Applications & Web Services using WSO2 Application Server & Carbon Studio
2. Monitoring Web Applications & Services
3. Securing Web Applications & Services
4. Best practices in writing/deploying Web applications & services on WSO2 platform

Get yourself registered.

WSO2Con2011 : 13 days more.... are you attending?

2011-08-30T12:57:00.000+05:30

33 speakers

12 tutorial sessions

3 tracks

29 talks

4 keynotes

2 panel discussions

13 days more

Are you attending?

WSO2Con2011: Data in your SOA: From SQL to NoSQL and Beyond

2011-08-30T11:23:00.002+05:30

Puzzled by the recent developments in data storage technologies? Trying to decide how to store data in your next project? Join me @ "Data in your SOA: From SQL to NoSQL and Beyond" session in WSO2Con 2011 where I discuss various approaches available for data storage.
Abstract is as follows.

Abstract:
Data Persistence is a key need for any type of application. The data being persisted could vary from input captures from users, pushed by some other systems, generated as a result of internal processing, auditing/logging & many more.Solution Architecture changes from problem to problem. So does the need to efficiently store data. Your applications deal with different types of data. Each of these has it's own access patterns, structure, life cycle, archival needs and many more attributes/behaviors. Thus, choosing the right storage mechanism matters a lot to the ultimate performance and success of your application.

For most applications, relational database has been the traditional choice for storage. This is a decision simply taken based on wide use of relational databases & readily available skill set. Along the same line, Internet based applications have redefined where & how we keep our data. The document, spreadsheet or presentation we locally saved on our computer can now be stored some where on the Internet. And, sky is the limit for number of users who can get registered for these applications. Thus, many are Internet based applications are continuously pushed to find more efficient & scalable storage architectures. The NoSQL (Not Only SQL) moment got it's adrenaline due to this need & has given a new awakening to discussions on data storage space.

This talk focuses on choices available when selecting a data storage engine, their trade offs, facts to consider when making the choice & examples of different usages.

Productivity Tips for using SSH

2011-08-18T17:57:00.000+05:30

This blog post (http://blogs.perl.org/users/smylers/2011/08/ssh-productivity-tips.html) provides invaluable set of tips for increasing productivity when working with SSH. Found them to be very useful.

WSO2Con is back (2011)

2011-08-14T18:02:00.002+05:30

WSO2Con 2011 - Colombo is happening this year too after a highly successful WSO2Con 2010 (pictures). More than 25 speakers from 10 countries are included in this year's agenda where we talk about more recent developments on middleware landscape.

Get yourself registered. Early bird discounts are now available.

Update:

If your not convinced yet, read this great post from Sanjiva - http://sanjiva.weerawarana.org/2011/08/are-you-attending-wso2con-2011.html.

coming home to you ...

2011-08-14T17:22:00.001+05:30

"I left you long ago, to search for a better life,
I've travelled around the world, have seen amazing sites,
Though I thought I had it made, you were always on my mind,
I'm coming home I'm coming home to you"

Came across the song "coming home to you" by Ruklanka while reading Sanjiva's blog post on "10 years since returning to Sri Lanka".

Both blog post from Sanjiva & song from Ruklanka are very much in sync with the prevailing positive environment Sri Lanka after the war. And reflect some of the things I have been telling few of my friends who choose to **migrate** to other countries in search of greener pastures.

Another post I am longing to do for sometime is of infrastructure development activities happening all over the country. I am still waiting for some images to finish it..

MacOSX : How to open a new terminal tab in the same directory?

2011-08-12T18:28:00.000+05:30

Lack of this functionality on Mac OS X out of the box has been a great menace & a productivity killer for me. finally found an acceptable solution.

Original Source: http://procbits.com/2011/07/18/open-a-new-tab-in-terminal-app-in-the-same-directory-on-mac-os-x/

Switch to super user mode
Create a new file in '/usr/local/bin' called 'nt' (nt = new tab)

Add following content

#!/bin/bash

osascript -e 'tell application "Terminal"' \

-e 'tell application "System Events" to tell process "Terminal" to keystroke "t" using command down' \

-e "do script with command \"cd `pwd`;clear\" in selected tab of the front window" \

-e 'end tell' &> /dev/null

Grant execute permissions using 'chmod +x /usr/local/bin/nt'
Open a new terminal, browser to a directory & type 'nt' from there !!

How to find repeating blocks in a file?

2011-07-26T01:00:00.000+05:30

Content of the file. Note that there are multiple repeating SQL statements.

Finding lines that repeat.(using sort sql-stmt.log | uniq -c | awk '$1 !~/1/')

Setting conditional breakpoints in Eclipse

2011-07-26T00:38:00.000+05:30

Step 1:
Start the server in debug mode.

Step 2:
Connect to server using Eclipse debug configuration

Step 3:
Create a breakpoint & go to it's properties

Step 4:
Enter conditional criteria for the break point.

Step 5:
Now all set.Take a not of '[conditional]' in break point display. Your break point will be trigger when the condition is satisfied.

WSO2 Stratos Introduction - Screencast

2011-05-23T02:04:00.000+05:30

Worked with Dakshitha & Harindu from WSO2, on a screen cast of WSO2 Stratos - a PaaS offering of WSO2's comprehensive middleware stack.
I think we came up with a pretty decent output after only few hours of script writing, recording & editing. Checkout the screen cast here... Appreciate feedback & planning to do few more of these soon.

"Men run software world" - what prevents women?

2011-05-23T01:42:00.000+05:30

I had a discussion (turned into a debate :-) ) with two of my female colleagues on "women not given equal rights in the world of software". My argument was, there is no such thing as gender which prevents women from going to the top.
But my colleagues insisted there are many more, with a special mention to 'Men engineers' waiting for an error done by 'women engineers'. This could be true, but I do strongly believe women pull themselves down by thinking lesser of their abilities. So does Facebook's COO. Read more ...

Palo Alto, California - here comes WSO2

2011-05-23T01:24:00.000+05:30

After closing $6.5M funding round lead by Quest Software, WSO2 opened a new office in Palo Alto, California on 19th May. Plan is to drive North American Marketing from here & provide more efficient support responses within this timezone.

Bradby Express - 2011

2011-05-19T12:02:00.000+05:30

If anyone is interested..heard the last time was great !!!

Scale of Universe !!

2011-05-19T09:56:00.000+05:30

Chck this - http://www.primaxstudio.com/stuff/scale_of_universe/
Couple of things to learn out of this creation.

The content (the intention of the creator :) )
Concept of how big picture is created
How big picture changes along every step
How to present complex set of data using a simple image

How is SSL hopelessly broken?

2011-04-21T12:12:00.002+05:30

“It's pretty crappy, but it's what it is now,” White Hat Security CTO Jeremiah Grossman said, referring to the SSL system. “It is definitely weak. It could fall down at anytime.” ®

Pretty scary news on so called trust system on top of which we send all our valuable data. I used to have high regard on CAs. But looks like they need to do lot of homework . Read more here .

Best Wishes for Sri Lankan Cricket team !!

2011-04-02T10:34:00.007+05:30

In the jungle, the mighty jungle

The LIONS ROAR tonight

In the jungle, the mighty jungle

The LIONS WIN tonight

Image source : http://www.icccricketworldcup-2011.com/

Update:
After a competitive display of gentleman's cricket by both teams, India won the ICC world cup 2011. Congratulations to India.
Except for some poor fielding, Sri Lanka team (still our heros) put up a good fight. Sanga looked bit tensed throughout the match. But he very well knows no Sri Lankan will touch his or any of the team member's homes :D .. well done boys.. You guys are still our heros. welcome back home ....

Update 2:
A warm welcome for Sri Lankan players :
"Sri Lanka's cricketers had lost the final game sometime past 11 p.m. on Saturday. In under 12 hours, as a hot Sunday sun blazed in all its glory, Sri Lankan people, by the grace and warmth they displayed in the face of a defeat, unlike all over the sub-continent, proved that they can rise above: cricket, after all is a game. All that matters is that the game is played well and in its true spirit. In doing so, the people have set an example to the rest of south Asia. - The Hindu"

Good read for Sri Lankan University Students

2011-02-25T15:40:00.003+05:30

Bellow image contains an article(in Sinhalese) appeared on a weekend news paper in Sri Lanka sometime back.
Abstract:
Sri Lanka provides free education up to university/college degree. But portion of the young generation
that enters the university system becomes a target of political agendas of different political parties. The result is these talented young lives loosing their chance to have free education & getting lost in different political campaigns. Ultimately they lose most valuable time of their lives and end up miserably. Usually it's too late when they realize the ply they fell into.

Thought of posting this article written by Sriyani Perera since I believe every university must student read & understand this dilemma.

Interesting article on 1996 Cricket world cup win..

2011-02-24T01:09:00.002+05:30

Few interesting things (never heard b4 as well) that caught my eye are..

"Upul came running towards me and said; ‘Let me go to the deep and let Aravinda come to the ring.’ I asked why? Then he said; ‘Well, if he gets hit, we might lose him for the final.’ He took a chance and went to the deep."

"They said there’s a carpet exhibition downstairs and they have all gone to see the carpet exhibition..... That’s the time I told myself; ‘Look, the Aussies are very tensed, our guys are not putting any pressure on themselves and this is ideal.’ That’s the time I thought, well, what’s the point in me getting stressed and I also started bargaining. I realised at that point that we were going to win this World Cup."

Read the full article here.

SVN checkout fails - Windows max path

2011-01-21T12:43:00.004+05:30

Windows has limitation on max file path. Due to this a svn checkout on a long folder structure will fail. For example,

E:\abc\pqr\xyz>svn co http://svn.wso2.org/repos/wso2/branches/carbon/3.1.0/components/governance test

will fail giving following message.

svn: Can't open file 'test\org.wso2.carbon.governance.gadgets\3.1.0\org.wso2.car
bon.governance.gadgets.impactanalysis\src\main\java\org\wso2\carbon\governance\g
adgets\impactanalysis\internal\.svn\tmp\text-base\GovernanceImpactAnalysisServic
eComponent.java.svn-base': The system cannot find the path specified.

Paul gave a solution for this. Alter the checkout path as follows.

svn co http://svn.wso2.org/repos/wso2/branches/carbon/3.1.0/components/governance \\?\E:\abc\pqr\xyz\abc\pqr\xyz\test

It works like magic now.

One less password to remember !!!

2011-01-20T10:58:00.001+05:30

But, make you more vulnerable for single point of attack. Get exposed @ one place & your bound to loose everything.

How to display a value of Maven Property?

2011-01-18T19:56:00.007+05:30

Have following 'plugin' snippet inside the 'build' section of your pom.xml


<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<version>1.1</version>
<executions>
<execution>
 <phase>validate</phase>
 <goals>
  <goal>run</goal>
 </goals>
 <configuration>
  <tasks>
   <echo>******** Displaying value of property ********</echo>
   <echo>${project.basedir}</echo>
  </tasks>
 </configuration>
</execution>
</executions>
</plugin>

How to find modified/conflicted files from svn stat?

2010-12-25T09:36:00.002+05:30

Following command will filter out modified & conflicted files from a 'svn stat' output.This can be quite handy if your svn stat output is quite large.

svn stat | sed -n '/^[CM]/p'

^[CM] looks for lines starting with either 'C' or 'M'.

eg:

trunk$ svn stat | sed -n '/^[CM]/p'
M products/ds/modules/distribution/src/assembly/bin.xml
C components/pom.xml
M dependencies/pom.xml
M features/pom.xml
M dependencies/axiom/pom.xml

Google Fonts Directory

2010-12-15T10:43:00.002+05:30

Came across 'Google Fonts Directory' through Hiranya.
Nice looking set of fonts. Try - http://code.google.com/webfonts

WSO2 Business Activity Monitor - v1.2.0 released..

2010-12-11T08:46:00.006+05:30

Close to real time insight into what's happening in a business is an invaluable asset to any entrepreneur. But most of the Business Activity Monitoring (BAM) tools available in the market today are way too expensive.

WSO2 BAM is an open source tool that can be used to,

Collect
Summarize
Archive
Visualize

business activities happening through an application deployment. As of current status, it is specialized in collecting statistics related to SOA based deployments. However, there is a highly extensible framework around WSO2 Business Activity Monitor that let's you pump any type of business statistics into it.

This model is powered by WS-Eventing based publish/subscribe model to send & receive data.
WSO2 released v1.2.0 of it's Business Activity Monitor yesterday. Some of the major attractions in the new release are the steps taken towards,

Activity Monitoring & Message co-relation
Message Tracing
Property based Message Filtering

Let me summarize how we tackle these new areas.

Activity Monitoring & Message co-relation
A single message entering your SOA deployment can flow through/or split into multiple logic flows simultaneously. But from a business transaction point of view this is still a single business operation. It is important to be able to view all the messages related to a single activity together. A concept of an activity id is being used to co-related between multiple messages belonging to a single activity.

Message Tracing
Facilitates tracing a message flow through a system. Each & every point a message passed through a system can be recorded, queried & presented.

Property based Message Filtering
Each message might contain certain attributes/properties that can be of interest. For example OrderID, CustomerID might be interested properties in the case of an Order Message passing through the system. WSO2 BAM allows you to specify these properties, extract out the values, store them & query based on those values.

WSO2 comes out with flying colors @ NBQSA 2010

2010-10-01T23:15:00.002+05:30

WSO2 won 4 awards @ NBQSA 2010 (National Best Quality Software Awards) competition held for 12th consecative year in Sri Lanka. This competition was organized by BCS's Sri Lankan Chapter.
This is the first time we took part in NBQSA & we are thrilled to walk away with 4 awards including the BEST for overall categories. Although only three products took part & won, these awards are made possible by passion, committment, guidence & innovation we practiced for last 5 years.

Every member (past & present) of WSO2 is responsible for these recognitions. Awards we won are as follows.

Gold Award (Overall) - WSO2 Enterprise Service Bus
Gold Award (Tools & Infrastructure) - WSO2 Enterprise Service Bus
Silver Award (R & D) - WSO2 Gadget Server
Bronze Award (Tools & Infrastructur) - WSO2 Data Services Server

All of these three products have been selected to represent Sri Lanka in APICTA competition to be held KL,Malaysia next week.

The Fink Project

2010-08-21T21:14:00.004+05:30

Ubuntu users are so much accustomed to 'apt-get install xxxxxx' whenever some new feature is required. But rarely we realize that this comes with the generosity of open source community & the software this community has shared with rest of the world for free. I felt power & flexibility offered by 'apt-get' more when I switched to Mac OS X. But soon I came across 'Fink' project.

"The Fink project wants to bring the full world of Unix Open Source software to Darwin and Mac OS X."

By having "Fink" installed on your Mac, you can install any of your favorite Unix/Linux programs on top of Mac. Follow the installation instruction mentioned here.

After having 'Fink' installed, you can issue commands like following....

fink install gimp
fink install pstree
fink install axel

Where is 'make' on Mac OS X?

2010-08-21T20:51:00.003+05:30

If your trying to install a software by compiling source on a default Mac OS X (v10.6.3)installation, the very first error you will be getting is "-bash: make: command not found" .

Unlike on Ubuntu, you do not get these development utilities installed by default on Mac OS X. In order to get these tools, you need to download "Xcode Developer Tools" from Apple Connect.

Ubuntu users guide to Mac OS

2010-08-21T20:08:00.007+05:30

Recently I migrated from Ubuntu 10.04 to Mac OS X 10.6.3 . While Mac provides a new user experience, I found myself looking for alternatives for features I have been using on Ubuntu. I also found few basic features (lock screen,viewing folder path on Finder Window) missing OOB.

Being very new to Mac @ this point, I am not at all familiar with user forums, popular download sites,documentation, blogs for Mac OS. And I am sure there have been many discussions on getting started with Mac OS X. But I thought of having my new journey documented as I explore more in to Mac. All the related posts will be having the tag 'Mac-OS-X'.
Updates

How to capture screen shot? (Command-Shift-4 -> Space Bar -> Click)
How to change prompt? (export PS1="\W$ ")

The illustrated guide to a Ph.D.

2010-08-21T19:56:00.002+05:30

The illustrated guide to a Ph.D. .... an interesting way of presenting.. :-)

Java - not free any more?

2010-08-14T12:21:00.002+05:30

Recently Oracle sued Google for "knowingly, directly and repeatedly infringed Oracle's Java-related intellectual property" in Android smartphone software. But as per JavaSE FAQ "Java SE continues to be available under the Sun Binary Code License (BCL) with no additional cost for its use".

It's pathetic to see Oracle trying to control(monetize) over something that was meant to be free through it's acquisition. May be it's high time to develop/ship products on a JDK other than that of Sun's (Oracle's rather).

Birthday !!!

2010-08-04T23:47:00.000+05:30

"WSO2 is celebrating 5 Yrs on earth today."

Webinar : Build your first application in WSO2 Cloud

2010-07-27T12:37:00.003+05:30

WSO2 Stratos improves collaboaration. With Stratos, you can publish the service you just wrote with a handful of clicks & make it accessible to other developers in your team. The team can be located @ other part of the world. But the process of sharing your services/work still remains the same. Will all of this & many more, we believe WSO2 Stratos adds new paradigm to traditional software development lifecycle.

I will be doing a webinar on this. It will cover,

Webapp deployment
Service hosting
Gadget hosting
Service composition & aggregation
Security

functionality available on WSO2 Stratos. There will be two sessions on following times.

Wednesday, 4th August 2010 [ 9:00 AM - 10:00 AM (GMT)]
Wednesday, 4th August 2010 [10:00 AM - 11:00 AM (PDT)]

Link to registration page is http://wso2.org/library/webinars/2010/08/build-your-first-application-wso2-cloud .

WSO2 Stratos ...

2010-07-27T12:30:00.003+05:30

WSO2 recently announced a new offering. It's called "Stratos".

What is "WSO2 Stratos"?

It's a complete SOA deployment platform.
It's multi-tenant capable, meaning that you can run a single server offering services to multiple tenants/organizations in a complete isolated manner.
It's on an elastic run-time enabling you to boot up more servers of the same type based on load.
It can run of public & private cloud infrastructures.
We already have a running instance on Amazon. You can try it out by registering (free!!) @ https://cloud.wso2.com .
Read what Paul & Samisa thinks about Stratos here & here.

15 ways of using Ping command

2010-07-26T17:17:00.001+05:30

Check out the original article @ http://bit.ly/7ZX2nk.

Test picture directly from phone

2010-07-26T12:04:00.000+05:30

Test picture directly from phone!!!

First post from email

2010-07-26T11:58:00.000+05:30

Trying out posting capabilities from email as well. This time thought
of attaching a random picture as well. Let's see how blogger positions
this.

--
Sent from my mobile device

First post from mobile

2010-07-26T11:38:00.001+05:30

Thought of testing blogger's phone blog posting capability. Here comes the very first blog post from my phone!!!..

Murali - THE LEGEND

2010-07-22T15:51:00.008+05:30

Muttiah Muralitharan (Murali) picked his 800th Test wicket short while ago during last day of first Test match against India in Galle. With this achievement, Murali became the first player in the Test Cricket history to take 800 wickets.

[Picture courtesy of http://www.thetwenty20cricket.com]

This achievement is more special as Murali will be retiring from Test cricket today. It's been great to watch him spin (and bat too :-)) over these years. Let's hope Murali will lead a gentleman's life unlike some of the past cricketing heroes who could not live up to the standards they hoisted themselves during sporting careers.
Well done Murali & thanks for the entertainment. Good luck with your new life !!!!

Developer to developer: Tooling - milestone12

2010-07-19T02:34:00.012+05:30

What makes a tool successful one?
How to make a successful/useful tool?
How to build sleek UI wizards that build complex applications?
How to get feedback, change & deliver back to the community?
How to make multiple deployment strategies to work from a tool?

These are some of the questions that go through every member of WSO2 Tooling team these days. And yes..... We are writing the history books with a creation of Developer Studio for WSO2 platform. The initial work is very promising & we would like to hear you feedback as well..

Download latest milestone version of Carbon Studio (still debating over name !!!) from here & tell us what you think.

Too lazy to download & install?.... Let the online documentation motivate you.

WSO2 Business Activity Monitor 1.1.0 Released!

2010-07-19T01:41:00.009+05:30

In large business organizations, IT related services are considered a background process which enables the business to run smoothly. These services should facilitate the need to monitor the business by the people who understand the business in a business friendly manner. WSO2 Business Activity Monitor (BAM) is a tool which can provide such view point to business experts.

Recently WSO2 released v1.1.0 of BAM. It's new features include:

Support for Oracle back-ends
Error category monitoring for WSO2 ESB mediation data
Improvements to analytics summary calculations, storage and visualization
Support for deployment on JBoss, Apache Tomcat, and WebLogic Application servers
Improved P2 based provisioning and feature management support
Support for deleting Monitored Servers

You can download the latest version of BAM from here & explore the full feature list.

Swimming Pools: Then & Now ..

2010-07-19T00:33:00.010+05:30

Then:	Now:

Visit to ancient city Polonnaruwa

2010-07-18T23:19:00.008+05:30

Visited ancient city of Polonnaruwa, Sri Lanka today. It was an unplanned visit. I was in the area & suddenly thought of paying a visit. Took some pictures for the pleasure of those of you who haven't visited & planning to visit.
"Polonnaruwa is the second most ancient of Sri Lanka's kingdoms. It was first declared the capital city by King Vijayabahu. Today the ancient city of Polonnaruwa remains one of the best planned Archeological relic sites in the country, standing testimony to the discipline and greatness of the Kingdom's first rulers. It has been declared a World Heritage site by UNESCO.[Source http://en.wikipedia.org/wiki/World_Heritage_Sites_of_Sri_Lanka#Polonnaruwa]"

g-Eclipse

2010-04-16T11:40:00.004+05:30

g-Eclipse project aims to build an integrated workbench framework to access the power of existing Grid infrastructures. It can either be used as a graphical client for accessing existing Grid infrastructures or as an API for writing applications for Grids.

JSDL(Job Submission Description Language) is used to define jobs & find jobs available on Grid resources. Attached image(extracted from 'g-Eclipse final Architecture') shows project explorer for a Grid project.

More screen shots on example applications are available @ http://www.geclipse.eu/index.php?id=67 . Visit project home page http://www.geclipse.eu here.

Subqueries are not allowed in this context. Only scalar expressions are allowed

2010-02-23T16:44:00.002+05:30

Above error is caused by using a wrong subquery syntax on SQLs sent to MSSQL server.

For example let's assume you have a SQL similar to following:


INSERT INTO USER_ROLE (USER_ID, ROLE_ID, TENANT_ID) 
VALUES(
(SELECT ID FROM USER WHERE USER_NAME=? AND TENANT_ID=?),
(SELECT ID FROM ROLE WHERE ROLE_NAME=? AND TENANT_ID=?), 
?)

On MSSQL this will result in an error with a message similar to "Subqueries are not allowed in this context. Only scalar expressions are allowed.". The query should be modified to match following syntax.


INSERT INTO USER_ROLE (ROLE_ID, USER_ID, TENANT_ID) 
SELECT 
(SELECT ID FROM ROLE WHERE ROLE_NAME=? AND TENANT_ID=?),
(SELECT ID FROM USER WHERE USER_NAME=? AND TENANT_ID=?),
(?)

when variables do not initialize ...

2010-02-02T22:45:00.003+05:30

Picture says it all ... :-)

will iPad revolutionize my everyday life?

2010-01-30T00:43:00.004+05:30

I have been constantly in search of a digital device which seamlessly fits into everyday work.

One of my most critical requirement is note taking/writing & searching @ at a future time. This can be while I am @ garage, service station, waiting in a queue, meeting or simply taking down the shopping list mother/wify delivers over phone. The other is reading while I am @ any of above places. The material can be a PDF or an on line resource.

Obviously, both requirements demand substantial battery power and I do not want the device to be a burden to carry either.

While many would argue latest mobile phones or a kindle would do, as per my experience they lack one thing or the other. For example, I find iPhone too delicate thing to handle due to it's sleek design. Every time I use one, I feeling like it's going to slip through my hands.

I came across this blog post(hence the subject) in which John D. Halamka, MD lists down his wish list for 'ideal clinical device'.

Looks like I am not the only unsatisfied user waiting for the perfect device to arrive. :-)

#1071 - Specified key was too long; max key length is 767 bytes

2010-01-24T21:44:00.001+05:30

Recently I encountered above error (MySQL - #1071) while working on a MySQL database script. I was having an index on a varchar(2000) column. The solution was to reduce the size of the column to somewhere close to varchar(750). From Googling, I figured that it's a very common error.

Personally, I think it's better to re-think the necessacity to have a column of this length & an index on it.

Private vs Shared clouds - what's your choice?

2010-01-24T20:45:00.006+05:30

Should your organization have a private cloud computing environment or should you seek the services offered by public cloud service offerings?

On this blog post James Hamilton is expressing his views on disadvantages of setting up a private cloud & advantages of using a shared one. Alistair Croll is also sharing the same thought as James on informationweek.

I found following comment from Alistair very interesting..

"In many ways, cloud computing is the child in the Emperor's New Clothes, pointing out that the monarch is naked. So now enterprise IT is doing what it should have been for a long time: deploying those disruptive technologies. That's great news; it'll save money, and help the company focus on its core business rather than being delayed by IT obstacles.

But it doesn't make internal IT a cloud. Because of this, enterprise cloud computing is a temporary, transient thing that's just a stepping stone to clouds-as-a-business-model...."

And new development on cooling strategies seem to be fascinating as well..

"Cloud operators, meanwhile, are getting much, much more efficient in other ways. They're negotiating with cities for electrical pricing. Filing patents on boats that can sail the seas, cooling themselves with seawater. Building data centers near dams. Writing their own operations software. Building custom switches....."

Are they correct in their views? I don't know.. I guess we should wait for few more years to witness benefits enterprises gain by adopting either one of these options.

Streaming capability helps WSO2 DS to support large XML payloads

2010-01-07T00:49:00.003+05:30

Along with v2.2.0 release of WSO2 Data Services (DS) server comes a drastic enhancement to response payload size. The core engine has been modified to support XML streaming. This has resulted in two notable improvements.

Efficient use of server memory - No matter how large the payload is, server memory does not grow propotional to it. The streaming capabilies push data to client side as and when needed

Improved response time

In this tutorial, Anjana Fernando demonstrates how a payload of size 1G (1024MB) is transfered to a client residing on same host in 92 seconds. (http://wso2.org/library/knowledge-base/large-scale-data-transfer-data-services-streaming)

All these improvements are possible due to powerful XML processing capabilities offered by Apache AXIOM library.

Oracle Ref cursor support in WSO2 Data Services

2010-01-07T00:30:00.003+05:30

Oracle Ref cursor is a Data type. A variable created using this data type is usually called as a Cursor Variable.
Some of the primary advantages of using a ref cursor are,

ability to pass resultset between sub programs (eg: functions, stored procedures)
dynamic queries
efficient memory utilization

It's possible to use Oracle Ref cursor as a return type in a function or a stored procedure.

From v2.2.0 release onwards WSO2 Data Services supports functions/stored procedures having Ref cursors as return types. Kalpanie Wanigaratne has taken a simple ref cursor example to demonstrate this new enhancement here. (http://wso2.org/library/tutorials/use-oracle-ref-cursors-wso2-data-services-server)

WSO2 Data Services now supports content filtering

2010-01-06T23:41:00.008+05:30

Data Services lets you expose data as SOA style web services or REST style resources. But what if you want some part of this data set to be visible only to a selected set of users only? Content filtering similar to this is a pressing need when it comes to exposing data via APIs.

With it's latest release version(v2.2.0) of WSO2 Data Services Server is capable of content filtering based on service consuming user's roles. In this tutorial Anajana Fernando explains how to configure a data service for content filtering.

For those of you who are familiar with WSO2 Data Services, following is a quick preview of what needs to change in your data service configuration. (Take a note of the highlighted attribute).

  <result element="Orders" rowName="Order" defaultNamespace="">
   <element name="OrderNumber" column="ORDERNUMBER"/>
   <element name="OrderDate" column="ORDERDATE"/>
   <element name="Status" column="STATUS"/>
   <element name="CustomerName" column="CUSTOMERNAME"/>
   <element name="Amount" column="AMOUNT" requiredRoles="Finance" />
</result>

Parts Diagram of Toyota Hilux 2005 - 2008

2010-01-04T11:00:00.004+05:30

If anyone is looking for Parts Diagram of Toyota Hilux 2005 - 2008 models, following site might be of great resource. It's quite handy to have this before you visit a garage for repairs.

http://www.toyotahilux.info/partsdiagrams.html

If anyone has Service Manual, Parts Diagram for Toyota Hilux LN 106 - 1996 model, I am desperately in need of one.

How to monitor your business using WSO2 Business Activity Monitor

2010-01-04T09:18:00.002+05:30

This tutorial demonstrates how WSO2 Business Activity Monitor & WSO2 Data Services Server can be integrated to provide live statistics of online business services.

I have taken a simple Order Management System as a base system & explained how it's data can be service enabled using WSO2 Data Services. Then I have extended the scenario on to monitoring & demonstrated how WSO2 Business Activity Monitor can be utilized for monitoring these business services.

Check out the tutorial here (http://wso2.org/blog/sumedha/9008).

WSO2 Business Activity Monitor v1.0.0 released..

2010-01-04T09:13:00.003+05:30

WSO2 released v1.0.0 of it's Business Activity Monitor few days ago.

This Business Activity Monitor is capable of receiving event-based metrics, such as server response times and polling data using a periodic query to a running system. Collected data is then collated for presentation.

The presentation is handled by WSO2's Gadget Server component, which is powered by Apache Software Foundation's Shindig project, the reference implementation for Google Gadgets and OpenSocial specifications.

Gadget Server is capable of displaying customizable dashboards to end users.Gadgets in the dashboards can analyze and display the information in standard or user-customized ways.

Check out the WSO2 Business Activity Monitor (a.k.a. BAM) here.

Help to save MySQL

2010-01-02T11:34:00.005+05:30

I am a keen enthusiast user of MySQL, checking out every time a new version is announced & running my applications on top of MySQL. All that liberty is possible because MySQL is available as an open source database...

[In April 2009, Oracle announced that it had agreed to acquire Sun. Since Sun had acquired MySQL the previous year, this would mean that Oracle, the market leader for closed source databases, would get to own MySQL, the most popular open source database..

If Oracle acquired MySQL on that basis, it would have as much control over MySQL as money can possibly buy over an open source project.....]

read more & sign the petition here - http://www.helpmysql.org

If your interested, there's also this blog post from Michael Widenius,main author of open source MySQL.

WSO2 Data Services v2.2.0 released ..

2009-12-31T21:40:00.004+05:30

WSO2 Data Services v2.2.0 is out. This new release includes following new features that you might want to try out.

Support for large XML outputs
Google Spreadsheets as a data source
Content filtering based on user roles
Support for named parameters
Ability to configure schema type for output elements
Mixing multiple data sources in nested queries
Excel 2007 support
Support for Oracle Ref Cursor
Bug fixes for issue reported againtest v2.0.0

The full release note is available @ http://ds4soa.blogspot.com/2009/12/wso2-data-services-v220-released.html. The team @ WSO2 is taking a well deserved break to start 2010. As soon as we open for 2010 you will see several articles being published on these new features on our Oxygen Tank.

We will link them from our Data Services HOWTOs page as well.

In the meantime, download the v2.2.0 release from here & send us your feedback to ds-java-user - at - wso2.org .

Openfire 3.6.4 Server-Login to Admin console fails

2009-12-28T23:20:00.003+05:30

Thought of putting this up so that someone else will not waste time as I did while setting up openfire 3.6.4 server.

I have been using Openfire(formerly known as Wildfire) for sometime. But with it's latest public release (v3.6.4), I have noticed a strange behavior. Whenever I install a new server instance, I go through the setup wizard & go to the login screen for Admin console. But openfire does not allow me to login to the console. It keeps on complaining "Login failed: make sure your username and password are correct and that you're an admin or moderator." no matter how many times I enter the correct username/password combination.

The solution is simple. Re-start the openfire server (v3.6.4) in order to login successfully after initial setup wizard. Trivial bug it looks like. Wonder if it has been listed as an known issue. I did not bother to look ;-) .. Nevertheless kudos to openfire team for putting up an amazingly simple XMPP server.

How to clear Google Gadget Cache?

2009-12-28T11:01:00.004+05:30

Google Gadget API's makeRequest(..) method caches content by default. This is mainly to improve performance of your gadgets. But chances are very much that you need to have most up-to-date data fetched & displayed on your gadget. Here's how to tell Gadget engine not to cache content.


   var params = {};
   params[gadgets.io.RequestParameters.CONTENT_TYPE] 
         = gadgets.io.ContentType.DOM;
   var url = "http://localhost:9764/services/OrderService/GetAllOrders";
   var refreshInterval = 0;        
   var sep = "?";
   if (url.indexOf("?") > -1) {
    sep = "&";
   }
   url = [ url, sep, "nocache=", 
        refreshInterval ].join("");
   gadgets.io.makeRequest(url, readFeed, params);

WSO2 Data Services 2.2.0 is almost done !!!

2009-12-15T15:20:00.002+05:30

Await the release of WSO2 Data Services 2.2.0 ...

This release includes some interesting features as well as a solid foundation for next major release coming up in Q1, 2010....

Introducing Ozone - Private Cloud in a Box ...

2009-10-29T01:48:00.004+05:30

What is WSO2 Ozone?
The subject says it all !!! - Private Cloud in a box.
And .. It's a tool we built because we needed it. Its not a WSO2 product - its just a tool we find very useful and we believe others will too and hence is available as an open source project.

A more formal definition would be,

"WSO2 Ozone provides a portal based system for running a Xen-powered private cloud. Ozone has a node controller (running on Dom0) for managing VMs (for starting/stopping/creating DomUs), a cluster controller for managing a set of machines running the node controller and a Web portal to allow a person to start/stop/manage their own VMs."

Check out more interesting answers to,

Why not just use Xen?
Why not just use Eucalyptus?
Why yet another cloud platform?
Why not just use Amazon EC2?

at project home page. And most importantly, download & use it.

Web Services Framework for C++(WSO2 WSF/C++) 2.1.0

2009-10-17T11:36:00.003+05:30

Looking for way to consume and write web services in C++? Look no more. WSO2 WSF/C++ 2.1.0 is here to stay.
The library provides,

Client API for consume Web Services
Service API for writing Web Services
Attachment support using MTOM
WS-Addressing,WS-Policy, WS-Security support
Stub generation

Interested? checkout the project documentation ,project page & Dummy's Guide to Writing Web Services Clients with WSO2 WSF/C++.

WSO2 SOA Workshop - Santa Clara,CA

2009-10-12T19:03:00.003+05:30

Learn how to implement SOA at your organization from the experts. Click here to register.

WSO2 Technical update - 2009 Q2

2009-10-07T23:20:00.002+05:30

We at WSO2 are evolving all the time.... Hence it's utmost important to update our customers & users of the latest we have. Here's how we look like by the end of 2009 Q2. All the products mentioned here are open source.
Welcome to witness the creation of a giant yet light weight, feature rich yet open source SOA portfolio....

2009 Q2 WSO2 Technical Update

View more documents from WSO2.

Feel free to use this slide deck to introduce WSO2's array of products to your management/technical team.

How to check size of a Oracle database?

2009-10-02T22:02:00.005+05:30

I wanted to determine the size of a Oracle database used by an application. Of course following SQL plus command would do the job very easily.


select sum(bytes)/1024/1024 from dba_data_files;

But I wanted to get more statistics & most importantly in a formatted manner as well. Then I came across following script written by Jeffrey M. Hunter. This is a pretty clean script which outputs quite a few useful statistics of your Oracle database (very much similar to 'df' command). Hope this helps someone.


-- +----------------------------------------------------------------------------+
-- |                          Jeffrey M. Hunter                                 |
-- |                      jhunter@idevelopment.info                             |
-- |                         www.idevelopment.info                              |
-- |----------------------------------------------------------------------------|
-- |      Copyright (c) 1998-2008 Jeffrey M. Hunter. All rights reserved.       |
-- |----------------------------------------------------------------------------|
-- | DATABASE : Oracle                                                          |
-- | FILE     : dba_file_space_usage.sql                                        |
-- | CLASS    : Database Administration                                         |
-- | PURPOSE  : Reports on all data file usage. This script was designed to     |
-- |            work with Oracle8i or higher. It will include true TEMPORARY    |
-- |            tablespaces. (i.e. use of "tempfiles")                          |
-- | NOTE     : As with any code, ensure to test this script in a development   |
-- |            environment before attempting to run it in production.          |
-- +----------------------------------------------------------------------------+
 
SET LINESIZE 145
SET PAGESIZE 9999
SET VERIFY   OFF
 
COLUMN tablespace  FORMAT a18             HEADING 'Tablespace Name'
COLUMN filename    FORMAT a50             HEADING 'Filename'
COLUMN filesize    FORMAT 99,999,999,999  HEADING 'File Size'
COLUMN used        FORMAT 99,999,999,999  HEADING 'Used (in bytes)'
COLUMN pct_used    FORMAT 999             HEADING 'Pct. Used'
 
BREAK ON report
COMPUTE SUM OF filesize  ON report
COMPUTE SUM OF used      ON report
COMPUTE AVG OF pct_used  ON report
 
SELECT /*+ ordered */
    d.tablespace_name                     tablespace
  , d.file_name                           filename
  , d.file_id                             file_id
  , d.bytes                               filesize
  , NVL((d.bytes - s.bytes), d.bytes)     used
  , TRUNC(((NVL((d.bytes - s.bytes) , d.bytes)) / d.bytes) * 100)  pct_used
FROM
    sys.dba_data_files d
  , v$datafile v
  , ( select file_id, SUM(bytes) bytes
      from sys.dba_free_space
      GROUP BY file_id) s
WHERE
      (s.file_id (+)= d.file_id)
  AND (d.file_name = v.name)
UNION
SELECT
    d.tablespace_name                       tablespace 
  , d.file_name                             filename
  , d.file_id                               file_id
  , d.bytes                                 filesize
  , NVL(t.bytes_cached, 0)                  used
  , TRUNC((t.bytes_cached / d.bytes) * 100) pct_used
FROM
    sys.dba_temp_files d
  , v$temp_extent_pool t
  , v$tempfile v
WHERE 
      (t.file_id (+)= d.file_id)
  AND (d.file_id = v.file#)
/

Please respect the copyright message from the owner.

Free online training on WSO2 ESB

2009-09-27T18:10:00.002+05:30

We @ WSO2 will be once again hosting a free 3 hour online training on WSO2 Enterprise Service Bus. WSO2 ESB is rapidly becoming a default ESB of choice for many companies switching to Open Source alternatives.

This specially designed course will help you to,

Evaluate how the ESB fits into your environment
Understand overall architecture of the system
Learn how to use the simple administration web interface

If interested, click here to register.

Keeping track of vehicle service records

2009-09-19T22:33:00.003+05:30

All this time I have been using a diary to keep track of service records of my daily runner. After going through this, a friend suggested that it would be nice to have an on-line version of it.

Interestingly, you cannot find many freely downloadable templates on the Internet either. So here comes a **very basic** electronic draft. Feel free to add features to this & let me know about it as well.

Initially I thought of creating a Google Gadget of this... may be some other day when I have time. This should make up a good sample for forth coming WSO2 Gadget Server too...

Open Source Data Services for Strategic SOA utilising WSO2 Data Services Server

2009-09-08T22:07:00.000+05:30

Open Source Data Services for Strategic SOA utilising WSO2 Data Services Server

View more documents from sumedha.r.

Open Source Integration with WSO2 Enterprise Service Bus

2009-09-08T21:01:00.000+05:30

Open Source Integration with WSO2 Enterprise Service Bus

View more documents from sumedha.r.

AFITC 2009

2009-09-04T17:19:00.001+05:30

Recently I did two presentations on WSO2 Enterprise Service Bus & WSO2 Data Services @ Air Force Information Technology Conference (2009) in Montgomery,Alabama.

The presentations are titled as follows.

Leveraging Open Source Integration with WSO2 Enterprise Service Bus

Open Source Data Services for Strategic SOA utilising WSO2 Data Services Server

Hoping to put two separate posts containing presentations soon..

Best of WSO2 SOA Summer School

2009-08-13T18:37:00.003+05:30

Back by popular demand, we @ WSO2 will be hosting a wrap-up session of WSO2's SOA Summer School 2009 today. This 3 hours session brings you the best of all sessions conducted over the past 8 weeks. Click bellow widget for registration.

YNR - Yet Another Release !!!

2009-07-11T11:29:00.002+05:30

After several months of hard work, we have released v2.1.0 of WSO2 ESB(Enterprise Service Bus) & v3.1.0 of WSAS (Web Services Application Server).

ESB 2.1.0 contains several notable new features. My personal favourites are,
* Rule based mediation via Drools
* Fine grained authorization for services via the Entitlement mediator
* Reliable-Messaging specification 1.1 support
* Enhanced WS-Eventing support and Event Sources making it an even broker
* Enhanced AJAX based sequence, endpoint and proxy service editors
* Enhanced transport configuration management through the graphical console
* Enhanced integrated registry and search functionalities with versioning, notifications, rating of resources, and commenting
* Enhanced REST/GET and other HTTP method support
* P2 based OSGi feature support, for optional features like service management, runtime governance and so on..

It can be downloaded from here.

WSAS 3.1.0 is a stabilization release. It contains a highly improved core platform & several bug fixes since last release. WSAS 3.0.1 can be downloaded from here.

First Few Milliseconds of a HTTPS Connection..

2009-06-23T11:03:00.002+05:30

Nice explanation on "The First Few Milliseconds of a HTTPS Connection" by Jeff Moser.

Short and Simple Story of the Credit Crisis

2009-03-14T19:53:00.002+05:30

Short and Simple Story of the Credit Crisis...very nicely explained.

The Crisis of Credit Visualized from Jonathan Jarvis on Vimeo.

How to unlock Huawei E220 HSDPA Modem

2009-02-15T13:40:00.000+05:30

Came across this post through Azeez's blog.
Read Seejay's blog to find out how you can unlock a network locked Huawei E220 HSDPA Modem.

Creating source distribution in a componentized project

2009-02-14T12:57:00.003+05:30

In a componentized project, functionality comes from different components. A full source distribution for such project needs to contain sources from each of these components. Hence it's somewhat challenging when it comes to creating a source distribution.

In my article Here, you can find out one possible solution.

Challenges faced when building componentized projects

2009-02-14T12:10:00.014+05:30

(More updated version may be available here.)
Using pluggable componentized model to add up features to a project has many advantages. But it sure does introduce many challenges at every aspect of development, marketing, pricing, etc. I have been closely associated with new componentized model(OSGi based) we recently announced @ WSO2. While there can be many, here are some of the evident challenges we faced when creating the distributions.

Assembling/packaging different components together

Functionality comes from components located under different svn paths. We need to identify the set of components needed by a product, their dependencies & pack them together.

Creating full source distribution
Since functionality is coming from different components, source from each of these component needs to be included in the full source distribution.

Creating documentation distribution
Each component has a documentation of its own. This documentation need to be included & there should be product specific master documentation which glues together these component specific documentation.

The images inside these components should come with minimal UI theming effects, as UI theming can be drastically different in two projects using the same component.

Although the documentation is coming from different components written by different authors, the ultimate user/reader of this full documentation pack should not feel that difference when moving between documentation of multiple components.

A consistent UI theming should be applied to all the documentation coming from different components.

Packaging Licenses together
Each component can contain dependencies to third party libraries having compatible licenses. We should be cable of providing this full license list, if an end user requests for it.

Keeping final product size under control
Maintaining the self sustainability & avoid duplication is an art to master. Too much duplication will send the size of the final distribution sky-high.

Blog on Enterprise Content Management

2009-02-14T10:11:00.002+05:30

Amila Hendahewa is a good friend of mine & a very enthusiastic scout. We went to same school & have been in constant touch ever since.
He is in to ECM(Enterprise Content Management) & recently started a blog on the subject. I am sure it will be of great source of experience in time to come.

Installing Firefox 3.0 on Ubuntu 7.10

2009-02-11T14:49:00.003+05:30

Found out about Ubiquity add-on to Firefox from Dhanushka. But when I tried to install it, Firefox started complaining about an older browser version. So I had to upgrade to a new Firefox version. So I did,

apt-get install firefox-3.0

But the installation got crashed in the middle giving following error.

firefox-3.0: Depends: xulrunner-1.9 (>= 1.9.0.1) but it is not going to be installed

Googling for a solution did not provide any satisfactory solutions, hence I started to figure out a away out by myself. Apparently xulrunner-1.9.0.1 is being used by 'yelp' package on Ubuntu Gutsy. Since I do not use yelp (GNOME online help browser application) at all, I decided to remove it.

apt-get remove yelp

And then I tried installing Firefox 3.0. WHOA... It worked!!!!..
Hope someone finds this useful.

Making good SOA great in a tight economy

2009-02-10T10:55:00.004+05:30

The economic crisis has surely hit your IT budget & you need to find cheaper alternatives to get going during these tough times. If your applications are based on SOA, recent Open Source(Apache licensed) offerings from WSO2 might be of high interest to you.

WSAS 3.0 (Web Services Application Server)
WSAS 3.0 succeeds award winning WSAS 2.3. WSAS is the first application server to support the full XML-based WS-* stack and REST in a simple, lightweight model that makes it easier to develop, deploy, manage, and monitor Web services. It's based on Apache Axis2, one of the fastest web services engines in the world.

ESB 2.0 (Enterprise Service Bus)
WSO2 ESB is an ultra fast, light-weight enterprise-ready open source product based on the Apache Synapse ESB. With WSO2 ESB, you can connect, manage and transform service interactions between Web services, REST/POX services and legacy systems. ESB v2.0 succeeds the highly used, much appreciated v1.7.1.

Registry 2.0
WSO2 Registry is an enterprise-ready open source product for governing SOA deployments. With WSO2 Registry, you can store, catalog, index and manage your enterprise metadata in a simple and scalable model.
The current version has major performance improvements over the last public release(v1.1).

BPS 1.0 (Business Process Server)
BPS is the newest addition to WSO2's family of open source offerings. It is an easy-to-use open source business process server that executes business processes written using the WS-BPEL standard. It is powered by Apache ODE and provides a complete Web-based graphical console to deploy, manage and view processes in addition to managing and viewing process instances.

All these latest releases are based on a new OSGi based platform called 'Carbon' on which I have blogged here. If your interested to know more about 'WSO2 Carbon', Paul & Samisa has nice explanations too.

How to build a UI component for WSO2 Carbon?

2009-02-07T10:54:00.003+05:30

This is the first one in a series of tutorials explaining how to build UI components to be deployed on top of WSO2's carbon platform. You can access the full content from here. Feel free to try out & send in your comments.

Formatting is bit weird :-(.The WYSIWYG editor is not keeping up to it's promises.

Coming up in the series,

How to add a menu item?
Accessing Services deployed
How to do AJAX calls?
How to register a service?
How to i18n your pages?
How to add a breadcrumb?

How to get rid of spaces when adding a table to blogger?

2009-02-07T08:29:00.003+05:30

If you ever try adding a HTML table to blogger, chances are very much that you will see a huge spacing between last line of text and start of the table. And if you take a look @ the HTML source you will see large number of <br/> tags being placed in.

The blogger template seems to be adding these <br/> tags for each occurrence we have a line break in our HTML code for the table.

Solution : Go to 'Edit Html' view & remove all line breaks, save & see how it looks.

P.S. : 'Preview' will not show these <br/> tags.

Component based product buidling

2009-02-07T00:44:00.017+05:30

"Kulu Ganga Wana Nivahana" - Camping Site, Knukles Mountain Range - Sri Lanka

1	2	3
4	5	6
7	8	9
10	11	12

Corbet’s Gap - Sri Lanka

2009-02-07T00:02:00.011+05:30

Corbet’s Gap - ("Attala-mettuwa" in Sinhalese) named after the Englishman who discovered it, is a vast split in the range of Kandyan hills.
On a clear day an amazing view is possible through this gap. There are quite a few trekking trails available in the area as well. Corbet's View Finchamsland is one of the lodging places situated in the area.
Myself & bunch of friends visited there recently & I was quite attracted to the silence & wilderness in the surrounding jungle. It's an ideal place to visit if you love wild life,nature & traveling. Here are some of the memories brought back in a SD card.

The road

The entrance

The view

If your interested in finding more details on activities, accommodation,etc Lakdasun is a good starting place.