Contract-First Web Services in practice - part 1 of 5.

I've recently been working on a SaaS project where I'm integrating different ISVs (Independent Software Vendors) into a SaaS aggregation platform with a market place environment. Users come to the marketplace, purchase subscriptions to the ISVs' applications and realtime provisioning is expected. An exciting opportunity to test a SOA paradigm as SOAP web-services are a crucial medium of the SOA model and was the first solution to jump to my mind.

Once my approach was selected, my next task was to define a service specification that I would expect the ISVs to implement. This started out as a document with a list of methods and attributes encompassing the functionality I would expect the ISVs to expose. Once this was done, I needed to create a reference implementation in .NET, Java and PHP, and that's where the problems began.

My aim was to create a single WSDL which could easily be implemented in the 3 main programming languages so that only a single client is written. The main issues I had to address are summarized below:

• Creating a consistent doc/literal format style WSDL
• Working effectively with arrays in XSD schema
• Working effectively with typed exceptions - what is the best way of creating Soap Faults?
• JAX-WS, JAX-RPC, Axis, .NET ASMX, WCF, PHP-SOAP, NU-SOAP - just a snapshot of the different SOAP stacks in common use.
• Contract-First SOA

I'll be detailing the challenges I faced in a series of 5 tutorials on how to create a non-trivial WSDL and implement it in the 3 programming languages.

Joomla! Displaying Links to Blogger posts.

My website, http://www.kimenye.com, is powered by Joomla! (one of the world's most popular open source content management system). Not being a fun of its usability as a blogging platform, I made the decision to host this blog in Google's Blogger service. Having done that, I found I needed to display links to my blog from my website in a dynamic manner, that didn't require me to manually add links when I add new content. As part of my website, I have a panel where I intend to display the Titles and Links to the blog posts. This article describes how I achieved this.

Pre-Requisites

1. Apache Web Server (preferably 2.0)
2. PHP 5
3. Zend GData PHP Library

Configuration of Zend PHP Library

We will be connecting to the Google Data Services to retrieve posts for the blog. This is done using the Zend GData PHP Library.  See this article on the Google Code pages with detailed information on how to configure the libary. There's also a video on the page by one of Google's Software Engineers with step by step instructions. If you are on shared hosting, your mileage may vary following those instructions. I'll outline the failed instructions briefly.

1. Upload the Zend Framework library. From the unpacked archive, we need everything in the folder library (i.e the folder XXX\library\Zend).
2. Edit the .htaccess file to include the Zend Library folder in the include path. 

This did not work for me, as my hosting provider has moved to using PHPSUEXEC and hence don't allow any php directives in the .htaccess file. These have to be moved to a php.ini file created in your public_html folder. I did this, however, this php.ini file was not getting parsed and hence my includes were failing. I checked my configuration using the phpinfo() method and saw that new php.ini file was not being parsed and hence the include_path variable was not being updated.

Using the Zend GData Library

view plaincopy to clipboardprint?

1. <?php
4. // no direct access
5. defined('_JEXEC') or die('Restricted access');
7. //This is the path of where you have uploaded your Zend GData library
8. $path = '/home/your_user_name/public_html/libraries/';
10. //This method updates the include path to include the Zend GData libaries
11. //This is useful if you are in a shared hosting environment where it is not
12. //easy to update the include path
13. set_include_path(get_include_path() . PATH_SEPARATOR . $path);
15. //require the Zend Loader file
16. require_once (dirname(__FILE__).DS.'Zend'.DS.'Loader.php');
18. /**
19. * @see Zend_Gdata
20. */
21. Zend_Loader::loadClass('Zend_Gdata');
23. /**
24. * @see Zend_Gdata_Query
25. */
26. Zend_Loader::loadClass('Zend_Gdata_Query');
28. /**
29. * @see Zend_Gdata_ClientLogin
30. */
31. Zend_Loader::loadClass('Zend_Gdata_ClientLogin');
33. /**
34. * This is the class to use to connect to the Google Blogger Service
35. */
36. class BloggerHelper
37. {
38. function getList()
39. {
40. $count = 5; //this is the number of items to be returned
41. $blogId = 123456789; //this is your blog id
43. $email = 'someone@gmail.com'; //This is the email to log into Blogger
44. $password = 'password'; //This is the password to log into Blogger
46. $client = Zend_Gdata_ClientLogin::getHttpClient($email, $password, 'blogger');
47. $gdClient = new Zend_Gdata($client);
49. $query = new Zend_Gdata_Query('http://www.blogger.com/feeds/' . $blogId . '/posts/default');
50. $query->setMaxResults($count);
51. $feed = $gdClient->getFeed($query);
53. $i = 0;
54. $lists = array();
55. foreach($feed->entries as $entry) //This is a temp hack to remove drafts
56. {
57. $lists[$i]->link = htmlspecialchars($entry->link[1]->text);
58. $lists[$i]->text = $entry->title->text;
59. $i++;
60. }
62. return $lists;
63. }
64. }

<?php // no direct access defined('_JEXEC') or die('Restricted access'); //This is the path of where you have uploaded your Zend GData library $path = '/home/your_user_name/public_html/libraries/'; //This method updates the include path to include the Zend GData libaries //This is useful if you are in a shared hosting environment where it is not //easy to update the include path set_include_path(get_include_path() . PATH_SEPARATOR . $path); //require the Zend Loader file require_once (dirname(__FILE__).DS.'Zend'.DS.'Loader.php'); /** * @see Zend_Gdata */ Zend_Loader::loadClass('Zend_Gdata'); /** * @see Zend_Gdata_Query */ Zend_Loader::loadClass('Zend_Gdata_Query'); /** * @see Zend_Gdata_ClientLogin */ Zend_Loader::loadClass('Zend_Gdata_ClientLogin'); /** * This is the class to use to connect to the Google Blogger Service */ class BloggerHelper { function getList() { $count = 5; //this is the number of items to be returned $blogId = 123456789; //this is your blog id $email = 'someone@gmail.com'; //This is the email to log into Blogger $password = 'password'; //This is the password to log into Blogger $client = Zend_Gdata_ClientLogin::getHttpClient($email, $password, 'blogger'); $gdClient = new Zend_Gdata($client); $query = new Zend_Gdata_Query('http://www.blogger.com/feeds/' . $blogId . '/posts/default'); $query->setMaxResults($count);<br /> $feed = $gdClient->getFeed($query);<br /> <br /> $i = 0;<br /> $lists = array();<br /> foreach($feed->entries as $entry) //This is a temp hack to remove drafts<br /> {<br /> $lists[$i]->link = htmlspecialchars($entry->link[1]->text);<br /> $lists[$i]->text = $entry->title->text;<br /> $i++;<br /> }<br /> <br /> return $lists;<br /> }<br />}

The code shown above requires you to know your blog id. This can be obtained by looking at the request parameters in the url for your blog. A simple way to find this is to log into your Blogger account, click on the blog you want to access and click on the settings link. Now look at the url on your blog. It should have something like "http://www.blogger.com/blog-options-basic.g?blogID=1234567890". The numbers after the string "blogID=" is your blog id. 

The getList() method returns an array of objects with the link as the url to the blog post and the text property as the title of the blog. This array of items is then used by a Joomla! module to display this information on the screen.

I've not found an elegant way to distinguish the drafts from the published posts, hence the current way of using the URL will have to do for now. In the future I intend to add some additional logic to do searching based on key words, so it works like an aggregator as well as functionality to display posts published in the last 30 days. Hope this is a nice starting place.

Google App Engine Wicket ModificationWatcher

What is the ModificationWatcher?

ModificationWatcher is a Wicket class that checks to see if resources have been modified and reloads them without restarting the servlet container (where possible). This is made possible by setting up a polling frequency in the init() method of the WebApplication class for your project. Unfortunately, Google App Engine does not allow the spawning of new threads (on which the ModificationWatcher class depends on), hence a work-around is necessary. First we need to look at the WicketApplication class, as shown in the example below:

Example WicketApplication Class

view plaincopy to clipboardprint?

1. import org.apache.wicket.Application;
2. import org.apache.wicket.protocol.http.HttpSessionStore;
3. import org.apache.wicket.protocol.http.WebApplication;
4. import org.apache.wicket.protocol.http.WebRequest;
5. import org.apache.wicket.session.ISessionStore;
6. import org.apache.wicket.util.lang.PackageName;
8. public class WicketApplication extends WebApplication {
10. @Override
11. protected void init() {
12. super.init();
13. // getResourceSettings().setResourcePollFrequency(null);
14. }
16. protected WebRequest newWebRequest(HttpServletRequest servletRequest) {
17. getResourceSettings().getResourceWatcher(true).start(
18. getResourceSettings().getResourcePollFrequency());
19. return super.newWebRequest(servletRequest);
20. }
22. @Override
23. public String getConfigurationType() {
24. return Application.DEVELOPMENT;
25. // return Application.DEPLOYMENT;
26. }
28. @Override
29. protected ISessionStore newSessionStore() {
30. return new HttpSessionStore(this);
31. // return new SecondLevelCacheSessionStore(this, new DiskPageStore());
32. }
33. }

<br />import org.apache.wicket.Application;<br />import org.apache.wicket.protocol.http.HttpSessionStore;<br />import org.apache.wicket.protocol.http.WebApplication;<br />import org.apache.wicket.protocol.http.WebRequest;<br />import org.apache.wicket.session.ISessionStore;<br />import org.apache.wicket.util.lang.PackageName;<br /><br />public class WicketApplication extends WebApplication {<br /><br /> @Override<br /> protected void init() {<br /> super.init();<br /> // getResourceSettings().setResourcePollFrequency(null);<br /> }<br /><br /> protected WebRequest newWebRequest(HttpServletRequest servletRequest) {<br /> getResourceSettings().getResourceWatcher(true).start(<br /> getResourceSettings().getResourcePollFrequency());<br /> return super.newWebRequest(servletRequest);<br /> }<br /><br /> @Override<br /> public String getConfigurationType() {<br /> return Application.DEVELOPMENT;<br /> // return Application.DEPLOYMENT;<br /> }<br /><br /> @Override<br /> protected ISessionStore newSessionStore() {<br /> return new HttpSessionStore(this);<br /> // return new SecondLevelCacheSessionStore(this, new DiskPageStore());<br /> }<br />}<br /><br />

The first 'hack' to getting Wicket working in GAE is to set the session store type for Wicket to HttpSessionStore, instead of using the SecondLevelCacheSessionStore as the latter directly refers to the file system and is not allowed in the app engine sandbox. This is done by overriding the newSessionStore() method [line 29 in the code extract shown above].

Now, back to the ModificationWatcher. If you want to completely disable it, you can set the poll frequency to null in the init method of your WebApplication class [this has been commented out in line 13 above]. However, if you are in development, this will mean that any changes to the source pages [in my experience its really been the HTML pages], you will have to restart your servlet container to pick-up the changes, which is not very efficient for development purposes. This work-around by-passes the polled watching for file changes and checks to see if the file has changed every time it is requested for. This is achieved by overriding the newWebRequest() method [see line 16 above]. This is fine for development purposes, and MUST be disabled for production, for obvious reasons. 

And for this to work, you need a version of the ModificationWatcher class that does not spawn any threads. ModificationWatcher was declared a final class and hence cannot be extended, and so you need to override the Wicket provided version with one of your own. This is easily achieved by creating a similar package structure in your source code and creating the file : org.apache.wicket.util.watch.ModificationWatcher. The code for this class is as below:

ModificationWatcher

view plaincopy to clipboardprint?

1. /**
2. * Monitors one or more <code>IModifiable</code> objects, calling a
3. * {@link IChangeListener IChangeListener} when a given object's modification time changes.
4. *
5. * @author Jonathan Locke
6. * @since 1.2.6
7. */
8. public final class ModificationWatcher
9. {
10. /** logger */
11. private static final Logger log = LoggerFactory.getLogger(ModificationWatcher.class);
13. /** maps <code>IModifiable</code> objects to <code>Entry</code> objects */
14. private final Map modifiableToEntry = new ConcurrentHashMap();
16. /** the <code>Task</code> to run */
17. //private Task task;
19. /**
20. * Container class for holding modifiable entries to watch.
21. */
22. private static final class Entry
23. {
24. // The most recent lastModificationTime polled on the object
25. Time lastModifiedTime;
27. // The set of listeners to call when the modifiable changes
28. final ChangeListenerSet listeners = new ChangeListenerSet();
30. // The modifiable thing
31. IModifiable modifiable;
32. }
34. /**
35. * Default constructor for two-phase construction.
36. */
37. public ModificationWatcher()
38. {
39. }
41. /**
42. * Constructor that accepts a <code>Duration</code> argument representing the poll frequency.
43. *
44. * @param pollFrequency
45. * how often to check on <code>IModifiable</code>s
46. */
47. public ModificationWatcher(final Duration pollFrequency)
48. {
49. start(pollFrequency);
50. }
52. /**
53. * Adds an <code>IModifiable</code> object and an <code>IChangeListener</code> object to
54. * call when the modifiable object is modified.
55. *
56. * @param modifiable
57. * an <code>IModifiable</code> object to monitor
58. * @param listener
59. * an <code>IChangeListener</code> to call if the <code>IModifiable</code> object
60. * is modified
61. * @return <code>true</code> if the set did not already contain the specified element
62. */
63. public final boolean add(final IModifiable modifiable, final IChangeListener listener)
64. {
65. // Look up entry for modifiable
66. final Entry entry = (Entry)modifiableToEntry.get(modifiable);
68. // Found it?
69. if (entry == null)
70. {
71. if (modifiable.lastModifiedTime() != null)
72. {
73. // Construct new entry
74. final Entry newEntry = new Entry();
76. newEntry.modifiable = modifiable;
77. newEntry.lastModifiedTime = modifiable.lastModifiedTime();
78. newEntry.listeners.add(listener);
80. // Put in map
81. modifiableToEntry.put(modifiable, newEntry);
82. }
83. else
84. {
85. // The IModifiable is not returning a valid lastModifiedTime
86. log.info("Cannot track modifications to resource " + modifiable);
87. }
89. return true;
90. }
91. else
92. {
93. // Add listener to existing entry
94. return entry.listeners.add(listener);
95. }
96. }
98. /**
99. * Removes all entries associated with an <code>IModifiable</code> object.
100. *
101. * @param modifiable
102. * an <code>IModifiable</code> object
103. * @return the <code>IModifiable</code> object that was removed, else <code>null</code>
104. */
105. public IModifiable remove(final IModifiable modifiable)
106. {
107. final Entry entry = (Entry)modifiableToEntry.remove(modifiable);
108. if (entry != null)
109. {
110. return entry.modifiable;
111. }
112. return null;
113. }
115. /**
116. * Starts watching at a given <code>Duration</code> polling rate.
117. *
118. * @param pollFrequency
119. * the polling rate <code>Duration</code>
120. */
121. public void start(final Duration pollFrequency)
122. {
123. // Construct task with the given polling frequency
124. //task = new Task("ModificationWatcher");
126. //task.run(pollFrequency, new ICode()
127. //{
128. // public void run(final Logger log)
129. // {
130. // Iterate over a copy of the list of entries to avoid
131. // concurrent
132. // modification problems without the associated liveness issues
133. // of holding a lock while potentially polling file times!
134. for (final Iterator iterator = new ArrayList(modifiableToEntry.values()).iterator(); iterator
135. .hasNext();)
136. {
137. // Get next entry
138. final Entry entry = (Entry)iterator.next();
140. // If the modifiable has been modified after the last known
141. // modification time
142. final Time modifiableLastModified = entry.modifiable.lastModifiedTime();
144. if (modifiableLastModified.after(entry.lastModifiedTime))
145. {
146. // Notify all listeners that the modifiable was modified
147. entry.listeners.notifyListeners();
149. // Update timestamp
150. entry.lastModifiedTime = modifiableLastModified;
151. }
152. }
153. // }
154. //});
155. }
157. /**
158. * Stops this <code>ModificationWatcher</code>.
159. */
160. public void destroy()
161. {
162. //if (task != null)
163. //{
164. // task.stop();
165. // task.interrupt();
166. //}
167. }
169. /**
170. * Retrieves a key set of all <code>IModifiable</code> objects currently being monitored.
171. *
172. * @return a <code>Set</code> of all <code>IModifiable</code> entries currently maintained
173. */
174. public final Set getEntries()
175. {
176. return modifiableToEntry.keySet();
177. }
178. }

/**<br /> * Monitors one or more <code>IModifiable</code> objects, calling a<br /> * {@link IChangeListener IChangeListener} when a given object's modification time changes.<br /> * <br /> * @author Jonathan Locke<br /> * @since 1.2.6<br /> */<br />public final class ModificationWatcher<br />{<br /> /** logger */<br /> private static final Logger log = LoggerFactory.getLogger(ModificationWatcher.class);<br /><br /> /** maps <code>IModifiable</code> objects to <code>Entry</code> objects */<br /> private final Map modifiableToEntry = new ConcurrentHashMap();<br /><br /> /** the <code>Task</code> to run */<br /> //private Task task;<br /><br /> /**<br /> * Container class for holding modifiable entries to watch.<br /> */<br /> private static final class Entry<br /> {<br /> // The most recent lastModificationTime polled on the object<br /> Time lastModifiedTime;<br /><br /> // The set of listeners to call when the modifiable changes<br /> final ChangeListenerSet listeners = new ChangeListenerSet();<br /><br /> // The modifiable thing<br /> IModifiable modifiable;<br /> }<br /><br /> /**<br /> * Default constructor for two-phase construction.<br /> */<br /> public ModificationWatcher()<br /> {<br /> }<br /><br /> /**<br /> * Constructor that accepts a <code>Duration</code> argument representing the poll frequency.<br /> * <br /> * @param pollFrequency<br /> * how often to check on <code>IModifiable</code>s<br /> */<br /> public ModificationWatcher(final Duration pollFrequency)<br /> {<br /> start(pollFrequency);<br /> }<br /><br /> /**<br /> * Adds an <code>IModifiable</code> object and an <code>IChangeListener</code> object to<br /> * call when the modifiable object is modified.<br /> * <br /> * @param modifiable<br /> * an <code>IModifiable</code> object to monitor<br /> * @param listener<br /> * an <code>IChangeListener</code> to call if the <code>IModifiable</code> object<br /> * is modified<br /> * @return <code>true</code> if the set did not already contain the specified element<br /> */<br /> public final boolean add(final IModifiable modifiable, final IChangeListener listener)<br /> {<br /> // Look up entry for modifiable<br /> final Entry entry = (Entry)modifiableToEntry.get(modifiable);<br /><br /> // Found it?<br /> if (entry == null)<br /> {<br /> if (modifiable.lastModifiedTime() != null)<br /> {<br /> // Construct new entry<br /> final Entry newEntry = new Entry();<br /><br /> newEntry.modifiable = modifiable;<br /> newEntry.lastModifiedTime = modifiable.lastModifiedTime();<br /> newEntry.listeners.add(listener);<br /><br /> // Put in map<br /> modifiableToEntry.put(modifiable, newEntry);<br /> }<br /> else<br /> {<br /> // The IModifiable is not returning a valid lastModifiedTime<br /> log.info("Cannot track modifications to resource " + modifiable);<br /> }<br /><br /> return true;<br /> }<br /> else<br /> {<br /> // Add listener to existing entry<br /> return entry.listeners.add(listener);<br /> }<br /> }<br /><br /> /**<br /> * Removes all entries associated with an <code>IModifiable</code> object.<br /> * <br /> * @param modifiable<br /> * an <code>IModifiable</code> object<br /> * @return the <code>IModifiable</code> object that was removed, else <code>null</code><br /> */<br /> public IModifiable remove(final IModifiable modifiable)<br /> {<br /> final Entry entry = (Entry)modifiableToEntry.remove(modifiable);<br /> if (entry != null)<br /> {<br /> return entry.modifiable;<br /> }<br /> return null;<br /> }<br /><br /> /**<br /> * Starts watching at a given <code>Duration</code> polling rate.<br /> * <br /> * @param pollFrequency<br /> * the polling rate <code>Duration</code><br /> */<br /> public void start(final Duration pollFrequency)<br /> {<br /> // Construct task with the given polling frequency<br /> //task = new Task("ModificationWatcher");<br /><br /> //task.run(pollFrequency, new ICode()<br /> //{<br /> // public void run(final Logger log)<br /> // {<br /> // Iterate over a copy of the list of entries to avoid<br /> // concurrent<br /> // modification problems without the associated liveness issues<br /> // of holding a lock while potentially polling file times!<br /> for (final Iterator iterator = new ArrayList(modifiableToEntry.values()).iterator(); iterator<br /> .hasNext();)<br /> {<br /> // Get next entry<br /> final Entry entry = (Entry)iterator.next();<br /><br /> // If the modifiable has been modified after the last known<br /> // modification time<br /> final Time modifiableLastModified = entry.modifiable.lastModifiedTime();<br /><br /> if (modifiableLastModified.after(entry.lastModifiedTime))<br /> {<br /> // Notify all listeners that the modifiable was modified<br /> entry.listeners.notifyListeners();<br /><br /> // Update timestamp<br /> entry.lastModifiedTime = modifiableLastModified;<br /> }<br /> }<br /> // }<br /> //});<br /> }<br /><br /> /**<br /> * Stops this <code>ModificationWatcher</code>.<br /> */<br /> public void destroy()<br /> {<br /> //if (task != null)<br /> //{<br /> // task.stop();<br /> // task.interrupt();<br /> //}<br /> }<br /><br /> /**<br /> * Retrieves a key set of all <code>IModifiable</code> objects currently being monitored.<br /> * <br /> * @return a <code>Set</code> of all <code>IModifiable</code> entries currently maintained<br /> */<br /> public final Set getEntries()<br /> {<br /> return modifiableToEntry.keySet();<br /> }<br />}

All that has been done is that the Threading references have been taken out so no new threads are spawned and the check for the resources is done in the same thread as that the one that is servicing the user request. This is not the most elegant solution, but it works.

This hack is based on the following sources: 

1. http://stronglytypedblog.blogspot.com/2009/04/wicket-on-google-app-engine.html
2. http://lborupj.blogspot.com/2009/04/wicket-on-google-app-engine.html