Status Report for 11/28/2007 -- 12/26/2007

Progress

1. Design for server side (updated)

To do it step by step, we first design the client side for one user who can register/monitor one variable/sensor.

The basic idea is that write new data to an XML file when it is available one the server side and return the file’s URL to the client.

2. Implementation

At current stage, logic in server side is not difficult. A program written by JSP, which is called by client, would do the following steps:

• Register a variable based on the parameters from client
• Write the new data to an XML file
• Return the URL of the file to the client

Discussion

1. Register multiple sensors for one user

The user first registers variable called A-var, the server can get the new coming data from some callback function. After that, he wants to register another variable B-var, is that true that we can obtain both data from the same function? There is a problem in register/unregister.

2. Register multiple sensors for multiple users

If multiple users register the same variable, which is better, the arrary or file? How to maintain the session?

Reference

• Bulletproof Ajax, Keith, Jeremy
• Ajax in practice, Crane, Dave
• JSP : the complete reference, Hanna, Phil
• JavaScript bible, Goodman, Danny
• Ajax for web application developers, Hadlock, Kris
• Ajax design patterns, Mahemoff, Michael
• Ajax : creating Web pages with asynchronous JavaScript and XML, Woychowsky, Edmond
• Ajax bible, Holzner, Steven

Status Report for 10/31/2007 -- 11/28/2007

Progress

1. Design for Ajax-based client side (updated)

To do it step by step, we first design the client side for one user who can register/monitor one variable/sensor.

The basic idea is that fetch an XML file when it is available one the server side, parse its contents, and generate markup to display the content details in the Web page.

2. Implementation

In the previous implementation, we set up the communication between client and server by calling the basic functions provided by JavaScript.

(1) Creating a new instance of an XMLHttpRequest object (xhr = new XMLHttpRequest()) is used to make a Http request from client to server. In most browsers, it is very straight-forward, but in IE, we need to create a new instance of an ActiveX object. If xhr equals to false, it means that the browser executing the script does not have Ajax Capabilities. Otherwise, you will get an instance of XMLHttpRequest or ActiveXObject.

In addition, in the callback function which is an attched function to the onreadystatchange event handler, we need to check the status properly to make sure that the response was successfully returned from the server before we process the data from the server.

In a word, we have to take care of a lot of details if we do it from sctatch. So there are a few of open source toolkits that can make our lives a bit easier.

In Ajax.Request class provided by Prototype, all we have to provide is a URL, the HTTP method, and a call back function. Internally, Ajax.Request works out how to create the XHR object and fills in the blanks when shaping the request. It also simplifies the callback semantics considerably, allowing us to supply a funtion that would only be called once on completion of the response, and that therefore only have to deal with the application logic.

(2) In this application, we wish to obtain and display data from the server when they are available, so we need to keep the data constantly up-to-date. The Prototype Ajax.PeriodicalUpdater class can automatically keep the display of the server-provided data up-to-date at a frequency that best suits our page. So it is easy for us to schedule the automatic updates without having to code any scheduling logic or write timeout handlers.

(3) For simplicity, currently on the server side we just write the new data to a file while the client side keeps polling the request to read the file and process it.

Discussion

In using Prototype Ajax.PeriodicalUpdater class, we have to take the following factors into account :

• Choosing what to automatically update and at what interval
• The expected number of simultaneous visitors
• The size of the update response
• The process required to generate that response
• The load-handling capability of the server configuration

Reference

• Bulletproof Ajax, Keith, Jeremy
• Ajax in practice, Crane, Dave
• JSP : the complete reference, Hanna, Phil
• JavaScript bible, Goodman, Danny
• Ajax for web application developers, Hadlock, Kris
• Ajax design patterns, Mahemoff, Michael
• Ajax : creating Web pages with asynchronous JavaScript and XML, Woychowsky, Edmond
• Ajax bible, Holzner, Steven

Status Report for 10/17/2007 -- 10/31/2007

Progress

Prototype architecture for new CIMAClient project

1. Main Frame

2. Implementation of some parts

On server side, main steps for (1), (2) and (3) shown in above figure:

(1) Start the server;

(2) Start handling CIMA http/REST requests

• interact with a CIMA service
• get data from a handle (registration)
• Status information about gateway

(3) Get stream data in "myHandler" method in src/edu/indiana/extreme/www/cima/channel/sink/Channel_sinkImpl.java

On client side, main steps for (4) shown in above figure:

(1) Create a new instance of XMLHttpRequest;

(2) If successful, attach a function to the onreadystatchange event handler, that function will be executed EVERY TIME the server pings the client with an update. The function will be executed every time the readystatechange event is triggered;

In this function,

(a) Check the status properly to make sure that the response was successfully returned from the server;

(b) Extract data from parcel(an XML file);

(c) Generate content;

(d) Create the text for SensorName, TimeStamp and DoubleData to go inside the paragraph;(e) Join these nodes together, and insert the markup just created;

(3) Send the request;

Reference

• Bulletproof Ajax, Keith, Jeremy
• Ajax in practice, Crane, Dave
• JSP : the complete reference, Hanna, Phil
• JavaScript bible, Goodman, Danny
• Ajax for web application developers, Hadlock, Kris
• Ajax design patterns, Mahemoff, Michael
• Ajax : creating Web pages with asynchronous JavaScript and XML, Woychowsky, Edmond
• Ajax bible, Holzner, Steven

Status Report for 09/12/2007 -- 10/17/2007

Progress

Prototype architecture for Ajax-based CIMAClient project

Summary of the comparison with different main frames

Figure 1

Figure 1 shows the traditional model of the web application. For the CIMA web services architecture, once the CIMA sensors have been registered to the client machine, the data stream will be kept sending to the web service in CIMA Client, but the data cannot reach client machine automatically. So the client needs to send the HTTP request to retrieve the data manually. In this case, the method of sending data with respect to the user’s actions and the method of sending data with respect to the browser’s actions are synchronous, so we cannot get the real time data. On the other hand, the server will send back the entire page. Rinse and repeat.

Figure 2

Figure 2 shows the Ajax model based on polling. For the CIMA web services architecture, first the client needs to send register request to ask the CIMA web service to send the variable data just registered to the web service in CIMA Client, then a periodic polling will be done by the Ajax layer, and the client can get the updated data. In this model, the method of sending data with respect to the user’s actions is asynchronous while the method of sending data with respect to the browser’s actions is still synchronous. The advantage is that user interface is never blocked and the bandwidth usage is potentially lower than the traditional model even if it is still high.

In our research work, we found that there were some “smarter” Ajax models which have the features of high update frequency, true real time, low bandwidth usage and low load on the network infrastructure. For example, Ajax-Comet model based on smart polling which is asynchronous polling and the variable polling frequency is controlled by server; another example is Ajax-Comet model based on streaming which implements fully asynchronous push. We might employ all the Ajax models to our application and find out which is the best according to our data features.

Status Report for 08/29/2007 -- 09/12/2007

Progress

Prototype architecture for new CIMAClient project

1. Main Frame

2. Requirements

Get data from CIMA instruments/CIMA web services, and show them on an Ajax-based browser. The main difference between this project and the Atom Feed for IUMSC is as follows:

(1) In “Atom Feed” project, the users can browse the data they requested at anytime at anywhere as long as they have a standard Atom Feed Reader and the URL for the feed(s), and CIMAClientPlatform acts as an adaptor. Since we cannot set the frequency of incoming data in almost all Atom Feed reader at browser (as far as I know), the main frame is called “push model”. So it is not a real-time monitor, and there might have some data missed;

(2) In this project, the users need to monitor near real-time data, so they need to send http requests at some rate to get data instead of getting date passively. So the main frame is called "pull model". Additionally, Ajex can make user interface more friendly and fancy.

3. Work Flow

(1) Startup CIMAClientPlatform to send register requests for all variables/sensors to CIMA web service no matter if the users need them or not;

(2) CIMA web service will send stream data() back including all the registered data;

(3) Users send Http requests to Web service interface at anytime. A http request includes session ID, sensor data and how many data the user-side wants to get(the most recent one or all);

(4) Web service interface sends the data back to the browser.

4. Problems

(1) It is supposed to be used by multiple users, so is there any limitation for the number of users because we cannot assume the data column is small enough for server to process ( The user might request some ccd image and the transfer rate is in the range of 4K to 8M/sec);

(2) The problem in garbage collection in Javascript;

(3) Data format:

The format of return data should be XML or JSON or both. Ajax-based client application can accept JSON much easier than XML.

Discussion

In “Problems” section

Status Report for 08/08/2007 -- 08/29/2007

Progress

Draft of prototype architecture for new project

1. Main Frame

2. Requirements

Get data from CIMA instruments/CIMA web services, and show them on an Ajax-based browser.

(Difference between this with Atom Feed for IUMS? any special requirements or restrictions for that?)

3. Work Flow

(1) Users send HTTP requests to register a variable(Bay1Temp, for example) to be shown in the browser(for example, http://shim.addr/register?Name=[cima_service]%Source=[variable_name])

(2) Users get a reply with the user ID(for example, http://shim.addr/UUID). This UUID will be used to identify different users and get data from the web service;

(3) Web service interface will create a thread for that request/user;

(4) Web service interface(or The thread?) then send the register request to CIMA web service (with UUID?)(Socket);

(5) CIMA web service will return stream data() back(Socket);

(6) The parcel will be stored in a ring buffer in SVC(???);

(7) Users send Poll URL to Web service interface(or The thread?) (when will it happen?);

(8) Web service interface(or The thread?) sends the data back to the browser.

4. Problems

(1) It is supposed to be used by multiple users, so is there any limitation for the number of users because we cannot assume the data column is small enough for server to process( The user might request some ccd image and the transfer rate is in the range of 4K to 8M/sec);

(2) Compare the following two cases in how to distinguish clients:

(a) Create one thread for every registration;

(b) One service with IDs to distinguish clients;

- Distinguish which parcel goes to which client

- Overhead matching IDs to parcels(?)

(3) Data format:

The format of return data should be XML or JSON or both. Ajax-based client application can accept JSON much easier then XML.

Discussion

In “Problems” section

Status Report for 08/01/2007 -- 08/08/2007

Progress

Summary of data structure design for IUMSC 12 sensors Atom Feed

1. Requirements

Create Atom Feed for IUMSC 12 sensors

• All sensors’ data in one feed
• Each sensor data in one feed
• Out-of-Range data from all sensors in one feed

2. Work Flow

Implementation is in Method void myHandler(String parcel) in Class Channel_sinkImpl in Package edu.indiana.extreme.www.cima.channel.sink.

(1) Read and parse the parcel and get SensorName, TimeStamp and DoubleData for the incoming sensor data;

(2) Find the SensorStorage for this parcel and update the according SensorStorage with the current parcel;

(3) Update the all-in-one SensorStorage with the current parcel;

(4) Update the SensorStorage for out-of-range data with the current parcel;

3. Data Structure Design

• Super Class

abstract class CachedSensorStorage {

private Feed createDefaultFeed; // Create "feed" part of a feed -- Author, Id, Title, Link

private Entry createDefaultEntry // Create "entry" part of a feed -- Author, Title, Id, Updated

protected abstract void updateCachedEntryContent;

protected abstract String generateContentString;

public boolean updateData {

Call Method createDefaultFeed to create basic feed part

Call Method createDefaultEntry to add entry to the feed;

Call Method updateCachedEntryContent which is synchronized to update/append content based on the incoming parcel in cached Entry Content

Call Method FeedDoc.writeFeedDoc provided by AtomSphere to write feed. The method is synchronized too.

}

• Sub Classes

(1) class IndividualCachedSensorStorage extends CachedSensorStorage {

ArrayList cachedEntryContent; // An In-Memory data structure to store latest parcel;

protected String generateContentString; // Generate the parcel content string from the cachedEntryContent. It overrides the method in its superclass.

protected void updateCachedEntryContent // Update data in the In-Memory data structure based on incoming parcel

}

(2) class AllInOneCachedSensorStorage extends CachedSensorStorage;

(3) class AbnormalCachedSensorStorage extends CachedSensorStorage;

Class IndividualCachedSensorStorage, Class AllInOneCachedSensorStorage and AbnormalCachedSensorStorage extend their super class and override two methods generateContentString and updateCachedEntryContent based on their different ways to process the In-Memory data.

4. URLs for all Atom Feeds

For all sensors' data :

http://hagar.cs.indiana.edu:8181/IUMSCAtomFeed.xml

For each sensor's data :

http://hagar.cs.indiana.edu:8181/IUMSCAtomFeed_Bay1Humid.xml

http://hagar.cs.indiana.edu:8181/IUMSCAtomFeed_Bay1Temp.xml

http://hagar.cs.indiana.edu:8181/IUMSCAtomFeed_CampCWInTemp.xml

http://hagar.cs.indiana.edu:8181/IUMSCAtomFeed_CrystalTemp.xml

http://hagar.cs.indiana.edu:8181/IUMSCAtomFeed_DEHumid.xml

http://hagar.cs.indiana.edu:8181/IUMSCAtomFeed_DETemp.xml

http://hagar.cs.indiana.edu:8181/IUMSCAtomFeed_FrameBuffer_D_Usage.xml

http://hagar.cs.indiana.edu:8181/IUMSCAtomFeed_FrameBuffer_E_Usage.xml

http://hagar.cs.indiana.edu:8181/IUMSCAtomFeed_FrameBuffer_F_Usage.xml

http://hagar.cs.indiana.edu:8181/IUMSCAtomFeed_LabCWInTemp.xml

http://hagar.cs.indiana.edu:8181/IUMSCAtomFeed_LabCWOutTemp.xml

http://hagar.cs.indiana.edu:8181/IUMSCAtomFeed_LN2Levl.xml

For abnormal data from all sensors:

http://hagar.cs.indiana.edu:8181/IUMSCAtomFeed_Abnormal.xml

Future work

1. SSO;

2. Netvibes Widgets.

Status Report for 07/18/2007 -- 08/01/2007

Progress

Summary of problems and solutions in Multithreaded Programming for IUMSC 12 sensors Atom Feed

1. Exception in thread occurred for the first time

• Background & Problem

CIMA Client can receive parcels from all 12 sensors. When a parcel arrives, the in-memory cache which stores parcels for each sensor will be updated. Then the content of cache will be written to a Feed.

After CIMA Client ran for about 15 to 20 minutes, there was an exception in Thread and no parcel came afterward.

The error message is shown below:

Exception in thread "Thread-3" java.lang.IllegalStateException:

IllegalStateException:

at org.apache.axis.components.threadpool.ThreadPool.addWorker(ThreadPool.java:103)

at org.apache.axis.transport.http.SimpleAxisServer.run(SimpleAxisServer.java:243)

at java.lang.Thread.run(Thread.java:595)

• Solution

(1) It seems that the program ran out of threads. It was addWorker that throws this when the thread pool runs out of threads.

(2) Check that the default maximum thread count (org.apache.axis.components.threadpool.ThreadPool, public static final int DEFAULT_MAX_THREADS) is set to 100 when the instance of ThreadPool is created.

(3) Change SimpleAxisServer() to SimpleAxisServer(int maxPoolSize, int

maxSessions). Both of the constructor parameters default to 100. Now we make them 1000 instead.

2. Random occurred exception in thread

• Problem

No thread exceptions occurred for 1 hour after changes described above, but the problems were found by observing the output window in eclipse. That is, if only a few parcels came at the same time, the program had enough time to write them to the Feed and receive new incoming parcels. But sometimes many parcels came at the same time, and parcels should be written to the Feed one by one, thus during this period the program could not receive any parcel and it seemed the program had been dead since then.

Restart the CIMA Client, exception in thread arose randomly.

• Solution – Step 1

Variable “maxPoolSize” and “maxSessions” were kept to 1000 and 1000, and set a "threshold" in myHandler() to 990, which meant that once there were 990 or more parcels were waiting for being processed the program would skip the latest one. Doing so was aiming to relieve the possible threads burst. As a result, there was no exception in thread for a longer time, but unfortunately the threadpool was full eventurally.

• Solution – Step 2

Removed the writing to Feed part from myHandler(), and added/activated a thread which was just responsible for writing parcels to Feed. The result was that the speed of getting parcels and writing them to a remote disk did not match forever no matter how large the buffer we set and no matter whether the thread was added or not. So the burst of threadpool occurred at last.

• Solution – Step 3

In myHandler(), the "writing to feed" part was commented and set variable maxPoolSize and maxSessions to only 5 and 5, and set a "threshold" in myHanlder() to 0. The result was that and it worked very well -- it could receive all the parcels and had never been dead. Then I wrote the Feed to /tmp/foo.xml(local disk, local file), and set variable maxPoolSize and maxSessions still to 5 and 5, and it still worked very well!

So I drew a conclusion that the bottleneck might be the "writing to feed" part which was not local operation and the speed of writing to feed did not match the speed of receiving the parcels, so resulted in the burst of the threadpool.

• Solution – Step 4

Wrote the Feed to local disk and set a symbolic link to the remote one. – It worked very well and never had been dead for 3 days( Since I need to add one more functionality to CIMA Client, so I stopped it).

Discussion

1. Functionalities for users to select different sensors and/or just show the sensors with abnormal data.

Future work

1. Get sensor values based on the user’s requirement;

2. Show only values not in normal range;

3. The issues described in Discussion section

Status Report for 07/11/2007 -- 07/18/2007

Progress

Some problems and solutions in detail design for IUMSC 12 sensors Atom Feed

1. Synchronization in writing data to Atom Feed

• Problem

It is possible that data from 12 sensors could come simultaneously, but each sensor data should be written to the Feed sequentially. In that case, if the data is being written to Feed, the other data should not be written to the Feed. On the other hand, the incoming data is updated in memory first, then written to the Feed. So the updating should be synchronized too.

• Solution

Put the Feed updating statements in a synchronized block and this synchronized block should be as small as possible.

2. Exception in thread

• Problem

After CIMA Client ran for about 15 to 20 minutes, there was an exception in Thread and no parcel comes afterward.

Error message is shown below:

Exception in thread "Thread-3" java.lang.IllegalStateException:

IllegalStateException:

at org.apache.axis.components.threadpool.ThreadPool.addWorker(ThreadPool.java:103)

at org.apache.axis.transport.http.SimpleAxisServer.run(SimpleAxisServer.java:243)

at java.lang.Thread.run(Thread.java:595)

• Solution

(1) It seems that the program ran out of threads. It was addWorker that throws this when the thread pool runs out of threads.

(2) Check that the default maximum thread count (org.apache.axis.components.threadpool.ThreadPool, public static final int DEFAULT_MAX_THREADS) is set to 100 when the instance of ThreadPool is created.

(3) Change SimpleAxisServer() to SimpleAxisServer(int maxPoolSize, int

maxSessions). Both of the constructor parameters default to 100. Now we make them 1000 instead.

3. A problem caused by Synchronization

• Problem

It has been no thread exceptions for 1 hour, but I found another problem by observing the output window in eclipse. That is, if only a few sensor data come at the same time, the program has enough time to write the data to the Feed and receive new incoming data after that. If sometimes many sensor data come at the same time, the program needs to write these data one by one, during this period the program cannot receive any parcel and it seems the program has been dead for twenty or more minutes.

• Possible solution

Instead of the critical sections using "synchronized" around the whole feed writing process we need some other way to serialize the writing of the feed. Maybe we should add a queue data structure into which each change is added, then the writer reads everything available in the queue, writes the feed, and reads the next batch of changes. This would limit the critical section to updating and reading an in-memory data structure rather than reading and writing a file.

Discussion

1. Functionalities for users to select different sensors and/or just show the sensors with abnormal data.

Future work

1. Get sensor values based on the user’s requirement;

2. Show only values not in normal range;

3. The issues described in Discussion section

Status Report for 05/16/2007 -- 05/30/2007

Progress

Detail design for IUMSC 12 sensors Atom Feed

1. Register IUMSC 12 sensors

Before reading all the sensor values from the real proxy box in the IUMSC lab, send requests to register all the sensors with CIMA service endpoint (http://129.79.85.21:7710). Sensor names and meanings are shown below.

• LabCWOutTemp Lab Chill water OUT
• LabCWInTemp Lab Chill water IN
• CampCWInTemp Campus Chill water IN
• LN2Levl Liquid Nitrogen level indicator
• Bay1Temp Bay1 Temperature
• Bay1Humid Bay1 Humidity
• DETemp Diffractometer Enclosure Temperature
• DEHumid Diffractometer Enclosure Humidity
• CrystalTemp Crystal Temperature
• FrameBuffer_D_Usage Disk Usage Disk D
• FrameBuffer_E_Usage Disk Usage Disk E
• FrameBuffer_F_Usage Disk Usage Disk F

Note: The sensor value used before was from test-proxy box (CIMA service endpoint (http://129.79.85.20:7710)) with only two probes (Bay1Temp & LabCWOutTemp) connected to it.

2. Write data to the feed

Part1: Read the parcel into DOM

Get a Parser and parse the parcel

Part2: Update/Append feed

If no feed exists, create the "feed" part of the feed including the following elements: Author, Id, Title and Link and Updated;

Otherwise add an Entry to the feed

If no entry in the feed, create basic elements for the entry including the following elements : Author, Title, Id and Updated

Otherwise get the existing entry handle(there should be only one entry) in the feed

Get the current content in the Content

Set the receiving data to the Content

3. Read feed

Open http://www.netvibes.com/, add the URL of IUMSC Atom Feed(http://hagar.cs.indiana.edu:8181//IUMSCAtomFeed.xml) to netvibes

4. IUMSC Atom Feed in netvibes

The screen shot below shows the Atom Feed in netvibes.

Discussion & Questions

1. The data from 12 sensors come in at different times, that is to say, sometimes nothing comes during 10 to 15 seconds while sometimes data from some sensors come almost at the same time. This might cause that some data might be left out

-- I need to provide file locking function by using the FileLock class before updating, then unlock. Or make the routine that writes the feed "synchronized" or put the feed updating statements in a synchronized block(not as desirable as a synchronized routine).

2. In netvibes, after refreshing the current webpage, new data don’t come, only the old data were shown.

-- I kept clicking the fresh button in the feed area, but it did not work

Future work

1. Get sensor values based on the user’s requirement;

2. Show only values not in normal range;

3. Adjust feed format;

4. The issues described in Discussion section

Status Report for 04/25/2007 -- 05/16/2007

Progress

Detail design for Bay1Temp Atom Feed (method1 shown in the last report is used here)

1. Create an atom feed document

• New a feed

Feed theFeed = new Feed();

• Add "xmlns" attribute to the feed

theFeed.addAttribute(new Attribute("xmlns","http://www.w3.org/2005/Atom"));

• Add Author

theFeed.addAuthor(new Author("Yu(Carol) Deng"));

• Set a universally unique Id for the feed

theFeed.setId(new Id("urn:uuid:" + new UID().toString()));

• Add Title

Title feedTitle = new Title("text"); // Set title type for the feed

feedTitle.setText("Bay1Temp Atom Feed"); // Set title content

theFeed.setTitle(feedTitle); // Set the title

• Add Link

Link feedLink = new Link(); // New a Link in the feed

feedLink.setRel(new Attribute("rel", "self")); // Set "rel" attribute of the link

feedLink.setType(new Attribute("type", "application/atom+xml")); //Set "type" attribute of the link

feedLink.setHref(new Attribute("href", FeedHref)); //Set "href" attribute of the link

theFeed.addLink(feedLink); //Add the link

• Set Updated to the entry

theFeed.setUpdated(new Updated(new Date()));

2. Add an entry document to the feed

• New an Entry

parcelEntry = new Entry();

• Add Author

parcelEntry.addAuthor(new Author("Yu(Carol) Deng"));

• Add Title

Title parcelTitle = new Title("text"); // Set title type for the feed

parcelTitle.setText("SensorName, TimeStamp, DoubleData"); // Set title content

parcelEntry.setTitle(parcelTitle); // Set the title

• Set a universally unique Id for the entry

parcelEntry.setId(new Id("urn:uuid:" + new UID().toString()));

• Set Updated to the entry

Calendar cal = new GregorianCalendar();

parcelEntry.setUpdated(new Updated(cal.getTime()));

• Set the current data to the Content

parcelEntry.setContent(nodeSensorName + nodeTimeStamp + nodeDoubleData);

• Add the Entry to the feed

currentFeed.addEntry(parcelEntry);

3. Atom Feed for Bay1Temp

?>

- <feed xmlns="http://www.w3.org/2005/Atom">

<id>urn:uuid:-7012692:11296d2cefc:-8000id>

<updated>2007-05-17T01:15:29.205-05:00updated>

<title type="text">Bay1Temp Atom Feedtitle>

- <author>

<name>Yu(Carol) Dengname>

author>

<link rel="self" type="application/atom+xml" href="http://hagar.cs.indiana.edu:8181/foo.xml" />

- <entry>

<id>urn:uuid:-7012692:11296d2cefc:-7fffid>

<updated>2007-05-16T17:37:17.66-05:00updated>

<title type="text">SensorName, TimeStamp, DoubleDatatitle>

- <author>

<name>Yu(Carol) Dengname>

author>

<content type="html">Bay1Temp 2007-05-17 05:14:48Z 25.5
Bay1Temp 2007-05-17 05:15:08Z 25.5
Bay1Temp 2007-05-17 05:15:28Z 25.5content>

entry>

feed>

4. Bay1Temp Atom Feed in iGoogle

Add the url of Bay1Temp Atom Feed to iGoogle, the screen shot below shows the Atom Feed in iGoogle.

Discussion & Questions

1. In iGoogle, it shows 3 links for Bay1Temp in the feed area

-- I tried to clean some cookies or cache, but they are still there.

2. In iGoogle, after refreshing the current webpage, click the content of feed which had content, is not available for the data.

-- I kept clicking the "+" button in the feed area, data is still not available.

Future work

1. Get all sensors from a lab going to individual feeds;

2. The issues described in Discussion section

Status Report for 04/04/2007 -- 04/25/2007

Progress

Detail design for Atom Feed for CIMA Web 2.0

1. Method I

Basic idea:

• On the server side:

(1) A program keeps running to get Parcel from CIMA Service every several seconds;

(2) Every time the program get Parcel it will call XML Parser to wrap the data into format of item in the Atom Feed;

(3) Write the new-come item to the Atom Feed, and delete the oldest one at the same time;

(4) Once there is any feed request, it will read the Atom feed (file)

• On the client side:

(1) User sends feed request to the server;

(2) Show feed content to the user

2. Method II

Basic idea:

• On the server side:

(1) Once there is any feed request, a serverlet will be started to listen to the Parcel from CIMA service;

(2) If no data comes, return a message to the user; otherwise, get Parcel and call XML Parser. This step keeps doing this way until it gets enough Parcels based on users requirements(For example, user wants to watch the latest 10 data each time) and to wrap the them into format of item in the Atom Feed

(3) Write these items to the Atom Feed;

(4) Once there is any feed request, it will read the Atom feed (file)

• On the client side:

(1) User sends feed request to the server;

(2) Show feed content to the user

3. Comparison
(1) Method I is simple and it can update information at the same rate they receive information, but it needs to run the program all the time even if no feed request comes;

(2) Method II saves resource and it gets the data just upon the user request, but it is a little bit complicated and the user may need to wait for some time for server get enough data

Discussion

1. Comparison of two implementations

2. Request frequency settings in Sage

Future work

1. The issues described in Discussion section

Status Report for 03/07/2007 -- 04/04/2007

Progress

1. Workflow for Atom Feed for CIMA Web 2.0

(1) CIMA persons subscribe all the available parameters by Sage on the browser side;

(2) CIMA Client gets the http request and registers a CIMA service;

(3) CIMA Service sends back a parcel with the service information users required;

(4) XML Parser converts the parcel format into Atom Syndication format and writes to Atom Feed;

(5) Send results back to users

2. Detail Design

(1) Install Sage at http://sage.mozdev.org/ (Sage is a lightweight RSS and Atom feed reader extension for Mozilla Firefox);

(2) Make CIMAClientPlatform application run. This application a skeleton application to register with a CIMA service and get data (Parcel) back. In a Parcel, it contains many data, currently, the user needs “SensorName”, “TimeStamp” and “DoubleData”;

(3) In the class src/edu/indiana/extreme/www/cima/channel/sink/Channel_sinkImpl.java,

put some code for parsing Parcel to Atom Syndication format and writes to Atom Feed in the "myHandler" method. The basic steps are shown below:

(a) Read XML data into a DOM;

(b) Configure the Factory;

(c) Instantiate the Factory;

(d) Get a parser and parse the parcel;

(e) Write the data required to Atom Feed;

(4) Copy the Atom Feed to an available URL manually;

(5) On browser side, create a new bookmark in Sage for the Atom Feed just created and Check the feed;

(6) The content/data in the Atom Feed will show on the right panel

Discussion

1. Currently, I manually copied the Atom Feed from hagar.cs.indiana.edu/Web2.0/...... to

http://www.cs.indiana.edu/~ydeng/atomfeedcima.xml, so I am wondering if I need to use some TOMCAT programming to move these files automatically.

2. How often the Sage retrieves the data from the Atom Feed.

Future work

1. The issues described in Discussion section

Status Report for 02/14/2007 -- 03/07/2007

Progress

1. Atom Feed for CIMA Web 2.0

(1) CIMA persons click the link to feeds which will pop up on all the parameters for them to subscribe;

(2) CIMA Client gets the http request and registers a CIMA service;

(3) CIMA Service sends a parcel with the service information users required;

(4) Atom Feed parses the parcel and convert service information into Atom Syndication format and send them back to users

2. Research methods

(1) Create a simple atom feed (xml file) sample by hand

(2) Make a service with and Create an atom feed from step(1) by using atmosphere Java library;

A brief instruction to create an atom feed:

(a) requirements: JDK 1.4.2 or 1.5

(b) download the following jar and war files at http://www.colorfulsoftware.com/projects/atomsphere

• atmosphere which includes atomsphere-1.0.1.0.jar, atomsphere-1.0.1.0-javadoc.jar, atomsphere-1.0.1.0-sources.jar
• atmosphere-taglib which includes atomsphere-taglib-1.0.1.0.jar, atomsphere-taglib-1.0.1.0-sources.jar
• atmosphere-webapp which includes atomsphere-webapp-1.0.1.0.war
• atomphere-weblib which includes atomsphere-weblib-1.0.1.0.jar, atomsphere-weblib-1.0.1.0-sources.jar
• jsr173_1.0_api.jar, sjsxp_101.class, sjsxp_fcs_1_0.class, StAX Utilities Project

(c) export all the jar files above in the java project

(d) API(JavaDoc) http://www.colorfulsoftware.com/projects/atomsphere/api/

(e) create an atom feed by atmosphere library (sample code is shown below)

Feed feed = new Feed();
feed.addAuthor(new Author("Yu(Carol) Deng"));
feed.setId(new Id("tag:www.xxxxxxxx.com"));
feed.setTitle(new Title("Cool Atom Feed"));
feed.setUpdated(new Updated(new Date(2005,3,2)));

Entry firstEntry = new Entry();
firstEntry.setTitle(new Title("Some interesting stuff"));
firstEntry.setId(new Id("tag:www.XMML.com/2004/12/08.html"));
firstEntry.setUpdated(new Updated(new Date(2006,3,1)));
feed.addEntry(firstEntry);

FeedDoc.writeFeedDoc("foo.xml",feed,FeedDoc.encoding,FeedDoc.xml_version);

(3) Repeat step(2), use xml beans (apache project) instead;

I tried to write a program in XML beans (http://xmlbeans.apache.org/) to create an atom feed. But I found that the schema of Atom 1.0 feed follows RELAX NG Compact Schema while xml beans supports XML Schema not RELAX NG Compact Schema. I tried to find the conversion between these two, but failed. I am wondering if there is some kind of conversion between RELAX NG Compact Schema and XML Schema.

Discussion

1. The format of parcel from CIMA Service

2. The storage of the Atom feed

Future work

1. The issues described in Discussion section

Status Report for 01/31/2007 -- 02/14/2007

Progress

1. RSS/Atom Feed for CIMA Web 2.0

2. Research methods

(1) Create an atom feed (atom.xml) sample by hand;

(2) Make a service with atomsphere Java library and produce atom.xml from step(1), that is to say, make a service that responds with the atom feed in the HTTP response;

(3) Repeat step(2), use xml beans (apache project) instead;

(4) Compare the two methods above.

3. Research steps

(1) make feeds;

(2) do flicke;

(3) do pipe

4. Read some papers and books related to RSS/Atom feed

(1) Web sites

• http://en.wikipedia.org/wiki/RSS_(file_format)#Example_-_RSS_2.0

• http://www.colorfulsoftware.com/projects/atomsphere

• http://xmlbeans.apache.org/

• http://www.timfanelli.com/item/howto_create_atom_feed

• http://www.russellbeattie.com/notebook/1007853.html

(2) Books

• Developing Feeds with Rss and Atom by Ben Hammersley
• Beginning RSS and Atom Programming by Danny Ayers and Andrew Watt
• Hacking RSS and Atom by Leslie M. Orchard

Discussion

1. RSS for CIMA

2. XML Bean for CIMA

Future work

1. The issues described in Discussion section