Progress
Summary of problems and solutions in Multithreaded Programming for IUMSC 12 sensors Atom Feed
1. Exception in thread occurred for the first time
- Background & Problem
CIMA Client can receive parcels from all 12 sensors. When a parcel arrives, the in-memory cache which stores parcels for each sensor will be updated. Then the content of cache will be written to a Feed.
After CIMA Client ran for about 15 to 20 minutes, there was an exception in Thread and no parcel came afterward.
The error message is shown below:
Exception in thread "Thread-3" java.lang.IllegalStateException:
IllegalStateException:
at org.apache.axis.components.threadpool.ThreadPool.addWorker(ThreadPool.java:103)
at org.apache.axis.transport.http.SimpleAxisServer.run(SimpleAxisServer.java:243)
at java.lang.Thread.run(Thread.java:595)
- Solution
(1) It seems that the program ran out of threads. It was addWorker that throws this when the thread pool runs out of threads.
(2) Check that the default maximum thread count (org.apache.axis.components.threadpool.ThreadPool, public static final int DEFAULT_MAX_THREADS) is set to 100 when the instance of ThreadPool is created.
(3) Change SimpleAxisServer() to SimpleAxisServer(int maxPoolSize, int
maxSessions). Both of the constructor parameters default to 100. Now we make them 1000 instead.
2. Random occurred exception in thread
- Problem
No thread exceptions occurred for 1 hour after changes described above, but the problems were found by observing the output window in eclipse. That is, if only a few parcels came at the same time, the program had enough time to write them to the Feed and receive new incoming parcels. But sometimes many parcels came at the same time, and parcels should be written to the Feed one by one, thus during this period the program could not receive any parcel and it seemed the program had been dead since then.
Restart the CIMA Client, exception in thread arose randomly.
- Solution – Step 1
Variable “maxPoolSize” and “maxSessions” were kept to 1000 and 1000, and set a "threshold" in myHandler() to 990, which meant that once there were 990 or more parcels were waiting for being processed the program would skip the latest one. Doing so was aiming to relieve the possible threads burst. As a result, there was no exception in thread for a longer time, but unfortunately the threadpool was full eventurally.
- Solution – Step 2
Removed the writing to Feed part from myHandler(), and added/activated a thread which was just responsible for writing parcels to Feed. The result was that the speed of getting parcels and writing them to a remote disk did not match forever no matter how large the buffer we set and no matter whether the thread was added or not. So the burst of threadpool occurred at last.
- Solution – Step 3
In myHandler(), the "writing to feed" part was commented and set variable maxPoolSize and maxSessions to only 5 and 5, and set a "threshold" in myHanlder() to 0. The result was that and it worked very well -- it could receive all the parcels and had never been dead. Then I wrote the Feed to /tmp/foo.xml(local disk, local file), and set variable maxPoolSize and maxSessions still to 5 and 5, and it still worked very well!
So I drew a conclusion that the bottleneck might be the "writing to feed" part which was not local operation and the speed of writing to feed did not match the speed of receiving the parcels, so resulted in the burst of the threadpool.
- Solution – Step 4
Wrote the Feed to local disk and set a symbolic link to the remote one. – It worked very well and never had been dead for 3 days( Since I need to add one more functionality to CIMA Client, so I stopped it).
Discussion
1. Functionalities for users to select different sensors and/or just show the sensors with abnormal data.
Future work
1. Get sensor values based on the user’s requirement;
2. Show only values not in normal range;
3. The issues described in Discussion section
No comments:
Post a Comment