Maintaining the LinBox Webservice
On this page I document exactly how the LinBox web service was
developed and how it may be updated and maintained. If you have any further
questions, feel free to email me at matt.fendt@gmail.com. Please prefix the
subject line of your email with [LINBOX].
-- Matthew Fendt, January 2008
Page Contents
Building the Server
Building the Clients
The linalg Client
Updating and Maintaining the Service
New to Summer '08
New to Winter '09
Building the Server
A "server" is the computer on which the web service will be hosted.
It is also where the LinBox library is installed and where the computations
are done. The following contains the step that were taken to install the server on hmrg.
Some things to note:
- Currently only configured for Fendt's directory on "hmrg.pc.cis.udel.edu"
- Web service resides in /home/fendt/apache-tomcat-6.0.18/webapps/
axis2/WEB-INF/services
- Apache is currently listening on port 2000. This is verified in
/WEB-INF/server.xml The "Connector port" is set to 2000. This
is at line ~50.
- Apache startup/shutdown locations:
STARTUP: /home/fendt/apache-tomcat-6.0.18/bin/startup.sh
SHUTDOWN: /home/fendt/apache-tomcat-6.0.18/bin/shutdown.sh
Requirements:
- Apache Ant 1.7.0
- Apache Tomcat 6.0.18
- Axis2 1.4.1 installed under Tomcat
- JDK version 1.6.0_07
- SWIG version 1.3.31 (For updating LinBox functionality)
- LinBoxWebservice directory
- LinBox computational library itself
Setup:
- Install Ant, Tomcat, Axis2, and Java
- Set CATALINA_HOME to Tomcat directory (eg /home/fendt/apache-tomcat-6.0.18)
- Set AXIS2_HOME to Axis 2 directory (eg /home/fendt/axis2-1.4.1)
- Set ANT_HOME to Ant directory (eg /home/fendt/apache-ant-1.7.0)
- Set JAVA_HOME to Java directory (eg /usr/lib/jvm/java-6-sun)
- Make sure Apache is shut down.
- Svn and configure the LinBox library. This was done on hmrg.pc.cis.
udel.edu in the directory /home/fendt/linbox. This is the copy of LinBox that
the web service will use.
- Copy the LinBoxWebservice directory, located at /home/fendt/LinBoxWebservice.
- In LinBoxWebservice directory, type "ant generate.service" to ready the
service.
- Start up Apache with ./home/fendt/apache-tomcat-6.0.18/bin/startup.sh
- The web service is now installed in the "services" directory on Apache,
is listening on port 2000, and ready for client calls.
Return to top
Building the Clients
A "client" is any computer that gives information to a "server" computer in order to find the result of a computation. The following can be done to install
a client activity on any computer, be it an online web form or a stand alone program.
Requirements:
- Apache Ant 1.7.0
- Axis2 1.4.1, unzipped
- Java version 1.6.0_07
- LinBoxWebservice directory
Setup:
- Install Ant, Axis2, and Java
- Set AXIS2_HOME to Axis 2 directory (eg /home/fendt/axis2-1.4.1)
- Set ANT_HOME to Ant directory (eg /home/fendt/apache-ant-1.7.0)
- Set JAVA_HOME to Java directory (eg /usr/lib/jvm/java-6-sun)
- Copy the LinBoxWebservice directory, located at /home/fendt/LinBoxWebservice.
There are currently four different kinds of clients that can be installed:
- A stand-alone client that is run from the command line
(TransferAgentStandalontClient).
- A dependent client that relies on another class to receive the answer
(TransferAgentDependentClient). An example of this second client
is the linalg client.
- A prototype client that does not call LinBox directly, but rather waits in a
queue to be processed (TransferAgentUserClient). This is still being
developed.
- A prototype client that listends for enqueue messages and makes the calls to
LinBox. This is still being developed.
To Install and Run the Stand Alone Client:
- In the LinBoxWebservice directory, type "ant generate.standalone.client"
- The client can now be run in the LinBoxWebservice directory, for example: "ant run.standalone.client
-Darg1=rank -Darg2=matrices/mat2.txt"
To Install and Run the Dependent Client:
- In the LinBoxWebservice directory, type "ant generate.dependent.client"
- The class is compiled to LinBoxWebservice/build/client/build/classes/samples/
quickstart/clients/TransferAgentDependentClient.class and can be imported by other classes.
To Install and Run the Experimental User Client:
- In the LinBoxWebservice directory, type "ant generate.user.client"
- The client can now be run in the LinBoxWebservice directory, for example: "ant
run.user.client -Darg1=rank -Darg2=matrices/mat2.txt"
To Install and Run the Experimental Queue Client:
- In the LinBoxWebservice directory, type "ant generate.queue.client"
- The client can now be run in the LinBoxWebservice directory, for example: "ant
run.queue.client"
Return to top
The linalg Client
The linalg client is a particular instance of the dependent client build
described above. The client is hosted on linalg and processes data from a
web form on linalg.org. The client then sends the data to hmrg to
compute the answer.
Getting the software
- Get LinBoxWebservice directory from fendt@hmrg to desired location, eg
/usa/fendt/LinBoxWebservice
- Unzip axi2-1.2.zip, to eg /usa/fendt/axis2-1.2
- Unzip commons-fileupload-1.2-bin.zip, to eg /home/www/usa/linbox
- Unzip commons-io-1.3.2-bin.zip, to eg /home/www/usa/linbox
- The zip files for 3) and 4) are found on the apache web site, www.apache.org.
- Install tomcat in /home/www/usa/linbox/apache-tomcat-6.0.13
Modifying the software
- Set JAVA_HOME variable to /usr/jdk/jdk1.5.0_07 NOTE: The service will not work if
this variable is set to an older version of Java. If the service is not working,
check to see if there is a newer version of Java.
- Set AXIS2_HOME to /usa/fendt/axis2-1.2
- Change port to 8080, in tomcat's conf/server.xml at line 50
Setting up the web client
- Make the following directories in /home/www/usa/linbox/apache-tomcat-6.0.13/webapps/ROOT/WEB-INF/
- lib
- classes/samples/quickstart/service/adb
- classes/samples/quickstart/clients
- Run "ant.generate.dependent.client" in LinBoxWebservice directory
- Copy build/client/build/classes/samples/quickstart/clients/TransferAgentDependentClient.class to /home/www/usa/linbox/apache-tomcat-6.0.13/webapps/ROOT/WEB-INF/classes/samples/quickstart/clients
- Copy all files from build/client/build/classes/samples/quickstart/service/adb/ to /home/www/usa/linbox/apache-tomcat-6.0.13/webapps/ROOT/WEB-INF/classes/samples/quickstart/service/adb
- Copy build/client/build/lib/Services-test-client.jar to /home/www/usa/linbox/apache-tomcat-6.0.13/webapps/ROOT/WEB-INF/lib
- Copy commons-io-1.3.2.jar to /home/www/usa/linbox/apache-tomcat-6.0.13/webapps/ROOT/WEB-INF/lib
- Copy commons-fileupload-1.2.jar to /home/www/usa/linbox/apache-tomcat-6.0.13/webapps/ROOT/WEB-INF/lib
- Copy "LinalgComputingClient.jsp" and "SubmitComputation.jsp" to /home/www/usa/linbox/apache-tomcat-6.0.13/webapps/ROOT/
Running the web client
- Make sure the web service on hmrg is running!!!
- Start up tomcat.
- The web client is ready to go. Note that when tomcat is first started up
that the web page may need to be "reloaded" several times before the service
starts running.
- If you manage to kill my service on linalg, you may need to restart the
web service on hmrg before the linalg web client can work again. :)
Return to top
Updating and Maintaining the Service
Compilation of LinBox Functional .so From Default Directory on Hmrg
- cd to /home/fendt/LinBoxWebservice/src/samples/quickstart/service/adb/xsd
- In makefile: Make sure that the path of the LinBox library is correct.
It is currently -I/home/fendt/linbox
- Type "make liblinboxfunctions.so"
- The .so should be ready to be used.
Relocating the LinBox Function .so
- In makefile, set the path to the LinBox library, eg "-I/home/fendt/linbox"
- Follow compilation instructions as above
To Add Additional LinBox Functionality
- In linboxfunctions.i: Add both the computation function, eg
- extern bool det(std::istream& matrix_in, std::ostream& det_out)
and the
i/o function, eg - extern char* detFiles(char *matfile)
both inside and
outside the %{ ... %} block
- In linboxfunctions.C: Update the "Currently supported methods" section,
#include the appropriate files from the linbox library, and add the
appropriate "file" variable. Add new computation and i/o function.
- Run the .i file through SWIG (swig -java -c++ -package
samples.quickstart.service.adb.xsd linboxfunctions.i).
This will generate the necessary interfacing for the C++ linbox code to work
with the Java web service.
- Type "make liblinboxfunctions.so"
- The .so should be ready to be used. Note that if only "linboxfunctions.C" needs to be changed, step 3 can be ommited.
- The resources/META-INF/services.xml file needs to be updated; add a new "operation" tag to the file.
- The service should be reinstalled on hmrg so that the new service is added.
- The client code also needs to be updated to accomidate the new functionality.
Return to top
New to Summer '08
Previously, the web service calls could only be done synchronously and one at a time. I have changed this
so that they are asynchronous and the service can handle more than one request at a time. I did this
by seperating the model into three parts: The client, the middleman/queue, and the service.
The client is now called the TransferAgentUserClient. It does not call the LinBox service directly, but
rather calls the middleman (which itself is a web service). It is asynchronous and currently has the
following functions:
- Rank
- Run with ant run.user.client -Darg1=rank -Darg2=(matrix-file)
- Determinant
- Run with ant run.user.client -Darg1=determinant -Darg2=(matrix-file)
- Trace
- Run with ant run.user.client -Darg1=trace -Darg2=(matrix-file)
- Valence
- Run with ant run.user.client -Darg1=valence -Darg2=(matrix-file)
- SmithNormalForm
- Run with ant run.user.client -Darg1=smithNormalForm -Darg2=(matrix-file)
- QueueStatus, which checks how many operations are currently waiting in the queue
- Run with ant run.user.client -Darg1=queuestatus
- CheckID, which checks the number of operations that are in front of your operation
in the queue.
- Run with ant run.user.client -Darg1=checkID -Darg2=(your-operation-ID)
- GetTimeEstimate, which returns an estimate of how long your operation will take to compute,
and how long until it will be returned to you.
- Run with ant run.user.client -Darg1=gettimeestimate -Darg2=(your-operation-ID)
- EstimateLength, which does a quick calculation on how long your operation will take.
This does not call a web service, but executes a small program on the user's side.
- Run with ant run.user.client -Darg1=estimatelength -Darg2=(matrix-file) -Darg3=(operation)
The middleman is a new intermediate web service. It is what the user will now call instead of the
original web service. When it recieves a request, it enqueues the request in a queue. It then
waits for the queue manager to give it back the answer. When it does, the middleman returns
the answer to the user.
The Queue client (also called the Queue manager) is first run before any LinBox calls are made.
It creates a persistant queue which holds the middleman's web service calls. The Queue client then
dequeues from the queue and calls the original LinBox web service. When it gets the answer back from
the LinBox web service, it stores the information for the middleman to retrieve. Note that the
middleman web service and the Queue client must be on the same machine, because they communication
via Java RMI. However, these two do not have to be on the same machine as the LinBox web service,
since the Queue client and the LinBox web service communicate via web service calls.
The LinBox web service is virtually unchanged. It is simply called by the Queue client instead of
the original StandAlone or Dependent clients.
[OUTDATED, USE THE INSTRUCTION FROM THE WINTER, BELOW] Copy the LinBoxWebservice directory as before. Then in that directory:
- Generate the LinBox web service with ant generate.service
- Generate the middleman web service with ant generate.middleman
- Start up apache with /home/fendt/apache-tomcat-6.0.18/bin/startup.sh
- Generate the user client with ant generate.user.client
- Generate the queue client with ant generate.queue.client
- Start up RMI with /usr/lib/jvm/java-6-sun/bin/rmiregistry
- Start the queue client with ant run.queue.client
- Start the queue client with ant run.user.client
- You can now make web service calls with the user client (see list of functions above).
Return to top
New to Winter '09
-
At the end of the summer, the hmrg computer was updated to Ubuntu, and this caused some things to stop
working in the service. During this winter, I fixed these issues, and both the queued command line
server and the web client are running.
-
The wrapper code for LinBox no longer writes the answer to a file and then immediately opens and sends back the answer. It writes directly to a ostringstream instead.
-
Axis2 was upgraded to version 1.4. This helped with the issue above, but introduced a new warning:
[WARN] Unable to generate EPR for the transport : http
This is seen during the wsdl generation of the middleman and TransferAgent services. It is uncertain that this warning has any effect.
-
A warning:
[java] log4j:WARN No appenders could be found for logger (org.apache.axis2.description.AxisService).
[java] log4j:WARN Please initialize the log4j system properly.
occured when running the queue and user clients. This was solved by adding the directory containing a log4j file to the runtime directory in the Ant file.
-
The TransferAgentMiddleman web service and the queue client were moved over to linalg.org from hmrg. Everything still works.
-
-
Sometimes when the answer is too larged (>50 digits or so), the service will return a null pointer instead of the answer. This does not cause the service to crash, nor does it happen all of the time. The problem was traced to a specific area in the Swig auto-generated code. Perhaps Java is garbage collecting the memory before it is returned. This is a significant bug.
The command line client can now be run as follows:
To Run the Service With the New Model
Copy the LinBoxWebservice directory to hmrg and linalg. Then
- Shut down apache on hmrg with /home/fendt/apache-tomcat-6.0.18/bin/shutdown.sh
- Generate the LinBox web service on hmrg with ant generate.service
- Start up apache on hmrg with /home/fendt/apache-tomcat-6.0.18/bin/startup.sh
- Generate the middleman web service on linalg with ant generate.middleman
- Generate the queue client on linalg with ant generate.queue.client
- Generate the user client on hmrg with ant generate.user.client
- Start up RMI on linalg with //usr/jdk/jdk1.5.0_15/bin/rmiregistry
- Start the queue client on linalg with ant run.queue.client
- Start the queue client on hmrg with ant run.user.client
- You can now make web service calls on hmrg with the user client (see list of functions above).
Return to top
Return to server page