Setting up the Rule Game Server on a DoIT Shared Hosting host

This document contains notes for setting up and updating the Rule Game Server web site on the University Wisconsin-Madison DoIT Shared Hosting platform (powered by Plesk software). This is in addition to the generic instructions for using the Plesk platform provided by the DoIT staff.

Introduction

While the hosts provided by DoIT Shared Hosting (to which we will refer to as Plesk hosts) have the most essential software components (such as the Apache Tomcat server and the MySQL server) that we have on our own in-house servers, the manner in which you deploy and control your applications on Plesk hosts is somewhat different from how you'd do it "in house". This document explains the additional steps one would need to take, as compared to the standard operating procedures outlined in our SOP documentation for "in-house" deployment ( Preparing and running an experiment; Maintaining and deploying server code and server data files; Set up your own Rule Game Server).

The DoIT Shared Hosting provides us with two virtual hosts, wwwtest.rulegame.wisc.edu and rulegame.wisc.edu. (Before we have given a full get-go for a global DNS change, the latter host can be accessed as preview.rulegame.wisc.edu; alternatively, you can modify the /etc/hosts file to associate the host name rulegame.wisc.edu with the IP address provided by the DoIT people for this Plesk host). Physically, each of the two virtual hosts is hosted on a server with several dozens other virtual hosts (production ones or test ones, respectively). The software platform for managing them is called Plesk.

Plesk offers several ways for you to control what goes on at each host. For this discussion, we'll be talking about wwwtest.rulegame.wisc.edu, but the same applied to the production host as well. To carry out any of the operations discussed below, you need to have the Global Protect VPN running, connecting to the main UWM VPN gateway (not the CAE VPN gateway), unless you are working on a PC on the UWM internal network.

Secure FTP. You can connect to the host in question with sftp to transfer files to or from that host. You connect to test-rulegame@wwwtest.rulegame.wisc.edu or rulegame@rulegame.wisc.edu, as the case may be, using the password that was provided to you by email. Once you have logged in, you will see that your home directory on the remote host is /var/www/vhosts/wwwtest.rulegame.wisc.edu (or /var/www/vhosts/rulegame.wisc.edu, on the production host), and you can work with files in that directory or its subdirectories. (And if you type "ls ..", you can see what other virtual hosts are hosted on this physical host).
"Chrooted" shell. You can log in to the DoIT shared hosting's "Site Administrarion control panel, https://webhosting.doit.wisc.edu/panel. Depening on where you are, and what you have been doing this day, the logging process may involve several stages. You may be initially sent to the general UW authentication page (where you log in using your UW netid and password), before you are shown the DoIT-specific "ADMIN CONTROL PANEL LOGIN" screen. At that screen, you enter the host name (such as wwwtest.rulegame.wisc.edu or rulegame.wisc.edu) as the "login", and the DoIT-shared-hosting-specific password for our hosts (that was provided to you via email). This gets you to the "Websites and Domains" screen. On that screen, click on the "SSH" link. That gives you a web browser window in which you can type shell commands, much like you were using a normal ssh connection to the remote host.
However, what you're interacting with is not a "normal" shell but a "chrooted" one. (The name comes from the name of the UNIX command, chroot, i.e. "change root".) This means that when you use that shell, what appears as the root of the file system visible to you (and can be said to be your home directory) is actually /var/www/vhosts/wwwtest.rulegame.wisc.edu (or /var/www/vhosts/rulegame.wisc.edu, as the case may be). You don't see the rest of the file system. You also can use only a very small number of shell commands, and some of them, such as scp, do not work.
You obviously can't do all that much in this shell, but at least you can move files around, check time stamps, create and unpack ZIP files, etc.
From the "Websites and Domains" screen you can access a number of other tools. This includes the "Files" tool for managing files stored in your space; th tool for controlling the MySQL databases running on your server.
The Tomcat server has a manager screen at https://wwwtest.rulegame.wisc.edu/manager (or https://rulegame.wisc.edu/manager , as the case may be). You can use it to start and stop your applications.

The sections of this document more or less parallel the setup steps described in the Set up your own Rule Game Server guide, Option C (full setup).

Databases

The initial setup of databases for wwwtest.rulegame.wisc.edu

Added database named "game", with "none" as the "related site". (Could have picked wwwtest.rulegame.wisc.edu as the related site) (eventually, assigned the database to the site)

Created user "game" on that database, with the password as specified in persistence.xml

Importing data from the existing database (on sapir) to the DoIT host

Export on sapir:

mysqldump --databases game > sapir-dump.sql

vmenkov@SAPIR:~$ ls -l sapir-dump.sql 
-rw-rw-r-- 1 vmenkov vmenkov 11236564 Nov  1 19:38 sapir-dump.sql

Then copied the dump file directly into the database on the DoIT hist via the web UI.

The web UI shows:

Connection Information for game
You can use these database connection details, for example, when you are asked to specify ones during installation of a web app.

Host
localhost:3306
Database name
game
User name
game
Password
******

(Initial) The master config file

The master configuration file for the use on the Plesk hosts was created as follows.

#--- This is the master configuration file for the Rule Game Server.
#--- It is located in /opt/w2020/w2020.conf

#--- The Rule Game server input data directory. The Game Server will read
#--- experiment control files from it
FILES_GAME_DATA = "/opt/w2020/game-data";

#----The Rule Game server saved data directory.
#--- The Game Server will write transcript files etc to it.
FILES_SAVED =  "/opt/w2020/saved";

#---- The user name for the MySQL server
JDBC_USER = "game"
#---- The password that we have set for this MySQL server user
JDBC_PASSWORD = "....the password goes here....";

#---- The URL for the Rule Game GUI Client (prod and dev versions)
GUI_PROD = "/rule-game/prod/";
GUI_DEV = "/rule-game/dev/";

In the code above, the GUI_PROD and GUI_DEV URLs have no host name or port name, since the client will be deployed to the same web server (the Plesk host) with the Rule Game Server that uses it.

(Initial) Installing the MySQL connector

When we're using the DoIT Shared Hosting, Tomcat is already set up for us.

For the Tomcat web server to work with the MySQL database, it needs an aixiliary JAR file with the drivers. We have copied it from one of our existing installations using sftp.

    sftp test-rulegame@wwwtest.rulegame.wisc.edu
    cd tomcat/lib
    put mysql-connector-j-8.0.31.jar

(Initial and ongoing) The web app code (server side)

It would be interesting to find out a way to compile the Game Server code directly on a Plesk host. At this point, I don't know how to set this up (the chrooted shell obviously is not equipped with the toold we'd need for that). Instead, the current approach is to build a WAR file (w2020.war) on some other host (could be your own PC, could be sapir or the CAE host) and then copy it over to the Plesk host. (It's a large file -- 36 MB -- so copying it from a host within the UW network is probably preferable to copying it from your home machine).

Step by step:

Build a WAR file of the Game Server application (w2020.war) on your PC or on some other host (such as sapir), from our source code and the necessary third-party libraries. For detailed instructions see e.g. e Rule Game Server build Guide.
Use sftp to move the WAR file to the Plesk host. Let's suppose you put it there to the directiory tmp under your home directory.
Use the manager app ( https://wwwtest.rulegame.wisc.edu/manager/ ) to deploy a local file ("WAR or Directory path"): /var/www/vhosts/wwwtest.rulegame.wisc.edu/tmp/w2020.war (in the "Context path" box, you should enter /w2020.) Now, our app is visible at https://wwwtest.rulegame.wisc.edu/w2020/ !

(Initial and ongoing) The GUI client

We do the same with the GUI client: build it on "our own" host (this could be your own PC or sapir), pack it into a WAR file, and then deploy the WAR file to the Plesk host.

On a host that you have full control of (such as your PC or sapir), download the source code of the GUI Client; build, install and deploy.
On sapir, build the WAR file (based on what's deployed under rule-game/prod in Apache HTTPd server's directory)
```
 cd /var/www/rule-game
 jar cvf ~/tmp/rule-game.war prod/* dev/*
 cd
 cd tmp
 jar tf rule-game.war 
```
(Of course, you don't actually have to deploy the client on sapir before deploying it on the Plesk host; you can build the WAR file right in the build directory. The client build instructions have a sample shell script for that.
sftp the war file to the Plesk host, placing it e.g. in your tmp directory
Deploy the client WAR file using the manager tool. In the mananger ( https://wwwtest.rulegame.wisc.edu/manager ) deploy from the WAR path: /var/www/vhosts/wwwtest.rulegame.wisc.edu/tmp/rule-game.war. This will make the client show under https://wwwtest.rulegame.wisc.edu/rule-game/prod

Experiment control files

(Initial) set up auto-deployment from the GitHub repository

Set up auto-deployment from https://github.com/RuleGame/Rule-Game-game-data via the Git menu.

repo URL = https://github.com/RuleGame/Rule-Game-game-data.git

Deployment: master branch automatically to /opt/w2020/game-data

(Ongoing) Updating experiment control files

After you have created experiment control files for a new experiment and checked them into the GitHub repository, you have to effect a "pull" on the DoIT site. The process is very similar to that for the landing pages (see the write-up below), with the repository name RuleGame/Rule-Game-game-data instead of RuleGame/Rule-Game-top-pages.

The landing page

(Initial) Set-up

Via the GIT menu, I set up the deployment of the GIT repository https://github.com/RuleGame/Rule-Game-top-pages.git to /tomcat/webapps/ROOT

(Ongoing) Updating the landing page

This section explains how to update the web pages visible at the top-level of the web site, i.e. at https://wwwtest.rulegame.wisc.edu/ and elsewhere in the top-level directory.

When you first start working with the top-level pages, check out the current version of these pages from the GitHub repository at https://github.com/RuleGame/Rule-Game-top-pages to your computer. Let's suppose that you have decided to work with these files in the directory named landing under your home directory on your personal computer:
```
	    mkdir landing
	    cd landing
	    git init
	    git remote add origin https://github.com/RuleGame/Rule-Game-top-pages.git
	    git pull origin master	   
```
Subsequently, whenever you resume working with these files and have a reason to thing that some of the files have changed in the repository (e.g. because one of your co-PIs has checked in an update to one of the files), you just need to do the "pull" from the repository to your PC:
```
	      cd landing
	      git pull origin master	   
```
Edit the desired HTML files. (You can, of course, edit the files in some other directory using your favorite WYSIWYG HTML editing software, such as BlueGrifon, and then simply copy the saved files to the landing directory).
The main file (the one which the web server will display at https://wwwtest.rulegame.wisc.edu/ or the analogous "landing page" place at another server you're deploying it to) is index.html. As with any web site, you can create other files (e.g. papers.html with a list of ArXiv papers) and put links to them into index.html.
Once you are ready to deploy the updated pages, check them in and commit; that will send them to the GitHub repository:
```
	    git commit -a -m 'This is my new update'
	    git push origin master
	  
```
You may want to verify that your updated files have been received by the GitHub server by taking a look at your repository there.
Unless you are working on a personal computer on the UW internal network, you must start the Global Protect VPN, connecting to the main UW VPN server (uwmadison.vpn.wisc.edu) (and not to the Engineering (CAE) VPN), before you can deploy the pages to the Plesk server provided to us by DoIT Shared Hosting. This will likely involve a step that requires using the smartphone Duo application for the "two-factor authentication".
Once you are ready to deploy the pages to the Plesk server (and are either working on a personal computer on the UW internal network, or using the UW VPN), please log in to the shared hosting Site Administration control panel, https://webhosting.doit.wisc.edu/panel. Depening on where you are, and what you have been doing this day, the logging process may involve several stages. You may be initially sent to the general UW authentication page (where you log in using your UW netid and password), before you are shown the DoIT-specific "ADMIN CONTROL PANEL LOGIN" screen. At that screen, you enter the host name (such as wwwtest.rulegame.wisc.edu or rulegame.wisc.edu) as the "login", and the DoIT-shared-hosting-specific password for our hosts (that was provided to you via email).
Once the login is successful, you'll see the "Websites and Domains" screen. On that screen, click on the "Git" link. That will take you to the screen with the list of the GitHub repositories associated with the web site. Find the section corresponding to the relevant repository (Rule-Game-top-pages.git). In that section, there are two buttons, "Pull now" and "Deploy now". Click on the first on them, Pull now. If everything goes well (that is, the Plesk service managing the DoIT sharing hosting machines successfully connects to GitHub and gets the data from there), you will see a pop-up window with the title "Deployment task is running", which then becomes "Deployment task has completed". After that, if you look at the "Latest commit" table in your repository's section of the DoIT control panel screen, you should see that the commit on the top of the table is the one with the date and commit message corresponding to your most recent git commit.
You don't need to click on the "Deploy now" button, because that step is automated. Once you go in your web browser to the main page of your web site (https://wwwtest.rulegame.wisc.edu/ or https://rulegame.wisc.edu/ , as the case may be), you should see your recent updates "live" on the site.

Making use of experimental data

See

Pulling data from remote server: how to import data from a remote host (e.g. a Plesk host) to a host where you can analyze them (e.g. the CAE host or your laptop)