Getting started with PSU software
1: Set up your connection to the EDOG server.
Email Nicholas Moore (nsm5081@psu.edu) and Chris Hirsh (cdh5049@psu.edu) asking for access to the EDOG server and a folder in the users folder. CC your adviser.
2: Download Open OnDemand
Follow these instructions to install Open OnDemand.
3: Download a file transfer utility
This will allow you to transfer files to and from the cluster. I recommend WinSCP. Use the same log in information as you do on Exceed.
4: Check out other PSU computer resources
- PSU Software: Free software such as Office, Windows, and Solidworks, as well as discounted MATLAB, Mathematica, Adobe, etc.
- PSU Web Apps and PSU Web Files: You can access common programs like Solidworks, MATLAB, Mathematica, and office. They can be a little slower than a local computer, but if you aren’t in the lab or don’t have the programs yourself, they work quite well.
Connecting to the EDOG computers
Setting up remote connection
We use the Remote Desktop Connection app to connect remotely to desktop machines in the lab. In order to do that, you should get connected to the university network first, which is done by installing and setting up the GlobalProtect app. The Remote Desktop Connection is a Microsoft Windows built-in app that should be already installed on your machine. You can think of GlobalProtect as a tool to get connected to the university Virtual Private Network (VPN). There are a few different university VPNs, however the most commonly used are the College of engineering VPN, the students VPN, and the general university VPN.
College of engineering has a tutorial to install and set the GlobalProtect app here:
https://www.engr.psu.edu/ncts-protected/Protected/vpn/coevpn.aspx
Once you are done, you will see the global project icon in the bottom right area (system tray) in your taskbar. It looks like a discolored globe with an X on it when it is not connected, or a full colored globe with a check mark when connected. Click on it and then click on Connect in the dialogue box that appears. Sometimes, especially the first time, it will ask you for your username (without @psu.edu), password, and a two-factor authentication.
You are almost ready.
To access the computers, get connected through the Remote Desktop Connection app.
- Make sure GlobalProtect is “Connected”
- Go to Start à Remote Desktop Connection
- Enter the full computer name. The computer names are on top of each computer’s CPU.
For example, if the computer name is “EDOG23”, enter “e5-me-edog23.ad.psu.edu” - Then select “Connect”
- Enter your username as username@ad.psu.edu (e.g. abc123@ad.psu.edu)
- Enter your password, and two-factor authentication if asked.
Using the Clusters
Setting up your software
Tutorials and training can be found on the ICS Website.
Open Exceed on Demand, log in to the server and use your PSU id/password. In “Settings” at the top, “Show Startup Manager”. Select “Seamless_Mode.cfg” and “Terminal.xs”. Press Run.
At the top left, Edit-Profiles-Edit. Here you can change settings, such as the icon and font size. I always recommend changing color schemes to a black background and light text. It reduces eye strain.
Open WinSCP. You can configure it to be like a regular windows explorer window, but I recommend the two column format (Commander). Go to Options-Preferences-Panels. Show hidden files.
To run software on Hammer, you must use a command to load it, then launch it. See this link for the list of available software. For example, to use comsol, go to the Exceed on Demand command prompt, type “module load comsol/5.2”, then type “comsol”. Note the “/4.4” at the end.
Hammer will not by default load the newest release of the program, nor will you be notified of a new version release. Type “module avail (program)” to see a list of available versions, including the default one that would load without putting a version number.
If you are going to use a program every time you go on Hammer (e.g., Comsol), you can put the load command in a startup file. In your home directory on WinSCP (gpfs/home/yourPSUid), open “.bashrc”. At the bottom, add the “module load (program)” command. Then, whenever you log into Hammer, all you have to type is “comsol”.
General Notes
- At the end of a session, typing “exit” into the command prompt will close your session
- To open another program, in the Hammer terminal go to File-Open Tab.
- If you load the COMSOL module, it automatically loads the MATLAB module.
- Your primary work should be done in folders in your work directory. There is plenty of room. Files in your scratch directory will be deleted if inactive for 30 days, and your home directory only allows 8GB of memory.
- Sometimes when software fails, it creates an error log in your home directory. It can be over 1GB. Your home directory can only hold 8GB of data, so a few error logs can cause memory errors. Without home memory, some programs won’t run. You can delete these huge log files, and that should fix memory errors.
- Type “top” while in Hammer to see the current Hammer node usage. You can log into different nodes from the startup of Exceed by putting a number between “hammer” and “.rcc.psu.edu”.
Using the Lion Clusters
Hammer is meant to be used interactively. If you have a simulation that may take a long time (30+ min), you should submit to a cluster. Cluster jobs can be sent for up to 24 hours of computation, and as many CPUs/ as much RAM as needed. However, more time and hardware will take longer to get through the que. For programs such as COMSOL, it needs to share memory (can only be on 1 node), 2-4 processors and 4-6 GB/processor is more than enough. The following is how to use the clusters. Specifically, I will describe Comsol, however most of this is applicable to other programs.
- Make your model, set it up as if you were going to submit. Make sure it runs for a few minutes and no erros pop up.
- DO NOT CLEAR ALL SOLUTIONS. This might cause the solver to clear all solutions after running.
- If you are using Matlab, before you submit to cluster make sure there is “quit” at the bottom of your matlab script.
- Either close your program or open a new tab on Hammer.
- Type: “ssh -Y lionxg.rcc.psu.edu”. Note that “xg” is replacable. I usually use xg, xj, or xf. They are all interchangeable; if you log in and the queue is long, log out and try another.
- After you log in, you can see how long the queue is by typing “qstat”. If there aren’t many “Q” next to the cluster name, then you may not wait long.
- Change your directory to the directory your file is in. (eg. “cd work/ga”)
- Create a pbs script in your local folder. Open up a notepad file (recommend you download Notepad ++ and customize it. An example of Comsol and Matlab pbs scripts are given below.
- Transfer the file to the remote directory using WinSCP.
- Type “qsub scriptname.pbs” into the Exceed prompt. If it gives an error about it being a dos file, type “dos2unix filename.pbs”.
- It will give you a job number. You can type “qstat -u “psuID”” into the command prompt to see its status. See the table below for commands.
- The final line batch command can be used by matlab to run a simulation in batch mode. Have it output the results to a .csv file, then have Matlab read them in. Remember to have Matlab save variables in a file, otherwise you won’t get any results.
Cluster information can be found here.
PBS Scripts
Comsol: 4 Processors, 4GB/processor, 20 hour run time
#PBS -l nodes=1:ppn=4
#PBS -l pmem=4gb
#PBS -l walltime=20:00:00
#PBS -j oe
cd $PBS_O_WORKDIR
echo PBS: qsub is running on $PBS_O_HOST
echo PBS: job identifier is $PBS_JOBID
echo PBS: job name is $PBS_JOBNAME
module load comsol/5.2
export COMSOL_NUM_THREADS=4
comsol batch -np 4 -inputfile filename.mph -outputfile filename_out.mph
Matlab: 2 processors, 2GB/processor, 2 hour run time.
#PBS -l nodes=1:ppn=2
#PBS -l pmem=2gb
#PBS -l walltime=02:00:00
#PBS -j oe
cd $PBS_O_WORKDIR
echo PBS: qsub is running on $PBS_O_HOST
echo PBS: job identifier is $PBS_JOBID
echo PBS: job name is $PBS_JOBNAME
module load matlab
matlab -nodesktop -nosplash -r filename
echo PBS: End Time is `date`
exit
General Linux Commands
Command | Description | Example |
cd | Change directory | cd work cd .. cd /gpfs/work/x/xyz123 |
clear | Clears command window | |
dos2unix | Convert dos file to unix. This is necessary if you make a pbs script on windows and upload to hammer | dos2unix pbsscript.pbs |
module load | Load a program to your session. This DOES NOT execute the program, you do this before executing the program | module load comsol/5.2 |
rm -rf ~/.Trash/* | Clear recycle bin on Hammer | |
ssh -Y lionxg.rcc.psu.edu | Connect to a lion cluster (swap xg with any other lion cluster if you want) | |
qsub | Submit a pbs script to cluster (must log in to cluster first, does not work on hammer) | qsub pbsscript.pbs |
qstat | See the list of what has been submitted to the cluster. | |
qstat -u xyz123 | See your current qued jobs, job#, time, etc | |
qdel | Cancel a job using the job number | qdel 8675309 |
qdel $(qselect -u xyz123) | Cancel all current jobs for your username | |
mkdir | Make a directory | mkdir backup mkdir backup/v0-01 |
cp | Copy a file | cp btscm.mph backup/v0-02 |
mv | Move a file | mv btscm.mph backup/v0-01 |