Submitting NAMD Jobs

Submitting NAMD Jobs

Prerequisites


Ideally you should be able to run your model locally before submitting to GridMarkets. GridMarkets will run your compute-intensive model on our powerful servers, releasing your local machine for further modelling.
Before submitting a NAMD job, please ensure you have installed Envoy and logged-in.

Preparing Your Files

Assemble all your input files into a directory and create a text file (the "job file") in that directory with the relevant parameters needed for the submission.  A submission is one or more jobs that will be run sequentially.  If a job fails or is stopped then subsequent jobs will not be run.  For NAMD running a namd2 job, each line of of this "job file" specifies each job to be run sequentially and should be in the format:
example.in
Where example.in is the input file to be run. On submission the GridMarkets system will run
/usr/local/NAMD_2.14_Linux-x86_64-multicore-CUDA/charmrun.cuda /usr/local/NAMD_2.14_Linux-x86_64-multicore-CUDA/namd2.cuda +devices 0 +auto-provision example.in
and the contents of the stdout will be available in Envoy as the job progresses.  Note that the "namd2.cuda +devices 0 +auto-provision" is automatically populated so please do not add to the job file - it only needs the input filename. 
Therefore please do NOT include any switches or redirects in the job file else your submission may fail and/or you may not be able to monitor its progress.  If you have any special requirements then please contact us at pharma@gridmarkets.com
If the log file exceeds 5Mb then just the last 1000 lines will be displayed.

Submitting via the Submission GUI or Command Line

There are two ways to submit an NAMD job to GridMarkets.  The first involves using a Graphical User Interface (GUI) and the second involves using a Command Line Interface (CLI).  The GUI simplifies the submission process and is recommended for use by first time users.  The CLI is designed more for power users and for programmatic submissions.  Jump to "Submitting with the Submission GUI" to learn more about the GUI option and to "Submitting with the CLI" to learn more about the CLI option.

Submitting with the Submission GUI

Download the GridMarkets Submission GUI for your OS.  Once installed and running:
  1. Select "NAMD" from the Application dropdown
  2. "2.14" from the Version dropdown as currently that is only supported
  3. Select the job file using the BROWSE button
  4. Provide a "Project Name" under which the files will be uploaded to
NAMD submission interface
Note that if you make a new submission with the same Project Name then the files will be overwritten, as will any results when the job runs.  However, this can be useful if you just need to make modifications to some of the input files as only files that have changed will be uploaded, thus saving you time.

You can change the Submission Name from the default of the job file name to better track your jobs.  You may also optionally request to auto-download all resulting files.  Finally, select the following from the drop-downs:
  1. Machine Type = please see our website for the current options, their specifications and pricing
  2. Job Budget & Action (optional) = a credit limit for each job - if any job consumes the specified number of credits then you can elect to:
    1. Alert = receive an email;
    2. Terminate = immediately kill the job, but you will not receive any resulting files, so please contact pharma@gridmarkets.com if you wish them to be returned;
    3. Suspend = allow the job to complete, but do not run any subsequent jobs in the sequence.
Click SUBMIT to have Envoy submit the job to GridMarkets servers.

Submitting with the CLI

The precompiled binary can be downloaded and extracted for Windows, Linux or Mac and placed in your preferred location. This binary can be added to your Linux or Mac PATH by running:

export PATH=$PATH:/path/to/binary
Since NAMD only uses one GPU device per machine and doesn't benefit from running on multiple machines in parallel, the command line to submit would be:
gridmarkets-cli submit --job-file path/to/files_for_submission/job_file --project-name my-project-name 
--product namd --product-version 2.14 --machine-type pharma-low --machines 1 --credits-budget 0 --upload-all-files --confirm
Where:
  1. my-project-name = the name of the project that all the files will be under.  Note that you can make new submissions under the same project name if you want to reuse any files without having to re-upload them, but they will overwrite any files that have already been uploaded so do not do this if you are currently running a job using that project's files.  You can also just submit under a new project name.
  2. path/to/files_for_submission/job_file = the full path to the job file you created above.  All the files and sub-directories in that directory will be uploaded when submitting the job with --upload-all-files, else only the job file will be uploaded.
  3. Note that we are assuming version 2.14 (--product-version 2.14), but you can skip this option to be presented in the next step with the available options to run.
You will then be prompted by the CLI for:
  1. The type of machine you'd like to use - please see the pricing page for more information on the options; set machine_type to pharma-low, pharma-med or pharma-high
  2. The submission name - a user-friendly name for this submission so you can easily identify it in Envoy, or accept the default date & time
  3. An optional Job Budget & Action = a credit limit for each job (or set --credits-budget to zero to skip) - if any job consumes the specified number of credits then you can elect to:
    1. Alert = receive an email;
    2. Terminate = immediately kill the job, but you will not receive any resulting files, so please contact pharma@gridmarkets.com if you wish them to be returned;
    3. Suspend = allow the job to complete, but do not run any subsequent jobs in the sequence.
  4. Confirmation that you wish to submit, so that can you cancel by hitting "n" if needed, or ENTER or "y" to submit
The CLI will then instruct Envoy to upload all the files in the directory and submit the job to the GridMarkets system.

Monitoring & Controlling Your Job

Please see the Envoy video on how to track your job's progress, stopping & restarting, viewing logs, etc.

Getting Your Results

Any generated files will automatically be downloaded by Envoy to the same directory as the inputs, but this will only be on the machine it was submitted from.  You can always go to the "Files" section of the project in Envoy to manually download any files.

Walkthrough Example

In this section we will walk you through a practical example where we run the NAMD

DHFR 4fs NVE benchmark from the Amber 22 Benchmark Suite.  Specifically, we will be running the ~23K atom Dihydrofolate Reductase Enzyme simulation in explicit water:


DHFR NPT HMR 4fs = 23,558 atoms
This is a more realistic version of the JAC (DHFR) benchmark
run with what would be typical NPT production parameters. It uses shake with hydrogen mass repartitioning and a 4fs timestep. It has a 8 angstrom cutoff, runs in the NPT ensemble and writes to the output and trajectory file every 2500 steps (10ps). Additionally it writes to the restart file every 50000 steps. NSTLIM is set to 100,000 by default but should be increased for larger processor count runs. For performance reasons it is set to produce NETCDF binary trajectory files.
# Her1-Her1

set restart T
set restraint F

set p         1.0
set T         300.0
set num_steps 1000

set new      465K_Her1-Her1
set previous relres

# CHARMM force field
structure         H1H1.psf      # must be in X-PLOR format!
paratypecharmm    on
parameters        par_all27_prot_lipid.prm
coordinates       H1H1.pdb
exclude           scaled1-4
1-4scaling        1.0

# use restraints for setup
if { $restraint == "T" } {
  constraints       on
  consexp           2
  consref           H1H1.pdb
  conskfile         H1H1.pdb
  conskcol          B
}

# restart options
if { $restart == "T" } {
  binvelocities       $previous.vel
  bincoordinates      $previous.coor
  ExtendedSystem      $previous.xsc
  firsttimestep       0
} else {
  temperature   $T
}

# temperature coupling
langevin             on
langevinHydrogen     on
langevinTemp         $T
langevinDamping      1

# pressure coupling
useGroupPressure     yes

LangevinPiston       on
LangevinPistonTarget $p
LangevinPistonPeriod 200
LangevinPistonDecay  100
LangevinPistonTemp   $T

useFlexibleCell      yes
useConstantArea      yes

# output
outputname         $new
outputEnergies     100
outputPressure     100
outputTiming       100
binaryoutput       yes

# write restart
restartname        $new
restartfreq        1000
binaryrestart      yes

# trajectory file
DCDfile            $new.dcd
DCDUnitCell        yes
DCDfreq            5000

# eXtended System Trajectory
XSTfile            $new.xst
XSTfreq            1000

# wrap coordinates
wrapAll            on
wrapNearest        on

# multipe time step settings
nonbondedFreq      1
fullElectFrequency 1

# rigid bonds
useSettle          on
rigidBonds         all
rigidIterations    100

# cutoffs and non-bonded pair lists
switchdist         8
cutoff             10
switching          on
pairlistdist       12
pairlistsPerCycle  1
stepspercycle      20

# particle mesh ewald
PME                on
PMEGridSpacing     1.0

# cell bassis vectors are read from restart file
if { $restart != "T" } {
  cellBasisVector1   194 0   0
  cellBasisVector2   0   149 0
  cellBasisVector3   0   0   157
}
timestep 2.0 # in fs
run $num_steps


The required input files are as follows:
    H1H1.in
    H1H1.pdb 
    H1H1.psf
    par_all27_prot_lipid.prm
    relres.coor
    relres.vel
    relres.xsc

In order to run this in GridMarkets we first create a folder in which we place our input files referenced in the table above.  We also need to create the GridMarkets job file which is a text file that contains the executable(s) to run and the command line options, so simply just:
H1H1.in
Next, we start up both Envoy and the Submission GUI and submit our job file:


Once the files have been uploaded, we can see the status of the example submission running in Envoy:
Example NAMD job - running
We can see the simulation has finished in less than 2 minutes and only required 0.02 credits.
Example NAMD job - completed

Once the job has completed, the results can be downloaded manually, or the Auto-download results checkbox can be used during submission for automatic retrieval.
Resulting files after successful job completion


















    • Related Articles

    • Submitting Amber Jobs

      Prerequisites Ideally you should be able to run your model locally before submitting to GridMarkets. GridMarkets will run your compute-intensive model on our powerful servers, releasing your local machine for further modelling. All users must obtain ...

    • Submitting GROMACS Jobs

      Prerequisites Ideally you should be able to run your model locally before submitting to GridMarkets. GridMarkets will run your compute-intensive model on our powerful servers, releasing your local machine for further modelling. Please ensure you have ...

    • Running multiple jobs simultaneously

      You can run any number of jobs on any number of machines simultaneously up to the number of machines available for your Service Level. GridMarkets dynamically scales to increase the overall number of machines available if you change your service ...

    • Restarting stopped or suspended jobs

      When you run out of credits, your jobs will either be stopped or suspended if they are parallel jobs (i.e. parts that are running will be allowed to complete, but non-started ones will not run), but your files will remain.  You can top up your ...

    • Use your application for other work after submission

      Once you have submitted your project to Envoy you can return to using your application as it will transfer all files and submit your job in the background.  Once you see the submission in Envoy you can even turn off your machine.  Envoy will ...