Submitting Amber Jobs

Submitting Amber Jobs

Prerequisites

Ideally you should be able to run your model locally before submitting to GridMarkets. GridMarkets will run your compute-intensive model on our powerful servers, releasing your local machine for further modelling.  All users must obtain an Amber22 license if they don't already have one - please see https://ambermd.org/GetAmber.php#amber.  Non-commerical users can obtain a license for free by clicking on the "Accept non-commercial license and download" button; commercial users need to follow the instructions under "Getting Amber22 for commercial use."
Please ensure you have installed Envoy and are logged-in before submitting.

Preparing Your Files

Assemble all your input files into a directory and create a text file (the "job file") in that directory with the relevant parameters needed for the submission.  A submission is one or more jobs that will be run sequentially.  If a job fails or is stopped then subsequent jobs will not be run.  For Amber running a pmemd job, each line of of this job file specifies each job to be run sequentially and should be in the format:
pmemd.cuda -O -i mdin -p prmtop -c inpcrd -r restrt -x mdcrd -inf mdinfo -ref refc
Where:
  1. pmemd.cuda = the specific Amber executable to run such as pmemd.cuda [defaults to pmemd.cuda_SPFP], pmemd.cuda_DPFP, pmemd.cuda_SPFP, sander, etc.
  2. -O = overwrite any existing files with the same name
  3. -i mdin = the Amber input file [input, required]
  4. -p prmtop = the parameter/topology specification file [input, required]
  5. -c inpcrd = the coordinate / previous restart file [input, required]
  6. -r restrt = the restart file [output]
  7. -x mdcrd = the trajectory file to be written [output]
  8. -inf mdinfo = the job progress files [output, optional]
  9. -ref refc = the reference coordinates when using restraints [input, optional]
On submission the GridMarkets system will run (using the above example)
$AMBERHOME/bin/pmemd.cuda -O -i mdin -o mdout -p prmtop -c inpcrd -r restrt -x mdcrd -inf mdinfo -ref refc
and the contents of the "mdout" log file will be available in Envoy as the job progresses, and available under a unique job ID for download from Envoy once the job completes.
Note that "-o mdout" switch is automatically added to the command line, so please do NOT include any competing switches or redirects in the job file else your submission may fail and/or you may not be able to monitor its progress.  If you have any special requirements then please contact us at pharma@gridmarkets.com
Please note that MPI execution including multi-GPU and Replica Exchange type runs is not currently supported. We are working on adding MPI support. For now if you have any special requirements for MPI runs then please contact us at pharma@gridmarkets.com

Running Chained Sequential Jobs

If you want to run multiple calculations sequentially, for example to carry out heating and then multiple steps of equilibration you simply specify each job to be run on a separate line of the job file - e.g.
pmemd.cuda -O -i heat.mdin -p prmtop -c minimized_struct.inpcrd -r heat.rst -x heat.mdcrd -inf heat.mdinfo
pmemd.cuda -O -i equil.mdin -p prmtop -c heat.rst -r equil1.rst -x equil1.mdcrd -inf equil1.mdinfo
pmemd.cuda -O -i equil.mdin -p prmtop -c equil1.rst -r equil2.rst -x equil2.mdcrd -inf equil2.mdinfo

Submitting with the GUI

First, download the GridMarkets Submission GUI for your OS. Once installed and running:
  1. Select "Amber" from the Application dropdown
  2. "2022" from the Version dropdown as currently only Amber 22 is supported
  3. Select the job file using the BROWSE button
  4. Provide a "Project Name" under which the files will be uploaded to

Note that if you make a new submission with the same Project Name then the provided input files will be overwritten, as will any results when the job runs.  However, this can be useful if you just need to make modifications to some of the input files as only files that have changed will be uploaded, thus saving you time.

You can change the Submission Name from the default of the job file name to better track your jobs.  You may also optionally request to auto-download all resulting files.  Finally, select the following from the drop-downs:
  1. Machine Type = please see our website for the current options, their specifications and pricing - performance should approximately match the Amber GPU Benchmarks by GPU type
  2. Job Budget & Action (optional) = a credit limit for each job - if any job consumes the specified number of credits then you can elect to:
    1. Alert = receive an email;
    2. Terminate = immediately kill the job, but you will not receive any resulting files, so please contact pharma@gridmarkets.com if you wish them to be returned;
    3. Suspend = allow the job to complete, but do not run any subsequent jobs in the sequence.
Click SUBMIT to have Envoy submit the job to GridMarkets servers.

Submitting with the CLI

The precompiled CLI binary can be downloaded and extracted for Windows, Linux or Mac and placed in your preferred location. The binary can be added to your Linux or Mac PATH by running:

export PATH=$PATH:/path/to/binary
Since Amber only uses one GPU device per machine and doesn't benefit from running on multiple machines in parallel, the command line to submit would be:
gridmarkets-cli submit --job-file path/to/files_for_submission/job_file --project-name my-project-name 
--product amber --product-version 2022 --machine-type pharma-low --machines 1 --credits-budget 0 --upload-all-files --confirm
Where:
  1. my-project-name = the name of the project that all the files will be under.  Note that you can make new submissions under the same project name if you want to reuse any files without having to re-upload them, but they will overwrite any files that have already been uploaded so do not do this if you are currently running a job using that project's files.  You can also just submit under a new project name.
  2. path/to/files_for_submission/cmd_file = the full path to the job file you created above.  All the files and sub-directories in that directory will be uploaded when submitting the job with --upload-all-files, else only the job file will be uploaded.
  3. Note that we are assuming version 2022 (--product-version 2022), but you can skip this option to be presented in the next step with the available options to run.
You will then be prompted by the CLI for:
  1. The type of machine you'd like to use - please see our website for the current options, their specifications and pricing - performance should approximately match the Amber GPU Benchmarks by GPU type or set machine_type to pharma-low, pharma-med or pharma-high
  2. The submission name = a user-friendly name for this submission so you can easily identify it in Envoy, or accept the job file name as the default
  3. An optional Job Budget & Action = a credit limit for each job (or set --credits-budget to zero to skip) - if any job consumes the specified number of credits then you can elect to:
    1. Alert = receive an email;
    2. Terminate = immediately kill the job, but you will not receive any resulting files, so please contact pharma@gridmarkets.com if you wish them to be returned;
    3. Suspend = allow the job to complete, but do not run any subsequent jobs in the sequence.
  4. Confirmation that you wish to submit, so that can you cancel by hitting "n" if needed, or ENTER or "y" to submit.
The CLI will then instruct Envoy to upload all the files in the directory and submit the job to the GridMarkets system.

Monitoring & Controlling Your Submission

Please refer to the Envoy video on how to track your submission progress, stopping & restarting jobs, viewing logs, etc.

Getting Your Results

If you selected the option to automatically download results then any files generated by each jobs as it completes will be downloaded by Envoy to the same directory as the inputs, but this will only be on the machine it was submitted from.  If you connect to Envoy from a different machine you will need to manually download outputs from the "Files" section of the project.


Walkthrough Example

In this section we will walk you through a practical example where we run the AMBER DHFR 4fs NVE benchmark from the Amber 22 Benchmark Suite.  Specifically, we will be running the ~23K atom Dihydrofolate Reductase Enzyme simulation in explicit water.  HERE are sample project files that you can get started with.


DHFR NPT HMR 4fs = 23,558 atoms
This is a more realistic version of the JAC (DHFR) benchmark
run with what would be typical NPT production parameters. It uses shake with hydrogen mass repartitioning and a 4fs timestep. It has a 8 angstrom cutoff, runs in the NPT ensemble and writes to the output and trajectory file every 2500 steps (10ps). Additionally it writes to the restart file every 50000 steps. NSTLIM is set to 100,000 by default but should be increased for larger processor count runs. For performance reasons it is set to produce NETCDF binary trajectory files.
Typical Production MD NVE with reasonable energy conservation, HMR, 4fs.
 &cntrl
    ntx=5, irest=1,
    ntc=2, ntf=2,
    nstlim=100000,
    ntpr=2500, ntwx=2500,
    ntwr=50000,
    dt=0.004, cut=8.,
    ntt=1, tautp=10.0,
    temp0=300.0,
    ntb=2, ntp=1, barostat=2,
    ioutfm=1,
 /


The required input files are as follows:
      Amber mdin file: dhfr_npt_4fs.mdin
      Topology file: dhfr_hmass.prmtop
      Input Coordinate File: dhfr_npt_4fs.inpcrd

In order to run this in GridMarkets we first create a folder in which we place our input files referenced in the table above.  We also need to create the GridMarkets job file which is a text file that contains the executable(s) to run and the command line options:
pmemd.cuda -O -i dhfr_npt_4fs.mdin -p dhfr_hmass.prmtop -c dhfr_npt_4fs.inpcrd -r dhfr_npt_4fs.rst -x dhfr_npt_4fs.mdcrd -inf dhfr_npt_4fs.mdinfo
Next, we start up both Envoy and the Submission GUI and submit our cmdfile:

Once the files have been uploaded, we can see the status of the example submission running in Envoy:
Amber example job - running
We can see the simulation has finished in less than 2 minutes and only required 0.03 credits.
Amber example job - completed
Once the job has completed, the results can be downloaded manually, or the Auto-download results checkbox can be used during submission for automatic retrieval.
Resulting files after successful job completion




















    • Related Articles

    • Submitting NAMD Jobs

      Prerequisites Ideally you should be able to run your model locally before submitting to GridMarkets. GridMarkets will run your compute-intensive model on our powerful servers, releasing your local machine for further modelling. Before submitting a ...

    • Submitting GROMACS Jobs

      Prerequisites Ideally you should be able to run your model locally before submitting to GridMarkets. GridMarkets will run your compute-intensive model on our powerful servers, releasing your local machine for further modelling. Please ensure you have ...

    • Running multiple jobs simultaneously

      You can run any number of jobs on any number of machines simultaneously up to the number of machines available for your Service Level. GridMarkets dynamically scales to increase the overall number of machines available if you change your service ...

    • Restarting stopped or suspended jobs

      When you run out of credits, your jobs will either be stopped or suspended if they are parallel jobs (i.e. parts that are running will be allowed to complete, but non-started ones will not run), but your files will remain.  You can top up your ...

    • Use your application for other work after submission

      Once you have submitted your project to Envoy you can return to using your application as it will transfer all files and submit your job in the background.  Once you see the submission in Envoy you can even turn off your machine.  Envoy will ...