Skip to content

Latest commit

 

History

History
74 lines (57 loc) · 9.25 KB

0_Prerequisites.md

File metadata and controls

74 lines (57 loc) · 9.25 KB

Prerequisites

Assumptions

Throughout this tutorial-style introduction to working with HPC codes, we will assume that the reader:

  • Is familiar with the C programming language.
  • Is familiar with using a command line in a Linux environment. Since the supercomputers used for this course are accessed exclusively through a terminal, this is essential. This tutorial assumes that the shell used is Bash.
  • Can efficiently navigate documentation and reference material, both offline, e.g. man pages or language specifications, and online, e.g. discussion forums or mailing lists, in order to independently find answers to questions and solve programming and system usage issues.

What this tutorial is not

While there may be some overlap with the lectures, particularly concerning tools and programming frameworks, it is important to keep in mind that this tutorial is not a replacement for the lectures. It also does not aim to:

  • Give a complete course on any of the programming languages and frameworks discussed
  • Provide a "walkthrough" for the coursework
    • In particular, copy-pasting all the commands into your terminal will not get you closer to the goals of this unit. Do not type anything into a terminal until you've understood exactly what it does and why it is useful to you.
  • Explain how to write a good report (although this will be covered later in class)
  • Replace engagement in labs, including any feedback, suggestions, and clarifications
  • Deter students from taking novel approaches or in any way limit their use of technologies not discussed in these pages
    • For example, even though this repository shows sample code in C, feel free to write Fortran!
  • Substitute for interactive learning, e.g. in lectures, labs, and on the forum

Required (and useful) tools

To connect to the BlueCrystal supercomputer, you will need an SSH client. If you use any Unix derivative, chances are you have a working ssh command for this purpose; if you're on Windows, some of your options are listed below.

To write your code, we suggest you use an editor that you can customise to your liking, e.g. indentation or syntax highlighting. Many editors have out-of-the-box support for the languages you will likely use throughout this course; these include:

  • Visual Studio Code and Atom, which are modern open-source text editors with rich plugin support, although they are written in JavaScript.
  • gedit or Kate, as your DE's packaged text editor.
  • Vim and Emacs, in the terminal.

If you want to be able to compile and run your code locally (i.e. on your own laptop, or on a machine in the lab), you will need a compiler supporting your languages and frameworks. This is not required, but it may ease and speed up debugging and correctness testing. For C, both GCC and LLVM/Clang are available for virtually all Linux distributions and support OpenMP. As as student, you can also get access to a copy of the Intel Compiler. To compile MPI programs, you will need to install an MPI implementation, with the most common choices being Open MPI and MPICH; it does not matter which one you choose, but make sure you are not following the documentation for the other choice!

To transfer files between your machine and the supercomputer, you can use scp, which is part of OpenSSH and likely already available on your Linux box. There is also rsync, which may speed up repeated trasnfers. However, manually transferring files every time you make a change is cumbersome, so a better alternative is to set up SSHFS to mount a remote directory as a virtual drive on your local machine. Another option is to set up your text editor to automatically sync with a remote folder (example for VS Code).

We provide the starting code and some of the examples through GitHub repositories. Although you can obtain the files without using git, we strongly encourage you to use version control for your assignment. Without such a system, you will find yourself saving multiple copies of your files with attempted optimisations, and you risk losing track of which changes stay and which go. If you are not familiar with Git, a good starting point is the Atlassian tutorial series—use it, it may well save you a great deal of wasted effort!

Suggestions for Windows Users

While we encourage you to use a Linux machine, or at least a *nix environment, this is not strictly required. If you use Windows, you are advised to:

  • Set up a more Linux-like environment, in order to avoid many (sometimes subtle!) inter-platform issues. Avoid using cmd, as it lacks features; some better options are (in rough order of "niceness"):
    • Make use of Windows Subsystem for Linux (WSL) to obtain a full Unix shell. This will give you an environment which is virtually indistinguishable from running full Linux for the purpose of this course. WSLtty is a very good terminal for use with WSL; ConEmu is also good, but you may run into text display issues.
    • If you are not on Windows 10 (although arguably you should be!), Cygwin can provide a Linux-like environment that makes many Linux tools available on Windows.
    • Install Linux in a Virtual Machine. Both VirtualBox and VMWare provide free virtualisation software. If in doubt which Linux distribution to choose, go with Ubuntu (for large community support), Fedora (for a stable an up-to-date distribution), or Debian (for long-term stability).
    • Install Git for Windows and choose the option to use the packaged terminal (mintty). This will have Bash and OpenSSH packaged in alongside git.
    • ConEmu (mentioned above) includes a packaged, but minimal, version of Bash that you can use without either WSL or Cygiwn. This may be enough for your needs. cmder is ConEmu with more sane defaults and Bash and OpenSSH built-in.
    • Use Microsoft's OpenSSH for Windows or a stand-alone SSH client (PuTTY is a reliable and featureful choice) to allow you to connect to the supercomputer remotely. Note that if you choose this option only, you will not be able to test your code on your own machine. This likely needs to be paired with a SFTP client, such as WinSCP.
  • Read about issues with line endings and make sure you work around them. Also keep in mind that on Windows NTFS is case-insensitive (but case-preserving!), so a file that looks as if it's named Makefile may actually be MaKEfILe—take care when (re)naming files.
  • Use a text editor designed with programming in mind, and not Notepad or—even worse—Word or WordPad. This will help you avoid silly mistakes such as spaces instead of tabs in Makefiles. Visual Studio Code is a very good (extensible) editor, and Notepad++ is a more lightweight alternative. There is a related issue with file extensions: if you have those hidden (which is the default in Windows), make sure you editor doesn't automatically add one, e.g. Makefile.txt.

Note that the rest of this tutorial assumes a Linux environment, and so will only show command examples for Linux. If you use Windows, or software that differs from our standard choices, you are responsible for setting it up in such a way that you can follow the tutorial. As long as you choose reasonably popular software, there should be plenty of support available online, and we will do our best to highlight any issues that we know might impact non-Linux platforms.