The Research Computing User's Guide
   Next

The Research Computing User's Guide

September 22, 2024

Jason W. Bacon

Instructor
 University of Wisconsin -- Milwaukee
 Department of Computer Science

Table of Contents

I. Research Computing
1. Computational Science
2. Where do I get the Software?
3. Using Unix
4. Unix Shell Scripting
5. Data Management
II. Parallel Computing
6. Parallel Computing
7. Job Scheduling
8. Job Scheduling with LPJS
9. Job Scheduling with SLURM
10. Job Scheduling with HTCondor
11. Job Scheduling with PBS (TORQUE)
12. Job Scheduling with LSF
III. High Performance Programming
13. Computer Hardware
14. Software Development
15. Data Representation
16. Introduction to High-Performance Programming
17. Data Types
18. Basic Terminal Input/Output (I/O)
19. Conditional Execution
20. Loops (Iterative Execution)
21. Subprograms
22. Building with Make
23. Memory Addresses and Pointers
24. Arrays
25. Strings
26. File I/O
27. Matrices
28. Software Performance
29. Structures
30. Object Oriented Programming
31. The Preprocessor
32. Software Project Management
IV. Parallel Programming
33. Parallel Programming
34. Programming for HTC
35. Programming for HPC
V. Systems Management
36. Systems Management
37. Popular Unix Platforms in Science
38. Platform Selection
39. System Security
40. Software Management
41. Running Multiple Operating Systems

List of Figures

2.1. FreeBSD Ports Build Options
2.2. Visualizing Gene Neighborhoods with Matplotlib
3.1. Hot Keys
3.2. Sample of a Unix File system
3.3. File streams
3.4. Standard streams
3.5. Redirecting standard output
3.6. Colorized grep output
4.1. Editing a script in APE
15.1. Monte Carlo Dart Board
16.1. Compilation
19.1. Flowchart for a simple conditional
32.1. Getting the Gitlab Clone URL
37.1. FreeBSD Installer Disk Partition Screen
37.2. FreeBSD Installer ZFS Options
37.3. FreeBSD Installer ZFS RAID Level Selection
37.4. FreeBSD Installer ZFS Disk Selection
37.5. FreeBSD Installer ZFS Commit
37.6. FreeBSD ZFS Status
41.1. Windows as a Guest under VirtualBox on a Mac Host
41.2. CentOS 7 with Gnome Desktop as a Guest under VirtualBox
41.3. FreeBSD with Lumina Dekstop as a Guest under VirtualBox

List of Tables

1.1. Computation Time Line
3.1. Partial List of Unix Operating Systems
3.2. Pkgsrc Build Times
3.3. Default Key Bindings in some Shells
3.4. Globbing Symbols
3.5. Special Directory Symbols
3.6. Common hot keys in more
3.7. Unix Commands
3.8. Common Extensions
3.9. Standard Streams
3.10. Redirection Operators
3.11. Run times of pipes with cat
3.12. RE Patterns
3.13. Column headers of top command
3.14. Reserved Environment Variables
4.1. Conventional script file name extensions
4.2. Shell Start Up Scripts
4.3. Printf Format Specifiers
4.4. Special Character Sequences
4.5. Test Command Relational Operators
4.6. Test command file operations
4.7. C Shell Relational Operators
4.8. Shell Conditional Operators
4.9. Compression tool for each filename extension
13.1. Example Memory Map
13.2. The Memory Hierarchy
14.1. Example MIPS Instruction
14.2. Selection Sort of 100,000 Integers on AMD Opteron
14.3. Selection Sort of 100,000 Integers on Intel i7
15.1. Basic Binary Units
15.2. Prefixes for larger quantities
15.3.
15.4. Highest power of each base <= 439
15.5. Binary/Octal Conversion
15.6. Binary/Hex Conversion
15.7. Unsigned Binary Integer Ranges
15.8. Powers of 2
15.9. Powers of 2
15.10. 8-bit sign-magnitude format
15.11. Two's Complement Integer Ranges
15.12. IEEE 32-bit Format
15.13. IEEE 64-bit Format
17.1. C Data Types
17.2. Fortran 90 Data Types
17.3. Memory Map
17.4. C Constants and Types
17.5. C Escape Sequences
17.6. Fortran 90 Constants and Types
17.7. Basic Math Operators in C
17.8. Bitwise Operators in C
17.9. Basic Math Operators in Fortran
17.10. Explicit Conversion Functions
18.1. Standard Stream Names
18.2. Format specifiers for printf()
18.3. Format specifiers for scanf()
18.4. Standard Stream Names
18.5. Common Format Descriptors
19.1. Relational Expressions
19.2. Relational Operators
19.3. Boolean Operators
21.1. Program Sizes
21.2. Pass by Value
21.3. Pass by Reference
22.1. Standard Make Variables
24.1. Memory Map of an Array
24.2. Memory Map of list and i
27.1. Time Table
27.2. Temperature
27.3. Generic 2D Matrix
28.1. Common Orders of Operation
40.1. Computation Time Line
40.2. Package Manager Comparison

List of Examples

2.1. A Typical Caveman Install
3.1. Practice Break
3.2. Practice Break
3.3. Practice Break
3.4. White space in an Argument
3.5. Practice Break
3.6. Practice Break
3.7. Practice Break
3.8. Practice Break
3.9. Practice Break
3.10. Practice Break
3.11. Practice Break
3.12. Practice Break
3.13. Practice Break
3.14. Practice Break
3.15. Practice Break
3.16. Practice Break
3.17. Simple Pipe
3.18. Filters
3.19. Practice Break
3.20. Practice Break
3.21. Practice Break
3.22. Practice Break
3.23. Practice Break
3.24. Practice Break
3.25. Practice Break
3.26. Practice Break
3.27. Practice Break
3.28. Practice Break
3.29. Practice Break
3.30. Practice Break
3.31. Practice Break
3.32. Practice Break
3.33. Pushing data with rsync
3.34. Pulling data with rsync
3.35. Practice Break
3.36. Practice Break
3.37. Practice Break
4.1. A Simple Bourne Shell Script
4.2. A Simple C-shell Script
4.3. Multiple File Downloads
12.1. Simple LSF Batch Script
12.2. Batch Interactive Script
12.3. Batch Parallel Script
12.4. LSF and OpenMPI
15.1. A fixed-point system
15.2. A modulo-100 System
15.3. Conversion from Decimal
15.4. 3710 to binary
15.5. 13510 to octal
15.6. 13510 to hex
15.7. A 4-bit unsigned binary number system
15.8. Improving Speed with Integers
16.1. A Simple C Program
16.2. A Simple Fortran Program
17.1. Precedence and Order of Evaluation
20.1. While Loop
20.2. A More Flexible Sine Printer
20.3. Unstructured Fortran Loop
20.4. Unstructured C Loop
20.5. Infinite Loop
21.1. Ordering Pizza
21.2. Sorting
21.3. Sample C Code Passing by Value
27.1. Comparison of Row-major and Col-major Access
28.1. Selection Sort
28.2. Binary Search

1. Acknowledgments

Many thanks to (in alphabetical order) Aishwarya Shrestha, Anisha Tasnim, and Shamar Rhadre Webster, for extensive feedback and corrections.

   Next
   2. Practice Problem Instructions