Technically a modern, multi processor supercomputer is a network of computers working together in parallel to solve a problem.
First of all find out the hardware components & resources needed. We shall need one head node, at least a dozen identical compute nodes, a power distribution unit, an Ethernet switch & a rack. Find out the electrical demand, cooling & space needed. We have to decide also on what IP address we want for our private networks, what name should be given to the nodes, what software packages we want to install, and which type of technology we want to provide the parallel computing capabilities.
We shall need to assemble the computing nodes or arrange pre build servers. We shall select a computer server chassis which can maximize the space, cooling & energy efficiency. Or we can use twelve or so used, outdated servers which will outweigh the total of their parts yet save us a reasonable amount. All processors, network adapters, and motherboards should be rated for the whole system to play together in good manner. We should not forget about RAM & storage for each node & at least one optical drive for the main or head node.
To install the servers into the rack, begin from the bottom because rack is not top heavy. We shall need a companion to help us to handle it as the dense servers may be heavy & leading them into the rails holding them into the rack is difficult.
To install the Ethernet switch above the server chassis, for the configuration of the switch, permit for jumbo frame sizes of 9000 bytes, we have to set up the IP address to the static address we decided on in step 1, and turn off unessential routing protocols such as SMTP snooping.
To install the Power Distribution Unit (PDU), depends on how much current our nodes may require at max. Load, we may require 220 volts for high performance computing.
With everything installed, we can begin the configuration process. Linux is the de facto Operating System for HPC clusters which not only is it the ideal environment for scientific computations, but it does not cost anything to install it on thousands of nodes. We should imagine how much it would cost for installation of Windows on all those nodes. So, first of all, install the newest version of the motherboard BIOS & firmware, which should be the same on each node and then install other things on each node.
Install resource manager, the message passing interface & other essential libraries. We have to manually setup the necessary software to enable the parallel computing mechanisms.
Network the computing nodes together. The head node sends the computing tasks to the computing nodes which send the result back and also send messages to each other. The faster is better.
Test the cluster. We may want to do before releasing all this computing power to our users is to test its performance. High Performance Lynpack (HPL) benchmark is very famous for measuring the computational speed of cluster. We shall compile it from source with all possible optimizations our compiler offers for the architecture we chose.