Pay No Attention to the Server Behind the Curtain

Computer servers, hidden scaled complexity that works.


Servers? Desktops? They’re both computers, right? Well, yes, in the same sense that an elephant and a mouse are both mammals.

You’re familiar with a desktop or a laptop computer. What you may not know is there are uncounted ranks and ranks of computer servers behind the scenes providing you with, well, services. Servers are the things you reach over the network when you’re browsing to a web site or looking at a map online or sending an email. To even get to these servers, you unknowingly use other servers just to convert something like “” into a network address that other servers can use to route your request to the right server. There’s this whole infrastructure of servers upon servers upon servers that all manage to work together to get you where you want to go on the internet.

There’s not just one server waiting for you to contact it over the internet, either. For example,  one estimate from 2012 stated Facebook had 180,000 servers. That’s 180,000 computers working together just to do Facebook things for you and everyone else. That estimate is not counting all of the other servers working for you between Facebook and your desktop computer. When I think of it, I find myself astonished at how well our internet ecosystem actually works.

I can speak from experience about one server building that supplied high-performance computing services to a University. They had 5,500 servers on the middle floor of a three story building. The bottom floor was a refrigerated air intake for the entire middle floor, the top floor was the hot air exhaust from the entire middle floor. What really blew me away was that the air refrigeration units and internal building fans had not one, but two backup electrical generators. That seemed excessively paranoid until I was told that if the cold air stopped being supplied to the middle floor with its 5,500 servers for longer than 30 seconds, the microprocessors in all of these servers would be destroyed from their own waste heat. Just turning off all 5,500 servers immediately would not work since all of the existing heat has to go somewhere. Ohhhh. Maybe add a third backup electrical generator?

So, yes, servers on the internet are computers like your desktop computer, but on a scale difficult to fully grasp. And…hidden behind the curtain.

Computer Hardware vs. Software

Maybe a computer is a hardware and software system?

Once upon a time, there were three main types of computer people: the scientists, who used the Fortran programming language; the business people, who used the Cobol programming language; and the computer hardware designers. I was in the latter group.

The computer hardware designers used various extremely low-level assembly languages (and later the C programming language) to make their hardware designs work. Cobol and Fortran are high-level, rather abstract computer languages that require several layers of software infrastructure between them and the bare metal computer. These layers of system software infrastructure were created by the hardware designers at first. My point here is the computer hardware designers had to write the software to make their creations work for the scientists and business people.

Unsurprisingly, I came up through the ranks of the computer profession thinking that hardware and software development were two sides of the same coin. I viewed a computer as a system of hardware and software with a very fuzzy and malleable dividing line between the two.

I woke up one day and realized that somehow computer hardware and software had gotten separated and the two groups often viewed each other with distrust and even disdain. This often resulted in much finger-pointing and blaming when things at the hardware/software boundary did not work.

I tried sharing my view with people that a computer is a system that requires thoughtful design tradeoffs between its hardware and software aspects. This view was not well-received. The hardware/software distinction became even fuzzier (in my mind, at least) when hardware developers started using HDLs (Hardware Description Languages) to create their hardware instead of logic schematics. I’ve never been able to convince a hardware person that designing with an HDL looks a lot like software programming.

I returned to University and earned a degree in Computer Science in 1996, thinking that I would then be validated on both sides of the hardware/software coin. Much to my amusement, this software validation caused the hardware developers to distrust my views.

No worries, it is what it is. Hardware and software developers still ply their trade in separate groups and I still hold firm to the idea that a computer is a system that has two overlapping aspects to it. Although I remain amused that all (most?) of my profession makes an arbitrary distinction between computer hardware and software.

The Heart of the Beast

How do computers do so many different things?

Computers run spreadsheets, web browsers, games, take pictures, do email and a host of many other (hopefully) useful things. Computer servers, desktops and mobile devices do the many things they do because of a chip at their heart that executes instructions. These instructions are collectively known as a program. (More on programs in a later post.) This chip is called a microprocessor or a CPU (Central Processing Unit), and it executes several millions of instructions per second. Each instruction is rather simple, so it takes a rather horrifying number of executed instructions to, say, open a web page in your browser. Good thing the microprocessor is really, really fast.

There are many, many ways to describe the guts of a microprocessor. A computational instruction execution engine, an uncountable number of transistors, a set of defined functional units, and a serious power hog and heat generator are a few of the ways of looking at a microprocessor. Let’s look at a microprocessor from the point of view of it being a set of functional units for now.

Let’s warp the idea of a household kitchen for a moment and view it though the lens of it being a set of functional units. You’ve got your refrigerator, freezer, sink, faucet, blender, oven, microwave, dishwasher, cupboards, toaster and so on. Each “functional unit” in your kitchen does one thing really well. A faucet is good at producing water and a toaster is great at toasting bagels. A faucet is maybe not so good at toasting bagels and I sincerely hope your toaster does not produce water. All of these functional unit things together make up your kitchen. A microprocessor is similar in that it has math units for computation, caches for temporary storage of instructions and data, instruction execution units to execute programs, memory management units to keep memory straight (wish my brain had one), and so on. Fewer functional units in total than your kitchen, actually. A bit smaller than your kitchen, too.

The good news is that many years of experience, experimentation and observation have paid off in that we have microprocessors today that can do many, many things reasonably well. It’s more of a generic efficiency apartment kitchen than a Uno’s Pizzeria and Grill kitchen. Nothing wrong with that, I don’t need an Uno’s kitchen when I make dinner tonight.

The bad news is that the functional units in a microprocessor are fixed at what they are. Let’s say I needed another freezer in my generic efficiency apartment kitchen, it’s not so easy to add one. Same thing with a microprocessor. They typically have four floating point math functional units to, well, do math. If my program only needs four at a time for execution, all is well. But let’s say I have a weather prediction program I want to execute and I want to predict the weather tomorrow. A weather prediction program has a lot of math in it, as you may imagine. If I want the program to complete before tomorrow, I’d really like to have, say, a thousand floating point math functional units all running at the same time. It’s not so easy to add more math functional units to a microprocessor.

A microprocessor is a good general purpose instruction execution engine that has a fixed number of functional units to do the work it needs to do. In later posts, I will touch on the nature of programs, alternative ways and means to compute stuff,  programs that translate geek-readable text into microprocessor instructions and whatever else might appear in our wandering.