I’m Colten Webb, an MSCS student in the College of Computer Science at Georgia Tech.

Design of Operating Systems flashcards

Rummaging through my files I found some flashcards I assembled for CS3210: Design of Operating Systems, from when I took the course at GaTech. Here they are in csv form. I put them into Anki. These were only meant to reinforce proper “conceptual” understanding (which my professor ensured with his difficult exams).

LLM programmers

I forget where but I recently came across a discussion online about whether ChatGPT would allow companies to replace programmers in the near-future. The main response argued that since ChatGPT frequently makes mistakes that a businessperson would be unable to spot, programmers’ jobs are safe. The business people will still need programmers to make corrections to the code ChatGPT produces. In the same breath, the commenter observed that the more advanced the programmer, the more likely that ChatGPT helps them produce good code. That is, they are more productive with it compared when working without it. They suggested that the returns on ChatGPT are much smaller for novice devs, and maybe even detrimental if ChatGPT produces poor usage of APIs and they don’t know any better, but would if they had just read the docs. (The debate then devolved into talk about how coorporations would ban ChatGPT since copyright is so murky with it… as if they would throw away something that can make their best devs 10x more productive because their lawyers will have to work a little harder)

This debate was interesting to me because the two commenters had different ideas of what it meant to replace programmers. I think there are two ways to think about this, illustrated by two scenarios:

1. A business person wants a piece of software. They hire a dev. The dev makes the software for them.
2. A business person wants a piece of software. They hire a team of devs, since it’s complicated. The devs make the software for them.

In scenario 1, I agree that ChatGPT won’t replace the programmer (in the near-term). Even if ChatGPT could do a lot of the heavy lifting, it’s not good enough to make software from start to finish, unless it’s just a script and the business person is relatively tech savvy.

In scenario 2, ChatGPT could absolutely replace programmers. From my limited experience in industry, I’ve gathered that most teams have one or two senior devs and the rest are junior. The juniors are the codemonkeys and the seniors do most of the real decision-making. A senior developer that uses ChatGPT well could absolutely become as productive as they would be with one or two junior devs. In this case, a company would just hire the senior dev, and ChatGPT replaced two devs.

I can see this creating a strange equilibrium, where companies are even less interested in junior devs, and more interested in senior devs (this is already the case of course, but at least junior devs do produce code). But the only way to become a senior dev is through experience. So perhaps junior years works more like long-term internships? Senior devs get paid more, and junior devs paid less, until ChatGPT becomes strong enough to displace even the senior devs? Winner takes all? Or progress in LLM-powered learning platforms allows devs to get to senior level in less time?…

Typing fast and CS:GO

If you’ve ever played typeracer, then you understand how much of a skill fast typing is. Some people are able to type nearly 200 words per minute, but the average typist can only get around 60. The site measures typing speed by providing a short paragraph, and having you “type over it”, where you produce the text exactly as it is written, with capitalization, punctuation, spaces all included.

The implicit assumption here is that the proper way to measure typing speed is by limiting the domain of what you type to English sentences. The gap in typing speed, between the 60wpm’ers and the 200wpm’ers, closes almost entirely if the text is just random letters, or if the letters are presented to the typist one at a time as they go. This is means the mechanism for improving typing speed lies mostly in being able to form a series of gestalts, and letting muscle memory handle the rest.

A fun exercise, if you’ve played typeracer, is to try reproducing a C document as quickly as possible. Typing brackets and parenthensis will feel clumsy and awkward, even if you are an experienced programmer, because editors autocomplete them or because you don’t write code with a focus on speed – a more common bottleneck is finding APIs or logical reasoning. (I found that a weird exception for me is writing loops, specifically for (int i = 0 ; i < n ; i++), which I can hammer out like its English prose. Everything else was a mess!)

These effects are probably due to “chunking,” which I’ve heard of in the context of chess. Most experts at chess can memorize the position of pieces on a chess board almost effortlessly, as long as the setup is from a real game. If the same pieces are placed at random, they will do no better than anyone else. This is because chess experts have a rich fabric of learned patterns to help memorize a real configuration. If the configuration is pure noise, they lose their advantage.

I’m writing about all of this because I’ve noticed the same exact principle holds in Counter-Strike: Global Offensive (which I recently started playing again). Although players could stand in an almost unlimited number of positions, most will defend a control point from only a handful. An experienced player knows this, so they “pre-aim”, meaning they move their crosshair to where they expect enemy players to be before they expose themselves. With enough practice, a player can look in the right spot as they navigate the entire map, so that eliminating the enemy requires small corrections in aim, and rarely large and imprecise movements.

So to get better at CS:GO, it’s much better to learn where to anticipate the enemy, rather than to just aim faster. The largest gains come from intelligent anticipation, not improved reactions.

Biohacking advice creates a bad equilibrium

There is no shortage of experts online giving productivity advice. Most of productivity advice is tautological (if you wake up and do your most important thing, you will get more done than if you don’t do your most important thing), and so it’s really a thinly veiled mechanism for procrastination. Sometimes it provides a boost of motivation, which can be a useful catalyst, and sometimes it provides real advice about behavior change, but it looks more like therapy for people too proud for therapy. I’ve been guilty of reading a lot of this stuff, but you begin to see the patterns in the advice, the same mantras over and over again, and it gets boring. So I think most productivity advice is a problem, but the problem resolves itself.

But then biohacking became a thing. Experts would find a paper out of lab in Finland that says some caffenated ashwaganda amphetamine at this time of day is ideal for writing essays upside down, but only in a sensory deprivation tank. And then they throw you a bone, say “drink matcha tea,” because it has a precursor to whatever chemical was studied, and so will make you more productive.

This is not science, it’s optimisim. The problem is, optimism works. Trying something new will give you some motivation, and the placebo effect will do it’s magic. So people hear “drink matcha tea,” drink said matcha tea, and then become more productive.

That doesn’t sound so bad, except people are dumb. There will be people who try taking the obscure caffeinated ashwaganda amphetamine, jack up their dopamine response, and then deal with consequences for the rest of their life. Access to these substances usually has gatekeepers, namely doctors and pharmacists. The assumption people have is that these professionals are experts in linking symptoms to substance, and little more. Google can do that, so I can do that, the biohacker declares. The truth is that these professionals have a lot of context too. They’ve seen the effects these drugs, and similar drugs, have had on people over many years. They (should) have read much more about this than you have. They saw the meta-analysis, and the paper the meta-analysis ignored.

The secret is that these gatekeepers might know something that you don’t, and you won’t even know to google for it.

Doctors have a responsibility to their patients, morally and by law. They’re job is to help you do what’s in your best interest. An internet expert in biohacking has no such responsibility to their audience. Maybe at first they’ll tell you something respectable, like try caffeine as a stimulant. But they need to keep producing content, so after a while the advice will be backed by less and less evidence. For a while it won’t be harmful advice, just unsupported. But then, when every expert gives the same harmless but ineffective advice, they need to find a way to differentiate themselves. So they start to say things that can be harmful, because others aren’t saying it. Bonus points if they have a PhD to command some authority. And it doesn’t matter, because they only need to grow their audience, and they can always create a wall of deniability (“this isn’t medical advice”). They have no exposure, it’s betting someone else’s money.

My point here is that the equilibrium for biohacking experts is to give advice just risky enough to build an audience without getting in trouble. Such experts will be charismatic, irresponsible and careerist. So instead of listening to them, look for experts who focus on providing context, not simple rules and advice. That’s what their expertise really qualifies them to do, anyway.

Interpreting congruence identities as statements about redundancy

Let’s look at the most basic congruence identity, the addition identity:

$$a \equiv b \text{ and } c\equiv d \implies a+c\equiv b+d \pmod m$$

To me, it isn’t immediately obvious how this relates to the typical use of mod I’ve seen, which is computing solutions without integer overflow, and using the Chinese Remainder Theorem to reconstruct the true solution. For example, a typical (simplified) use of congruence looks something like this:

ans = 0
ans = (ans + a) % m
ans = (ans + b) % m
...
ans = (ans + z) % m

print(ans)

This avoid integer overflow, whereas print((a + b + ... + z) % m) does not. Thus when I’ve used congruence identities, I’ve thought of it as a way to insert additional mod operations, which is more obvious in the following statement:

$$a\ \mathrm{mod}\ m + c\ \mathrm{mod}\ m \equiv a+c \pmod m$$

The more general addition identity is useful for proofs, but I think this “where can I add mod operations without changing the answer” interpretation provides useful intuition.