Lessons You Learn by Writing Shell Scripts by Hand

I've seen social media personalities claim LLMs mean you can stop learning shell scripting. I believe in learning and loving the details of your tools. Here are two small lessons I learned by writing shell scripts.

You Have To Check Your Errors

When I began shell scripting, I assumed a failed command would halt a shell script. It doesn't:

#!/bin/sh
ls /does-not-exist
echo "This line is reached after the failed command"

$ ./a-failing-command-does-not-stop-your-script.sh
ls: /does-not-exist: No such file or directory
This line is reached after the failed command 

If you follow the bad advice of not learning shell scripting, you won't learn that proceeding after a failed command is expected behaviour, not a bug.

You'll probably get error handling from an LLM by default. If you don't know that set -e is checking a process' exit status, then you won't know when you should embed set +e to turn the check off, or why you might run ls /does-not-exist || true, and I think that's heartbreaking. These behaviours are our history.

Different Kernels Parse a Shebang Differently

Linux is a member of the family of Unix-compatible operating systems. Every Unix-compatible operating system has a Bourne-compatible shell, sed, awk and a host of other tools. Some behave differently on different operating systems, the kernel behaves differently, and by usage you learn that.

In this example I'll run a script first on macOS, then on Alpine Linux:

#!/bin/sh -e -o pipefail

echo ">>> hello, world!"

mac$ ./kernels-parse-shebangs-differently.sh
>>> hello, world!

alpine$ ./kernels-parse-shebangs-differently.sh
/bin/sh: illegal option -

This runs on macOS and fails on Linux because macOS' XNU kernel parses a shebang line differently to Linux. MacOS passes the shebang arguments as separate strings, Linux passes the arguments as a single string. From the docs:

On Linux, the entire string following the interpreter name is passed as a single argument to the interpreter, and this string can include white space.

from the Linux execve man page

XNU parses all of the arguments separately and hands them to the interpreter separately, as we intuitively would expect.

If any optional args are specified, they become the first (second, ...) argument to the interpreter. The name of the originally execve()'d file becomes the subsequent argument; otherwise, the name of the originally execve()'d file is the first argument

from the Apple execve man page.

These differences are there because Unix-like OSes implement POSIX, a standard that doesn't specify a lot of details, and underspecifies what it does describe. That gives OS implementors flexibility, but means you and I have details to learn.

I'm confident that a short LLM-generated script will work correctly on your laptop. It might work on your target OS for production, but when it doesn't the errors are terse. You might be able to paste in the error, but you're less likely to get curious about what causes the difference.

In this case, why don't the OSes treat a shebang similarly? Strangely, because the Unix wars of the 1980s predate POSIX, the standard that Unix-like operating systems implement. Enough Unix-es existed before standardisation that some behaviours could not reasonably be standardised.

Summary

Don't cheat yourself of the joy of learning how this stuff works. The small details add up to a larger picture; good habits mean an error in Production is an opportunity to learn more; learning develops into mastery.

Do know, I'm not trying to say this is the best possible way Unix-like operating systems could have evolved. By noticing these inconsistent details, though, you come to appreciate that the engineers were human beings, learning in real time, experimenting.

I find this encouraging: if the legends made a bit of a mess, maybe it's okay for my own experimentation to be messy.

Not confident you know how to map out a learning plan? Reach out on my contact form and I'll help you. No sales pressure or anything, I just want to help.

Want to learn more about Unix in general? Check out my free Newsletters. An Evening with Berferd covers a weird detail in an excellent whitepaper, and you'll learn about chroot, an isolation primitive that's a precursor to containers.