Building my own shell in Python — day 2

Day 1 was the loop and the builtins, and it ended in the middle of parsing. Today I went after a different piece entirely: redirection. The thing you do a hundred times a week without thinking — command > file.txt — and never once ask how it works.

So I went and researched it properly before writing a line. That turned out to be the right order to do it in.

What redirection actually is

Every process starts life with three streams already wired up, and they are numbered:

• 0 — standard input, where it reads from
• 1 — standard output, where normal results go
• 2 — standard error, where problems go

Normally all three point at your terminal, which is why output and errors both just appear on screen. Redirection isn't the shell adding a feature to a command — it's the shell quietly pointing one of those numbered streams somewhere else before the command starts. The command itself has no idea it happened.

The moment I understood that, the syntax stopped looking arbitrary:

>   file     stdout to file, overwrite
1>  file     the same thing — the 1 is just implied
2>  file     stderr to file, overwrite
>>  file     stdout to file, append
2>> file     stderr to file, append

The number in front is the stream you're redirecting, and it defaults to 1 when you leave it out. Doubling the arrow switches from truncating the file to appending to it. That's the whole grammar. It had looked like memorised trivia for years; it's really just two independent choices — which stream, and replace or add.

Then I tried to implement it

This is where the day got interesting, because Python's subprocess module had an opinion I didn't know about.

I reached for capture_output=True, since capturing the output is obviously what I wanted. It worked — right up until a command failed. Then the error message just disappeared. Nothing on screen, nothing in the file. Gone.

The reason is sitting in the name, and I had skimmed straight past it. capture_output=True doesn't capture the output. It captures both streams — it's shorthand for sending stdout and stderr to pipes at the same time. So when I redirected stdout to a file, stderr was still being swept into a pipe that nothing ever read from or printed. I hadn't redirected the errors. I had binned them.

And that is wrong behaviour for a shell. Run this in a real terminal:

$ ls /nope > out.txt
ls: cannot access '/nope': No such file or directory

Only stdout was redirected, so the error still lands on your screen. That is the entire point of having two separate streams: you can send the results somewhere without losing sight of the problems.

The thing that made it click

The fix wasn't a bigger capture. It was the opposite — stop capturing things you have no plans for.

Instead of grabbing everything and sorting it out afterwards, you give each stream its own destination up front, before the process starts. And here's the piece I was missing: leaving a stream alone means the child process inherits it from its parent. It isn't discarded and it isn't captured — it flows straight through to the terminal, exactly as if you had never been involved.

Which is precisely the behaviour you want for a stream nobody redirected. So the logic falls out cleanly:

• Saw > or 1> — point stdout at the file, leave stderr inherited.
• Saw 2> — point stderr at the file, leave stdout inherited.
• Saw both — point each at its own file.
• Saw neither — leave both alone and let them through.

And append versus overwrite never needed any shell logic at all. It's just how you open the file: append mode instead of write mode. The operator maps to a file mode, and the rest takes care of itself.

Where I am now

Redirection works — both streams, both modes, independently. Which means I can finally do the thing that started all this: send a command's errors to their own log file while its output goes somewhere else entirely.

Two days in, the pattern is already clear: every one of these features looks like magic from the outside and turns out to be plumbing on the inside.