Ross Burton's Tip Of The Week "complains" that after

if [ $? -ne 0 ]; then

it happens that "the act of checking $? causes $? to be set to the result of the test". Which is "success", hence zero.

Of course it is so! $? will hold the exit status of the latest "whatever": command, function, script, including the test using the built-in construct [ (which is still a command!), or the test with [[.

The man page for Bash is horrendously long, but a more practical reference is the Advanced Bash-Scripting Guide (I am too lazy to use O'Reilly's bash Quick Reference which I bought not for $9.99, but rather for $6.50 because of the discount code DSUG). See 7.1. Test Constructs.

Since it was about $-things in Bash, I collected here a list of the common "buck-variables":

$0, $1, $2, etc. : positional parameters, passed from command line to script, passed to a function, or set to a variable.
$# : number of command line arguments or positional parameters.
$* : all of the positional parameters, seen as a single word. "$*" must be quoted.
$@ : same as $*, but each parameter is a quoted string, that is, the parameters are passed on intact, without interpretation or expansion. This means, among other things, that each parameter in the argument list is seen as a separate word. "$@" should be quoted.

NOTE: $* and $@ differ only when between double quotes. They sometimes display inconsistent and puzzling behavior, depending on the setting of $IFS (internal field separator). $IFS defaults to whitespace (space, tab, and newline), but may be changed, for example, to parse a comma-separated data file. Note that $* uses the first character held in $IFS.

$- : flags passed to script.
$! : process ID of last job run in background.
$_ : the last argument of previous command executed.
$$ : process ID (PID) of the script itself.

NOTE: $$ variable often finds use in scripts to construct "unique" temp file names in simpler ways than invoking mktemp(1).

$? : exit status of a command, function, or the script itself.

NOTE: $? reads the exit status of the last command executed. After a function returns, $? gives the exit status of the last command executed in the function. This is Bash's way of giving functions a "return value." After a script terminates, a $? from the command line gives the exit status of the script, that is, the last command executed in the script, which is, by convention, 0 on success or an integer in the range 1-255 on error.