Lab 03: Shell Scripting

Exercise 1: Balancing Numbers with tr练习1：使用tr平衡数字

Task任务

Convert digits using tr: 0-4 → <, 6-9 → >, 5 unchanged.

使用 tr 转换数字：0-4 → <, 6-9 → >, 5 保持不变。

The Solution解决方案

$ cat balancing_numbers.sh
#!/bin/dash
tr '01234' '<' | tr '6789' '>'

How it Works工作原理

Part部分	Meaning含义
`tr '01234' '<'`	Map characters 0,1,2,3,4 to <将字符 0,1,2,3,4 映射为 <
`\|`	Pipe output to next command将输出通过管道传给下一个命令
`tr '6789' '>'`	Map characters 6,7,8,9 to >将字符 6,7,8,9 映射为 >

Example示例

$ echo "1 234 5 678 9" | ./balancing_numbers.sh
< <<< 5 >>> >

💡 Key Insight: tr automatically repeats the last character in SET2 to match the length of SET1. So tr '01234' '<' works the same as tr '01234' '<<<<<'.

💡 关键洞察： tr 会自动重复 SET2 的最后一个字符来匹配 SET1 的长度。所以 tr '01234' '<' 等同于 tr '01234' '<<<<<'。

Exercise 2: Echo with Conditions (if/while)练习2：带条件的Echo（if/while）

Task任务

Write echon.sh that prints a string N times with proper error handling.

编写 echon.sh，打印字符串 N 次，并正确处理错误。

Key Concepts核心概念

1. Shebang (#!/bin/dash)1. Shebang (#!/bin/dash)

#!/bin/dash

Tells the system to use dash shell to execute this script. Must be the first line, no spaces before #.

告诉系统使用 dash shell 执行此脚本。必须是第一行，# 前不能有空格。

2. Special Variables2. 特殊变量

Variable变量	Meaning含义
`$#`	Number of arguments参数个数
`$1, $2, ...`	1st, 2nd, ... argument第1个、第2个...参数
`$@`	All arguments所有参数
`$?`	Exit status of last command上一个命令的退出状态

3. if Statement Syntax3. if 语句语法

$ # Basic if
if [ condition ]; then
    # commands
fi
$ # if-elif-else
if [ condition ]; then
    # commands
elif [ condition ]; then
    # commands
else
    # commands
fi

4. Test Operators ([ ])4. 测试运算符 ([ ])

Operator运算符	Meaning含义	Example示例
`-eq`	Equal (numbers)等于（数字）	`[ $a -eq 5 ]`
`-ne`	Not equal (numbers)不等于（数字）	`[ $# -ne 2 ]`
`-lt`	Less than小于	`[ $i -lt 10 ]`
`-gt`	Greater than大于	`[ $1 -gt 0 ]`
`-f`	Is a regular file是普通文件	`[ -f "$file" ]`
`-d`	Is a directory是目录	`[ -d "$dir" ]`

5. while Loop5. while 循环

i=0
while [ $i -lt $1 ]; do
    echo "$2"
    i=$((i + 1))  # Arithmetic: i = i + 1
done

6. case Statement6. case 语句

case "$1" in
    *[!0-9]*)
        echo "Not a number"
        exit 1
        ;;
esac

*[!0-9]* matches if the string contains any non-digit character.

*[!0-9]* 匹配字符串中包含任何非数字字符的情况。

Complete Solution完整解决方案

$ cat echon.sh
#!/bin/dash
# Check argument count
if [ $# -ne 2 ]; then
    echo "Usage: ./echon.sh  "
    exit 1
fi
# Check if first argument is a non-negative integer
case "$1" in
    *[!0-9]*)
        echo "./echon.sh: argument 1 must be a non-negative integer"
        exit 1
        ;;
esac
# Loop and print
i=0
while [ $i -lt $1 ]; do
    echo "$2"
    i=$((i + 1))
done

⚠️ Common Mistakes:

⚠️ 常见错误：

[ ] must have spaces inside: [ $i -lt 10 ] not [$i -lt 10][ ] 内部必须有空格：[ $i -lt 10 ] 而不是 [$i -lt 10]
Variable assignment: i=0 not i = 0 (no spaces!)变量赋值：i=0 而不是 i = 0（不能有空格！）
Every if needs a fi, every case needs an esac每个 if 需要一个 fi，每个 case 需要一个 esac

Exercise 3: File Sizes (for/wc)练习3：文件大小（for/wc）

Task任务

Categorize files by line count: small (<10), medium (10-99), large (≥100).

按行数对文件分类：小文件（<10行）、中文件（10-99行）、大文件（≥100行）。

Key Concepts核心概念

1. for Loop1. for 循环

for file in *; do
    echo "$file"
done

* is a wildcard that matches all files in the current directory.

* 是通配符，匹配当前目录下的所有文件。

2. wc -l (Count Lines)2. wc -l（计算行数）

$ wc -l < filename
5

< redirects file to stdin, so wc only outputs the count.

< 将文件重定向到标准输入，所以 wc 只输出计数。

3. Command Substitution $()3. 命令替换 $()

lines=$(wc -l < "$file")

Captures command output into a variable.

将命令输出捕获到变量中。

4. String Concatenation4. 字符串拼接

small="$small $file"

Append $file to the small variable.

将 $file 追加到 small 变量。

Complete Solution完整解决方案

$ cat file_sizes.sh
#!/bin/dash
small=""
medium=""
large=""
for file in *; do
    if [ -f "$file" ]; then
        lines=$(wc -l < "$file")
        
        if [ $lines -lt 10 ]; then
            small="$small $file"
        elif [ $lines -lt 100 ]; then
            medium="$medium $file"
        else
            large="$large $file"
        fi
    fi
done
echo "Small files:$small"
echo "Medium-sized files:$medium"
echo "Large files:$large"

💡 Why [ -f "$file" ]? The * wildcard matches both files AND directories. We use -f to check if it's a regular file before processing.

💡 为什么用 [ -f "$file" ]？ * 通配符匹配文件和目录。我们用 -f 在处理前检查它是否是普通文件。

Exercise 4: Scraping JSON APIs (curl/jq)练习4：抓取JSON API（curl/jq）

Task任务

Fetch course data from UNSW Handbook API and filter by prefix.

从 UNSW Handbook API 获取课程数据并按前缀过滤。

Key Concepts核心概念

1. curl - Fetch Data1. curl - 获取数据

$ curl -sL "https://api.example.com/data"

Option选项	Meaning含义
`-s`	Silent mode (no progress bar)静默模式（无进度条）
`-L`	Follow redirects跟随重定向

2. jq - Parse JSON2. jq - 解析JSON

$ # Basic structure
jq -r '.data.results[] | select(...) | "..."
                
3. jq Operators3. jq 运算符
                    
                                Operator运算符
                                Meaning含义
                                Example示例
                            
                                .
                                Current object当前对象
                                .data
                            
                                []
                                Iterate array遍历数组
                                .results[]
                            
                                |
                                Pipe (jq internal)管道（jq内部）
                                .data | .results
                            
                                select()
                                Filter过滤
                                select(.type == "Course")
                            
                                startswith()
                                String starts with字符串以...开头
                                startswith("COMP")
                            
                                map()
                                Transform array elements转换数组元素
                                map(ascii_downcase)
                            
                                any()
                                Check if any matches检查是否有任何匹配
                                any(. == "x")
                            
4. Injecting Shell Variables into jq4. 将Shell变量注入jq# Use '"'"$var"'"' pattern inside jq string
jq -r '... | startswith("'"$2"'") | ...'

Operator运算符	Meaning含义	Example示例
`.`	Current object当前对象	`.data`
`[]`	Iterate array遍历数组	`.results[]`
`\|`	Pipe (jq internal)管道（jq内部）	`.data \| .results`
`select()`	Filter过滤	`select(.type == "Course")`
`startswith()`	String starts with字符串以...开头	`startswith("COMP")`
`map()`	Transform array elements转换数组元素	`map(ascii_downcase)`
`any()`	Check if any matches检查是否有任何匹配	`any(. == "x")`

Complete Solution完整解决方案

$ cat scraping_courses.sh
#!/bin/dash
if [ $# -ne 2 ]; then
    echo "Usage: ./scraping_courses.sh  "
    exit 1
fi
if ! echo "$1" | grep -qE '^[0-9]+$'; then
    echo "./scraping_courses.sh: argument 1 must be an integer between 2019 and 2026"
    exit 1
fi
if [ "$1" -lt 2019 ] || [ "$1" -gt 2026 ]; then
    echo "./scraping_courses.sh: argument 1 must be an integer between 2019 and 2026"
    exit 1
fi
url="https://handbook-proxy.cse.unsw.edu.au/current/api/search-all?searchType=advanced&siteId=unsw-prod-pres&query=&siteYear=$1"
curl -sL "$url" | jq -r '.data.results[] | select(.contentTypeLabel == "Course") | select(.lines | map(ascii_downcase) | any(. == "undergraduate" or . == "postgraduate")) | select(.code | startswith("'"$2"'")) | "\(.code) \(.title)"' | tr -s ' ' | sort | uniq

💡 Debugging Tips:

💡 调试技巧：

Run bash -n script.sh to check syntax运行 bash -n script.sh 检查语法
Add set -x at the top to trace execution在顶部添加 set -x 跟踪执行
Use echo "DEBUG: $var" to see variable values使用 echo "DEBUG: $var" 查看变量值
Test API URL in browser first to see JSON structure先在浏览器中测试 API URL 查看 JSON 结构

Summary: Key Patterns总结：关键模式

Pattern模式	When to Use何时使用	Example示例
`if [ ]`	True/false conditions真/假条件	`if [ $x -gt 5 ]`
`case`	Multiple value matching多值匹配	`case "$1" in ... esac`
`while`	Loop until condition false循环直到条件为假	`while [ $i -lt $n ]`
`for`	Iterate over items遍历项目	`for file in *`
`$()`	Command substitution命令替换	`lines=$(wc -l < file)`
`$((expr))`	Arithmetic算术运算	`i=$((i + 1))`

📚 Theory Reference — Quiz Knowledge Points理论速查表 — 测验知识点

Every concept the Lab 03 quiz tests, organised by category. Use this as a final-pass study sheet before taking the quiz. Lab 03 测验考查的每一个知识点，按类别整理。做测验前用这个表过一遍即可。

1. `tr` — Translate / Squeeze / Delete1. `tr` — 翻译 / 压缩 / 删除

Concept	Detail	Example
Positional mapping	SET1 char at position i → SET2 char at position iSET1 第 i 个字符 → SET2 第 i 个字符	`tr "abc" "123"` → b becomes 2
SET2 shorter than SET1	tr auto-repeats SET2's last char to match SET1 lengthtr 自动用 SET2 最后一个字符填充	`tr "01234" "<"` → all map to `<`
`-d` delete	Delete every char in SET1删除 SET1 中所有字符	`tr -d "0-9"` removes all digits
`-s` squeeze	Collapse repeated occurrences of SET1 chars to one将 SET1 中重复出现的字符压缩为一个	`tr -s " "` → multiple spaces become one
Pipeline order matters	Each tr operates on the output of the previous one每个 tr 处理上一个的输出	`echo 12345 \| tr "0-4" "<" \| tr "6-9" ">"` → `<<<<5`

2. Shebang & Variables2. Shebang 与变量

Symbol	Meaning	Notes
`#!/bin/dash`	Shebang — tells OS which interpreter to useShebang — 告诉系统使用哪个解释器	Must be the first line, no leading spaces必须是第一行，不能有空格
`$0`	Script name脚本名	—
`$1`, `$2`, …	1st, 2nd positional argument第 1、第 2 位置参数	Always quote: `"$1"`永远加引号 `"$1"`
`$#`	Argument count参数个数	—
`$@`	All arguments所有参数	—
`name=value`	Variable assignment变量赋值	No spaces around `=``=` 两边不能有空格
`$var`	Read variable value读取变量值	—
`$((expr))`	Arithmetic expansion (math)算术扩展（数学运算）	`i=$((i + 1))`
`$(cmd)`	Command substitution — capture stdout as a string命令替换 — 把 stdout 抓成字符串	`lines=$(wc -l < file)`

3. `if` / `case` Conditionals3. `if` / `case` 条件语句

The [ ] rule: spaces are required inside [ ]. [$x -eq 5] is wrong; [ $x -eq 5 ] is correct.[ ] 规则：方括号内必须有空格。[$x -eq 5] 错；[ $x -eq 5 ] 对。

Block endings are reverse spellings: if → fi, case → esac.代码块结束用反向拼写：if → fi，case → esac。

Operator	Meaning	Use on
`-eq`	equal	numbers
`-ne`	not equal	numbers
`-lt`	less than	numbers
`-le`	less than or equal	numbers
`-gt`	greater than	numbers
`-ge`	greater than or equal	numbers
`=`	equal	strings
`!=`	not equal	strings
`-z`	string is empty	strings
`-n`	string is non-empty	strings
`-f path`	path exists AND is a regular file	file tests
`-d path`	path exists AND is a directory	file tests
`-e path`	path exists (any type)	file tests

case patterns:case 模式：

* — match anything (zero or more chars)匹配任意字符
? — match exactly one char匹配恰好一个字符
[abc] — one of a, b, ca、b、c 中的一个
[!0-9] — one char that is NOT a digit一个非数字字符
*[!0-9]* — string contains any non-digit char字符串包含任何非数字字符
;; — end of a pattern's command block (like break)一个模式命令块的结束（类似 break）

4. Loops — `while` & `for`4. 循环 — `while` 与 `for`

Form	Pattern	Notes
`while [ cond ]; do … done`	condition-controlled loop	Need `do` + `done`; spaces inside `[ ]`需要 `do` + `done`；`[ ]` 内有空格
`for var in list; do … done`	iterate over a list	`for file in *` — wildcards expand
`for i in 1 2 3 4 5`	POSIX-portable count	Works in `dash`; `for ((;;))` is bash-only`dash` 兼容；`for ((;;))` 只支持 bash
`i=$((i + 1))`	increment counter	No `i++` in POSIX shellPOSIX shell 没有 `i++`
`small="$small $file"`	append to a string	String concatenation, with space separator字符串拼接（带空格分隔）
`break` / `continue`	exit / skip iteration	—

5. `curl` & `jq` — APIs & JSON5. `curl` 与 `jq` — API 与 JSON

Tool / Flag	Meaning	Example
`curl -s`	silent (no progress)静默（无进度条）	—
`curl -L`	follow redirects跟随重定向	—
`curl -sL`	silent + follow redirects (typical pairing)静默 + 跟随重定向（常用组合）	`curl -sL https://api/...`
`jq -r`	raw output (no JSON quotes)原始输出（不带 JSON 引号）	—
`.field`	access JSON field访问 JSON 字段	`.data.results`
`.array[]`	iterate over array elements遍历数组元素	`.data.results[]`
`select(cond)`	filter — keep items where cond is true过滤 — 保留条件为真的项	`select(.type == "Course")`
`startswith("X")`	true if string begins with X字符串以 X 开头返回 true	`select(.code \| startswith("COMP"))`
Inject shell var into jq	Break out of jq's single-quoted string, inject `"$var"`, re-enter跳出 jq 单引号串，插入 `"$var"`，再回到串内	`jq '... \| .name == "'"$2"'"'`

6. Debugging Shell Scripts6. Shell 脚本调试

Command / Issue	Effect / Cause
`bash -n script.sh`	Syntax check only — does not execute仅语法检查 — 不实际运行
`set -x` (or `bash -x script.sh`)	Trace mode — print every command before running跟踪模式 — 运行前打印每条命令
`set -e`	Exit immediately on any command failure任一命令失败就立即退出
"Permission denied" running `./script.sh`	Missing execute bit — fix with `chmod +x script.sh`缺少执行权限 — 用 `chmod +x script.sh` 修复
Run without exec bit不加执行权限运行	Invoke interpreter explicitly: `dash script.sh` or `bash script.sh`显式调用解释器：`dash script.sh` 或 `bash script.sh`

⚡ Top mistakes graders see⚡ 阅卷常见错误

Spaces around = in assignment (x = 5 ✗ → x=5 ✓)赋值时 = 两边带空格（x = 5 ✗ → x=5 ✓）
Missing spaces inside [ ][ ] 内部缺空格
Forgetting fi / esac / done忘了 fi / esac / done
Unquoted "$1" when filenames may contain spaces文件名可能含空格时没给 "$1" 加引号
Using i++ (not POSIX) instead of i=$((i + 1))用 i++（非 POSIX）而不是 i=$((i + 1))

Lab 03: Shell ScriptingLab 03: Shell脚本编程

Exercise 1: Balancing Numbers with tr练习1：使用tr平衡数字

Task任务

The Solution解决方案

How it Works工作原理

Example示例

Exercise 2: Echo with Conditions (if/while)练习2：带条件的Echo（if/while）

Task任务

Key Concepts核心概念

1. Shebang (#!/bin/dash)1. Shebang (#!/bin/dash)

2. Special Variables2. 特殊变量

3. if Statement Syntax3. if 语句语法

4. Test Operators ([ ])4. 测试运算符 ([ ])

5. while Loop5. while 循环

6. case Statement6. case 语句

Complete Solution完整解决方案

Exercise 3: File Sizes (for/wc)练习3：文件大小（for/wc）

Task任务

Key Concepts核心概念

1. for Loop1. for 循环

2. wc -l (Count Lines)2. wc -l（计算行数）

3. Command Substitution $()3. 命令替换 $()

4. String Concatenation4. 字符串拼接

Complete Solution完整解决方案

Exercise 4: Scraping JSON APIs (curl/jq)练习4：抓取JSON API（curl/jq）

Task任务

Key Concepts核心概念

1. curl - Fetch Data1. curl - 获取数据

2. jq - Parse JSON2. jq - 解析JSON

3. jq Operators3. jq 运算符

4. Injecting Shell Variables into jq4. 将Shell变量注入jq

Complete Solution完整解决方案

Summary: Key Patterns总结：关键模式

📚 Theory Reference — Quiz Knowledge Points理论速查表 — 测验知识点

1. tr — Translate / Squeeze / Delete1. tr — 翻译 / 压缩 / 删除

2. Shebang & Variables2. Shebang 与变量

3. if / case Conditionals3. if / case 条件语句

4. Loops — while & for4. 循环 — while 与 for

5. curl & jq — APIs & JSON5. curl 与 jq — API 与 JSON

6. Debugging Shell Scripts6. Shell 脚本调试

1. `tr` — Translate / Squeeze / Delete1. `tr` — 翻译 / 压缩 / 删除

3. `if` / `case` Conditionals3. `if` / `case` 条件语句

4. Loops — `while` & `for`4. 循环 — `while` 与 `for`

5. `curl` & `jq` — APIs & JSON5. `curl` 与 `jq` — API 与 JSON