Files, JSON, and paths

with open(...) — always

with open(path, "r", encoding="utf-8") as f: opens the file and guarantees it closes, even if the block raises. Never call open() without with in real code.

Always pass encoding="utf-8". Without it, Python uses the OS default, and a script that works on Linux CI crashes on a Windows laptop the moment a document contains an accented character or emoji.

Modes: "r" read, "w" write (truncates), "a" append, add "b" for bytes.

JSON and JSONL

json.dumps(obj) turns a dict into a string; json.loads(s) parses it back. The file variants are json.dump(obj, f) and json.load(f).

JSONL (one JSON object per line) is the standard format for fine-tuning datasets and eval sets, because you can stream it line by line without parsing the whole file.

json.dumps(obj, indent=2) for human-readable config. Use ensure_ascii=False when your text contains non-English characters, otherwise they are escaped into unreadable \u sequences.

pathlib over string concatenation

Use from pathlib import Path. Join with /: Path("data") / "train.jsonl". It produces correct separators on every OS.

Useful methods: .exists(), .mkdir(parents=True, exist_ok=True), .suffix, .stem, .read_text(), .write_text(), and .glob("*.jsonl") to list matching files.

Copy-paste examples

Copy into your own editor, or load one into the compiler below and press Run.

import json

config = {"model": "gpt-4.1-mini", "temperature": 0.2, "tools": ["search", "calc"]}

with open("config.json", "w", encoding="utf-8") as f:
    json.dump(config, f, indent=2)

with open("config.json", "r", encoding="utf-8") as f:
    loaded = json.load(f)

print(loaded)
print("tools:", loaded["tools"])
print(json.dumps(loaded, indent=2))

import json

rows = [
    {"prompt": "What is MLOps?", "completion": "Operating ML in production."},
    {"prompt": "What is RAG?", "completion": "Retrieval plus generation."},
]

with open("train.jsonl", "w", encoding="utf-8") as f:
    for row in rows:
        f.write(json.dumps(row, ensure_ascii=False) + "\n")

with open("train.jsonl", "r", encoding="utf-8") as f:
    for i, line in enumerate(f, start=1):
        record = json.loads(line)
        print(i, record["prompt"], "->", record["completion"])

from pathlib import Path

data_dir = Path("artifacts") / "run-12"
data_dir.mkdir(parents=True, exist_ok=True)

metrics_path = data_dir / "metrics.json"
metrics_path.write_text('{"accuracy": 0.91}', encoding="utf-8")

print("exists:", metrics_path.exists())
print("suffix:", metrics_path.suffix, "| stem:", metrics_path.stem)
print("content:", metrics_path.read_text(encoding="utf-8"))
print("files:", [str(p) for p in data_dir.glob("*.json")])

Takeaways

• Always use with open(..., encoding="utf-8") — it closes the file and avoids OS encoding bugs.
• JSONL is the standard for training and eval datasets because it streams.
• Build paths with pathlib, not string concatenation.