StreamRecorder/TODO.md
2026-08-06 00:20:54 -05:00

10 KiB
Raw Blame History

Return-from-a-year-away Audit — 2026-08-05

Scope reviewed: main (checked out), manifest, cleanup (all local + origin equivalents). Remote: https://git.aaronic.cc/kacarmichael/StreamRecorder.git

Branch situation (read this first)

The three branches are not parallel efforts — they're one linear line of development that main never caught up to:

main (0f4a39f, 2025-07-26) ──┐  stale, single-file-era downloader only
                              │
manifest (718d69c, 2025-08-10) ── +restructured cmd/pkg layout, config-less
                              │
cleanup  (7d67cec, 2025-08-11) ── +pkg/config, tests, CI, Makefile, deployment docs

manifest is literally an ancestor of cleanup (same commit 718d69c). cleanup is the most complete state that ever existed and was the last thing touched (2025-08-11) before the year gap. None of this reached main. If you pick this project back up, decide this before anything else:

  • Merge cleanup → main (recommended — it's a strict superset: config validation, pkg/nas, pkg/processing, unit tests, CI workflow, DEPLOYMENT.md/TESTING.md). Then delete manifest (fully subsumed) and re-evaluate cleanup for deletion once merged.
  • Or, if cleanup's direction (FFmpeg concatenation, NAS-mirroring processing pipeline) is no longer where you want to go, explicitly archive it and restart main from a clean design — but do this as a decision, not by default.

🔴 Critical — fix/rotate before running this again

  1. Leaked NAS credentials, committed in plaintext. pkg/config/config.go (branch cleanup) hardcodes a real default:

    Username: "NASAdmin",
    Password: "s3tkY6tzA&KN6M",
    

    This is in git history on a remote (git.aaronic.cc) regardless of whether you later change the default. Rotate this NAS account's password now — changing the code doesn't un-leak a password that's sat in a repo for a year. Then remove the credential entirely (fail closed: require NAS_USERNAME/NAS_PASSWORD env vars with no default, error out if unset when transfer is enabled) rather than shipping a working default.

  2. Guaranteed crash: double mutex unlock in pkg/transfer/cleanup.go. In ExecuteCleanup (present on main too, not just cleanup):

    cs.mu.Lock()
    ...
    cs.mu.Unlock()        // line 72 — releases the lock
    ...
    batch := make([]string, batchSize)
    copy(batch, cs.pendingFiles[:batchSize])
    cs.pendingFiles = cs.pendingFiles[batchSize:]   // mutated WITHOUT the lock held
    cs.mu.Unlock()        // line 79 — unlocking an already-unlocked mutex
    

    An unlock of an unlocked sync.Mutex is a fatal, unrecoverable runtime error (fatal error: sync: unlock of unlocked mutex) — it kills the whole process, not just a goroutine. Since Cleanup.AfterTransfer defaults to true and RetainHours: 0 (immediate eligibility), this fires the first time any file finishes transferring and cleanup runs. This is very likely why any end-to-end run with NAS transfer enabled never survived long. Fix: hold the lock for the whole slice mutation, or copy-then-release-then-mutate-under-a-second-short-lock, but don't unlock twice.

  3. Nil-pointer crash in pkg/media/stream.go (VariantDownloader, main branch):

    playlist, err := LoadMediaPlaylist(variant.URL)
    seq := playlist.SeqNo      // <-- dereferenced before the err check
    if err != nil {
        ...
        goto waitTick
    }
    

    LoadMediaPlaylist returns nil, err on any HTTP/parse failure, so playlist is a typed-nil pointer here — playlist.SeqNo panics. A panic in a goroutine takes down the entire process (Go doesn't isolate goroutine panics). Practically: any single transient network hiccup on any variant's playlist fetch kills the whole download session, mid-recording. Move the err != nil check above the SeqNo read.


🟠 High-priority bugs / correctness

  1. PriorityQueue.Less in pkg/transfer/queue.go sorts newest-first (pq[i].Timestamp.After(pq[j].Timestamp)), making the transfer queue LIFO. For a live-recording pipeline that keeps producing new segments, this means the oldest queued segments can be starved indefinitely as long as new ones keep arriving — the opposite of what you'd want (drain oldest first so nothing waits forever). Worth confirming this was intentional; if not, flip the comparison.

  2. Retry backoff runs before the first attempt, not just retries — pkg/transfer/queue.go, processItem:

    for attempt := 1; attempt <= maxRetries; attempt++ {
        if attempt > 0 {   // always true, since attempt starts at 1
            ... backoff ...
    

    Every single transfer sleeps 1s before even trying once. Should be attempt > 1.

  3. pkg/constants/constants.go (main) hardcodes a live FloSports auth token directly in source (MasterURL with token=..., hmac2=...). It's expired now (epoch ~2025-07), so no live exposure, but it's the same pattern as #1 — secrets/tokens don't belong in source. Worth a rule going forward: URLs/tokens come from CLI args or env, never constants.

  4. http.Client{} created with no timeout in pkg/media/stream.go, playlist.go, and segment.go. A stalled TCP connection (dead CDN edge, network partition) can hang a fetch forever — nothing in the surrounding code will time it out except the 10s context wrapped specifically around segment downloads in VariantDownloader. Playlist-fetch calls (GetAllVariants, LoadMediaPlaylist) have no such wrapper. Give the shared client a sane Timeout (e.g. 15–30s) as a backstop.


🟡 Cleanup / dead weight

  1. cmd/proc/main.go is a one-line stub (package proc) on main — dead entry point, no func main(). Either finish it or delete it; right now go build ./... would fail on this if it were ever wired as a real main package (it currently silently isn't, which is its own trap for future-you).
  2. Built binaries committed to git (bin/flo_download, bin/flo_download.exe, ~18MB, on cleanup/manifest). The .gitea/workflows/build.yml already cross-compiles and uploads artifacts — there's no reason binaries should also live in the repo/history. Remove and add bin/ to .gitignore.
  3. scratch.txt is tracked despite being in .gitignore. Adding a path to .gitignore after it's already tracked does nothing — it's still in the repo (and it's just a raw debug log dump). git rm --cached scratch.txt.
  4. .idea/* (JetBrains project files) are committed. Low stakes solo-repo, but standard practice is to gitignore the whole .idea/ directory rather than cherry-pick.

🔵 Program / architecture — bigger picture

  1. The project's own CLAUDE.md states the goal is "a listening REST API [that] take[s] in m3u8 urls, parse[s] them, and eventually send[s] to a conversion service." None of the three branches implement any HTTP server — all three are CLI-only, one-shot-per-invocation tools. This is the largest gap between stated intent and actual code. Worth deciding: is the REST-API framing still the goal, or has the project's actual direction (CLI recorder + NAS mirror + FFmpeg concat, per cleanup's CLAUDE.md) superseded it? Right now the docs and the code disagree with each other.
  2. pkg/nas/nas.go's "connection" is a Windows-only shell-out to net use (EstablishConnection), while CI (build.yml) cross-compiles for linux/darwin/windows. On non-Windows the UNC-path logic (\\server\share) and net use simply don't apply — there's no SMB client abstraction, so the Linux/macOS builds can produce a binary that can't actually do NAS transfer. If cross-platform deployment is real (not just "it happens to compile"), this needs a real cross-platform SMB/CIFS story (e.g., assume the NAS is pre-mounted by the OS and just treat it as a local path, which is actually simpler and already half-true for the Linux CIFS example in DEPLOYMENT.md).
  3. Zero tests on main. cleanup has real coverage (pkg/config, pkg/utils, pkg/constants, pkg/httpClient, pkg/media/manifest, pkg/processing) plus a Makefile (test, test-coverage, lint, security via gosec) and a TESTING.md. All of this is stuck on a branch that was never merged — it's not a "write tests" TODO so much as a "land the tests you already wrote" TODO.
  4. CI never actually runs tests, lint, or gosec despite the Makefile defining a full ci target (fmt tidy lint security test build). .gitea/workflows/build.yml only cross-compiles. Wire make ci (or its steps) into the pipeline so a broken build/test doesn't silently ship a binary artifact.
  5. Unbounded seen map in VariantDownloader (pkg/media/stream.go) grows for the lifetime of a variant's polling loop with no eviction — fine for a few-hour event, worth a note if this is ever meant to run continuously/24-7.

🟢 Deployment

  1. DEPLOYMENT.md (on cleanup) documents Docker/Compose/Kubernetes deployment in detail, but there's no actual Dockerfile committed anywhere in the repo — it's a doc-only example. If containerized deployment is the real target, commit the Dockerfile and at least smoke-test the build.
  2. No secrets-management story beyond "use env vars" — fine for a home lab, but the doc's own "Security" section says "use secret management systems," which isn't backed by anything concrete (no .env.example, no sealed-secrets/vault reference). Given finding #1, this is worth actually doing, not just documenting.
  3. Health checks, metrics/log shipping are listed as "considerations" in DEPLOYMENT.md but nothing is implemented — the app is a fire-and-forget CLI process with log.Printf to stdout only. If this runs unattended (e.g., in a container watching for the next event), you have no way to know it's alive short of watching logs.

Suggested order of attack

  1. Rotate the NAS credential (#1) — this is time-sensitive regardless of anything else.
  2. Fix the two crash bugs (#2, #3) — nothing else is worth doing until a full recording session can survive.
  3. Decide the branch-merge question and land cleanup → main (or consciously don't).
  4. Wire CI to actually test/lint (#15) so regressions get caught going forward.
  5. Then tackle the architecture question (#12: REST API vs. CLI-recorder-as-is) before investing more in either direction.