Bug fixes
This commit is contained in:
parent
0f4a39f426
commit
11b2ba8a8b
111
TODO.md
Normal file
111
TODO.md
Normal file
@ -0,0 +1,111 @@
|
|||||||
|
# Return-from-a-year-away Audit — 2026-08-05
|
||||||
|
|
||||||
|
Scope reviewed: `main` (checked out), `manifest`, `cleanup` (all local + `origin` equivalents).
|
||||||
|
Remote: `https://git.aaronic.cc/kacarmichael/StreamRecorder.git`
|
||||||
|
|
||||||
|
## Branch situation (read this first)
|
||||||
|
|
||||||
|
The three branches are **not parallel efforts** — they're one linear line of development that `main` never caught up to:
|
||||||
|
|
||||||
|
```
|
||||||
|
main (0f4a39f, 2025-07-26) ──┐ stale, single-file-era downloader only
|
||||||
|
│
|
||||||
|
manifest (718d69c, 2025-08-10) ── +restructured cmd/pkg layout, config-less
|
||||||
|
│
|
||||||
|
cleanup (7d67cec, 2025-08-11) ── +pkg/config, tests, CI, Makefile, deployment docs
|
||||||
|
```
|
||||||
|
|
||||||
|
`manifest` is literally an ancestor of `cleanup` (same commit `718d69c`). `cleanup` is the most complete state that ever existed and was the last thing touched (2025-08-11) before the year gap. **None of this reached `main`.** If you pick this project back up, decide this before anything else:
|
||||||
|
|
||||||
|
- [ ] **Merge `cleanup` → `main`** (recommended — it's a strict superset: config validation, `pkg/nas`, `pkg/processing`, unit tests, CI workflow, `DEPLOYMENT.md`/`TESTING.md`). Then delete `manifest` (fully subsumed) and re-evaluate `cleanup` for deletion once merged.
|
||||||
|
- [ ] Or, if `cleanup`'s direction (FFmpeg concatenation, NAS-mirroring processing pipeline) is no longer where you want to go, explicitly archive it and restart `main` from a clean design — but do this as a decision, not by default.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔴 Critical — fix/rotate before running this again
|
||||||
|
|
||||||
|
1. **Leaked NAS credentials, committed in plaintext.** `pkg/config/config.go` (branch `cleanup`) hardcodes a real default:
|
||||||
|
```go
|
||||||
|
Username: "NASAdmin",
|
||||||
|
Password: "s3tkY6tzA&KN6M",
|
||||||
|
```
|
||||||
|
This is in git history on a remote (`git.aaronic.cc`) regardless of whether you later change the default. **Rotate this NAS account's password now** — changing the code doesn't un-leak a password that's sat in a repo for a year. Then remove the credential entirely (fail closed: require `NAS_USERNAME`/`NAS_PASSWORD` env vars with no default, error out if unset when transfer is enabled) rather than shipping a working default.
|
||||||
|
|
||||||
|
2. **Guaranteed crash: double mutex unlock in `pkg/transfer/cleanup.go`.** In `ExecuteCleanup` (present on **`main` too**, not just `cleanup`):
|
||||||
|
```go
|
||||||
|
cs.mu.Lock()
|
||||||
|
...
|
||||||
|
cs.mu.Unlock() // line 72 — releases the lock
|
||||||
|
...
|
||||||
|
batch := make([]string, batchSize)
|
||||||
|
copy(batch, cs.pendingFiles[:batchSize])
|
||||||
|
cs.pendingFiles = cs.pendingFiles[batchSize:] // mutated WITHOUT the lock held
|
||||||
|
cs.mu.Unlock() // line 79 — unlocking an already-unlocked mutex
|
||||||
|
```
|
||||||
|
An unlock of an unlocked `sync.Mutex` is a **fatal, unrecoverable runtime error** (`fatal error: sync: unlock of unlocked mutex`) — it kills the whole process, not just a goroutine. Since `Cleanup.AfterTransfer` defaults to `true` and `RetainHours: 0` (immediate eligibility), this fires the first time any file finishes transferring and cleanup runs. This is very likely why any end-to-end run with NAS transfer enabled never survived long. Fix: hold the lock for the whole slice mutation, or copy-then-release-then-mutate-under-a-second-short-lock, but don't unlock twice.
|
||||||
|
|
||||||
|
3. **Nil-pointer crash in `pkg/media/stream.go` (`VariantDownloader`, `main` branch):**
|
||||||
|
```go
|
||||||
|
playlist, err := LoadMediaPlaylist(variant.URL)
|
||||||
|
seq := playlist.SeqNo // <-- dereferenced before the err check
|
||||||
|
if err != nil {
|
||||||
|
...
|
||||||
|
goto waitTick
|
||||||
|
}
|
||||||
|
```
|
||||||
|
`LoadMediaPlaylist` returns `nil, err` on any HTTP/parse failure, so `playlist` is a typed-nil pointer here — `playlist.SeqNo` panics. A panic in a goroutine takes down the entire process (Go doesn't isolate goroutine panics). Practically: any single transient network hiccup on any variant's playlist fetch kills the whole download session, mid-recording. Move the `err != nil` check above the `SeqNo` read.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🟠 High-priority bugs / correctness
|
||||||
|
|
||||||
|
4. **`PriorityQueue.Less` in `pkg/transfer/queue.go` sorts newest-first** (`pq[i].Timestamp.After(pq[j].Timestamp)`), making the transfer queue LIFO. For a live-recording pipeline that keeps producing new segments, this means the *oldest* queued segments can be starved indefinitely as long as new ones keep arriving — the opposite of what you'd want (drain oldest first so nothing waits forever). Worth confirming this was intentional; if not, flip the comparison.
|
||||||
|
|
||||||
|
5. **Retry backoff runs before the *first* attempt, not just retries** — `pkg/transfer/queue.go`, `processItem`:
|
||||||
|
```go
|
||||||
|
for attempt := 1; attempt <= maxRetries; attempt++ {
|
||||||
|
if attempt > 0 { // always true, since attempt starts at 1
|
||||||
|
... backoff ...
|
||||||
|
```
|
||||||
|
Every single transfer sleeps `1s` before even trying once. Should be `attempt > 1`.
|
||||||
|
|
||||||
|
6. **`pkg/constants/constants.go` (main) hardcodes a live FloSports auth token** directly in source (`MasterURL` with `token=...`, `hmac2=...`). It's expired now (epoch ~2025-07), so no live exposure, but it's the same pattern as #1 — secrets/tokens don't belong in source. Worth a rule going forward: URLs/tokens come from CLI args or env, never constants.
|
||||||
|
|
||||||
|
7. **`http.Client{}` created with no timeout** in `pkg/media/stream.go`, `playlist.go`, and `segment.go`. A stalled TCP connection (dead CDN edge, network partition) can hang a fetch forever — nothing in the surrounding code will time it out except the 10s context wrapped specifically around segment downloads in `VariantDownloader`. Playlist-fetch calls (`GetAllVariants`, `LoadMediaPlaylist`) have no such wrapper. Give the shared client a sane `Timeout` (e.g. 15–30s) as a backstop.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🟡 Cleanup / dead weight
|
||||||
|
|
||||||
|
8. **`cmd/proc/main.go` is a one-line stub** (`package proc`) on `main` — dead entry point, no `func main()`. Either finish it or delete it; right now `go build ./...` would fail on this if it were ever wired as a real `main` package (it currently silently isn't, which is its own trap for future-you).
|
||||||
|
9. **Built binaries committed to git** (`bin/flo_download`, `bin/flo_download.exe`, ~18MB, on `cleanup`/`manifest`). The `.gitea/workflows/build.yml` already cross-compiles and uploads artifacts — there's no reason binaries should also live in the repo/history. Remove and add `bin/` to `.gitignore`.
|
||||||
|
10. **`scratch.txt` is tracked despite being in `.gitignore`.** Adding a path to `.gitignore` after it's already tracked does nothing — it's still in the repo (and it's just a raw debug log dump). `git rm --cached scratch.txt`.
|
||||||
|
11. `.idea/*` (JetBrains project files) are committed. Low stakes solo-repo, but standard practice is to gitignore the whole `.idea/` directory rather than cherry-pick.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔵 Program / architecture — bigger picture
|
||||||
|
|
||||||
|
12. **The project's own `CLAUDE.md` states the goal is "a listening REST API [that] take[s] in m3u8 urls, parse[s] them, and eventually send[s] to a conversion service."** None of the three branches implement any HTTP server — all three are CLI-only, one-shot-per-invocation tools. This is the largest gap between stated intent and actual code. Worth deciding: is the REST-API framing still the goal, or has the project's actual direction (CLI recorder + NAS mirror + FFmpeg concat, per `cleanup`'s `CLAUDE.md`) superseded it? Right now the docs and the code disagree with each other.
|
||||||
|
13. **`pkg/nas/nas.go`'s "connection" is a Windows-only shell-out to `net use`** (`EstablishConnection`), while CI (`build.yml`) cross-compiles for `linux`/`darwin`/`windows`. On non-Windows the UNC-path logic (`\\server\share`) and `net use` simply don't apply — there's no SMB client abstraction, so the Linux/macOS builds can produce a binary that can't actually do NAS transfer. If cross-platform deployment is real (not just "it happens to compile"), this needs a real cross-platform SMB/CIFS story (e.g., assume the NAS is pre-mounted by the OS and just treat it as a local path, which is actually simpler and already half-true for the Linux CIFS example in `DEPLOYMENT.md`).
|
||||||
|
14. **Zero tests on `main`.** `cleanup` has real coverage (`pkg/config`, `pkg/utils`, `pkg/constants`, `pkg/httpClient`, `pkg/media/manifest`, `pkg/processing`) plus a `Makefile` (`test`, `test-coverage`, `lint`, `security` via `gosec`) and a `TESTING.md`. All of this is stuck on a branch that was never merged — it's not a "write tests" TODO so much as a "land the tests you already wrote" TODO.
|
||||||
|
15. **CI never actually runs tests, lint, or `gosec`** despite the `Makefile` defining a full `ci` target (`fmt tidy lint security test build`). `.gitea/workflows/build.yml` only cross-compiles. Wire `make ci` (or its steps) into the pipeline so a broken build/test doesn't silently ship a binary artifact.
|
||||||
|
16. **Unbounded `seen` map** in `VariantDownloader` (`pkg/media/stream.go`) grows for the lifetime of a variant's polling loop with no eviction — fine for a few-hour event, worth a note if this is ever meant to run continuously/24-7.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🟢 Deployment
|
||||||
|
|
||||||
|
17. `DEPLOYMENT.md` (on `cleanup`) documents Docker/Compose/Kubernetes deployment in detail, but there's no actual `Dockerfile` committed anywhere in the repo — it's a doc-only example. If containerized deployment is the real target, commit the Dockerfile and at least smoke-test the build.
|
||||||
|
18. No secrets-management story beyond "use env vars" — fine for a home lab, but the doc's own "Security" section says "use secret management systems," which isn't backed by anything concrete (no `.env.example`, no sealed-secrets/vault reference). Given finding #1, this is worth actually doing, not just documenting.
|
||||||
|
19. Health checks, metrics/log shipping are listed as "considerations" in `DEPLOYMENT.md` but nothing is implemented — the app is a fire-and-forget CLI process with `log.Printf` to stdout only. If this runs unattended (e.g., in a container watching for the next event), you have no way to know it's alive short of watching logs.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Suggested order of attack
|
||||||
|
|
||||||
|
1. Rotate the NAS credential (#1) — this is time-sensitive regardless of anything else.
|
||||||
|
2. Fix the two crash bugs (#2, #3) — nothing else is worth doing until a full recording session can survive.
|
||||||
|
3. Decide the branch-merge question and land `cleanup` → `main` (or consciously don't).
|
||||||
|
4. Wire CI to actually test/lint (#15) so regressions get caught going forward.
|
||||||
|
5. Then tackle the architecture question (#12: REST API vs. CLI-recorder-as-is) before investing more in either direction.
|
||||||
@ -110,12 +110,13 @@ func VariantDownloader(ctx context.Context, variant *StreamVariant, sem chan str
|
|||||||
default:
|
default:
|
||||||
}
|
}
|
||||||
|
|
||||||
|
var seq uint64
|
||||||
playlist, err := LoadMediaPlaylist(variant.URL)
|
playlist, err := LoadMediaPlaylist(variant.URL)
|
||||||
seq := playlist.SeqNo
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
log.Printf("%s: Error loading playlist playlist: %v", variant.Resolution, err)
|
log.Printf("%s: Error loading playlist playlist: %v", variant.Resolution, err)
|
||||||
goto waitTick
|
goto waitTick
|
||||||
}
|
}
|
||||||
|
seq = playlist.SeqNo
|
||||||
|
|
||||||
for _, seg := range playlist.Segments {
|
for _, seg := range playlist.Segments {
|
||||||
if seg == nil {
|
if seg == nil {
|
||||||
|
|||||||
@ -69,7 +69,6 @@ func (cs *CleanupService) ExecuteCleanup(ctx context.Context) error {
|
|||||||
if batchSize > len(cs.pendingFiles) {
|
if batchSize > len(cs.pendingFiles) {
|
||||||
batchSize = len(cs.pendingFiles)
|
batchSize = len(cs.pendingFiles)
|
||||||
}
|
}
|
||||||
cs.mu.Unlock()
|
|
||||||
|
|
||||||
log.Printf("Executing cleanup batch (size: %d)", batchSize)
|
log.Printf("Executing cleanup batch (size: %d)", batchSize)
|
||||||
|
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user