Health/status http endpoint, logging
Some checks failed
Build Go Binaries / build (push) Has been cancelled
Some checks failed
Build Go Binaries / build (push) Has been cancelled
This commit is contained in:
parent
ce2c5f3181
commit
309eeef59a
180
README.md
Normal file
180
README.md
Normal file
@ -0,0 +1,180 @@
|
||||
# m3u8-downloader
|
||||
|
||||
A Go command-line tool that records live HLS (HTTP Live Streaming) broadcasts. Point it at a master `.m3u8` playlist URL and it will:
|
||||
|
||||
1. **Download** every quality variant (1080p, 720p, ...) in real time as new segments appear.
|
||||
2. **Transfer** finished segments to NAS storage for archival, with retry logic and local cleanup.
|
||||
3. **Process** the archived segments into a single playable `.mp4` per event via FFmpeg.
|
||||
|
||||
**Download and transfer run together, in one invocation.** When NAS transfer is enabled (the default), starting a download also starts the transfer service in the same process: as soon as a segment is written to disk, a file watcher picks it up, transfers it to NAS, and deletes the local copy — all while the download keeps running. This keeps local disk usage bounded during a long recording instead of accumulating the whole event locally.
|
||||
|
||||
**Processing is a separate, later invocation** (see [Usage](#usage) below), because it reads finished segments *from NAS*, after transfer has fully drained — it can't run concurrently with the other two in any useful way.
|
||||
|
||||
There's also a standalone `-transfer` mode, but it's a catch-up/recovery tool, not the normal path: use it when a download session already ended with some files not yet confirmed on NAS (killed before the queue drained, or transfer was disabled during the recording and you want to sync afterward).
|
||||
|
||||
The project also ships structured JSON logging per subsystem and an HTTP status endpoint with a small web dashboard, so you can tell what's happening (and especially what's failing) without staring at a terminal.
|
||||
|
||||
> Project docs also refer to this as "StreamRecorder" (see `DEPLOYMENT.md`/`TESTING.md`) — same project, the Go module is named `m3u8-downloader`.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
|
||||
.m3u8 URL ──▶ │ DOWNLOAD │──────▶ │ TRANSFER │──────▶ │ PROCESSING │──▶ event.mp4
|
||||
│ (pkg/media) │ .ts │(pkg/transfer,│ NAS │(pkg/processing)
|
||||
└──────────────┘ files │ pkg/nas) │ files └──────────────┘
|
||||
│ └──────────────┘ │
|
||||
└──────────────┬─────────────────────────────────┘
|
||||
▼
|
||||
pkg/logging (JSON files) + pkg/health (HTTP /status, dashboard)
|
||||
```
|
||||
|
||||
### Download — `pkg/media`
|
||||
|
||||
- `stream.go`: `GetAllVariants()` fetches and parses the master playlist, extracting every quality variant. `VariantDownloader()` runs one per-variant polling loop (every `Core.RefreshDelay`, default 3s) that watches the variant's chunklist for new segments and dispatches downloads through a worker-limited semaphore (`Core.WorkerCount` per variant, default 4).
|
||||
- `playlist.go`: `LoadMediaPlaylist()` fetches and parses an individual variant's chunklist.
|
||||
- `segment.go`: `DownloadSegment()` downloads one `.ts` file, with a couple of retries (including a dedicated retry for transient HTTP 403s some CDNs return).
|
||||
- `manifest.go`: `ManifestWriter` tracks every segment downloaded per event and writes a JSON manifest (sequence number + resolution) at the end of a run.
|
||||
|
||||
Segments land on disk under `Paths.LocalOutput/<event>/<resolution>/`.
|
||||
|
||||
### Transfer — `pkg/transfer` + `pkg/nas`
|
||||
|
||||
- `service.go`: `TransferService` wires together the watcher, queue, and cleanup service, and owns the connection to NAS for the run.
|
||||
- `watcher.go`: `FileWatcher` (via `fsnotify`) watches the local event directory for new `.ts` files and queues them after a short settling delay (`Transfer.FileSettlingDelay`, default 5s), so a file isn't queued while still being written.
|
||||
- `queue.go`: `TransferQueue` is a priority-queued worker pool (`Transfer.WorkerCount`, default 2) that transfers queued files to NAS with retry + exponential backoff, and persists its state to `transfer_queue.json` so it survives a restart.
|
||||
- `nas.go` / `pkg/nas`: `NASService` establishes the NAS connection (on Windows, via `net use` for UNC paths) and does the actual file copy + size verification.
|
||||
- `cleanup.go`: `CleanupService` deletes local copies after a successful, verified transfer (`Cleanup.AfterTransfer`, default true).
|
||||
|
||||
During a normal download run, this whole stage starts automatically in the same process as the downloader (see the note above) — you don't invoke transfer separately for a live recording. The standalone `-transfer` mode exists only to catch up files that a prior download session ended without fully transferring.
|
||||
|
||||
### Processing — `pkg/processing`
|
||||
|
||||
- `ProcessingService` reads segments **from NAS** (not the local disk) for a given event, picks the highest-resolution variant available for each sequence number, writes an FFmpeg concat list, and shells out to FFmpeg to produce one `.mp4`.
|
||||
- This is why processing is normally the *last* step, run after transfer has fully drained — it only sees what's actually on NAS.
|
||||
- Requires a working `ffmpeg` on `PATH` (or `Processing.FFmpegPath` / `FFMPEG_PATH`), and connects to NAS on startup regardless of whether `Processing.Enabled` is set.
|
||||
|
||||
### Configuration — `pkg/config` + `pkg/constants`
|
||||
|
||||
`pkg/config/config.go` defines defaults and environment-variable overrides for every setting below. `pkg/constants.MustGetConfig()` lazily loads and caches this config as a process-wide singleton — most of the codebase reads config this way rather than passing it around explicitly.
|
||||
|
||||
### Logging — `pkg/logging`
|
||||
|
||||
A small factory (modeled after a structured-logging pattern used in another project) that hands out one `log/slog` JSON logger per pipeline stage: `App`, `Download`, `Transfer`, `Processing`. Each writes to its own file under `Logging.Dir` (default `data/logs/{app,download,transfer,processing}.log`). This is **additive** — every existing human-readable console log line is untouched, so you can still watch progress live in the terminal; the structured logs are there for after-the-fact debugging.
|
||||
|
||||
### Health / status — `pkg/health`
|
||||
|
||||
An in-memory registry that every subsystem reports into (active variants, segment success/failure counts, NAS connectivity, transfer queue stats, processing pipeline state), served over HTTP:
|
||||
|
||||
- `GET /status` — full JSON snapshot, `status: "healthy"` or `"degraded"` (degraded the moment anything has a recent error, NAS is disconnected while in use, or processing has failed).
|
||||
- `GET /healthz` — plain liveness check.
|
||||
- `GET /` — a small auto-refreshing dashboard (polls `/status` every 3s) showing the same information visually. No external dependencies (fonts/JS/CSS), so it works on a venue LAN with no internet access.
|
||||
|
||||
**Important scope note:** the health server only reflects whatever is live in the *current* process. During a normal download run, `/status` shows real-time download + transfer state together (they're the same process) with processing reported as `idle`. Running `-process` afterward is a separate process with its own fresh health server showing real-time processing state (and idle download/transfer). There's no single dashboard spanning download/transfer and processing at the same time, since processing only ever runs after the fact.
|
||||
|
||||
By default the health server binds all network interfaces (`:8090`) so it can be checked from another device on the same LAN (e.g. a phone) during an event. **There is no authentication or TLS** — this is fine on a trusted local network only, never expose it to the open internet.
|
||||
|
||||
## Configuration
|
||||
|
||||
All settings have sane defaults; override via environment variables as needed.
|
||||
|
||||
| Env var | Default | Meaning |
|
||||
|---|---|---|
|
||||
| `WORKER_COUNT` | `4` | Concurrent segment downloads per variant |
|
||||
| `REFRESH_DELAY_SECONDS` | `3` | How often each variant's chunklist is polled |
|
||||
| `ENABLE_NAS_TRANSFER` | `true` | Whether to run the transfer service alongside download |
|
||||
| `NAS_OUTPUT_PATH` | `\\HomeLabNAS\dci\streams` | UNC/NAS root path segments are copied to |
|
||||
| `NAS_USERNAME` / `NAS_PASSWORD` | *(see note below)* | NAS auth credentials |
|
||||
| `LOCAL_OUTPUT_DIR` | `data` | Local base directory for downloaded segments |
|
||||
| `PROCESS_OUTPUT_DIR` | `out` | Output directory for the final concatenated `.mp4` |
|
||||
| `FFMPEG_PATH` | `ffmpeg` | Path to the FFmpeg binary used by the processing stage |
|
||||
| `LOG_DIR` | `data/logs` | Directory for the per-subsystem JSON log files |
|
||||
| `LOG_LEVEL` | `info` | `debug` \| `info` \| `warn` \| `error` (also forced to `debug` by the `-debug` flag) |
|
||||
| `HEALTH_ENABLED` | `true` | Whether to start the health/status HTTP server |
|
||||
| `HEALTH_ADDR` | `:8090` | Bind address for the health server + dashboard |
|
||||
|
||||
> **Security note:** the default NAS credentials currently ship as hardcoded defaults in `pkg/config/config.go` for this home-lab setup. If you fork or redeploy this, set `NAS_USERNAME`/`NAS_PASSWORD` explicitly and rotate the shipped default — see `TODO.md` for the full flagged list of housekeeping items like this one.
|
||||
|
||||
## Building
|
||||
|
||||
```bash
|
||||
go build -o m3u8-downloader ./cmd/main
|
||||
# or, via the Makefile (produces stream-recorder(.exe)):
|
||||
make build
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
All modes are invoked through the single `cmd/main` binary:
|
||||
|
||||
```
|
||||
m3u8-downloader [-url <m3u8-url>] [-event <name>] [-debug] [-transfer] [-process]
|
||||
```
|
||||
|
||||
| Flag | Applies to | Meaning |
|
||||
|---|---|---|
|
||||
| `-url` | download mode | Master `.m3u8` playlist URL. If omitted (and neither `-transfer` nor `-process` is set), you'll be prompted for it interactively. |
|
||||
| `-event` | all modes | Event name — used to namespace local output, NAS destination, and the manifest file. **Always set this explicitly**; if left blank in download mode, segments land directly under the local output root instead of a per-event subfolder. In `-transfer`/`-process` mode, omitting it shows an interactive picker over existing event directories. |
|
||||
| `-debug` | download mode | Restricts download to the 1080p variant only (for faster iteration) and forces log level to `debug`. |
|
||||
| `-transfer` | transfer-only mode | Scans an already-downloaded event's local directory and transfers/re-transfers anything not yet confirmed on NAS, then runs the normal transfer service until shutdown. Requires `ENABLE_NAS_TRANSFER=true`. |
|
||||
| `-process` | process-only mode | Reads an event's segments from NAS and concatenates the highest-quality ones into a single `.mp4` via FFmpeg. Always attempts a NAS connection on startup, even if processing is otherwise disabled in config. |
|
||||
|
||||
### Which command for which situation
|
||||
|
||||
**Recording a live event** (the normal case — downloads and transfers to NAS concurrently, in one process, so local disk usage stays bounded rather than accumulating the whole event):
|
||||
```bash
|
||||
m3u8-downloader -url "https://example.com/live/master.m3u8" -event "2026-regionals"
|
||||
```
|
||||
Stop it cleanly with Ctrl+C (SIGINT/SIGTERM); it finishes in-flight downloads, saves transfer queue state, and writes the manifest before exiting.
|
||||
|
||||
**Quick test/dry run** (single variant, verbose logs, to confirm a playlist URL actually works before committing to a full multi-quality recording):
|
||||
```bash
|
||||
m3u8-downloader -url "https://example.com/live/master.m3u8" -event "test-run" -debug
|
||||
```
|
||||
|
||||
**Catching up NAS transfer after the fact** (recovery only — the recording process above already transfers live; use this if a session ended before every file made it to NAS, or transfer was disabled during the recording and you want to sync afterward):
|
||||
```bash
|
||||
m3u8-downloader -event "2026-regionals" -transfer
|
||||
```
|
||||
|
||||
**Producing the final video** (after transfer has fully drained — check `/status` or the NAS directory to confirm nothing's still pending):
|
||||
```bash
|
||||
m3u8-downloader -event "2026-regionals" -process
|
||||
```
|
||||
|
||||
**Checking on a running instance** (from any device on the same network as the recorder):
|
||||
```
|
||||
http://<recorder-host>:8090/ # dashboard
|
||||
http://<recorder-host>:8090/status # raw JSON
|
||||
```
|
||||
|
||||
## Data layout
|
||||
|
||||
```
|
||||
data/
|
||||
├── {event}.json # download manifest (sequence numbers + resolutions)
|
||||
├── {event}/
|
||||
│ ├── 1080p/ # downloaded segments, before NAS transfer + cleanup
|
||||
│ ├── 720p/
|
||||
│ └── ...
|
||||
├── logs/
|
||||
│ ├── app.log # per-subsystem structured JSON logs
|
||||
│ ├── download.log
|
||||
│ ├── transfer.log
|
||||
│ └── processing.log
|
||||
└── transfer_queue.json # persisted transfer queue state
|
||||
|
||||
\\<NAS>\...\{event}\ # mirrors the local structure once transferred
|
||||
├── 1080p/
|
||||
└── ...
|
||||
|
||||
out/
|
||||
└── {event}/
|
||||
└── {event}.mp4 # final FFmpeg output (process-only mode)
|
||||
```
|
||||
|
||||
## More documentation
|
||||
|
||||
- `DEPLOYMENT.md` — Docker/Kubernetes deployment examples and production considerations.
|
||||
- `TESTING.md` — test suite layout and how to run it (`make test`, `make test-coverage`, etc.).
|
||||
- `TODO.md` — known issues, pre-existing bugs, and roadmap items found during review, not yet addressed.
|
||||
@ -4,6 +4,8 @@ import (
|
||||
"context"
|
||||
"log"
|
||||
"m3u8-downloader/pkg/constants"
|
||||
"m3u8-downloader/pkg/health"
|
||||
"m3u8-downloader/pkg/logging"
|
||||
"m3u8-downloader/pkg/media"
|
||||
"m3u8-downloader/pkg/transfer"
|
||||
"m3u8-downloader/pkg/utils"
|
||||
@ -15,6 +17,8 @@ import (
|
||||
)
|
||||
|
||||
func Download(masterURL string, eventName string, debug bool) {
|
||||
appLog := logging.MustGetFactory().App
|
||||
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
|
||||
@ -24,11 +28,26 @@ func Download(masterURL string, eventName string, debug bool) {
|
||||
go func() {
|
||||
<-sigChan
|
||||
log.Println("Shutting down...")
|
||||
appLog.Info("shutdown signal received")
|
||||
cancel()
|
||||
}()
|
||||
|
||||
cfg := constants.MustGetConfig()
|
||||
|
||||
if cfg.Health.Enabled {
|
||||
healthSrv, err := health.Start(ctx, &cfg.Health, health.MustGetRegistry())
|
||||
if err != nil {
|
||||
appLog.Error("failed to start health server", logging.KeyError, err.Error())
|
||||
} else {
|
||||
appLog.Info("health server listening", "addr", cfg.Health.Addr)
|
||||
defer func() {
|
||||
shutdownCtx, shutdownCancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer shutdownCancel()
|
||||
_ = healthSrv.Shutdown(shutdownCtx)
|
||||
}()
|
||||
}
|
||||
}
|
||||
|
||||
var wg sync.WaitGroup
|
||||
var transferService *transfer.TransferService
|
||||
if cfg.NAS.EnableTransfer {
|
||||
@ -36,6 +55,7 @@ func Download(masterURL string, eventName string, debug bool) {
|
||||
if err != nil {
|
||||
log.Printf("Failed to create transfer service: %v", err)
|
||||
log.Println("Continuing without transfer service...")
|
||||
appLog.Error("failed to create transfer service, continuing without it", logging.KeyError, err.Error())
|
||||
} else {
|
||||
transferService = ts
|
||||
wg.Add(1)
|
||||
@ -43,9 +63,11 @@ func Download(masterURL string, eventName string, debug bool) {
|
||||
defer wg.Done()
|
||||
if err := transferService.Start(ctx); err != nil && err != context.Canceled {
|
||||
log.Printf("Transfer service error: %v", err)
|
||||
appLog.Error("transfer service error", logging.KeyError, err.Error())
|
||||
}
|
||||
}()
|
||||
log.Println("Transfer service started.")
|
||||
appLog.Info("transfer service started")
|
||||
}
|
||||
}
|
||||
|
||||
@ -53,11 +75,13 @@ func Download(masterURL string, eventName string, debug bool) {
|
||||
|
||||
eventPath := cfg.GetEventPath(eventName)
|
||||
if err := utils.EnsureDir(eventPath); err != nil {
|
||||
appLog.Error("failed to create event directory", logging.KeyError, err.Error())
|
||||
log.Fatalf("Failed to create event directory: %v", err)
|
||||
}
|
||||
|
||||
variants, err := media.GetAllVariants(masterURL, eventPath, manifestWriter)
|
||||
if err != nil {
|
||||
appLog.Error("failed to get variants", logging.KeyError, err.Error())
|
||||
log.Fatalf("Failed to get variants: %v", err)
|
||||
}
|
||||
log.Printf("Found %d variants", len(variants))
|
||||
@ -82,6 +106,7 @@ func Download(masterURL string, eventName string, debug bool) {
|
||||
|
||||
wg.Wait()
|
||||
log.Println("All variant downloaders finished.")
|
||||
appLog.Info("all variant downloaders finished")
|
||||
|
||||
if transferService != nil {
|
||||
shutdownCtx, shutdownCancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||
@ -90,7 +115,9 @@ func Download(masterURL string, eventName string, debug bool) {
|
||||
}
|
||||
|
||||
log.Println("All Services shut down.")
|
||||
appLog.Info("all services shut down")
|
||||
|
||||
manifestWriter.WriteManifest()
|
||||
log.Println("Manifest written.")
|
||||
appLog.Info("manifest written", logging.KeyEvent, eventName)
|
||||
}
|
||||
|
||||
@ -7,6 +7,8 @@ import (
|
||||
"m3u8-downloader/cmd/downloader"
|
||||
"m3u8-downloader/cmd/processor"
|
||||
"m3u8-downloader/cmd/transfer"
|
||||
"m3u8-downloader/pkg/constants"
|
||||
"m3u8-downloader/pkg/logging"
|
||||
"os"
|
||||
"strings"
|
||||
)
|
||||
@ -20,6 +22,14 @@ func main() {
|
||||
|
||||
flag.Parse()
|
||||
|
||||
// Must happen before the first logging.MustGetFactory() call anywhere in
|
||||
// the program, since the logging factory reads its level from this
|
||||
// config singleton once and caches it.
|
||||
if *debug {
|
||||
constants.MustGetConfig().Logging.Level = "debug"
|
||||
}
|
||||
logging.MustGetFactory().App.Info("m3u8-downloader starting", "debug", *debug)
|
||||
|
||||
if *transferOnly {
|
||||
transfer.RunTransferOnly(*eventName)
|
||||
return
|
||||
|
||||
@ -4,17 +4,39 @@ import (
|
||||
"context"
|
||||
"log"
|
||||
"m3u8-downloader/pkg/constants"
|
||||
"m3u8-downloader/pkg/health"
|
||||
"m3u8-downloader/pkg/logging"
|
||||
"m3u8-downloader/pkg/processing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func Process(eventName string) {
|
||||
appLog := logging.MustGetFactory().App
|
||||
log.Printf("Starting processing for event: %s", eventName)
|
||||
appLog.Info("starting processing", logging.KeyEvent, eventName)
|
||||
cfg := constants.MustGetConfig()
|
||||
|
||||
if cfg.Health.Enabled {
|
||||
healthSrv, err := health.Start(context.Background(), &cfg.Health, health.MustGetRegistry())
|
||||
if err != nil {
|
||||
appLog.Error("failed to start health server", logging.KeyError, err.Error())
|
||||
} else {
|
||||
appLog.Info("health server listening", "addr", cfg.Health.Addr)
|
||||
defer func() {
|
||||
shutdownCtx, shutdownCancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer shutdownCancel()
|
||||
_ = healthSrv.Shutdown(shutdownCtx)
|
||||
}()
|
||||
}
|
||||
}
|
||||
|
||||
ps, err := processing.NewProcessingService(eventName, cfg)
|
||||
if err != nil {
|
||||
appLog.Error("failed to create processing service", logging.KeyError, err.Error())
|
||||
log.Fatalf("Failed to create processing service: %v", err)
|
||||
}
|
||||
if err := ps.Start(context.Background()); err != nil {
|
||||
appLog.Error("processing service failed", logging.KeyError, err.Error())
|
||||
log.Fatalf("Failed to run processing service: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
@ -7,6 +7,8 @@ import (
|
||||
"log"
|
||||
"m3u8-downloader/pkg/config"
|
||||
"m3u8-downloader/pkg/constants"
|
||||
"m3u8-downloader/pkg/health"
|
||||
"m3u8-downloader/pkg/logging"
|
||||
"m3u8-downloader/pkg/transfer"
|
||||
"m3u8-downloader/pkg/utils"
|
||||
"os"
|
||||
@ -32,10 +34,12 @@ func getEventDirs(cfg *config.Config) ([]string, error) {
|
||||
}
|
||||
|
||||
func RunTransferOnly(eventName string) {
|
||||
appLog := logging.MustGetFactory().App
|
||||
cfg := constants.MustGetConfig()
|
||||
|
||||
// Check if NAS transfer is enabled
|
||||
if !cfg.NAS.EnableTransfer {
|
||||
appLog.Error("NAS transfer disabled, cannot run transfer-only mode")
|
||||
log.Fatal("NAS transfer is disabled in configuration. Please enable it to use transfer-only mode.")
|
||||
}
|
||||
|
||||
@ -79,30 +83,50 @@ func RunTransferOnly(eventName string) {
|
||||
go func() {
|
||||
<-sigChan
|
||||
log.Println("Shutting down transfer service...")
|
||||
appLog.Info("shutdown signal received")
|
||||
cancel()
|
||||
}()
|
||||
|
||||
if cfg.Health.Enabled {
|
||||
healthSrv, err := health.Start(ctx, &cfg.Health, health.MustGetRegistry())
|
||||
if err != nil {
|
||||
appLog.Error("failed to start health server", logging.KeyError, err.Error())
|
||||
} else {
|
||||
appLog.Info("health server listening", "addr", cfg.Health.Addr)
|
||||
defer func() {
|
||||
shutdownCtx, shutdownCancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer shutdownCancel()
|
||||
_ = healthSrv.Shutdown(shutdownCtx)
|
||||
}()
|
||||
}
|
||||
}
|
||||
|
||||
// Verify local event directory exists
|
||||
localEventPath := cfg.GetEventPath(eventName)
|
||||
if !utils.PathExists(localEventPath) {
|
||||
appLog.Error("local event directory does not exist", logging.KeyEvent, eventName)
|
||||
log.Fatalf("Local event directory does not exist: %s", localEventPath)
|
||||
}
|
||||
|
||||
// Create transfer service
|
||||
transferService, err := transfer.NewTrasferService(cfg.NAS.OutputPath, eventName)
|
||||
if err != nil {
|
||||
appLog.Error("failed to create transfer service", logging.KeyError, err.Error())
|
||||
log.Fatalf("Failed to create transfer service: %v", err)
|
||||
}
|
||||
|
||||
// Find and queue existing files
|
||||
if err := transferService.QueueExistingFiles(localEventPath); err != nil {
|
||||
appLog.Error("failed to queue existing files", logging.KeyError, err.Error())
|
||||
log.Fatalf("Failed to queue existing files: %v", err)
|
||||
}
|
||||
|
||||
// Start transfer service
|
||||
log.Println("Starting transfer service...")
|
||||
appLog.Info("starting transfer service", logging.KeyEvent, eventName)
|
||||
if err := transferService.Start(ctx); err != nil && err != context.Canceled {
|
||||
log.Printf("Transfer service error: %v", err)
|
||||
appLog.Error("transfer service error", logging.KeyError, err.Error())
|
||||
}
|
||||
|
||||
// Graceful shutdown
|
||||
@ -111,4 +135,5 @@ func RunTransferOnly(eventName string) {
|
||||
transferService.Shutdown(shutdownCtx)
|
||||
|
||||
log.Println("Transfer-only mode completed.")
|
||||
appLog.Info("transfer-only mode completed", logging.KeyEvent, eventName)
|
||||
}
|
||||
|
||||
@ -16,6 +16,8 @@ type Config struct {
|
||||
Transfer TransferConfig
|
||||
Cleanup CleanupConfig
|
||||
Paths PathsConfig
|
||||
Logging LoggingConfig
|
||||
Health HealthConfig
|
||||
}
|
||||
|
||||
type CoreConfig struct {
|
||||
@ -67,6 +69,16 @@ type PathsConfig struct {
|
||||
PersistenceFile string
|
||||
}
|
||||
|
||||
type LoggingConfig struct {
|
||||
Dir string
|
||||
Level string
|
||||
}
|
||||
|
||||
type HealthConfig struct {
|
||||
Enabled bool
|
||||
Addr string
|
||||
}
|
||||
|
||||
var defaultConfig = Config{
|
||||
Core: CoreConfig{
|
||||
WorkerCount: 4,
|
||||
@ -110,6 +122,14 @@ var defaultConfig = Config{
|
||||
ManifestDir: "data",
|
||||
PersistenceFile: "transfer_queue.json",
|
||||
},
|
||||
Logging: LoggingConfig{
|
||||
Dir: "data/logs",
|
||||
Level: "info",
|
||||
},
|
||||
Health: HealthConfig{
|
||||
Enabled: true,
|
||||
Addr: ":8090",
|
||||
},
|
||||
}
|
||||
|
||||
func Load() (*Config, error) {
|
||||
@ -167,6 +187,22 @@ func (c *Config) loadFromEnvironment() error {
|
||||
c.Processing.FFmpegPath = val
|
||||
}
|
||||
|
||||
if val := os.Getenv("LOG_DIR"); val != "" {
|
||||
c.Logging.Dir = val
|
||||
}
|
||||
|
||||
if val := os.Getenv("LOG_LEVEL"); val != "" {
|
||||
c.Logging.Level = val
|
||||
}
|
||||
|
||||
if val := os.Getenv("HEALTH_ENABLED"); val != "" {
|
||||
c.Health.Enabled = val == "true"
|
||||
}
|
||||
|
||||
if val := os.Getenv("HEALTH_ADDR"); val != "" {
|
||||
c.Health.Addr = val
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
@ -192,12 +228,16 @@ func (c *Config) resolveAndValidatePaths() error {
|
||||
if !filepath.IsAbs(c.Paths.PersistenceFile) {
|
||||
c.Paths.PersistenceFile = filepath.Join(c.Paths.BaseDir, c.Paths.PersistenceFile)
|
||||
}
|
||||
if !filepath.IsAbs(c.Logging.Dir) {
|
||||
c.Logging.Dir = filepath.Join(cwd, c.Logging.Dir)
|
||||
}
|
||||
|
||||
requiredDirs := []string{
|
||||
c.Paths.BaseDir,
|
||||
c.Paths.LocalOutput,
|
||||
c.Paths.ProcessOutput,
|
||||
c.Paths.ManifestDir,
|
||||
c.Logging.Dir,
|
||||
}
|
||||
|
||||
for _, dir := range requiredDirs {
|
||||
|
||||
323
pkg/health/dashboard.html
Normal file
323
pkg/health/dashboard.html
Normal file
@ -0,0 +1,323 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>m3u8-downloader status</title>
|
||||
<style>
|
||||
:root {
|
||||
--bg: #0f1115;
|
||||
--panel: #161922;
|
||||
--border: #262b38;
|
||||
--text: #e6e8ee;
|
||||
--muted: #8b909e;
|
||||
--green: #3ecf8e;
|
||||
--red: #f2545b;
|
||||
--amber: #f2b705;
|
||||
--blue: #4f8ef7;
|
||||
}
|
||||
* { box-sizing: border-box; }
|
||||
body {
|
||||
margin: 0;
|
||||
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif;
|
||||
background: var(--bg);
|
||||
color: var(--text);
|
||||
}
|
||||
header {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
flex-wrap: wrap;
|
||||
gap: 12px;
|
||||
padding: 16px 24px;
|
||||
border-bottom: 1px solid var(--border);
|
||||
}
|
||||
header h1 {
|
||||
font-size: 18px;
|
||||
margin: 0;
|
||||
font-weight: 600;
|
||||
}
|
||||
.header-meta {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 16px;
|
||||
color: var(--muted);
|
||||
font-size: 13px;
|
||||
}
|
||||
.badge {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 6px;
|
||||
padding: 4px 12px;
|
||||
border-radius: 999px;
|
||||
font-size: 13px;
|
||||
font-weight: 600;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.03em;
|
||||
}
|
||||
.badge-dot { width: 8px; height: 8px; border-radius: 50%; }
|
||||
.badge.healthy { background: rgba(62,207,142,0.15); color: var(--green); }
|
||||
.badge.healthy .badge-dot { background: var(--green); }
|
||||
.badge.degraded { background: rgba(242,84,91,0.15); color: var(--red); }
|
||||
.badge.degraded .badge-dot { background: var(--red); }
|
||||
.badge.unknown { background: rgba(139,144,158,0.15); color: var(--muted); }
|
||||
.badge.unknown .badge-dot { background: var(--muted); }
|
||||
|
||||
#conn-banner {
|
||||
display: none;
|
||||
background: var(--red);
|
||||
color: #fff;
|
||||
text-align: center;
|
||||
padding: 8px;
|
||||
font-size: 13px;
|
||||
font-weight: 600;
|
||||
}
|
||||
#conn-banner.show { display: block; }
|
||||
|
||||
main {
|
||||
padding: 24px;
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fit, minmax(340px, 1fr));
|
||||
gap: 20px;
|
||||
max-width: 1400px;
|
||||
margin: 0 auto;
|
||||
}
|
||||
.card {
|
||||
background: var(--panel);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 10px;
|
||||
padding: 18px 20px;
|
||||
}
|
||||
.card h2 {
|
||||
font-size: 14px;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.05em;
|
||||
color: var(--muted);
|
||||
margin: 0 0 14px 0;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
}
|
||||
table { width: 100%; border-collapse: collapse; font-size: 13px; }
|
||||
th, td { text-align: left; padding: 6px 8px; border-bottom: 1px solid var(--border); }
|
||||
th { color: var(--muted); font-weight: 500; font-size: 12px; }
|
||||
tr.failed-row { background: rgba(242,84,91,0.08); }
|
||||
tr.inactive-row { opacity: 0.5; }
|
||||
.dot { display: inline-block; width: 8px; height: 8px; border-radius: 50%; margin-right: 6px; }
|
||||
.dot.on { background: var(--green); }
|
||||
.dot.off { background: var(--red); }
|
||||
|
||||
.stat-grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(3, 1fr);
|
||||
gap: 10px;
|
||||
margin-bottom: 14px;
|
||||
}
|
||||
.stat {
|
||||
background: rgba(255,255,255,0.03);
|
||||
border-radius: 8px;
|
||||
padding: 10px 12px;
|
||||
}
|
||||
.stat .value { font-size: 20px; font-weight: 600; }
|
||||
.stat .label { font-size: 11px; color: var(--muted); text-transform: uppercase; letter-spacing: 0.03em; }
|
||||
.stat.failed .value { color: var(--red); }
|
||||
|
||||
.state-pill {
|
||||
display: inline-block;
|
||||
padding: 3px 10px;
|
||||
border-radius: 999px;
|
||||
font-size: 12px;
|
||||
font-weight: 600;
|
||||
text-transform: uppercase;
|
||||
}
|
||||
.state-idle { background: rgba(139,144,158,0.15); color: var(--muted); }
|
||||
.state-scanning, .state-running { background: rgba(79,142,247,0.15); color: var(--blue); }
|
||||
.state-completed { background: rgba(62,207,142,0.15); color: var(--green); }
|
||||
.state-failed { background: rgba(242,84,91,0.15); color: var(--red); }
|
||||
|
||||
.kv { display: flex; justify-content: space-between; padding: 4px 0; font-size: 13px; border-bottom: 1px solid var(--border); }
|
||||
.kv:last-child { border-bottom: none; }
|
||||
.kv .k { color: var(--muted); }
|
||||
|
||||
.error-list { margin: 10px 0 0 0; padding: 0; list-style: none; }
|
||||
.error-list li {
|
||||
font-size: 12px;
|
||||
padding: 6px 8px;
|
||||
margin-bottom: 4px;
|
||||
background: rgba(242,84,91,0.08);
|
||||
border-left: 3px solid var(--red);
|
||||
border-radius: 4px;
|
||||
color: var(--text);
|
||||
}
|
||||
.error-list .err-time { color: var(--muted); margin-right: 6px; }
|
||||
.empty-note { color: var(--muted); font-size: 13px; font-style: italic; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
|
||||
<div id="conn-banner">Lost connection to status endpoint - retrying...</div>
|
||||
|
||||
<header>
|
||||
<h1>m3u8-downloader</h1>
|
||||
<div class="header-meta">
|
||||
<span id="overall-badge" class="badge unknown"><span class="badge-dot"></span>unknown</span>
|
||||
<span id="uptime">uptime: -</span>
|
||||
<span id="last-updated">last updated: -</span>
|
||||
</div>
|
||||
</header>
|
||||
|
||||
<main>
|
||||
<section class="card">
|
||||
<h2>Download</h2>
|
||||
<table>
|
||||
<thead>
|
||||
<tr><th>Resolution</th><th>Active</th><th>OK</th><th>Failed</th><th>Last activity</th></tr>
|
||||
</thead>
|
||||
<tbody id="variant-rows"></tbody>
|
||||
</table>
|
||||
<div id="download-empty" class="empty-note" style="display:none;">No variants reporting yet.</div>
|
||||
<ul id="download-errors" class="error-list"></ul>
|
||||
</section>
|
||||
|
||||
<section class="card">
|
||||
<h2>Transfer</h2>
|
||||
<div class="kv"><span class="k">NAS connection</span><span><span id="nas-dot" class="dot off"></span><span id="nas-text">unknown</span></span></div>
|
||||
<div class="stat-grid" style="margin-top:12px;">
|
||||
<div class="stat"><div class="value" id="t-added">-</div><div class="label">Added</div></div>
|
||||
<div class="stat"><div class="value" id="t-completed">-</div><div class="label">Completed</div></div>
|
||||
<div class="stat failed"><div class="value" id="t-failed">-</div><div class="label">Failed</div></div>
|
||||
<div class="stat"><div class="value" id="t-pending">-</div><div class="label">Pending</div></div>
|
||||
<div class="stat"><div class="value" id="t-queue">-</div><div class="label">Queue size</div></div>
|
||||
<div class="stat"><div class="value" id="t-cleanup">-</div><div class="label">Cleanup pending</div></div>
|
||||
</div>
|
||||
<div class="kv"><span class="k">Transferred</span><span id="t-bytes">-</span></div>
|
||||
<ul id="transfer-errors" class="error-list"></ul>
|
||||
</section>
|
||||
|
||||
<section class="card">
|
||||
<h2>Processing</h2>
|
||||
<div class="kv"><span class="k">State</span><span id="p-state" class="state-pill state-idle">idle</span></div>
|
||||
<div class="kv"><span class="k">Event</span><span id="p-event">-</span></div>
|
||||
<div class="kv"><span class="k">Segments found</span><span id="p-segments">-</span></div>
|
||||
<div class="kv"><span class="k">Started</span><span id="p-started">-</span></div>
|
||||
<div class="kv"><span class="k">Finished</span><span id="p-finished">-</span></div>
|
||||
<ul id="processing-errors" class="error-list"></ul>
|
||||
</section>
|
||||
</main>
|
||||
|
||||
<script>
|
||||
const POLL_INTERVAL_MS = 3000;
|
||||
const connBanner = document.getElementById('conn-banner');
|
||||
|
||||
function fmtBytes(n) {
|
||||
if (!n || n <= 0) return '0 B';
|
||||
const units = ['B', 'KB', 'MB', 'GB', 'TB'];
|
||||
let i = 0;
|
||||
while (n >= 1024 && i < units.length - 1) { n /= 1024; i++; }
|
||||
return n.toFixed(i === 0 ? 0 : 1) + ' ' + units[i];
|
||||
}
|
||||
|
||||
function fmtDuration(sec) {
|
||||
sec = Math.floor(sec || 0);
|
||||
const h = Math.floor(sec / 3600);
|
||||
const m = Math.floor((sec % 3600) / 60);
|
||||
const s = sec % 60;
|
||||
const parts = [];
|
||||
if (h) parts.push(h + 'h');
|
||||
if (m || h) parts.push(m + 'm');
|
||||
parts.push(s + 's');
|
||||
return parts.join(' ');
|
||||
}
|
||||
|
||||
function fmtTime(iso) {
|
||||
if (!iso) return '-';
|
||||
const d = new Date(iso);
|
||||
if (isNaN(d.getTime()) || d.getFullYear() < 1971) return '-';
|
||||
return d.toLocaleTimeString();
|
||||
}
|
||||
|
||||
function renderErrors(listEl, errors) {
|
||||
listEl.innerHTML = '';
|
||||
(errors || []).slice().reverse().forEach(e => {
|
||||
const li = document.createElement('li');
|
||||
const t = document.createElement('span');
|
||||
t.className = 'err-time';
|
||||
t.textContent = fmtTime(e.time);
|
||||
li.appendChild(t);
|
||||
li.appendChild(document.createTextNode(e.message));
|
||||
listEl.appendChild(li);
|
||||
});
|
||||
}
|
||||
|
||||
function render(data) {
|
||||
const badge = document.getElementById('overall-badge');
|
||||
badge.className = 'badge ' + (data.status === 'healthy' ? 'healthy' : 'degraded');
|
||||
badge.innerHTML = '<span class="badge-dot"></span>' + data.status;
|
||||
|
||||
document.getElementById('uptime').textContent = 'uptime: ' + fmtDuration(data.uptime_seconds);
|
||||
document.getElementById('last-updated').textContent = 'last updated: ' + new Date().toLocaleTimeString();
|
||||
|
||||
// Download
|
||||
const variants = (data.download && data.download.variants) || [];
|
||||
const rows = document.getElementById('variant-rows');
|
||||
rows.innerHTML = '';
|
||||
document.getElementById('download-empty').style.display = variants.length ? 'none' : 'block';
|
||||
variants.forEach(v => {
|
||||
const tr = document.createElement('tr');
|
||||
if (v.segments_failed > 0) tr.classList.add('failed-row');
|
||||
if (!v.active) tr.classList.add('inactive-row');
|
||||
tr.innerHTML =
|
||||
'<td>' + v.resolution + '</td>' +
|
||||
'<td><span class="dot ' + (v.active ? 'on' : 'off') + '"></span>' + (v.active ? 'active' : 'stopped') + '</td>' +
|
||||
'<td>' + v.segments_ok + '</td>' +
|
||||
'<td>' + v.segments_failed + '</td>' +
|
||||
'<td>' + fmtTime(v.last_activity) + '</td>';
|
||||
rows.appendChild(tr);
|
||||
});
|
||||
renderErrors(document.getElementById('download-errors'), data.download && data.download.recent_errors);
|
||||
|
||||
// Transfer
|
||||
const t = data.transfer || {};
|
||||
const nasDot = document.getElementById('nas-dot');
|
||||
nasDot.className = 'dot ' + (t.nas_connected ? 'on' : 'off');
|
||||
document.getElementById('nas-text').textContent = t.nas_connected ? 'connected' : 'disconnected';
|
||||
document.getElementById('t-added').textContent = t.added ?? '-';
|
||||
document.getElementById('t-completed').textContent = t.completed ?? '-';
|
||||
document.getElementById('t-failed').textContent = t.failed ?? '-';
|
||||
document.getElementById('t-pending').textContent = t.pending ?? '-';
|
||||
document.getElementById('t-queue').textContent = t.queue_size ?? '-';
|
||||
document.getElementById('t-cleanup').textContent = t.cleanup_pending ?? '-';
|
||||
document.getElementById('t-bytes').textContent = fmtBytes(t.bytes_transferred);
|
||||
renderErrors(document.getElementById('transfer-errors'), t.recent_errors);
|
||||
|
||||
// Processing
|
||||
const p = data.processing || {};
|
||||
const stateEl = document.getElementById('p-state');
|
||||
const state = p.state || 'idle';
|
||||
stateEl.textContent = state;
|
||||
stateEl.className = 'state-pill state-' + state;
|
||||
document.getElementById('p-event').textContent = p.event_name || '-';
|
||||
document.getElementById('p-segments').textContent = p.segments_found ?? '-';
|
||||
document.getElementById('p-started').textContent = fmtTime(p.started_at);
|
||||
document.getElementById('p-finished').textContent = fmtTime(p.finished_at);
|
||||
renderErrors(document.getElementById('processing-errors'), p.recent_errors);
|
||||
}
|
||||
|
||||
async function poll() {
|
||||
try {
|
||||
const resp = await fetch('/status', { cache: 'no-store' });
|
||||
if (!resp.ok) throw new Error('status ' + resp.status);
|
||||
const data = await resp.json();
|
||||
connBanner.classList.remove('show');
|
||||
render(data);
|
||||
} catch (e) {
|
||||
connBanner.classList.add('show');
|
||||
}
|
||||
}
|
||||
|
||||
poll();
|
||||
setInterval(poll, POLL_INTERVAL_MS);
|
||||
</script>
|
||||
|
||||
</body>
|
||||
</html>
|
||||
352
pkg/health/registry.go
Normal file
352
pkg/health/registry.go
Normal file
@ -0,0 +1,352 @@
|
||||
package health
|
||||
|
||||
import (
|
||||
"sort"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
const maxRecentErrors = 10
|
||||
|
||||
type ErrorEntry struct {
|
||||
Time time.Time `json:"time"`
|
||||
Message string `json:"message"`
|
||||
}
|
||||
|
||||
type VariantStatus struct {
|
||||
Resolution string `json:"resolution"`
|
||||
Bandwidth uint32 `json:"bandwidth"`
|
||||
Active bool `json:"active"`
|
||||
SegmentsOK int `json:"segments_ok"`
|
||||
SegmentsFailed int `json:"segments_failed"`
|
||||
LastActivity time.Time `json:"last_activity"`
|
||||
LastError string `json:"last_error,omitempty"`
|
||||
}
|
||||
|
||||
// StatsProvider is satisfied by *transfer.QueueStats without pkg/health
|
||||
// needing to import pkg/transfer (which would create an import cycle, since
|
||||
// pkg/transfer calls into pkg/health to report status).
|
||||
type StatsProvider interface {
|
||||
GetStats() (added, completed, failed, pending int, bytes int64)
|
||||
}
|
||||
|
||||
type downloadState struct {
|
||||
variants map[string]*VariantStatus
|
||||
recentErrors []ErrorEntry
|
||||
}
|
||||
|
||||
type transferState struct {
|
||||
statsProvider StatsProvider
|
||||
queueSizeFn func() int
|
||||
cleanupPendingFn func() int
|
||||
nasConnectedFn func() bool
|
||||
nasConnected bool
|
||||
recentErrors []ErrorEntry
|
||||
}
|
||||
|
||||
type processingState struct {
|
||||
state string // idle | scanning | running | completed | failed
|
||||
eventName string
|
||||
segmentsFound int
|
||||
startedAt time.Time
|
||||
finishedAt time.Time
|
||||
lastError string
|
||||
recentErrors []ErrorEntry
|
||||
}
|
||||
|
||||
// Registry is the process-wide health/metrics store. All subsystems report
|
||||
// into it via the Record* methods; the health HTTP server reads it via
|
||||
// Snapshot(). Safe for concurrent use.
|
||||
type Registry struct {
|
||||
mu sync.RWMutex
|
||||
startedAt time.Time
|
||||
download downloadState
|
||||
transfer transferState
|
||||
processing processingState
|
||||
}
|
||||
|
||||
var (
|
||||
globalRegistry *Registry
|
||||
registryOnce sync.Once
|
||||
)
|
||||
|
||||
// MustGetRegistry returns the process-wide registry, mirroring the
|
||||
// constants.MustGetConfig() / logging.MustGetFactory() singleton style.
|
||||
func MustGetRegistry() *Registry {
|
||||
registryOnce.Do(func() {
|
||||
globalRegistry = New()
|
||||
})
|
||||
return globalRegistry
|
||||
}
|
||||
|
||||
func New() *Registry {
|
||||
return &Registry{
|
||||
startedAt: time.Now(),
|
||||
download: downloadState{
|
||||
variants: make(map[string]*VariantStatus),
|
||||
},
|
||||
processing: processingState{
|
||||
state: "idle",
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func appendBounded(errs []ErrorEntry, e ErrorEntry) []ErrorEntry {
|
||||
errs = append(errs, e)
|
||||
if len(errs) > maxRecentErrors {
|
||||
errs = errs[len(errs)-maxRecentErrors:]
|
||||
}
|
||||
return errs
|
||||
}
|
||||
|
||||
func errString(err error) string {
|
||||
if err == nil {
|
||||
return ""
|
||||
}
|
||||
return err.Error()
|
||||
}
|
||||
|
||||
// --- Download ---
|
||||
|
||||
func (r *Registry) RecordVariantStarted(resolution string, bandwidth uint32) {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
r.download.variants[resolution] = &VariantStatus{
|
||||
Resolution: resolution,
|
||||
Bandwidth: bandwidth,
|
||||
Active: true,
|
||||
LastActivity: time.Now(),
|
||||
}
|
||||
}
|
||||
|
||||
func (r *Registry) RecordVariantStopped(resolution string) {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
if v, ok := r.download.variants[resolution]; ok {
|
||||
v.Active = false
|
||||
}
|
||||
}
|
||||
|
||||
func (r *Registry) RecordSegmentDownloaded(resolution string) {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
v := r.ensureVariantLocked(resolution)
|
||||
v.SegmentsOK++
|
||||
v.LastActivity = time.Now()
|
||||
}
|
||||
|
||||
func (r *Registry) RecordSegmentFailed(resolution string, err error) {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
v := r.ensureVariantLocked(resolution)
|
||||
v.SegmentsFailed++
|
||||
v.LastActivity = time.Now()
|
||||
v.LastError = errString(err)
|
||||
r.download.recentErrors = appendBounded(r.download.recentErrors, ErrorEntry{
|
||||
Time: time.Now(),
|
||||
Message: resolution + ": " + errString(err),
|
||||
})
|
||||
}
|
||||
|
||||
func (r *Registry) RecordPlaylistError(resolution string, err error) {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
v := r.ensureVariantLocked(resolution)
|
||||
v.LastError = errString(err)
|
||||
r.download.recentErrors = appendBounded(r.download.recentErrors, ErrorEntry{
|
||||
Time: time.Now(),
|
||||
Message: resolution + " playlist: " + errString(err),
|
||||
})
|
||||
}
|
||||
|
||||
func (r *Registry) ensureVariantLocked(resolution string) *VariantStatus {
|
||||
v, ok := r.download.variants[resolution]
|
||||
if !ok {
|
||||
v = &VariantStatus{Resolution: resolution, Active: true}
|
||||
r.download.variants[resolution] = v
|
||||
}
|
||||
return v
|
||||
}
|
||||
|
||||
// --- Transfer ---
|
||||
|
||||
// AttachTransfer wires the registry to the live counters/state already
|
||||
// maintained by pkg/transfer instead of duplicating them. Called once from
|
||||
// TransferService construction.
|
||||
func (r *Registry) AttachTransfer(stats StatsProvider, queueSizeFn, cleanupPendingFn func() int, nasConnectedFn func() bool) {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
r.transfer.statsProvider = stats
|
||||
r.transfer.queueSizeFn = queueSizeFn
|
||||
r.transfer.cleanupPendingFn = cleanupPendingFn
|
||||
r.transfer.nasConnectedFn = nasConnectedFn
|
||||
}
|
||||
|
||||
func (r *Registry) RecordNASConnectivity(connected bool) {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
r.transfer.nasConnected = connected
|
||||
}
|
||||
|
||||
// RecordTransferError captures failures QueueStats' cumulative counters
|
||||
// don't individually retain (pre-queue failures, cleanup failures) so they
|
||||
// still surface in the bounded recent-errors list.
|
||||
func (r *Registry) RecordTransferError(sourcePath string, err error) {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
r.transfer.recentErrors = appendBounded(r.transfer.recentErrors, ErrorEntry{
|
||||
Time: time.Now(),
|
||||
Message: sourcePath + ": " + errString(err),
|
||||
})
|
||||
}
|
||||
|
||||
// --- Processing ---
|
||||
|
||||
func (r *Registry) RecordProcessingStageChange(state, eventName string) {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
r.processing.state = state
|
||||
if eventName != "" {
|
||||
r.processing.eventName = eventName
|
||||
}
|
||||
switch state {
|
||||
case "scanning":
|
||||
r.processing.startedAt = time.Now()
|
||||
r.processing.finishedAt = time.Time{}
|
||||
r.processing.lastError = ""
|
||||
case "completed", "failed":
|
||||
r.processing.finishedAt = time.Now()
|
||||
}
|
||||
}
|
||||
|
||||
func (r *Registry) RecordProcessingSegmentsFound(count int) {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
r.processing.segmentsFound = count
|
||||
}
|
||||
|
||||
func (r *Registry) RecordProcessingError(err error) {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
msg := errString(err)
|
||||
r.processing.lastError = msg
|
||||
r.processing.recentErrors = appendBounded(r.processing.recentErrors, ErrorEntry{
|
||||
Time: time.Now(),
|
||||
Message: msg,
|
||||
})
|
||||
}
|
||||
|
||||
// --- Snapshot ---
|
||||
|
||||
type DownloadSnapshot struct {
|
||||
Variants []*VariantStatus `json:"variants"`
|
||||
RecentErrors []ErrorEntry `json:"recent_errors"`
|
||||
}
|
||||
|
||||
type TransferSnapshot struct {
|
||||
NASConnected bool `json:"nas_connected"`
|
||||
QueueSize int `json:"queue_size"`
|
||||
CleanupPending int `json:"cleanup_pending"`
|
||||
Added int `json:"added"`
|
||||
Completed int `json:"completed"`
|
||||
Failed int `json:"failed"`
|
||||
Pending int `json:"pending"`
|
||||
BytesTransferred int64 `json:"bytes_transferred"`
|
||||
RecentErrors []ErrorEntry `json:"recent_errors"`
|
||||
}
|
||||
|
||||
type ProcessingSnapshot struct {
|
||||
State string `json:"state"`
|
||||
EventName string `json:"event_name,omitempty"`
|
||||
SegmentsFound int `json:"segments_found"`
|
||||
StartedAt *time.Time `json:"started_at,omitempty"`
|
||||
FinishedAt *time.Time `json:"finished_at,omitempty"`
|
||||
LastError string `json:"last_error,omitempty"`
|
||||
RecentErrors []ErrorEntry `json:"recent_errors"`
|
||||
}
|
||||
|
||||
type Snapshot struct {
|
||||
Status string `json:"status"`
|
||||
UptimeSec float64 `json:"uptime_seconds"`
|
||||
Download DownloadSnapshot `json:"download"`
|
||||
Transfer TransferSnapshot `json:"transfer"`
|
||||
Processing ProcessingSnapshot `json:"processing"`
|
||||
}
|
||||
|
||||
func (r *Registry) Snapshot() Snapshot {
|
||||
r.mu.RLock()
|
||||
defer r.mu.RUnlock()
|
||||
|
||||
variants := make([]*VariantStatus, 0, len(r.download.variants))
|
||||
for _, v := range r.download.variants {
|
||||
vCopy := *v
|
||||
variants = append(variants, &vCopy)
|
||||
}
|
||||
sort.Slice(variants, func(i, j int) bool { return variants[i].Resolution < variants[j].Resolution })
|
||||
|
||||
dl := DownloadSnapshot{
|
||||
Variants: variants,
|
||||
RecentErrors: append([]ErrorEntry(nil), r.download.recentErrors...),
|
||||
}
|
||||
|
||||
tr := TransferSnapshot{
|
||||
NASConnected: r.transfer.nasConnected,
|
||||
RecentErrors: append([]ErrorEntry(nil), r.transfer.recentErrors...),
|
||||
}
|
||||
if r.transfer.statsProvider != nil {
|
||||
added, completed, failed, pending, bytes := r.transfer.statsProvider.GetStats()
|
||||
tr.Added, tr.Completed, tr.Failed, tr.Pending, tr.BytesTransferred = added, completed, failed, pending, bytes
|
||||
}
|
||||
if r.transfer.queueSizeFn != nil {
|
||||
tr.QueueSize = r.transfer.queueSizeFn()
|
||||
}
|
||||
if r.transfer.cleanupPendingFn != nil {
|
||||
tr.CleanupPending = r.transfer.cleanupPendingFn()
|
||||
}
|
||||
if r.transfer.nasConnectedFn != nil {
|
||||
tr.NASConnected = r.transfer.nasConnectedFn()
|
||||
}
|
||||
|
||||
proc := ProcessingSnapshot{
|
||||
State: r.processing.state,
|
||||
EventName: r.processing.eventName,
|
||||
SegmentsFound: r.processing.segmentsFound,
|
||||
LastError: r.processing.lastError,
|
||||
RecentErrors: append([]ErrorEntry(nil), r.processing.recentErrors...),
|
||||
}
|
||||
if !r.processing.startedAt.IsZero() {
|
||||
t := r.processing.startedAt
|
||||
proc.StartedAt = &t
|
||||
}
|
||||
if !r.processing.finishedAt.IsZero() {
|
||||
t := r.processing.finishedAt
|
||||
proc.FinishedAt = &t
|
||||
}
|
||||
|
||||
snap := Snapshot{
|
||||
UptimeSec: time.Since(r.startedAt).Seconds(),
|
||||
Download: dl,
|
||||
Transfer: tr,
|
||||
Processing: proc,
|
||||
}
|
||||
snap.Status = computeStatus(dl, tr, proc)
|
||||
return snap
|
||||
}
|
||||
|
||||
// computeStatus gives one glanceable field: "degraded" the moment anything
|
||||
// worth investigating shows up, "healthy" otherwise. Transfer is only
|
||||
// considered for degradation once it's actually in use (Attached + has seen
|
||||
// activity), so a download-only run with NAS transfer disabled doesn't
|
||||
// falsely report degraded.
|
||||
func computeStatus(dl DownloadSnapshot, tr TransferSnapshot, proc ProcessingSnapshot) string {
|
||||
if len(dl.RecentErrors) > 0 || len(tr.RecentErrors) > 0 || len(proc.RecentErrors) > 0 {
|
||||
return "degraded"
|
||||
}
|
||||
if !tr.NASConnected && (tr.Added > 0 || tr.QueueSize > 0) {
|
||||
return "degraded"
|
||||
}
|
||||
if proc.State == "failed" {
|
||||
return "degraded"
|
||||
}
|
||||
return "healthy"
|
||||
}
|
||||
60
pkg/health/server.go
Normal file
60
pkg/health/server.go
Normal file
@ -0,0 +1,60 @@
|
||||
package health
|
||||
|
||||
import (
|
||||
"context"
|
||||
_ "embed"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"m3u8-downloader/pkg/config"
|
||||
"m3u8-downloader/pkg/logging"
|
||||
"net"
|
||||
"net/http"
|
||||
)
|
||||
|
||||
//go:embed dashboard.html
|
||||
var dashboardHTML []byte
|
||||
|
||||
// Start binds the health/status HTTP server and returns immediately once the
|
||||
// listener is bound, so callers see bind failures (e.g. port already in use)
|
||||
// synchronously rather than losing them in a background goroutine. The
|
||||
// returned *http.Server should be shut down by the caller via Shutdown(ctx).
|
||||
//
|
||||
// Security note: cfg.Addr is expected to be a bind-all address (e.g. ":8090")
|
||||
// so status can be checked from other devices on the local network during a
|
||||
// live event. There is no authentication or TLS on these endpoints - this is
|
||||
// only safe on a trusted local network, never exposed to the open internet.
|
||||
func Start(ctx context.Context, cfg *config.HealthConfig, reg *Registry) (*http.Server, error) {
|
||||
mux := http.NewServeMux()
|
||||
|
||||
mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||
w.Write(dashboardHTML)
|
||||
})
|
||||
|
||||
mux.HandleFunc("/status", func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
if err := json.NewEncoder(w).Encode(reg.Snapshot()); err != nil {
|
||||
http.Error(w, err.Error(), http.StatusInternalServerError)
|
||||
}
|
||||
})
|
||||
|
||||
mux.HandleFunc("/healthz", func(w http.ResponseWriter, r *http.Request) {
|
||||
w.WriteHeader(http.StatusOK)
|
||||
_, _ = w.Write([]byte("ok"))
|
||||
})
|
||||
|
||||
ln, err := net.Listen("tcp", cfg.Addr)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to bind health server to %s: %w", cfg.Addr, err)
|
||||
}
|
||||
|
||||
srv := &http.Server{Handler: mux}
|
||||
|
||||
go func() {
|
||||
if err := srv.Serve(ln); err != nil && err != http.ErrServerClosed {
|
||||
logging.MustGetFactory().App.Error("health server stopped unexpectedly", logging.KeyError, err.Error())
|
||||
}
|
||||
}()
|
||||
|
||||
return srv, nil
|
||||
}
|
||||
72
pkg/logging/factory.go
Normal file
72
pkg/logging/factory.go
Normal file
@ -0,0 +1,72 @@
|
||||
package logging
|
||||
|
||||
import (
|
||||
"log/slog"
|
||||
"m3u8-downloader/pkg/config"
|
||||
"m3u8-downloader/pkg/constants"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
)
|
||||
|
||||
// Factory holds one structured logger per pipeline stage, each writing JSON
|
||||
// lines to its own file under LoggingConfig.Dir. These are additive to the
|
||||
// existing stdlib log.* calls, not a replacement for them.
|
||||
type Factory struct {
|
||||
App *slog.Logger
|
||||
Download *slog.Logger
|
||||
Transfer *slog.Logger
|
||||
Processing *slog.Logger
|
||||
}
|
||||
|
||||
var (
|
||||
globalFactory *Factory
|
||||
factoryOnce sync.Once
|
||||
)
|
||||
|
||||
// MustGetFactory returns the process-wide logging factory, building it from
|
||||
// the config singleton on first use. Mirrors constants.MustGetConfig().
|
||||
func MustGetFactory() *Factory {
|
||||
factoryOnce.Do(func() {
|
||||
globalFactory = New(&constants.MustGetConfig().Logging)
|
||||
})
|
||||
return globalFactory
|
||||
}
|
||||
|
||||
func New(cfg *config.LoggingConfig) *Factory {
|
||||
level := parseLevel(cfg.Level)
|
||||
return &Factory{
|
||||
App: newSubsystemLogger(cfg.Dir, "app.log", "app", level),
|
||||
Download: newSubsystemLogger(cfg.Dir, "download.log", "download", level),
|
||||
Transfer: newSubsystemLogger(cfg.Dir, "transfer.log", "transfer", level),
|
||||
Processing: newSubsystemLogger(cfg.Dir, "processing.log", "processing", level),
|
||||
}
|
||||
}
|
||||
|
||||
// newSubsystemLogger opens (or creates) the subsystem's log file in append
|
||||
// mode. A logging misconfiguration must never take down an active recording,
|
||||
// so failure to open the file falls back to stderr instead of panicking.
|
||||
func newSubsystemLogger(dir, filename, component string, level slog.Level) *slog.Logger {
|
||||
path := filepath.Join(dir, filename)
|
||||
f, err := os.OpenFile(path, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0666)
|
||||
if err != nil {
|
||||
fallback := slog.New(slog.NewJSONHandler(os.Stderr, &slog.HandlerOptions{Level: level})).With("component", component)
|
||||
fallback.Error("failed to open log file, falling back to stderr", KeyFile, path, KeyError, err.Error())
|
||||
return fallback
|
||||
}
|
||||
return slog.New(slog.NewJSONHandler(f, &slog.HandlerOptions{Level: level})).With("component", component)
|
||||
}
|
||||
|
||||
func parseLevel(level string) slog.Level {
|
||||
switch strings.ToLower(strings.TrimSpace(level)) {
|
||||
case "debug":
|
||||
return slog.LevelDebug
|
||||
case "warn", "warning":
|
||||
return slog.LevelWarn
|
||||
case "error":
|
||||
return slog.LevelError
|
||||
default:
|
||||
return slog.LevelInfo
|
||||
}
|
||||
}
|
||||
13
pkg/logging/keys.go
Normal file
13
pkg/logging/keys.go
Normal file
@ -0,0 +1,13 @@
|
||||
package logging
|
||||
|
||||
const (
|
||||
KeyResolution = "resolution"
|
||||
KeyBandwidth = "bandwidth"
|
||||
KeySegment = "segment"
|
||||
KeySeq = "seq"
|
||||
KeyFile = "file"
|
||||
KeyBytes = "bytes"
|
||||
KeyEvent = "event"
|
||||
KeyError = "error"
|
||||
KeyAttempt = "attempt"
|
||||
)
|
||||
@ -4,6 +4,7 @@ import (
|
||||
"encoding/json"
|
||||
"log"
|
||||
"m3u8-downloader/pkg/constants"
|
||||
"m3u8-downloader/pkg/logging"
|
||||
"m3u8-downloader/pkg/utils"
|
||||
"os"
|
||||
"sort"
|
||||
@ -61,17 +62,20 @@ func (m *ManifestWriter) WriteManifest() {
|
||||
data, err := json.MarshalIndent(m.Segments, "", " ")
|
||||
if err != nil {
|
||||
log.Printf("Failed to marshal manifest: %v", err)
|
||||
logging.MustGetFactory().Download.Error("failed to marshal manifest", logging.KeyError, err.Error())
|
||||
return
|
||||
}
|
||||
|
||||
if err := utils.ValidateWritablePath(m.ManifestPath); err != nil {
|
||||
log.Printf("Manifest path validation failed: %v", err)
|
||||
logging.MustGetFactory().Download.Error("manifest path validation failed", logging.KeyError, err.Error())
|
||||
return
|
||||
}
|
||||
|
||||
file, err := os.Create(m.ManifestPath)
|
||||
if err != nil {
|
||||
log.Printf("Failed to create manifest file: %v", err)
|
||||
logging.MustGetFactory().Download.Error("failed to create manifest file", logging.KeyError, err.Error())
|
||||
return
|
||||
}
|
||||
|
||||
@ -80,6 +84,7 @@ func (m *ManifestWriter) WriteManifest() {
|
||||
_, err = file.Write(data)
|
||||
if err != nil {
|
||||
log.Printf("Failed to write manifest file: %v", err)
|
||||
logging.MustGetFactory().Download.Error("failed to write manifest file", logging.KeyError, err.Error())
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
@ -6,6 +6,7 @@ import (
|
||||
"io"
|
||||
"m3u8-downloader/pkg/constants"
|
||||
"m3u8-downloader/pkg/httpClient"
|
||||
"m3u8-downloader/pkg/logging"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
@ -33,6 +34,7 @@ func (j SegmentJob) Key() string {
|
||||
func DownloadSegment(ctx context.Context, client *http.Client, segmentURL string, outputDir string) error {
|
||||
for attempt := 0; attempt < 2; attempt++ {
|
||||
if attempt > 0 {
|
||||
logging.MustGetFactory().Download.Debug("retrying segment download", logging.KeyAttempt, attempt, "url", segmentURL)
|
||||
time.Sleep(300 * time.Millisecond)
|
||||
}
|
||||
req, err := http.NewRequestWithContext(ctx, "GET", segmentURL, nil)
|
||||
|
||||
@ -7,7 +7,9 @@ import (
|
||||
"github.com/grafov/m3u8"
|
||||
"log"
|
||||
"m3u8-downloader/pkg/constants"
|
||||
"m3u8-downloader/pkg/health"
|
||||
"m3u8-downloader/pkg/httpClient"
|
||||
"m3u8-downloader/pkg/logging"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"path"
|
||||
@ -53,12 +55,14 @@ func GetAllVariants(masterURL string, outputDir string, writer *ManifestWriter)
|
||||
req.Header.Set("Referer", constants.REFERRER)
|
||||
resp, err := client.Do(req)
|
||||
if err != nil {
|
||||
logging.MustGetFactory().Download.Error("failed to fetch master playlist", logging.KeyError, err.Error())
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
playlist, listType, err := m3u8.DecodeFrom(resp.Body, true)
|
||||
if err != nil {
|
||||
logging.MustGetFactory().Download.Error("failed to decode master playlist", logging.KeyError, err.Error())
|
||||
return nil, err
|
||||
}
|
||||
|
||||
@ -96,11 +100,17 @@ func GetAllVariants(masterURL string, outputDir string, writer *ManifestWriter)
|
||||
OutputDir: outputDir,
|
||||
})
|
||||
}
|
||||
logging.MustGetFactory().Download.Info("discovered variants", "count", len(variants))
|
||||
return variants, nil
|
||||
}
|
||||
|
||||
func VariantDownloader(ctx context.Context, variant *StreamVariant, sem chan struct{}, manifest *ManifestWriter) {
|
||||
log.Printf("Starting %s variant downloader (bandwidth: %d)", variant.Resolution, variant.Bandwidth)
|
||||
dlLog := logging.MustGetFactory().Download.With(logging.KeyResolution, variant.Resolution)
|
||||
registry := health.MustGetRegistry()
|
||||
dlLog.Info("variant downloader started", logging.KeyBandwidth, variant.Bandwidth)
|
||||
registry.RecordVariantStarted(variant.Resolution, variant.Bandwidth)
|
||||
|
||||
ticker := time.NewTicker(constants.RefreshDelay)
|
||||
defer ticker.Stop()
|
||||
client := &http.Client{}
|
||||
@ -109,6 +119,7 @@ func VariantDownloader(ctx context.Context, variant *StreamVariant, sem chan str
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
registry.RecordVariantStopped(variant.Resolution)
|
||||
return
|
||||
default:
|
||||
}
|
||||
@ -117,6 +128,8 @@ func VariantDownloader(ctx context.Context, variant *StreamVariant, sem chan str
|
||||
playlist, err := LoadMediaPlaylist(variant.URL)
|
||||
if err != nil {
|
||||
log.Printf("%s: Error loading playlist playlist: %v", variant.Resolution, err)
|
||||
dlLog.Warn("failed to load media playlist", logging.KeyError, err.Error())
|
||||
registry.RecordPlaylistError(variant.Resolution, err)
|
||||
goto waitTick
|
||||
}
|
||||
seq = playlist.SeqNo
|
||||
@ -149,6 +162,8 @@ func VariantDownloader(ctx context.Context, variant *StreamVariant, sem chan str
|
||||
|
||||
if err == nil {
|
||||
log.Printf("✓ %s downloaded segment %s", j.Variant.Resolution, name)
|
||||
dlLog.Debug("segment downloaded", logging.KeySegment, name)
|
||||
registry.RecordSegmentDownloaded(variant.Resolution)
|
||||
return
|
||||
}
|
||||
|
||||
@ -159,21 +174,27 @@ func VariantDownloader(ctx context.Context, variant *StreamVariant, sem chan str
|
||||
|
||||
if httpClient.IsHTTPStatus(err, 403) {
|
||||
log.Printf("✗ %s failed to download segment %s (403)", j.Variant.Resolution, name)
|
||||
dlLog.Warn("segment download failed (403)", logging.KeySegment, name)
|
||||
} else {
|
||||
log.Printf("✗ %s failed to download segment %s: %v", j.Variant.Resolution, name, err)
|
||||
dlLog.Error("segment download failed", logging.KeySegment, name, logging.KeyError, err.Error())
|
||||
}
|
||||
registry.RecordSegmentFailed(variant.Resolution, err)
|
||||
}(job)
|
||||
seq++
|
||||
}
|
||||
|
||||
if playlist.Closed {
|
||||
log.Printf("%s: Playlist closed (#EXT-X-ENDLIST)", variant.Resolution)
|
||||
dlLog.Info("playlist closed")
|
||||
registry.RecordVariantStopped(variant.Resolution)
|
||||
return
|
||||
}
|
||||
|
||||
waitTick:
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
registry.RecordVariantStopped(variant.Resolution)
|
||||
return
|
||||
case <-ticker.C:
|
||||
}
|
||||
|
||||
@ -6,6 +6,8 @@ import (
|
||||
"fmt"
|
||||
"log"
|
||||
"m3u8-downloader/pkg/config"
|
||||
"m3u8-downloader/pkg/health"
|
||||
"m3u8-downloader/pkg/logging"
|
||||
"m3u8-downloader/pkg/nas"
|
||||
"m3u8-downloader/pkg/utils"
|
||||
"os"
|
||||
@ -42,6 +44,7 @@ func NewProcessingService(eventName string, cfg *config.Config) (*ProcessingServ
|
||||
nasService := nas.NewNASService(nasConfig)
|
||||
|
||||
if err := nasService.TestConnection(); err != nil {
|
||||
logging.MustGetFactory().Processing.Error("failed to connect to NAS", logging.KeyError, err.Error())
|
||||
return nil, fmt.Errorf("failed to connect to NAS: %w", err)
|
||||
}
|
||||
|
||||
@ -76,18 +79,32 @@ func (ps *ProcessingService) GetEventDirs() ([]string, error) {
|
||||
}
|
||||
|
||||
func (ps *ProcessingService) Start(ctx context.Context) error {
|
||||
procLog := logging.MustGetFactory().Processing
|
||||
registry := health.MustGetRegistry()
|
||||
|
||||
if !ps.config.Processing.Enabled {
|
||||
log.Println("Processing service disabled")
|
||||
return nil
|
||||
}
|
||||
|
||||
registry.RecordProcessingStageChange("scanning", ps.eventName)
|
||||
|
||||
// fail centralizes the log/registry bookkeeping for every error return
|
||||
// site below instead of repeating it at each one.
|
||||
fail := func(err error) error {
|
||||
procLog.Error("processing failed", logging.KeyEvent, ps.eventName, logging.KeyError, err.Error())
|
||||
registry.RecordProcessingStageChange("failed", ps.eventName)
|
||||
registry.RecordProcessingError(err)
|
||||
return err
|
||||
}
|
||||
|
||||
if ps.eventName == "" {
|
||||
events, err := ps.GetEventDirs()
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to get event directories: %w", err)
|
||||
return fail(fmt.Errorf("failed to get event directories: %w", err))
|
||||
}
|
||||
if len(events) == 0 {
|
||||
return fmt.Errorf("no events found")
|
||||
return fail(fmt.Errorf("no events found"))
|
||||
}
|
||||
if len(events) > 1 {
|
||||
fmt.Println("Multiple events found, please select one:")
|
||||
@ -99,21 +116,23 @@ func (ps *ProcessingService) Start(ctx context.Context) error {
|
||||
input = strings.TrimSpace(input)
|
||||
index, err := strconv.Atoi(input)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to parse input: %w", err)
|
||||
return fail(fmt.Errorf("failed to parse input: %w", err))
|
||||
}
|
||||
if index < 1 || index > len(events) {
|
||||
return fmt.Errorf("invalid input")
|
||||
return fail(fmt.Errorf("invalid input"))
|
||||
}
|
||||
ps.eventName = events[index-1]
|
||||
} else {
|
||||
ps.eventName = events[0]
|
||||
}
|
||||
registry.RecordProcessingStageChange("scanning", ps.eventName)
|
||||
}
|
||||
procLog.Info("processing started", logging.KeyEvent, ps.eventName)
|
||||
|
||||
//Get all present resolutions
|
||||
dirs, err := ps.GetResolutions()
|
||||
if err != nil {
|
||||
return fmt.Errorf("Failed to get resolutions: %w", err)
|
||||
return fail(fmt.Errorf("Failed to get resolutions: %w", err))
|
||||
}
|
||||
|
||||
//Spawn a worker per resolution
|
||||
@ -131,25 +150,29 @@ func (ps *ProcessingService) Start(ctx context.Context) error {
|
||||
|
||||
segments, err := ps.AggregateSegmentInfo(ch)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Failed to aggregate segment info: %w", err)
|
||||
return fail(fmt.Errorf("Failed to aggregate segment info: %w", err))
|
||||
}
|
||||
registry.RecordProcessingSegmentsFound(len(segments))
|
||||
|
||||
aggFile, err := ps.WriteConcatFile(segments)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Failed to write concat file: %w", err)
|
||||
return fail(fmt.Errorf("Failed to write concat file: %w", err))
|
||||
}
|
||||
|
||||
// Feed info to ffmpeg to stitch files together
|
||||
outPath := ps.config.GetProcessOutputPath(ps.eventName)
|
||||
if err := utils.EnsureDir(outPath); err != nil {
|
||||
return fmt.Errorf("failed to create output directory: %w", err)
|
||||
return fail(fmt.Errorf("failed to create output directory: %w", err))
|
||||
}
|
||||
|
||||
registry.RecordProcessingStageChange("running", ps.eventName)
|
||||
concatErr := ps.RunFFmpeg(aggFile, outPath)
|
||||
if concatErr != nil {
|
||||
return concatErr
|
||||
return fail(concatErr)
|
||||
}
|
||||
|
||||
procLog.Info("processing completed", logging.KeyEvent, ps.eventName, "segments", len(segments))
|
||||
registry.RecordProcessingStageChange("completed", ps.eventName)
|
||||
return nil
|
||||
}
|
||||
|
||||
@ -217,6 +240,7 @@ func (ps *ProcessingService) AggregateSegmentInfo(ch <-chan SegmentInfo) (map[in
|
||||
|
||||
for segment := range ch {
|
||||
fmt.Printf("Received segment %s in resolution %s \n", segment.Name, segment.Resolution)
|
||||
logging.MustGetFactory().Processing.Debug("segment received", logging.KeySegment, segment.Name, logging.KeyResolution, segment.Resolution)
|
||||
current, exists := segmentMap[segment.SeqNo]
|
||||
if !exists || rank[segment.Resolution] > rank[current.Resolution] {
|
||||
segmentMap[segment.SeqNo] = segment
|
||||
@ -318,9 +342,11 @@ func (ps *ProcessingService) RunFFmpeg(inputPath, outputPath string) error {
|
||||
fileOutPath := utils.SafeJoin(outputPath, ps.eventName+".mp4")
|
||||
fmt.Println("Input path:", inputPath)
|
||||
fmt.Println("Output path:", fileOutPath)
|
||||
logging.MustGetFactory().Processing.Info("running ffmpeg", "input", inputPath, "output", fileOutPath)
|
||||
|
||||
path, err := ps.getFFmpegPath()
|
||||
if err != nil {
|
||||
logging.MustGetFactory().Processing.Error("failed to find ffmpeg", logging.KeyError, err.Error())
|
||||
return fmt.Errorf("failed to find FFmpeg: %w", err)
|
||||
}
|
||||
|
||||
@ -329,9 +355,11 @@ func (ps *ProcessingService) RunFFmpeg(inputPath, outputPath string) error {
|
||||
cmd.Stderr = os.Stderr
|
||||
|
||||
if err := cmd.Run(); err != nil {
|
||||
logging.MustGetFactory().Processing.Error("ffmpeg run failed", logging.KeyError, err.Error())
|
||||
return fmt.Errorf("failed to run ffmpeg: %w", err)
|
||||
}
|
||||
|
||||
fmt.Println("FFmpeg completed successfully")
|
||||
logging.MustGetFactory().Processing.Info("ffmpeg completed successfully", "output", fileOutPath)
|
||||
return nil
|
||||
}
|
||||
|
||||
@ -4,6 +4,7 @@ import (
|
||||
"context"
|
||||
"fmt"
|
||||
"log"
|
||||
"m3u8-downloader/pkg/logging"
|
||||
"os"
|
||||
"sync"
|
||||
"time"
|
||||
@ -41,6 +42,7 @@ func (cs *CleanupService) Start(ctx context.Context) error {
|
||||
}
|
||||
|
||||
log.Printf("Cleanup service started (retention: %v, batch: %d)", cs.config.RetentionPeriod, cs.config.BatchSize)
|
||||
logging.MustGetFactory().Transfer.Info("cleanup service started", "retention", cs.config.RetentionPeriod.String(), "batch_size", cs.config.BatchSize)
|
||||
|
||||
ticker := time.NewTicker(cs.config.CheckInterval)
|
||||
defer ticker.Stop()
|
||||
@ -97,6 +99,7 @@ func (cs *CleanupService) ExecuteCleanup(ctx context.Context) error {
|
||||
}
|
||||
|
||||
log.Printf("Cleanup batch completed (cleaned: %d, errors: %d)", cleanedCount, len(errors))
|
||||
logging.MustGetFactory().Transfer.Info("cleanup batch completed", "cleaned", cleanedCount, "errors", len(errors))
|
||||
|
||||
if len(errors) > 0 {
|
||||
for i, err := range errors {
|
||||
@ -105,6 +108,7 @@ func (cs *CleanupService) ExecuteCleanup(ctx context.Context) error {
|
||||
break
|
||||
}
|
||||
log.Printf("Error: %v", err)
|
||||
logging.MustGetFactory().Transfer.Warn("cleanup error", logging.KeyError, err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@ -4,6 +4,7 @@ import (
|
||||
"context"
|
||||
"fmt"
|
||||
"log"
|
||||
"m3u8-downloader/pkg/logging"
|
||||
"m3u8-downloader/pkg/nas"
|
||||
"os"
|
||||
"path/filepath"
|
||||
@ -32,6 +33,7 @@ func TransferFile(nt *nas.NASService, ctx context.Context, item *TransferItem) e
|
||||
}
|
||||
|
||||
log.Printf("File transfer completed: %s -> %s", item.SourcePath, destPath)
|
||||
logging.MustGetFactory().Transfer.Debug("nas copy completed", logging.KeyFile, item.SourcePath, "dest", destPath)
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
@ -6,6 +6,8 @@ import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"log"
|
||||
"m3u8-downloader/pkg/health"
|
||||
"m3u8-downloader/pkg/logging"
|
||||
"m3u8-downloader/pkg/nas"
|
||||
"os"
|
||||
"sync"
|
||||
@ -74,7 +76,10 @@ func (tq *TransferQueue) Add(item TransferItem) error {
|
||||
defer tq.mu.Unlock()
|
||||
|
||||
if tq.items.Len() >= tq.config.MaxQueueSize {
|
||||
return fmt.Errorf("Queue is full (max size: %d)", tq.config.MaxQueueSize)
|
||||
err := fmt.Errorf("Queue is full (max size: %d)", tq.config.MaxQueueSize)
|
||||
logging.MustGetFactory().Transfer.Error("queue full, dropping file", logging.KeyFile, item.SourcePath, logging.KeyError, err.Error())
|
||||
health.MustGetRegistry().RecordTransferError(item.SourcePath, err)
|
||||
return err
|
||||
}
|
||||
|
||||
heap.Push(tq.items, &item)
|
||||
@ -187,9 +192,11 @@ func (tq *TransferQueue) processItem(ctx context.Context, item TransferItem) {
|
||||
if tq.cleanup != nil {
|
||||
if err := tq.cleanup.ScheduleCleanup(item.SourcePath); err != nil {
|
||||
log.Printf("Failed to add file to cleanup list: %v", err)
|
||||
logging.MustGetFactory().Transfer.Warn("failed to schedule cleanup", logging.KeyFile, item.SourcePath, logging.KeyError, err.Error())
|
||||
}
|
||||
}
|
||||
log.Printf("File transfer completed: %s", item.SourcePath)
|
||||
logging.MustGetFactory().Transfer.Info("file transfer completed", logging.KeyFile, item.SourcePath, logging.KeyBytes, item.FileSize)
|
||||
return
|
||||
}
|
||||
|
||||
@ -197,11 +204,14 @@ func (tq *TransferQueue) processItem(ctx context.Context, item TransferItem) {
|
||||
item.RetryCount++
|
||||
|
||||
log.Printf("File transfer failed: %s (attempt %d/%d): %v", item.SourcePath, item.RetryCount, maxRetries, err)
|
||||
logging.MustGetFactory().Transfer.Warn("file transfer attempt failed", logging.KeyFile, item.SourcePath, logging.KeyAttempt, item.RetryCount, logging.KeyError, err.Error())
|
||||
|
||||
if attempt == maxRetries {
|
||||
item.Status = StatusFailed
|
||||
tq.stats.IncrementFailed()
|
||||
log.Printf("Transfer permanently failed for file: %s", item.SourcePath)
|
||||
logging.MustGetFactory().Transfer.Error("transfer permanently failed", logging.KeyFile, item.SourcePath, logging.KeyError, err.Error())
|
||||
health.MustGetRegistry().RecordTransferError(item.SourcePath, err)
|
||||
return
|
||||
}
|
||||
}
|
||||
@ -225,10 +235,12 @@ func (tq *TransferQueue) SaveState() error {
|
||||
"timestamp": time.Now(),
|
||||
}, "", " ")
|
||||
if err != nil {
|
||||
logging.MustGetFactory().Transfer.Error("failed to marshal queue state", logging.KeyError, err.Error())
|
||||
return fmt.Errorf("Failed to marshal queue state: %w", err)
|
||||
}
|
||||
|
||||
if err := os.WriteFile(tq.config.PersistencePath, data, 0644); err != nil {
|
||||
logging.MustGetFactory().Transfer.Error("failed to save queue state", logging.KeyError, err.Error())
|
||||
return fmt.Errorf("Failed to save queue state: %w", err)
|
||||
}
|
||||
|
||||
@ -241,6 +253,7 @@ func (tq *TransferQueue) LoadState() error {
|
||||
if os.IsNotExist(err) {
|
||||
return nil
|
||||
}
|
||||
logging.MustGetFactory().Transfer.Error("failed to load queue state", logging.KeyError, err.Error())
|
||||
return fmt.Errorf("Failed to load queue state: %w", err)
|
||||
}
|
||||
|
||||
|
||||
@ -5,6 +5,8 @@ import (
|
||||
"fmt"
|
||||
"log"
|
||||
"m3u8-downloader/pkg/constants"
|
||||
"m3u8-downloader/pkg/health"
|
||||
"m3u8-downloader/pkg/logging"
|
||||
nas2 "m3u8-downloader/pkg/nas"
|
||||
"m3u8-downloader/pkg/utils"
|
||||
"os"
|
||||
@ -36,8 +38,11 @@ func NewTrasferService(outputDir string, eventName string) (*TransferService, er
|
||||
nas := nas2.NewNASService(nasConfig)
|
||||
|
||||
if err := nas.TestConnection(); err != nil {
|
||||
logging.MustGetFactory().Transfer.Error("failed to connect to NAS", logging.KeyError, err.Error())
|
||||
return nil, fmt.Errorf("failed to connect to NAS: %w", err)
|
||||
}
|
||||
logging.MustGetFactory().Transfer.Info("connected to NAS", "path", outputDir)
|
||||
health.MustGetRegistry().RecordNASConnectivity(true)
|
||||
|
||||
cleanupConfig := CleanupConfig{
|
||||
Enabled: cfg.Cleanup.AfterTransfer,
|
||||
@ -66,6 +71,8 @@ func NewTrasferService(outputDir string, eventName string) (*TransferService, er
|
||||
return nil, fmt.Errorf("failed to create file watcher: %w", err)
|
||||
}
|
||||
|
||||
health.MustGetRegistry().AttachTransfer(queue.stats, queue.GetQueueSize, cleanup.GetPendingCount, nas.IsConnected)
|
||||
|
||||
return &TransferService{
|
||||
watcher: watcher,
|
||||
queue: queue,
|
||||
@ -109,6 +116,7 @@ func (ts *TransferService) Start(ctx context.Context) error {
|
||||
}()
|
||||
|
||||
log.Println("Transfer service started")
|
||||
logging.MustGetFactory().Transfer.Info("transfer service started")
|
||||
wg.Wait()
|
||||
|
||||
return nil
|
||||
@ -128,6 +136,9 @@ func (ts *TransferService) reportStats(ctx context.Context) {
|
||||
cleanupPending := ts.cleanup.GetPendingCount()
|
||||
|
||||
log.Printf("Transfer Stats: Added: %d, Completed: %d, Failed: %d, Pending: %d, Bytes: %d, Queue Size: %d, Cleanup Pending: %d", added, completed, failed, pending, bytes, queueSize, cleanupPending)
|
||||
logging.MustGetFactory().Transfer.Info("transfer stats",
|
||||
"added", added, "completed", completed, "failed", failed, "pending", pending,
|
||||
logging.KeyBytes, bytes, "queue_size", queueSize, "cleanup_pending", cleanupPending)
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -146,9 +157,12 @@ func (ts *TransferService) Shutdown(ctx context.Context) error {
|
||||
// Disconnect from NAS
|
||||
if err := ts.nas.Disconnect(); err != nil {
|
||||
log.Printf("Warning: failed to disconnect from NAS: %v", err)
|
||||
logging.MustGetFactory().Transfer.Warn("failed to disconnect from NAS", logging.KeyError, err.Error())
|
||||
}
|
||||
health.MustGetRegistry().RecordNASConnectivity(false)
|
||||
|
||||
log.Println("Transfer service shut down")
|
||||
logging.MustGetFactory().Transfer.Info("transfer service shut down")
|
||||
|
||||
return nil
|
||||
}
|
||||
@ -178,6 +192,8 @@ func (ts *TransferService) QueueExistingFiles(localEventPath string) error {
|
||||
relPath, err := filepath.Rel(localEventPath, path)
|
||||
if err != nil {
|
||||
log.Printf("Failed to get relative path for %s: %v", path, err)
|
||||
logging.MustGetFactory().Transfer.Warn("failed to get relative path", logging.KeyFile, path, logging.KeyError, err.Error())
|
||||
health.MustGetRegistry().RecordTransferError(path, err)
|
||||
return nil
|
||||
}
|
||||
|
||||
@ -218,6 +234,8 @@ func (ts *TransferService) QueueExistingFiles(localEventPath string) error {
|
||||
// Add to queue
|
||||
if err := ts.queue.Add(item); err != nil {
|
||||
log.Printf("Failed to queue file %s: %v", path, err)
|
||||
logging.MustGetFactory().Transfer.Error("failed to queue file", logging.KeyFile, path, logging.KeyError, err.Error())
|
||||
health.MustGetRegistry().RecordTransferError(path, err)
|
||||
} else {
|
||||
log.Printf("Queued file: %s (%s, %d bytes)", path, resolution, info.Size())
|
||||
fileCount++
|
||||
|
||||
@ -4,6 +4,8 @@ import (
|
||||
"context"
|
||||
"fmt"
|
||||
"log"
|
||||
"m3u8-downloader/pkg/health"
|
||||
"m3u8-downloader/pkg/logging"
|
||||
"math/rand"
|
||||
"os"
|
||||
"path/filepath"
|
||||
@ -45,11 +47,13 @@ func (fw *FileWatcher) Start(ctx context.Context) error {
|
||||
}
|
||||
|
||||
log.Printf("Starting file watcher on %s", fw.outputDir)
|
||||
logging.MustGetFactory().Transfer.Info("file watcher started", "dir", fw.outputDir)
|
||||
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
log.Println("File watcher shutting down...")
|
||||
logging.MustGetFactory().Transfer.Info("file watcher shutting down")
|
||||
return ctx.Err()
|
||||
|
||||
case event, ok := <-fw.watcher.Events:
|
||||
@ -63,6 +67,7 @@ func (fw *FileWatcher) Start(ctx context.Context) error {
|
||||
return fmt.Errorf("Watcher errors channel closed")
|
||||
}
|
||||
log.Printf("Watcher error: %v", err)
|
||||
logging.MustGetFactory().Transfer.Error("watcher error", logging.KeyError, err.Error())
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -144,6 +149,8 @@ func (fw *FileWatcher) processFile(filePath string) {
|
||||
info, err := os.Stat(filePath)
|
||||
if err != nil {
|
||||
log.Printf("Failed to stat file %s: %v", filePath, err)
|
||||
logging.MustGetFactory().Transfer.Warn("failed to stat file", logging.KeyFile, filePath, logging.KeyError, err.Error())
|
||||
health.MustGetRegistry().RecordTransferError(filePath, err)
|
||||
return
|
||||
}
|
||||
|
||||
@ -152,6 +159,8 @@ func (fw *FileWatcher) processFile(filePath string) {
|
||||
relPath, err := filepath.Rel(fw.outputDir, filePath)
|
||||
if err != nil {
|
||||
log.Printf("Failed to get relative path for file %s: %v", filePath, err)
|
||||
logging.MustGetFactory().Transfer.Warn("failed to get relative path", logging.KeyFile, filePath, logging.KeyError, err.Error())
|
||||
health.MustGetRegistry().RecordTransferError(filePath, err)
|
||||
return
|
||||
}
|
||||
|
||||
@ -167,8 +176,11 @@ func (fw *FileWatcher) processFile(filePath string) {
|
||||
|
||||
if err := fw.queue.Add(item); err != nil {
|
||||
log.Printf("Failed to add file to queue: %v", err)
|
||||
logging.MustGetFactory().Transfer.Error("failed to add file to queue", logging.KeyFile, filePath, logging.KeyError, err.Error())
|
||||
health.MustGetRegistry().RecordTransferError(filePath, err)
|
||||
} else {
|
||||
log.Printf("Added file to queue: %s", filePath)
|
||||
logging.MustGetFactory().Transfer.Debug("file queued", logging.KeyFile, filePath)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user