Composable Rust CLI for generating stable filesystem slices as path lists. Walk roots, apply filetype, metadata, content, size, WC, and ignore filters, then emit paths, JSONL, or NUL-delimited output for Unix pipelines.
captured-data
file-discovery
file-filter
filesystem
jsonl
pipe-friendly
pipe-utility
rust
terminal-ai
xdg
- Rust 100%
Andrew Briscoe
819af21b85
This change turns 'data-view' from an extension-only lister into a richer "filesystem slice" tool with stat, content-kind, and wc-based filtering, plus structured JSONL metadata.
**Capabilities**
- Added stat filters:
- '--modified-after/--modified-before' (RFC3339 or Unix epoch).
- '--birth-after/--birth-before' (creation time; entries without birth time don’t match).
- '--uid/--gid' (repeatable, Unix-only; OR semantics within each).
- '--size-min/--size-max' with K/M/G/T ('KiB' variants) parsing.
- '--executable' / '--non-executable' (Unix execute bits).
- Added content-kind filters:
- '--content any|text|binary' and '--text' / '--binary' aliases.
- Simple binary vs text detection to handle extensionless files (README, LICENSE, Makefile, etc.).
- Added wc-based filters and metrics:
- Filters: '--lines-min/max', '--words-min/max', '--bytes-min/max'.
- Optional '--wc' to emit 'bytes/lines/words' in JSONL.
- Added JSONL metadata:
- '--stat' includes 'size', 'modified', 'birth', 'uid', 'gid', 'executable', 'content' per path.
- '--wc' includes 'bytes', 'lines', 'words' per path.
- Plain path output remains the default, unchanged.
**Operations / Workflows**
- New workflows:
- "Recent small docs": 'data-view --path . --text --modified-after 2026-01-01T00:00:00Z --size-max 32K'.
- "Large binaries": 'data-view --path . --binary --size-min 10M'.
- "Readable text files": 'data-view --path . --text --lines-max 2000'.
- "Metadata inventory": 'data-view --path . --text --jsonl --stat --wc'.
- Existing flows (basic filetype+ignore slicing, path-only output, '--jsonl'/'--null') still work as before.
**Systems & Integration**
- Core binary ('src/main.rs') and the doc rust-script now share:
- New CLI flags listed above.
- New types: 'ContentKind', 'JsonStat', 'WcInfo', 'OutputEntry', 'ParsedFilters'.
- Helper functions for time/size parsing, executable and UID/GID detection (Unix), content detection, and wc counting.
- 'emit' now serializes 'JsonLine { path, stat?, wc? }' when '--jsonl' is used.
- 'Cargo.toml':
- Version bump: '0.1.0 → 0.1.1'.
- New dependency: 'chrono' for time parsing.
**Data & Information**
- JSONL schema:
- 'path: String'
- Optional 'stat { size, modified, birth, uid, gid, executable, content }'
- Optional 'wc { bytes, lines, words }'
- '--print-config' now exposes 'stat_filters', 'content_filter', 'wc_filters', and output flags ('stat', 'wc').
**Project Impact & Risks**
- Strengthens 'data-view' as a reusable backend for AI context building, TF-IDF indexing, and searchable inventories.
- Risks:
- wc computation reads full files (potentially heavy on huge files).
- Platform differences for birth time, UID/GID, and executability; non-Unix rejects unsupported filters.
- Follow-ups:
- Performance tuning for wc/content detection.
- More explicit documentation of platform semantics and edge cases.
|
||
|---|---|---|
| docs | ||
| src | ||
| .gitignore | ||
| Cargo.toml | ||