diff --git a/.gitignore b/.gitignore index 9106b2a..508691c 100644 --- a/.gitignore +++ b/.gitignore @@ -6,3 +6,13 @@ /pkg/ /spec/reports/ /tmp/ +*.gem + +# Editors and OS +*.swp +*.swo +*~ +.DS_Store + +# Scratch: picopackages installed while trying the tool out on this repo +*.bkup diff --git a/.standard.yml b/.standard.yml new file mode 100644 index 0000000..e87903e --- /dev/null +++ b/.standard.yml @@ -0,0 +1,9 @@ +# standardrb reads this, not .rubocop.yml — keep the two in step. +ruby_version: 3.1 + +ignore: + # Fixtures are deliberately malformed or copied verbatim from elsewhere; + # linting them defeats the point of having them. + - "test/files/**/*" + # The pre-refactor monolith, kept only for reference. + - "exe/pppkg" diff --git a/CHANGELOG.md b/CHANGELOG.md index 2283a4b..a1a8679 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,55 @@ ## [Unreleased] +### Added + +- Three-way merge on update, so a locally edited picopackage can still take + upstream changes. Conflicts are written to `.picopackage-merge` and the + original file is never touched. +- `base_checksum` metadata: the upstream payload a file was last reconciled + with. Doubles as the key into a content-addressed merge-base cache under + `$XDG_CACHE_HOME/picopackage` (`PICOPACKAGE_CACHE` overrides). +- Unrecognised metadata keys are preserved across rewrites instead of being + dropped. +- An upstream metadata block is explicitly optional. Any plain file is a valid + source; a block lets a publisher assert a canonical `url`, `filename`, + `payload_version` and licence, which are adopted as given. +- `ppkg verify`, `ppkg package --url`, `ppkg scan` output with urls, and + `ppkg help`. + +### Changed + +- Resolution is decided by checksum, never by mtime. A file whose payload + matches upstream is up to date however the timestamps compare. +- Merges run on the payload only; the metadata block is regenerated afterwards + rather than merged. +- A locally edited file whose upstream hasn't moved reports `current` instead of + running a no-op merge. It also restores the merge base to the cache, so a + cleared cache recovers on the next `update` rather than at the next conflict. +- Fetching uses `net/http` directly instead of `URI.open`, which treats a + leading `|` as a command to run. Redirects (max 5, `http`/`https` only), + `If-None-Match`, streamed size caps and typed errors come with it. +- `update` writes back to the local filename rather than upstream's preferred + one, so a renamed package doesn't install a second copy. +- Timestamps are ISO 8601 from every provider. + +### Fixed + +- Checksums are computed over the normalised payload everywhere. Written + packages previously failed their own verification, so every file looked + locally modified. +- Provider selection no longer aborts when a speculative provider guesses wrong, + so plain URLs reach `DefaultProvider`. A 404 now reports as a 404 rather than + "no provider could handle this". +- `Scanner` returns usable results and survives binary, unreadable and oversized + files. +- `ppkg verify` reads the checksum it actually writes; `ppkg package` saves. +- A metadata block containing only unmodelled keys is no longer treated as a + bare file and overwritten. +- Checksums are derived at write time, never adopted from an input. Installing + from an upstream whose block carried a stale `payload_checksum` produced a + file that failed its own verification and reported as locally modified before + anyone had edited it. + ## [0.2.0] - 2025-01-21 - Rename to from Picop to Picopackage diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..39cc438 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,45 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## What this is + +A Ruby gem (`picopackage`, CLI `ppkg`) for installing and updating single-file packages: ordinary source files that carry their own provenance in a commented YAML block at the end (`# @PICOPACKAGE_START` … `# @PICOPACKAGE_END`). See README.md for the format and notes.md for the resolution rules and open design questions. + +## Commands + +```bash +bin/setup # install dependencies +rake # tests + StandardRB (the default task) +rake test # tests only +rake test N=/pattern/ # run tests whose names match a pattern +ruby -Itest test/test_merge.rb # run a single test file +bundle exec standardrb # lint (config in .standard.yml, not .rubocop.yml) +bundle exec standardrb --fix +``` + +Releasing: bump `lib/picopackage/version.rb`, then `bundle exec rake release`. + +## Architecture + +Everything lives under `lib/picopackage/`; `lib/picopackage.rb` just requires the pieces and defines the shared error classes. + +The flow of an `install`/`update` is: **CLI → Fetch.fetch → Provider → Resolver → (Merge, Cache) → Package#save**. + +- **package.rb** — the data model. `Package` wraps a file's content and splits it into `Payload` (the code, normalized to `rstrip + "\n\n"`) and `Metadata` (a Struct parsed from the YAML block). `Metadata#extra` round-trips unknown keys so old tools can't corrupt packages written by newer ones. Key distinction: `modified?` (payload differs from its recorded `payload_checksum` — a human edited it) vs `diverged?` (payload differs from `base_checksum`, the upstream ancestor — true after a merge even when `modified?` is false); `diverged?` is what decides whether a fast-forward is safe. +- **provider.rb** — resolves a URL to a fetch strategy. `Provider.for` walks `PROVIDERS` (FileProvider, GithubGistProvider, OpenGistProvider, DefaultProvider); `handles_url?` returns true/false/`:maybe`, and `:maybe` providers prove themselves by fetching (they fetch in their constructor). Upstream metadata *claims* (url, version, licence) are adopted with `||=`; *records* (checksums) are always recomputed from the payload actually received. +- **fetch.rb** — two things: the HTTP fetcher (`Fetch`: 1 MB cap, 10s timeout, ≤5 redirects to http/https only, ETag support; deliberately `Net::HTTP`, never `URI.open`) and the `Resolver`, which contains all install/update decision logic and returns a `Result` with a state (`installed/adopted/current/updated/merged/conflict`) — deciding what to do about a conflict is the CLI's job. +- **merge.rb** — three-way merge shelled out to `git merge-file` or `diff3`; no Ruby reimplementation. +- **cache.rb** — content-addressed store of upstream payloads under `$XDG_CACHE_HOME/picopackage` (override: `PICOPACKAGE_CACHE`), keyed by checksum. This is where merge bases come from; a cache miss means the merge is refused, not guessed. +- **cli.rb** — argument parsing and reporting only; commands are `install update package verify inspect scan`. + +## Invariants (violating these is a bug, not a style choice) + +1. **The checksum decides, never the timestamp.** mtime is not evidence about content; equal payload checksums settle the content question regardless of timestamps. +2. **Merges run on the payload alone; the metadata block is derived, never merged.** It is regenerated on every write — `Package#generate_package` is the single choke point where `payload_checksum` is computed, so a stale checksum can never survive into a written file. +3. **Conflicts are non-destructive.** The user's file is never touched; conflict markers go to a sibling `.picopackage-merge`. +4. A bare local file whose payload differs from upstream is an *unrelated file*, not a modified package — refuse, don't merge. + +## Tests + +Minitest, in `test/test_*.rb`. `test/test_helper.rb` provides `PicopackageTest#with_dirs`, which gives every test its own upstream dir, project dir, and cache root (the cache is global state; never share one across tests). Fake an upstream by writing a real file and fetching it through the real `FileProvider` (`publish` + `upstream_package` helpers) rather than stubbing. Fixtures in `test/files/` are deliberately malformed or copied verbatim and are excluded from linting. diff --git a/Gemfile.lock b/Gemfile.lock index e5a35b6..df93d81 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -1,9 +1,8 @@ PATH remote: . specs: - picopackage (0.2.1) + picopackage (0.3.0) digest - open-uri (~> 0.5) yaml (~> 0.4) GEM @@ -23,11 +22,7 @@ GEM json (2.9.1) language_server-protocol (3.17.0.3) lint_roller (1.1.0) - minitest (5.25.4) - open-uri (0.5.0) - stringio - time - uri + minitest (5.27.0) parallel (1.26.3) parser (3.3.7.0) ast (~> 2.4.1) @@ -75,12 +70,9 @@ GEM lint_roller (~> 1.1) rubocop-performance (~> 1.23.0) stringio (3.1.2) - time (0.4.1) - date unicode-display_width (3.1.4) unicode-emoji (~> 4.0, >= 4.0.4) - unicode-emoji (4.0.4) - uri (1.0.2) + unicode-emoji (4.2.0) yaml (0.4.0) PLATFORMS diff --git a/README.md b/README.md index 28dc480..d5c84a9 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,11 @@ # Picopackage -A command line tool for installing and managing [Picopackages](https://picopackage.org). +A command line tool for installing and updating [picopackages](https://picopackage.org): +single files that carry their own provenance, so you can install one into your +project, edit it, and still pull upstream fixes later. + +For code that's too small to be a gem but copied often enough that pasting it +between projects hurts. ## Installation @@ -10,18 +15,121 @@ gem install picopackage ## Usage -`picopackage install ` +```bash +ppkg install https://gist.github.com/you/abc123 # fetch into . +ppkg install https://example.com/retry.rb lib/ # fetch into lib/ +ppkg update lib/retry.rb # re-fetch from its recorded url +ppkg package lib/mine.rb --url https://… # turn a local file into a picopackage +ppkg verify lib/retry.rb # check it against its checksum +ppkg scan . # list picopackages in a tree +ppkg inspect lib/retry.rb # print its metadata +``` + +## What a picopackage looks like + +An ordinary source file with a commented YAML block at the end: + +```ruby +def retry_with_backoff(attempts: 3) + # … +end + +# @PICOPACKAGE_START +# --- +# url: https://gist.github.com/you/abc123 +# filename: retry.rb +# payload_timestamp: '2026-08-12T14:06:38Z' +# payload_checksum: sha256:7b856b1b… +# base_checksum: sha256:7b856b1b… +# @PICOPACKAGE_END +``` + +Everything above the block is the **payload**. The block is regenerated on every +write. Metadata keys the tool doesn't recognise are preserved as-is, so optional +fields survive a tool that predates them. + +## Upstream doesn't need a block + +A plain file with no metadata at all is a valid picopackage source. You can +install someone's gist, blog snippet or raw file without them adopting anything +— everything the block would have said is derived from the fetch. + +When upstream *does* carry a block, it's opting in to extra guarantees, and only +half of it is upstream's to assert: + +| | Fields | Who decides | +| --- | --- | --- | +| **Claims** | `url`, `filename`, `payload_version`, `licence` | upstream — adopted as given | +| **Records** | `payload_checksum`, `base_checksum` | your machine — always recomputed | + +A claim is something only the author knows: the canonical URL that outlives the +mirror you fetched from, the version, the licence. A record describes one copy on +one disk, so upstream can't speak for yours. In particular an upstream +`payload_checksum` is never trusted — a checksum stored beside the content it +hashes proves nothing, and an author who edits without re-running the tool ships +a stale one. Adopting it would install a file that failed its own verification +and reported as locally modified before you touched it. + +## Updating a file you've edited + +The point of a picopackage is that it's *your* file — so editing it has to be +allowed, and updates have to cope with that. `ppkg update` does a three-way +merge against the version you originally installed: + +- **Unchanged locally** → fast-forwarded to upstream. +- **Edited locally, upstream unchanged** → nothing happens. No merge is + attempted; upstream is still the version you branched from. +- **Edited locally, no overlap with upstream's changes** → merged, keeping both. +- **Edited locally, overlapping changes** → refused. Your file is left exactly + as it is and a copy with conflict markers is written to + `.picopackage-merge` for you to work from. +- `--force` discards local changes and takes upstream wholesale. + +Two rules decide every case: + +1. **The checksum decides, never the timestamp.** An mtime says nothing about + content — a file re-uploaded unchanged gets a fresh one. +2. **Merges run on the payload alone.** The metadata block is regenerated + afterwards, so it never conflicts. + +The merge needs the common ancestor, which is kept in a content-addressed cache +under `$XDG_CACHE_HOME/picopackage` (override with `PICOPACKAGE_CACHE`). A file +installed on another machine, or one whose cache has been cleared, has no +ancestor available — `ppkg` says so and refuses rather than guessing. + +`git merge-file` is used when available, `diff3` otherwise. + +## Sources + +| Source | Handling | +| --- | --- | +| GitHub Gist | Resolved through the Gist API | +| Opengist | Resolved via its `.json` endpoint | +| Any URL | Fetched directly | +| Local path | Read from disk | + +Responses are capped at 1 MB and time out after 10s. Redirects are followed up +to 5 times and only to `http`/`https`. + +## What it deliberately doesn't do + +- **No dependencies.** Each file stands alone. There is no solver, no version + ranges, no transitive graph. +- **No registry.** The file's `url` is its identity; no central index exists. +- **No author verification.** A checksum stored beside the content it hashes + proves nothing about who wrote it — its job is detecting *local* modification. + Signing (SSH signatures, TOFU-pinned — see notes.md) is designed, not + implemented. Until then, the security model is that a picopackage is one + file and you can read it. ## Development -After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment. +`bin/setup` installs dependencies, `rake test` runs the tests, `rake` runs tests +and [Standard](https://github.com/standardrb/standard). `bin/console` gives you a +prompt. -To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org). +To release: update `lib/picopackage/version.rb`, then `bundle exec rake release`. -## Contributing +## Licence -Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/picop. - -## License - -The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT). +MIT. diff --git a/lib/picopackage.rb b/lib/picopackage.rb index 88b0f57..8015a93 100644 --- a/lib/picopackage.rb +++ b/lib/picopackage.rb @@ -1,18 +1,20 @@ # frozen_string_literal: true require_relative "picopackage/version" +require_relative "picopackage/package" +require_relative "picopackage/cache" +require_relative "picopackage/merge" require_relative "picopackage/fetch" require_relative "picopackage/provider" -require_relative "picopackage/package" require_relative "picopackage/scanner" require_relative "picopackage/cli" module Picopackage class Error < StandardError; end - class FileTooLargeError < StandardError; end + class FileTooLargeError < Error; end - class FetchError < StandardError; end + class FetchError < Error; end - class LocalModificationError < StandardError; end + class LocalModificationError < Error; end end diff --git a/lib/picopackage/cache.rb b/lib/picopackage/cache.rb new file mode 100644 index 0000000..464a1cd --- /dev/null +++ b/lib/picopackage/cache.rb @@ -0,0 +1,60 @@ +# frozen_string_literal: true + +require "fileutils" + +module Picopackage + # Content-addressed store of upstream payloads, kept so that updates have a + # merge base to work from. + # + # A three-way merge needs the common ancestor: the payload exactly as upstream + # last sent it. Metadata already records that payload's checksum in + # `base_checksum`, so the checksum doubles as the cache key and a package + # carries the pointer to its own ancestor. + class Cache + def self.default_root + ENV["PICOPACKAGE_CACHE"] || + File.join(ENV["XDG_CACHE_HOME"] || File.join(Dir.home, ".cache"), "picopackage") + end + + attr_reader :root + + def initialize(root: self.class.default_root) + @root = root + end + + # Returns the checksum the payload was stored under, for recording as a + # `base_checksum`. + def store(payload) + Payload.checksum(payload).tap do |checksum| + path = path_for(checksum) + next if File.exist?(path) + + FileUtils.mkdir_p(File.dirname(path)) + # Written via a sibling tempfile and renamed so a concurrent reader + # never sees a half-written ancestor. + tmp = "#{path}.#{Process.pid}.tmp" + File.write(tmp, payload) + File.rename(tmp, path) + end + end + + def fetch(checksum) + path = path_for(checksum) + File.read(path) if path && File.exist?(path) + end + + def include?(checksum) = !fetch(checksum).nil? + + private + + # "sha256:abcdef…" => /sha256/ab/cdef… + def path_for(checksum) + return nil if checksum.nil? || checksum.empty? + + algorithm, digest = checksum.split(":", 2) + return nil if digest.nil? || digest.length < 3 || !digest.match?(/\A[a-f0-9]+\z/i) + + File.join(root, algorithm, digest[0, 2], digest[2..]) + end + end +end diff --git a/lib/picopackage/cli.rb b/lib/picopackage/cli.rb index e41aed1..227d008 100644 --- a/lib/picopackage/cli.rb +++ b/lib/picopackage/cli.rb @@ -1,140 +1,146 @@ +# frozen_string_literal: true + require "optparse" module Picopackage class CLI + COMMANDS = %w[install update package verify inspect scan].freeze + def self.run(argv = ARGV) command = argv.shift + case command - when "scan" - options = {} - OptionParser.new do |opts| - opts.banner = "Usage: ppkg scan [options] DIRECTORY" - # opts.on('-v', '--verbose', 'Run verbosely') { |v| options[:verbose] = v } - end.parse!(argv) - - dir = argv.first || "." - Picopackage::Scanner.scan(dir).each { |f| puts f.file_path } - - when "init" - OptionParser.new do |opts| - opts.banner = "Usage: ppkg init FILE" - end.parse!(argv) - - file = argv.first - Picopackage::Package.from_file(file).init_metadata - - when "checksum" - OptionParser.new do |opts| - opts.banner = "Usage: ppkg checksum FILE" - end.parse!(argv) - file = argv.first - puts Picopackage::Package.from_file(file).checksum - - when "verify" - OptionParser.new do |opts| - opts.banner = "Usage: ppkg sign FILE" - end.parse!(argv) - - path = argv.first - source = Package.from_file(path) - - if source.metadata["content_checksum"].nil? - puts "⚠️ No checksum found in #{path}" - puts "Run 'ppkg sign #{path}' to add one" - exit 1 - end - - unless source.verify_payload - puts "❌ Checksum verification failed for #{path}" - puts "Expected: #{source.metadata["content_checksum"]}" - puts "Got: #{source.checksum}" - exit 1 - end - - puts "✅ #{path} verified successfully" - - when "inspect" - OptionParser.new do |opts| - opts.banner = "Usage: ppkg inspect FILE|DIRECTORY" - end.parse!(argv) - - path = argv.first - Picopackage::Package.from_file(path).inspect_metadata - - when "fetch" + when "install" options = {force: false} OptionParser.new do |opts| - opts.banner = "Usage: ppkg fetch [options] URI [PATH]" - opts.on("-f", "--force", "Force fetch") { |f| options[:force] = f } + opts.banner = "Usage: ppkg install [options] URL [DIRECTORY]" + opts.on("-f", "--force", "Overwrite local changes") { options[:force] = true } end.parse!(argv) url = argv.shift - path = argv.shift || "." # use '.' if no path provided + destination = argv.shift || "." + abort "Error: URL is required" if url.nil? - if url.nil? - puts "Error: URI is required" - exit 1 - end - - begin - Fetch.fetch(url, path, force: options[:force]) - rescue LocalModificationError => e - puts "Error: #{e.message}" - rescue => e - puts "Error: #{e.message}" - exit 1 - end + report Fetch.fetch(url, destination, force: options[:force]) when "update" options = {force: false} OptionParser.new do |opts| opts.banner = "Usage: ppkg update [options] FILE" - opts.on("-f", "--force", "Force update") { |f| options[:force] = f } + opts.on("-f", "--force", "Discard local changes") { options[:force] = true } end.parse!(argv) - file = argv.first - package = Package.from_file(file) - begin - Fetch.fetch(package.url, File.dirname(file), force: options[:force]) - rescue LocalModificationError => e - puts "Error: #{e.message}" - rescue => e - puts "Error: #{e.message}" + path = argv.shift + abort "Error: FILE is required" if path.nil? + + package = load_package(path) + abort "Error: #{path} has no url in its metadata, so there is nothing to update from" if package.url.nil? || package.url.empty? + + # Updates write back to the file the user named, not to whatever + # filename upstream would prefer — renaming a package locally is + # allowed and shouldn't silently install a second copy. + report Fetch.fetch(package.url, File.dirname(path), force: options[:force], filename: File.basename(path)) + + when "package", "init" + options = {} + OptionParser.new do |opts| + opts.banner = "Usage: ppkg package [--url URL] FILE" + opts.on("--url URL", "Where updates should be fetched from") { |url| options[:url] = url } + end.parse!(argv) + + path = argv.shift + abort "Error: FILE is required" if path.nil? + + package = load_package(path) + was_bare = package.bare? + package.init_metadata(filename: File.basename(path), url: options[:url]) + package.save(path) + Cache.new.store(package.payload) + + puts was_bare ? "Packaged #{path}" : "Updated metadata in #{path}" + warn "Note: no url recorded — 'ppkg update' won't work until one is set with --url" if package.url.nil? + + when "verify" + OptionParser.new { |opts| opts.banner = "Usage: ppkg verify FILE" }.parse!(argv) + + path = argv.shift + abort "Error: FILE is required" if path.nil? + + package = load_package(path) + + if package.payload_checksum.nil? + puts "⚠️ No checksum found in #{path}" + puts "Run 'ppkg package #{path}' to add one" exit 1 end + unless package.verify_payload + puts "❌ #{path} does not match its recorded checksum" + puts "Recorded: #{package.payload_checksum}" + puts "Actual: #{package.payload_digest}" + exit 1 + end + + puts "✅ #{path} matches its recorded checksum" + + when "inspect" + OptionParser.new { |opts| opts.banner = "Usage: ppkg inspect FILE" }.parse!(argv) + + path = argv.shift + abort "Error: FILE is required" if path.nil? + load_package(path).inspect_metadata + + when "scan" + OptionParser.new { |opts| opts.banner = "Usage: ppkg scan [DIRECTORY]" }.parse!(argv) + + found = Scanner.scan(argv.shift || ".") + found.each { |entry| puts [entry.path, entry.package.url].compact.join("\t") } + puts "No picopackages found" if found.empty? + + when "--version", "-v", "version" + puts Picopackage::VERSION + + when nil, "help", "--help", "-h" + usage + exit(command.nil? ? 1 : 0) + else - puts "Unknown command: #{command}" - puts "Available commands: fetch, update, scan, sign, inspect" + warn "Unknown command: #{command}" + usage exit 1 end - rescue OptionParser::InvalidOption => e - puts e.message - exit 1 - rescue => e - puts "Error: #{e.message}" - puts e.backtrace if ENV["DEBUG"] + rescue OptionParser::ParseError => e + abort e.message + rescue FetchError, Fetch::Error, ArgumentError => e + warn "Error: #{e.message}" + warn e.backtrace if ENV["DEBUG"] exit 1 end - def self.determine_script_source - # Get the full path of the currently executing script - current_path = File.expand_path($0) + def self.usage + puts <<~TEXT + Usage: ppkg [options] - # Check if script is in GEM_PATH - gem_paths = Gem.path.map { |p| File.expand_path(p) } + Commands: + install URL [DIR] Fetch a picopackage into DIR (default: .) + update FILE Re-fetch FILE from the url in its metadata + package FILE Add a metadata block to a local file + verify FILE Check FILE against its recorded checksum + inspect FILE Print FILE's metadata as JSON + scan [DIR] List picopackages under DIR - is_gem = gem_paths.any? { |path| current_path.start_with?(path) } + Run 'ppkg --help' for command options. + TEXT + end - if is_gem - # Running from gem installation - gem_name = File.basename(File.dirname(File.dirname(current_path))) - version = File.basename(File.dirname(current_path)) - {source: :gem, path: current_path, gem_name: gem_name, version: version} - else - # Running from local installation - {source: :local, path: current_path} - end + def self.load_package(path) + abort "Error: no such file: #{path}" unless File.file?(path) + Package.from_file(path) or abort "Error: could not read #{path}" + end + + def self.report(result) + puts result.message + exit 1 if result.conflict? end end end diff --git a/lib/picopackage/fetch.rb b/lib/picopackage/fetch.rb index 718ff5b..89e584e 100644 --- a/lib/picopackage/fetch.rb +++ b/lib/picopackage/fetch.rb @@ -1,168 +1,278 @@ +# frozen_string_literal: true + require "net/http" +require "openssl" require "fileutils" -require "tempfile" require "json" -require "debug" module Picopackage class Fetch class Error < StandardError; end - class HTTPError < Error; end - class FileTooLargeError < Error; end - class NotModifiedError < Error; end # Add this - class TooManyRedirectsError < Error; end # Add this - MAX_REDIRECTS = 5 # This constant is used but not defined + class HTTPError < Error; end + + class FileTooLargeError < Error; end + + class NotModifiedError < Error; end + + class TooManyRedirectsError < Error; end + + MAX_REDIRECTS = 5 + USER_AGENT = "picopackage/#{Picopackage::VERSION}" + SCHEMES = %w[http https].freeze def initialize(max_size: 1024 * 1024, timeout: 10) @max_size = max_size @timeout = timeout end - def fetch(uri) + def fetch(uri, etag: nil) + uri = URI.parse(uri.to_s) unless uri.is_a?(URI) case uri.scheme - when "http", "https" then fetch_http(uri) + when "http", "https" then fetch_http(uri, etag: etag) when "file" then fetch_file(uri) else - raise Error, "Unsupported scheme: #{uri.scheme}" + raise Error, "Unsupported scheme: #{uri.scheme.inspect}" end end - def self.fetch(url, destination, force: false) + # Fetches a URL and reconciles it with whatever is already on disk. + # Returns a Resolver::Result; deciding what to do about a conflict is the + # caller's business, not ours. + def self.fetch(url, destination, force: false, filename: nil, cache: Cache.new) raise ArgumentError, "Destination directory does not exist: #{destination}" unless Dir.exist?(destination) + provider = Provider.for(url) - package = provider.package - file_path = File.join(destination, package.filename) + raise FetchError, "No provider could handle #{url}" if provider.nil? - local_package = File.exist?(file_path) ? FileProvider.new(file_path).package : nil + remote = provider.package + target = filename || remote.filename + raise FetchError, "Could not determine a filename for #{url}" if target.nil? - resolver = Resolver.new(package, local_package, file_path, force: force).resolve + file_path = File.join(destination, File.basename(target)) + local = Package.from_file(file_path) - case resolver[:state] - when :kept, :updated - puts resolver[:message] - when :conflict - raise LocalModificationError, resolver[:message] - end - provider.package + Resolver.new(remote, local, file_path, force: force, cache: cache).resolve end private - def fetch_http(uri, etag = nil) - Net::HTTP.start(uri.host, uri.port, connection_options(uri)) do |http| - request = Net::HTTP::Get.new(uri.request_uri) + # Net::HTTP rather than URI.open, deliberately. This method's whole job is to + # dereference a URL a user handed us, and URI.open treats a leading "|" as a + # command to run. The scheme check in #fetch already blocks that, but a + # remote-code-fetching tool should not be one refactor away from shelling + # out, and doing it by hand is also what makes redirect and 304 handling + # possible. + def fetch_http(uri, etag: nil, redirects: MAX_REDIRECTS) + raise TooManyRedirectsError, "More than #{MAX_REDIRECTS} redirects from #{uri}" if redirects.negative? + + Net::HTTP.start(uri.host, uri.port, use_ssl: uri.scheme == "https", + open_timeout: @timeout, read_timeout: @timeout) do |http| + request = Net::HTTP::Get.new(uri) + request["User-Agent"] = USER_AGENT + request["Accept"] = "*/*" request["If-None-Match"] = etag if etag - response = http.request(request) - handle_response(response, uri) + http.request(request) do |response| + case response + when Net::HTTPNotModified + raise NotModifiedError, "#{uri} is unchanged" + when Net::HTTPRedirection + return fetch_http(redirect_target(uri, response), etag: etag, redirects: redirects - 1) + when Net::HTTPSuccess + return read_capped(response) + else + raise HTTPError, "#{response.code} #{response.message} for #{uri}" + end + end end + rescue Net::OpenTimeout, Net::ReadTimeout + raise HTTPError, "Timed out after #{@timeout}s fetching #{uri}" + rescue SocketError, SystemCallError, OpenSSL::SSL::SSLError => e + raise HTTPError, "Could not fetch #{uri}: #{e.message}" end - def fetch_file(uri) - File.read(uri.path) - end - - def connection_options(uri) - { - use_ssl: uri.scheme == "https", - read_timeout: @timeout, - open_timeout: @timeout - } - end - - def handle_response(response, uri) - case response - when Net::HTTPSuccess - { - body: read_body(response), - etag: response["ETag"] - } - when Net::HTTPNotModified - raise NotModifiedError.new("Resource not modified", etag: response["ETag"]) - when Net::HTTPRedirection - handle_redirect(response, uri) - else - raise HTTPError, "HTTP #{response.code}: #{response.message}" - end - end - - def handle_redirect(response, uri, redirect_count = 0) - raise TooManyRedirectsError if redirect_count >= MAX_REDIRECTS + def redirect_target(uri, response) location = response["location"] - new_uri = URI(location) - # Handle both relative paths and full URLs - new_uri = uri.merge(location) if new_uri.relative? - fetch(new_uri, redirect_count: redirect_count + 1) + raise HTTPError, "Redirect from #{uri} with no Location header" if location.nil? + + URI.join(uri, location).tap do |target| + raise Error, "Refusing to follow a redirect to #{target.scheme.inspect}" unless SCHEMES.include?(target.scheme) + end end - def read_body(response) + # Trusts Content-Length only to fail early — the streamed tally is what + # actually enforces the cap, since the header can lie or be absent. + def read_capped(response) + declared = response["content-length"]&.to_i + raise FileTooLargeError, "#{declared} bytes exceeds the #{@max_size} byte limit" if declared&.positive? && declared > @max_size + buffer = String.new(capacity: @max_size) response.read_body do |chunk| - raise FileTooLargeError, "Response would exceed #{@max_size} bytes" if buffer.bytesize + chunk.bytesize > @max_size + raise FileTooLargeError, "Response exceeds the #{@max_size} byte limit" if buffer.bytesize + chunk.bytesize > @max_size + buffer << chunk end buffer end + + def fetch_file(uri) = File.read(uri.path) end ## + # Decides what happens when an upstream payload meets a local file. + # + # Two rules drive every branch, and they are worth stating because they are + # what make the table in notes.md collapse to something small: + # + # 1. The checksum decides, never the timestamp. mtime is not evidence about + # content — a file re-uploaded unchanged gets a fresh one — so equal + # checksums end the content question outright, whatever the clocks say. + # 2. Merges run on the payload alone. The metadata block is derived, so + # merging it would conflict on every update; it is regenerated after. + # # States: - # - kept: local file was converted to a picopackage and kept - # - updated: local file was updated with remote picopackage - # - conflict: local and remote files differ - manually resolve or use -f to force + # - installed: no local file existed, upstream was written + # - adopted: a bare local file gained metadata; its content already matched + # - current: nothing to do + # - updated: local payload replaced with upstream + # - merged: local edits preserved, upstream changes applied on top + # - conflict: needs a human; nothing was written to the original file class Resolver - attr_reader :remote, :local, :local_path, :force - def initialize(remote_package, local_package, local_path, force: false) + STATES = %i[installed adopted current updated merged conflict].freeze + + Result = Struct.new(:state, :message, :path, keyword_init: true) do + def conflict? = state == :conflict + + def written? = %i[installed adopted updated merged].include?(state) + end + + MERGE_SUFFIX = ".picopackage-merge" + + attr_reader :remote, :local, :local_path, :force, :cache + + def initialize(remote_package, local_package, local_path, force: false, cache: Cache.new) @remote = remote_package @local = local_package @local_path = local_path @force = force - @same_checksum = @remote.payload_checksum == @local&.payload_checksum + @cache = cache end - STATES = %i[kept updated conflict].freeze - def resolve - validate_state_hash( - if @force - @remote.save(local_path) - {state: :updated, message: "Force mode: overwrote local file with remote package"} - elsif @local.nil? - @remote.save(local_path) - {state: :kept, message: "Saved Package as new file"} - elsif @remote.payload_version != @local.payload_version - {state: :conflict, message: "Version conflict. Local: #{@local.payload_version}, Remote: #{@remote.payload_version}"} - elsif @remote.payload_timestamp_as_time > @local.payload_timestamp_as_time - @remote.save(local_path) - {state: :updated, message: "Updated to newer version"} - elsif !@same_checksum - handle_checksum_mismatch - elsif @local.was_bare_file - debugger - @local.save(local_path) - {state: :kept, message: "Packaged existing file as Picopackage"} + validate( + if local.nil? + write(:installed, "Installed #{File.basename(local_path)}") + elsif force + write(:updated, "Forced: overwrote #{File.basename(local_path)} with upstream") + elsif same_payload? + reconcile_metadata + elsif local.bare? + # No metadata, no ancestor, and a different payload: this is an + # unrelated file that happens to share a name, not a modified + # picopackage. There is no shared history to merge along. + conflict("#{File.basename(local_path)} already exists, is not a picopackage, and differs from upstream. " \ + "Move it aside or use --force to overwrite it.") + elsif !local.diverged? + write(:updated, "Updated #{File.basename(local_path)} to upstream") + elsif upstream_unchanged? + keep_local_changes else - {state: :kept, message: "Local file is up to date"} + merge end ) end private - def validate_state_hash(hash) - raise "Invalid state" unless STATES.include?(hash[:state]) - raise "Missing message" unless hash[:message].is_a?(String) - hash + def same_payload? = remote.payload_digest == local.payload_digest + + # Same content on both sides, so the only open question is whose metadata + # to keep. A bare local file gets adopted; a package is already fine. + def reconcile_metadata + cache.store(remote.payload) + + if local.bare? + write(:adopted, "Adopted existing #{File.basename(local_path)} as a picopackage") + else + Result.new(state: :current, message: "#{File.basename(local_path)} is up to date", path: local_path) + end end - def handle_checksum_mismatch - if @force - @remote.save(local_path) # In force mode, remote wins - {state: :updated, message: "Overwrote local file with remote package"} - else - {state: :conflict, message: "Files differ. Use --force to convert both to packages"} + # Reached only when the payloads differ and the local one has diverged, so + # the difference is entirely ours: upstream still holds the ancestor we + # branched from. + def upstream_unchanged? = remote.payload_digest == local.ancestor_checksum + + # Nothing upstream to apply. Merging would be a correct no-op — base and + # theirs are the same text, so the result is ours — but it spawns a merge + # tool to prove it and reports `:merged` when nothing was merged. + # + # Storing the payload is not redundant: this is the one moment we are + # holding the ancestor itself, so a cache that has been cleared gets + # repopulated and the *next* update can still merge. + def keep_local_changes + cache.store(remote.payload) + Result.new(state: :current, + message: "#{File.basename(local_path)} is up to date; your local changes are untouched", + path: local_path) + end + + def merge + return conflict("#{File.basename(local_path)} has local changes and no merge tool (git or diff3) is available.") unless Merge.available? + + base = cache.fetch(local.ancestor_checksum) + if base.nil? + return conflict("#{File.basename(local_path)} has local changes and its original upstream version " \ + "isn't in the cache, so there is nothing to merge against. " \ + "Diff it against upstream by hand, or use --force to discard your changes.") end + + upstream_digest = remote.payload_digest + cache.store(remote.payload) + result = Merge.three_way(base: base, ours: local.payload, theirs: remote.payload) + + case result.status + when :clean + remote.payload = result.content + remote.base_checksum = upstream_digest + remote.save(local_path) + Result.new(state: :merged, message: "Merged upstream changes into #{File.basename(local_path)}, keeping your edits", path: local_path) + when :conflict + write_conflict_file(result) + else + conflict("Merge tool failed on #{File.basename(local_path)}.") + end + end + + # The original is never touched. A tool with no index and no `reset` has no + # business writing conflict markers into a file the user may not have + # committed, so the markers go to a sibling and the user drives. + def write_conflict_file(result) + path = local_path + MERGE_SUFFIX + File.write(path, result.content) + conflict("#{File.basename(local_path)} has local changes that conflict with upstream " \ + "(#{result.conflicts} #{(result.conflicts == 1) ? "conflict" : "conflicts"}). " \ + "Resolved-with-markers copy written to #{File.basename(path)}; your file is untouched.") + end + + def write(state, message) + # Metadata describes *this* file, so the filename follows the local name + # even when it differs from upstream's. `url` is what points upstream. + remote.filename = File.basename(local_path) + remote.base_checksum = remote.payload_digest + cache.store(remote.payload) + path = remote.save(local_path) + Result.new(state: state, message: message, path: path) + end + + def conflict(message) = Result.new(state: :conflict, message: message, path: local_path) + + def validate(result) + raise "Invalid state: #{result.state.inspect}" unless STATES.include?(result.state) + raise "Missing message" unless result.message.is_a?(String) + + result end end end diff --git a/lib/picopackage/merge.rb b/lib/picopackage/merge.rb new file mode 100644 index 0000000..516f2ba --- /dev/null +++ b/lib/picopackage/merge.rb @@ -0,0 +1,94 @@ +# frozen_string_literal: true + +require "tempfile" + +module Picopackage + # Three-way merge of payloads, delegated to `git merge-file` or POSIX `diff3`. + # + # Reimplementing diff3 in Ruby would be a few hundred lines of the most + # bug-prone code in the project, to produce output people already know how to + # read. Both external tools take their arguments in the same order — ours, + # base, theirs — and both report a clean merge with status 0 and conflicts + # with a positive status, so the two backends differ only in the command line. + module Merge + Result = Struct.new(:status, :content, :conflicts, keyword_init: true) do + def clean? = status == :clean + + def conflict? = status == :conflict + + def unavailable? = status == :unavailable + end + + LABELS = {ours: "local", base: "ancestor", theirs: "upstream"}.freeze + + module_function + + # Returns :git, :diff3, or nil. + def backend + return @backend if defined?(@backend) + + @backend = if command?("git") + :git + elsif command?("diff3") + :diff3 + end + end + + def available? = !backend.nil? + + def command?(name) + ENV.fetch("PATH", "").split(File::PATH_SEPARATOR).any? do |dir| + File.executable?(File.join(dir, name)) && !File.directory?(File.join(dir, name)) + end + end + + def three_way(base:, ours:, theirs:, labels: LABELS) + return Result.new(status: :unavailable, content: nil, conflicts: 0) unless available? + + with_temp_files(ours, base, theirs) do |ours_path, base_path, theirs_path| + merged, status = run(backend, ours_path, base_path, theirs_path, labels) + + # Negative (255 as unsigned) means the tool itself failed rather than + # finding conflicts — don't report that as a mergeable conflict. + if status.nil? || status > 128 + Result.new(status: :unavailable, content: nil, conflicts: 0) + elsif status.zero? + Result.new(status: :clean, content: merged, conflicts: 0) + else + Result.new(status: :conflict, content: merged, conflicts: status) + end + end + end + + def run(backend, ours_path, base_path, theirs_path, labels) + argv = case backend + when :git + ["git", "merge-file", "-p", "--diff3", + "-L", labels[:ours], "-L", labels[:base], "-L", labels[:theirs], + ours_path, base_path, theirs_path] + when :diff3 + ["diff3", "-m", + "-L", labels[:ours], "-L", labels[:base], "-L", labels[:theirs], + ours_path, base_path, theirs_path] + end + + output = IO.popen(argv, err: File::NULL, &:read) + [output, $?&.exitstatus] + rescue SystemCallError + [nil, nil] + end + + def with_temp_files(ours, base, theirs) + files = {ours: ours, base: base, theirs: theirs}.map do |name, content| + file = Tempfile.new(["picopackage-#{name}", ".txt"]) + file.binmode + file.write(content) + file.flush + file + end + yield(*files.map(&:path)) + ensure + files&.each { |f| f.close! } + end + end +end diff --git a/lib/picopackage/package.rb b/lib/picopackage/package.rb index 1ea3c70..c081076 100644 --- a/lib/picopackage/package.rb +++ b/lib/picopackage/package.rb @@ -1,33 +1,48 @@ +# frozen_string_literal: true + require "yaml" +require "json" require "digest" +require "time" require "forwardable" module Picopackage METADATA_PATTERN = /^\n*#\s*@PICOPACKAGE_START\n(.*?)^\s*#\s*@PICOPACKAGE_END\s*$/m - class Metadata < Struct.new(:url, :filename, :payload_version, :payload_timestamp, :payload_checksum, :etag, keyword_init: true) - # the #from_file method will create a new instance of Metadata from a file path, rather than read a package's metadata - def self.from_file(file_path, content: nil) - new(content: File.read(file_path)) - end - - def self.from_url_response(url, response) - end - + class Metadata < Struct.new(:url, :filename, :payload_version, :payload_timestamp, + :payload_checksum, :base_checksum, :etag, keyword_init: true) def self.from_content(content) - return new unless content =~ METADATA_PATTERN + return new unless (match = METADATA_PATTERN.match(content)) - yaml_content = $1.each_line.map { |line| line.sub(/^\s*#\s?/, "").rstrip }.join("\n") + parsed = YAML.safe_load(uncomment(match[1])) + return new unless parsed.is_a?(Hash) - # Load and transform in one chain - @metadata = new(**YAML.safe_load(yaml_content) - .slice(*Metadata.members.map(&:to_s)) - .transform_keys(&:to_sym)) - rescue - new # Return empty hash on any YAML/transformation errors + known, unknown = parsed.partition { |key, _| members.include?(key.to_s.to_sym) } + new(**known.to_h.transform_keys(&:to_sym)).tap { |metadata| metadata.extra = unknown.to_h } + rescue Psych::Exception, TypeError, ArgumentError + new end - def empty? = to_h.values.all?(&:nil?) + def self.from_file(path) = from_content(File.read(path)) + + def self.uncomment(block) + block.each_line.map { |line| line.sub(/^\s*#\s?/, "").rstrip }.join("\n") + end + + # Keys the struct doesn't model. The spec has a long tail of optional fields + # — licence, test_url, signature, parent_revision — and a tool that dropped + # the ones it didn't recognise would quietly destroy them the first time it + # rewrote a file. Round-tripping them costs nothing and means old tools + # can't corrupt packages written by newer ones. + attr_writer :extra + + def extra = @extra ||= {} + + def empty? = to_h.values.all?(&:nil?) && extra.empty? + + # Modelled members in declaration order, then unmodelled keys sorted, so a + # rewrite that changes one field produces a one-line diff. + def to_yaml_hash = to_h.compact.transform_keys(&:to_s).merge(extra.sort.to_h) end class Payload @@ -35,94 +50,149 @@ module Picopackage def self.normalize(payload) = payload.rstrip + "\n\n" - def self.normalized_from_content(content) = Payload.from_content(content).then { Payload.normalize(_1) } + def self.normalized_from_content(content) = normalize(from_content(content)) - def self.from_file(file_path) = normalized_from_content(File.read(file_path)) + def self.from_file(path) = normalized_from_content(File.read(path)) def self.checksum(payload) = "sha256:#{Digest::SHA256.hexdigest(payload)}" - def self.checksum_from_content(content) = checksum(from_content(content)) + # Always hashes the *normalised* payload. Hashing the raw payload here while + # verification hashed the normalised one meant freshly written packages + # failed their own checksum, so every file looked locally modified. + def self.checksum_from_content(content) = checksum(normalized_from_content(content)) end class Package extend Forwardable - attr_reader :content, :payload, :metadata, :was_bare_file + + attr_reader :content, :payload, :metadata def_delegators :@metadata, :url, :url=, :filename, :filename=, :payload_version, :payload_version=, :payload_timestamp, :payload_timestamp=, - :payload_checksum, :payload_checksum= + :payload_checksum, :payload_checksum=, + :base_checksum, :base_checksum=, + :etag, :etag= def self.from_file(file_path) - if File.exist?(file_path) - new(content: File.read(file_path)) - end + new(content: File.read(file_path)) if File.exist?(file_path) end def initialize(content:) @content = content - @payload = Payload.normalized_from_content(@content) - @metadata = Metadata.from_content(@content) - - if is_bare_file? - @was_bare_file = true - init_metadata - else - @was_bare_file = false - end + @had_metadata_block = METADATA_PATTERN.match?(content) + @payload = Payload.normalized_from_content(content) + @metadata = Metadata.from_content(content) end - def is_bare_file? = @metadata.empty? + # True when the file arrived with no metadata block at all: a hand-copied + # file, or an upstream that was never packaged. Deliberately not "its + # metadata parsed to nothing" — a block holding only optional keys we don't + # model would answer yes to that and get clobbered as though it were bare. + def bare? = !@had_metadata_block - def init_metadata - @metadata.url ||= url - @metadata.filename ||= filename - @metadata.payload_checksum ||= Payload.checksum_from_content(content) - @metadata.payload_timestamp ||= payload_timestamp - end + # What the payload hashes to right now. + def payload_digest = Payload.checksum(payload) - def save(path, filename = nil) - path = File.join(path, filename || @metadata.filename) if File.directory?(path) + # Local edits break the recorded checksum, and that divergence is the only + # modification signal available — mtime says nothing about content. + # + # Note the asymmetry with #verify_payload when no checksum is recorded: we + # have no *evidence of modification*, but we also cannot *verify*. Both + # answers are false, and they mean different things. + def modified? + return false unless recorded_checksum? - File.write(path, generate_package) + payload_digest != payload_checksum end def verify_payload - return false if metadata.payload_checksum.nil? || metadata.payload_checksum&.empty? - Payload.checksum(payload) == metadata.payload_checksum + return false unless recorded_checksum? + + payload_digest == payload_checksum + end + + # The upstream version this file was last reconciled with. Packages written + # before `base_checksum` existed, and hand-written metadata, have only + # `payload_checksum` — which meant exactly this before merges did. + def ancestor_checksum = base_checksum || payload_checksum + + # Whether the payload carries changes relative to upstream — from a human + # edit *or* from an earlier merge. + # + # This, not #modified?, is what decides whether upstream may simply + # overwrite the file. A merged file is unmodified (we wrote it, so its + # recorded checksum matches) while still holding local edits that a + # fast-forward would silently discard. + def diverged? + return false if ancestor_checksum.nil? || ancestor_checksum.empty? + + payload_digest != ancestor_checksum + end + + # Replaces the payload, e.g. with the result of a merge, and re-points the + # recorded checksum at it so the file reads as unmodified until a human + # touches it. `base_checksum` is left alone: it tracks the upstream ancestor, + # which is not what we just wrote. + def payload=(new_payload) + @payload = Payload.normalize(new_payload) + @content = @payload + @metadata.payload_checksum = Payload.checksum(@payload) + end + + # Fills in what's missing. `payload_checksum` is deliberately absent: it is + # derived at write time by #generate_package rather than carried over from + # an input, so a stale value can never survive into a file we write. + def init_metadata(filename: nil, url: nil) + @metadata.filename ||= filename + @metadata.url ||= url + @metadata.payload_timestamp ||= Time.now.utc.iso8601 + @metadata.base_checksum ||= payload_digest + self end def payload_timestamp_as_time - @metadata&.payload_timestamp ? Time.parse(@metadata.payload_timestamp) : nil + payload_timestamp ? Time.parse(payload_timestamp.to_s) : nil + rescue ArgumentError + nil end - def modified? = !verify_payload + # Returns the path written to. + def save(path, filename = nil) + path = File.join(path, filename || @metadata.filename) if File.directory?(path) + File.write(path, generate_package) + path + end - def inspect_metadata = puts JSON.pretty_generate(@metadata.to_h) - - private + def inspect_metadata = puts(JSON.pretty_generate(@metadata.to_yaml_hash)) def generate_package - @metadata.url = url.to_s - metadata_block = generate_metadata - if METADATA_PATTERN.match?(content) - content.sub(METADATA_PATTERN, "\n#{metadata_block}") + @metadata.url = url.to_s unless url.nil? + # The single choke point for the rule that a checksum is *derived*, never + # adopted or preserved. Every write goes through here, so every file we + # produce verifies against itself no matter which path built the package — + # including one built from an upstream block whose author edited the file + # without re-running the tool. + @metadata.payload_checksum = payload_digest + block = generate_metadata + + if METADATA_PATTERN.match?(@content) + @content.sub(METADATA_PATTERN, "\n#{block}") else - [content.rstrip, "\n#{metadata_block}"].join("\n") + [@content.rstrip, "\n#{block}"].join("\n") end end - # This will need a comment style one day, to work with other languages + private + + def recorded_checksum? = !(payload_checksum.nil? || payload_checksum.empty?) + + # This will need a comment style one day, to work with other languages. def generate_metadata - yaml_content = @metadata.to_h.transform_keys(&:to_s).to_yaml.strip - [ - "# @PICOPACKAGE_START", - yaml_content.lines.map { |line| "# #{line}" }.join, - "# @PICOPACKAGE_END", - "" - ].join("\n") + commented = @metadata.to_yaml_hash.to_yaml.strip.each_line.map { |line| "# #{line.chomp}".rstrip } + ["# @PICOPACKAGE_START", *commented, "# @PICOPACKAGE_END", ""].join("\n") end end end diff --git a/lib/picopackage/provider.rb b/lib/picopackage/provider.rb index 2c3cc9e..430c459 100644 --- a/lib/picopackage/provider.rb +++ b/lib/picopackage/provider.rb @@ -1,3 +1,5 @@ +# frozen_string_literal: true + require "time" require "pathname" @@ -6,34 +8,49 @@ module Picopackage def self.for(url) PROVIDERS.each do |provider| case provider.handles_url?(url) - when false + when false, nil next when true return provider.new(url) when :maybe - instance = provider.new(url) - return instance if instance.handles_body? + # A `:maybe` provider proves itself by fetching, and providers fetch in + # their constructor — so a speculative provider that guesses wrong + # (OpenGistProvider appends ".json" to every URL it sees) raises here. + # That has to fall through to the next candidate rather than abort the + # whole resolution, or no plain URL would ever reach DefaultProvider. + begin + instance = provider.new(url) + return instance if instance.handles_body? + rescue FetchError, Fetch::Error, URI::Error + next + end end end - nil # Return nil if no provider found + nil end end - # Base class for fetching content from a URL - # The variable `body` will contain the package_data retrieved from the URL - # The variable `package_data` will contain both and payload + metadata - this would be writen to a file. - # The variable `payload` will contain the payload extracted from `package_data` - # The variable `metadata` will contain the metadata extracted from `package_data` - - # Job of the Provider class is to fetch the body from the URL, and then extract the package_data - # and the filename from the body. The Package class will then take the body and split it into payload and metadata - + # Base class for fetching content from a URL. + # + # A provider's job is to fetch the body, and to pull the package content and a + # filename out of it. Splitting content into payload and metadata is the + # Package class's job, not a provider's. class DefaultProvider MAX_SIZE = 1024 * 1024 TIMEOUT = 10 + attr_reader :url, :package - def self.handles_url?(url) = :maybe + # Not `:maybe`. This is the last resort in PROVIDERS, so if it declined on a + # failed fetch the loop would fall off the end and report "no provider could + # handle this" for what is really a 404. Claiming any URL it understands the + # scheme of lets the actual HTTP error reach the user. + def self.handles_url?(url) + scheme = URI(url.to_s).scheme + %w[http https file].include?(scheme) + rescue URI::InvalidURIError + false + end def initialize(url, fetcher: Fetch.new(max_size: MAX_SIZE, timeout: TIMEOUT)) @url = transform_url(url) @@ -42,7 +59,7 @@ module Picopackage populate_metadata end - def transform_url(url) = URI(url) + def transform_url(url) = URI(url.to_s) def body @body ||= @fetcher.fetch(@url) @@ -56,78 +73,104 @@ module Picopackage raise FetchError, "Failed to parse JSON response" end - def payload_timestamp = Time.now.httpdate + # ISO 8601 throughout: sortable, unambiguous, and the same shape from every + # provider, which httpdate-here-iso-there was not. + def payload_timestamp = Time.now.utc.iso8601 def handles_body? - true - rescue FileTooLargeError, Net::HTTPError, RuntimeError + !body.nil? && !body.empty? + rescue FetchError, FileTooLargeError false end - # Implement in subclass - this come from the `body`. - # Spliting content into payload and metadata is the job of the Package class + # Implement in a subclass when the content is wrapped in something (JSON, + # HTML). Splitting payload from metadata is the Package class's job. def content = body - # Implement in subclass - this should return the filename extracted from the body - if it exists, but not from the metadata - def filename = File.basename @url + # Implement in a subclass when the body carries a filename. Not to be + # confused with the filename in a package's metadata. + def filename + candidate = File.basename(@url.path.to_s) + candidate.empty? ? nil : candidate + end + # Upstream may or may not carry a metadata block, and the two halves of one + # are not equally trustworthy: + # + # - *Claims* are upstream's to make — the canonical url (which outlives the + # mirror we happened to fetch from), payload_version, licence, signature. + # Those are adopted with `||=`. + # - *Records* describe a particular copy on a particular disk, so upstream + # cannot speak for ours. Checksums are recomputed from the payload we + # actually received. Adopting a stale payload_checksum installed a file + # that failed its own verification and read as locally modified before + # anyone had touched it. + # + # The checksum covers the extracted payload, not the raw body: hashing the + # body meant an upstream carrying a metadata block hashed its own metadata, + # so it never matched the same payload fetched from anywhere else. def populate_metadata @package.filename ||= filename - @package.url ||= @url + @package.url ||= @url.to_s @package.payload_timestamp ||= payload_timestamp - @package.payload_checksum ||= Payload.checksum(content) + @package.payload_checksum = @package.payload_digest + @package.base_checksum = @package.payload_checksum end end class GithubGistProvider < DefaultProvider - def self.handles_url?(url) = url.match?(%r{gist\.github\.com}) + def self.handles_url?(url) = url.to_s.match?(%r{gist\.github\.com}) def content = json_body["files"].values.first["content"] def filename = json_body["files"].values.first["filename"] def transform_url(url) - gist_id = url[/gist\.github\.com\/[^\/]+\/([a-f0-9]+)/, 1] - "https://api.github.com/gists/#{gist_id}" + gist_id = url.to_s[%r{gist\.github\.com/[^/]+/([a-f0-9]+)}, 1] + raise FetchError, "Could not extract a gist id from #{url}" if gist_id.nil? + + URI("https://api.github.com/gists/#{gist_id}") end def payload_timestamp - Time.parse(json_body["created_at"]) - rescue ArgumentError - nil + Time.parse(json_body["updated_at"] || json_body["created_at"]).utc.iso8601 + rescue ArgumentError, TypeError + super end end class OpenGistProvider < DefaultProvider - def handles_url?(url) = :maybe + # Speculative: it appends ".json" to anything and sees what comes back, so it + # must not inherit DefaultProvider's unconditional claim. + def self.handles_url?(url) = :maybe - def transform_url(url) = "#{url}.json" + def transform_url(url) = URI("#{url}.json") def content = json_body.dig("files", 0, "content") def filename = json_body.dig("files", 0, "filename") + # Only claims the URL if the JSON actually looks like a gist listing. def handles_body? - content && filename - rescue FileTooLargeError, Net::HTTPError, RuntimeError + !content.nil? && !filename.nil? + rescue FetchError, FileTooLargeError false end - # If we successfully fetch the body, and the body contains content and a filename, then we can handle the body end class FileProvider < DefaultProvider - def self.handles_url?(url) = File.exist?(url) + def self.handles_url?(url) = !url.to_s.empty? && File.exist?(url.to_s) - def transform_url(url) = Pathname(url) + def transform_url(url) = Pathname(url.to_s) def content = url.read def filename = url.basename.to_s def payload_timestamp - url.mtime.httpdate + url.mtime.utc.iso8601 rescue Errno::ENOENT - nil + super end end diff --git a/lib/picopackage/scanner.rb b/lib/picopackage/scanner.rb index 24bc6d5..0a960e2 100644 --- a/lib/picopackage/scanner.rb +++ b/lib/picopackage/scanner.rb @@ -1,11 +1,31 @@ +# frozen_string_literal: true + module Picopackage module Scanner + Found = Struct.new(:path, :package) + + # Scanning walks whole trees, so it has to survive binaries, unreadable + # files and anything large enough that reading it to look for a comment + # would be silly. + MAX_SCAN_SIZE = 1024 * 1024 + def self.scan(directory, pattern: "**/*") - Dir.glob(File.join(directory, pattern)).select do |file| - next unless File.file?(file) - content = File.read(file) - content.match?(Package::METADATA_PATTERN) - end.map { |file| Package.new(file) } + Dir.glob(File.join(directory, pattern)).sort.filter_map do |path| + content = readable_text(path) + next unless content&.match?(METADATA_PATTERN) + + Found.new(path, Package.new(content: content)) + end + end + + def self.readable_text(path) + return nil unless File.file?(path) + return nil if File.size(path) > MAX_SCAN_SIZE + + content = File.read(path, encoding: "UTF-8") + content.valid_encoding? ? content : nil + rescue SystemCallError, ArgumentError + nil end end end diff --git a/lib/picopackage/version.rb b/lib/picopackage/version.rb index 7ed6f47..1ab89a2 100644 --- a/lib/picopackage/version.rb +++ b/lib/picopackage/version.rb @@ -1,5 +1,5 @@ # frozen_string_literal: true module Picopackage - VERSION = "0.2.1" + VERSION = "0.3.0" end diff --git a/notes.md b/notes.md index d802afb..dfc54b9 100644 --- a/notes.md +++ b/notes.md @@ -1,25 +1,195 @@ -## Package Installation Flow +## Resolution rules -1. **Fetch URL** - └─> Download content - └─> Build Package Instance (Metadata & Payload) - └─> Check local file status +Two principles decide every case below: -2. **Local File Check** - ├─> If file doesn't exist: - │ └─> Save file - │ - └─> If file exists: - └─> Compare versions - ├─> If older/same: - │ └─> "Package already installed" - │ - └─> If newer: - └─> Check local modifications - ├─> If modified: - │ └─> "Local modifications detected" - │ └─> "Use 'update '" - │ - └─> If unmodified: - └─> "Update available" - └─> "Use 'update -f' to force update" +1. **The checksum decides, never the timestamp.** mtime is unreliable (a raw + file re-uploaded unchanged gets a new mtime; a gist mirrored elsewhere gets + an arbitrary one) and it is not evidence about content. If the payload + checksums agree, the payload is settled — there is nothing to resolve no + matter what the timestamps say. This is why every `????` in the old table + collapsed: all of those rows assumed equal checksums, so all of them are + metadata questions, not content questions. + +2. **Merge the payload, regenerate the metadata.** The metadata block is + derived, not authored, so it never takes part in a merge — merging it would + conflict on every single update. Three-way merges run on the payload alone + and the block is rewritten afterwards. + +### Terms + +- **payload** — the file with its metadata block removed, then normalised. +- **bare** — a file with no metadata block. A hand-copied file, or an upstream + that has never been packaged. +- **`payload_checksum`** — checksum of the payload as it sits in this file. + Local edits break it, which is how modification is detected. +- **`base_checksum`** — checksum of the upstream payload this file was last + reconciled with. Equal to `payload_checksum` on a clean install; they diverge + after a merge or a local edit. This is the merge base pointer and the cache + key. + +### Same payload checksum + +The content is identical, so the only question is which metadata to keep. + +| Remote | Local | Action | State | +|---------|---------|-------------------------------------------|-------------| +| Bare | missing | Write payload + generated metadata | `installed` | +| Package | missing | Write payload + upstream metadata | `installed` | +| Bare | Bare | Adopt: attach generated metadata | `adopted` | +| Package | Bare | Adopt: attach upstream metadata | `adopted` | +| Bare | Package | Nothing to do | `current` | +| Package | Package | Rewrite only if the metadata itself changed | `current` | + +mtime appears nowhere in this table. That is the point of principle 1. + +### Different payload checksum + +| Local | Modified | Base available | Action | State | +|--------------------|----------|----------------|-------------------------------------|-------------| +| missing | — | — | Write it | `installed` | +| Package | no | — | Fast-forward to upstream | `updated` | +| Package | yes | yes | Three-way merge, clean | `merged` | +| Package | yes | yes | Three-way merge, conflicted | `conflict` | +| Package | yes | no | Refuse — no ancestor to merge from | `conflict` | +| Bare | n/a | never | Refuse — unrelated file, same name | `conflict` | +| any | any | any | `--force` overwrites unconditionally | `updated` | + +A bare local file whose payload differs is not a modified picopackage — it is an +unrelated file that happens to share a name. There is no ancestor and no claim +that the two share a history, so merging would be guesswork. Refuse and say so. + +### Conflicts are non-destructive + +`git`/`diff3` conflict markers are written to `.picopackage-merge`, beside +the original. The original is never touched. A package manager with no `reset` +and no index has no business writing conflict markers into a file the user may +not have committed. + +## Merge base cache + +Three-way merge needs the common ancestor — the payload exactly as upstream last +sent it. It is content-addressed under `$XDG_CACHE_HOME/picopackage/sha256/ab/cdef…` +and written on every save, so a file installed by `ppkg` carries the pointer to +its own ancestor in `base_checksum`. + +Cache misses are expected and survivable: a file installed on another machine, a +cleared cache, a package predating this feature. The fallback is to refuse the +merge and say why, which is honest. A future `source_url` pinned to an immutable +revision would let us refetch the ancestor instead of refusing — see below. + +## Update UX and signing (designed 2026-08-13, not implemented) + +The security question splits in three, and today's answers are: **first +install** — pure trust in the URL, same as copy+paste, mitigated by "it's one +file and you can read it"; **update** — worse than copy+paste, because `ppkg +update` fetches whatever the mutable url serves and writes it with less +friction than a human re-pasting; **transport** — plain `http` is currently +allowed. The checksum contributes nothing adversarial: it lives inside the +file it hashes and is regenerated on every write, so it is integrity against +yourself, not against an attacker. The design below closes the update gap. + +### Diff by default + +`ppkg update` shows the diff and asks before writing. The diff shown is +**ancestor → upstream** — the untrusted delta. Local edits are not news to the +user, and the Resolver already holds all three payloads at that point, so this +is nearly free. We are not a diff tool: shell out (`git diff --no-index`, else +`diff -u`), respect `$PAGER` — the same philosophy as merge.rb. `-y`/`--yes` +skips the prompt for scripts; it never covers an identity change (below). + +### Identity: SSH signatures, TOFU-pinned + +SSH rather than Sigstore. A full Sigstore bundle is 4–10KB of JSON — routinely +larger than the payload it certifies — and drags a dependency tree into a +zero-dependency gem. An armored ed25519 ssh signature is ~0.5KB, embeds in the +metadata block, and the authors this format serves (gists, blogs) already hold +ssh keys, with `github.com/.keys` as a verification channel separate +from the file itself. What Sigstore would add — identity that survives key +loss, a transparency log — is real but not a different security class at this +scale. It stays a possible opt-in upgrade, not the price of entry. + +- The signature covers the **normalized payload + the claims** (url, filename, + payload_version). Signing the payload alone leaves `url` unsigned, so a + tampered first install could silently redirect every future update. The + claims/records split in provider.rb is the seam: sign the claims, keep + deriving the records. +- TOFU: the key fingerprint is pinned at install, or the first time a + signature appears on an already-installed file. Unsigned packages keep + working; signing is opt-in per file. + +### Key changes are a hard stop + +The known_hosts model: banner, refuse, distinct exit code — not a y/n prompt +someone can fat-finger through. The only door is `--accept-key `; +making the human transcribe the new fingerprint is what proves they looked. + +Routine updates and identity changes are different signals and get different +ceremonies. "Upstream changed, here's the diff" is routine: default-proceed, +informational. "The signing identity changed" is how account compromise +presents: default-refuse. Flattening both into one always-maximum-caution +ceremony is how alarm fatigue happens. + +### No rotation chains + +Old-key-signs-new-key succession (signify/TUF style) was considered and +dropped. If every key change is loud and manual anyway, a valid rotation proof +can only *downgrade* the alarm — and a stolen key holds everything needed to +mint that proof, so chains sharpen theft (a thief can rotate the author out of +their own package) while buying convenience only for planned rotation. At this +scale, "key changed, human reads the diff carefully, re-pins" *is* the +rotation ceremony, and it covers loss and theft with the same motion. + +### Exit codes are the contract + +Callers — scripts and reviewing agents — build policy on exit codes, not on +prose. An agent's policy may auto-accept a clean diff under a continuous key; +an identity change must be unrepresentable in that path — no flag an agent +routinely passes gets through it. + +| Code | Meaning | +|------|------------------------------------------------------------| +| 0 | written (installed/adopted/updated/merged) or already current | +| 1 | conflict — original untouched, as ever | +| 2 | declined — user said no to the diff | +| 3 | identity change — signing key differs from the pin | + +### Review is the content check; the signature is the identity check + +A signature answers "same author?" and says nothing about whether the code is +safe. Review answers "does this change look sane?" and says nothing about who +made it. They are complements, and single files invert the economics that make +library review hopeless: the whole unit — old file, new file, three-way diff, +the project it lives in — fits in one reading (or one context window), and +with no install hooks, build steps, or dependencies, what you review is +exactly what runs. + +The gem's job is to surface facts, not to judge: emit the diff plus the +machine-readable facts (`--json` — key changed, size delta, e.g. "adds +net/http") and exit accordingly. Any LLM stays out of the gem. Note for +whoever builds the reviewing agent: the payload is adversarial input — +prompt injection in comments, aimed at the reviewer, is an expected attack. + +### Order of work + +1. Refuse plain `http`, including via redirect — cheap, orthogonal. +2. `source_url` pinning (see open question below) — makes the update diff a + pointer change and recovers merge bases on cache miss. +3. Diff-by-default + the exit-code contract. +4. SSH signing and pinning. + +## Open questions + +- **Immutable source pinning.** `url` is mutable ("whatever is at this address + today"), which means builds aren't reproducible and "upstream improved" is + indistinguishable from "upstream was compromised". Wants splitting into a + pinned `source_url` (gist revision SHA, commit SHA) and a mutable `update_url` + used only for checking. This also recovers merge bases on a cache miss. +- **Multi-file packages.** `Jmap::Client` + `Jmap::Http` are one unit of reuse + across two files. Either the spec grows a flat file-set, or single-file + packages accept that they define several classes in one file and lose Zeitwerk + file-per-class layout. Unresolved, and the first real consumer hits it. +- **Comment styles.** `#` covers Ruby/Python/shell/YAML/Perl. Needs a table for + `//`, `--`, ``, `/* */`, `;` before the format is honestly + language-agnostic. +- **Unknown metadata keys** are preserved verbatim on rewrite (see + `Metadata#extra`), so optional spec fields survive a tool that predates them. diff --git a/picopackage.gemspec b/picopackage.gemspec index 1fa37a4..2ee6c15 100644 --- a/picopackage.gemspec +++ b/picopackage.gemspec @@ -33,7 +33,7 @@ Gem::Specification.new do |spec| spec.executables = spec.files.grep(%r{\Aexe/}) { |f| File.basename(f) } spec.require_paths = ["lib"] - spec.add_dependency "open-uri", "~> 0.5" + # Fetching is done with net/http directly — see the note in Fetch#fetch_http. spec.add_dependency "yaml", "~> 0.4" spec.add_dependency "digest" spec.add_development_dependency "debug" diff --git a/test/files/gist_v1.json b/test/files/gist_v1.json new file mode 100644 index 0000000..8d4a085 --- /dev/null +++ b/test/files/gist_v1.json @@ -0,0 +1,88 @@ +{ + "url": "https://api.github.com/gists/a06926360de8edf108be8591368ce1fb", + "forks_url": "https://api.github.com/gists/a06926360de8edf108be8591368ce1fb/forks", + "commits_url": "https://api.github.com/gists/a06926360de8edf108be8591368ce1fb/commits", + "id": "a06926360de8edf108be8591368ce1fb", + "node_id": "G_kwDNRzjaACBhMDY5MjYzNjBkZThlZGYxMDhiZTg1OTEzNjhjZTFmYg", + "git_pull_url": "https://gist.github.com/a06926360de8edf108be8591368ce1fb.git", + "git_push_url": "https://gist.github.com/a06926360de8edf108be8591368ce1fb.git", + "html_url": "https://gist.github.com/dkam/a06926360de8edf108be8591368ce1fb", + "files": { + "ipv6_in_sqlite.rb": { + "filename": "ipv6_in_sqlite.rb", + "type": "application/x-ruby", + "language": "Ruby", + "raw_url": "https://gist.githubusercontent.com/dkam/a06926360de8edf108be8591368ce1fb/raw/13e47fb5d3c981a407997fc954b163fe50a37b45/ipv6_in_sqlite.rb", + "size": 373, + "truncated": false, + "content": "class IpAddress < ApplicationRecord\n def ipv6\n (ipv6_high << 64) | ipv6_low\n end\n\n def ipv6=(addr)\n self.ipv6_high = addr >> 64\n self.ipv6_low = addr & 0xFFFFFFFFFFFFFFFF\n end\n\n def self.ipv6_range(start_ip, end_ip)\n where(ipv6_high: start_ip >> 64..end_ip >> 64)\n .where(ipv6_low: start_ip & 0xFFFFFFFFFFFFFFFF..end_ip & 0xFFFFFFFFFFFFFFFF)\n end\nend", + "encoding": "utf-8" + } + }, + "public": false, + "created_at": "2025-01-25T00:07:09Z", + "updated_at": "2025-01-25T00:07:09Z", + "description": "Store IPv6 address in SQLite, via Active Record", + "comments": 0, + "user": null, + "comments_enabled": true, + "comments_url": "https://api.github.com/gists/a06926360de8edf108be8591368ce1fb/comments", + "owner": { + "login": "dkam", + "id": 18232, + "node_id": "MDQ6VXNlcjE4MjMy", + "avatar_url": "https://avatars.githubusercontent.com/u/18232?v=4", + "gravatar_id": "", + "url": "https://api.github.com/users/dkam", + "html_url": "https://github.com/dkam", + "followers_url": "https://api.github.com/users/dkam/followers", + "following_url": "https://api.github.com/users/dkam/following{/other_user}", + "gists_url": "https://api.github.com/users/dkam/gists{/gist_id}", + "starred_url": "https://api.github.com/users/dkam/starred{/owner}{/repo}", + "subscriptions_url": "https://api.github.com/users/dkam/subscriptions", + "organizations_url": "https://api.github.com/users/dkam/orgs", + "repos_url": "https://api.github.com/users/dkam/repos", + "events_url": "https://api.github.com/users/dkam/events{/privacy}", + "received_events_url": "https://api.github.com/users/dkam/received_events", + "type": "User", + "user_view_type": "public", + "site_admin": false + }, + "forks": [ + + ], + "history": [ + { + "user": { + "login": "dkam", + "id": 18232, + "node_id": "MDQ6VXNlcjE4MjMy", + "avatar_url": "https://avatars.githubusercontent.com/u/18232?v=4", + "gravatar_id": "", + "url": "https://api.github.com/users/dkam", + "html_url": "https://github.com/dkam", + "followers_url": "https://api.github.com/users/dkam/followers", + "following_url": "https://api.github.com/users/dkam/following{/other_user}", + "gists_url": "https://api.github.com/users/dkam/gists{/gist_id}", + "starred_url": "https://api.github.com/users/dkam/starred{/owner}{/repo}", + "subscriptions_url": "https://api.github.com/users/dkam/subscriptions", + "organizations_url": "https://api.github.com/users/dkam/orgs", + "repos_url": "https://api.github.com/users/dkam/repos", + "events_url": "https://api.github.com/users/dkam/events{/privacy}", + "received_events_url": "https://api.github.com/users/dkam/received_events", + "type": "User", + "user_view_type": "public", + "site_admin": false + }, + "version": "7f52a7373edd4cbf186162dc71444792eece75b3", + "committed_at": "2025-01-25T00:07:09Z", + "change_status": { + "total": 15, + "additions": 15, + "deletions": 0 + }, + "url": "https://api.github.com/gists/a06926360de8edf108be8591368ce1fb/7f52a7373edd4cbf186162dc71444792eece75b3" + } + ], + "truncated": false +} diff --git a/test/files/gist_v2.json b/test/files/gist_v2.json new file mode 100644 index 0000000..484dd75 --- /dev/null +++ b/test/files/gist_v2.json @@ -0,0 +1,119 @@ +{ + "url": "https://api.github.com/gists/a06926360de8edf108be8591368ce1fb", + "forks_url": "https://api.github.com/gists/a06926360de8edf108be8591368ce1fb/forks", + "commits_url": "https://api.github.com/gists/a06926360de8edf108be8591368ce1fb/commits", + "id": "a06926360de8edf108be8591368ce1fb", + "node_id": "G_kwDNRzjaACBhMDY5MjYzNjBkZThlZGYxMDhiZTg1OTEzNjhjZTFmYg", + "git_pull_url": "https://gist.github.com/a06926360de8edf108be8591368ce1fb.git", + "git_push_url": "https://gist.github.com/a06926360de8edf108be8591368ce1fb.git", + "html_url": "https://gist.github.com/dkam/a06926360de8edf108be8591368ce1fb", + "files": { + "ipv6_in_sqlite.rb": { + "filename": "ipv6_in_sqlite.rb", + "type": "application/x-ruby", + "language": "Ruby", + "raw_url": "https://gist.githubusercontent.com/dkam/a06926360de8edf108be8591368ce1fb/raw/458a32b55c57e42d6e541f29317e9445a87d4e43/ipv6_in_sqlite.rb", + "size": 384, + "truncated": false, + "content": "# Comments\nclass IpAddress < ApplicationRecord\n def ipv6\n (ipv6_high << 64) | ipv6_low\n end\n\n def ipv6=(addr)\n self.ipv6_high = addr >> 64\n self.ipv6_low = addr & 0xFFFFFFFFFFFFFFFF\n end\n\n def self.ipv6_range(start_ip, end_ip)\n where(ipv6_high: start_ip >> 64..end_ip >> 64)\n .where(ipv6_low: start_ip & 0xFFFFFFFFFFFFFFFF..end_ip & 0xFFFFFFFFFFFFFFFF)\n end\nend", + "encoding": "utf-8" + } + }, + "public": false, + "created_at": "2025-01-25T00:07:09Z", + "updated_at": "2025-01-25T00:08:48Z", + "description": "Store IPv6 address in SQLite, via Active Record", + "comments": 0, + "user": null, + "comments_enabled": true, + "comments_url": "https://api.github.com/gists/a06926360de8edf108be8591368ce1fb/comments", + "owner": { + "login": "dkam", + "id": 18232, + "node_id": "MDQ6VXNlcjE4MjMy", + "avatar_url": "https://avatars.githubusercontent.com/u/18232?v=4", + "gravatar_id": "", + "url": "https://api.github.com/users/dkam", + "html_url": "https://github.com/dkam", + "followers_url": "https://api.github.com/users/dkam/followers", + "following_url": "https://api.github.com/users/dkam/following{/other_user}", + "gists_url": "https://api.github.com/users/dkam/gists{/gist_id}", + "starred_url": "https://api.github.com/users/dkam/starred{/owner}{/repo}", + "subscriptions_url": "https://api.github.com/users/dkam/subscriptions", + "organizations_url": "https://api.github.com/users/dkam/orgs", + "repos_url": "https://api.github.com/users/dkam/repos", + "events_url": "https://api.github.com/users/dkam/events{/privacy}", + "received_events_url": "https://api.github.com/users/dkam/received_events", + "type": "User", + "user_view_type": "public", + "site_admin": false + }, + "forks": [ + + ], + "history": [ + { + "user": { + "login": "dkam", + "id": 18232, + "node_id": "MDQ6VXNlcjE4MjMy", + "avatar_url": "https://avatars.githubusercontent.com/u/18232?v=4", + "gravatar_id": "", + "url": "https://api.github.com/users/dkam", + "html_url": "https://github.com/dkam", + "followers_url": "https://api.github.com/users/dkam/followers", + "following_url": "https://api.github.com/users/dkam/following{/other_user}", + "gists_url": "https://api.github.com/users/dkam/gists{/gist_id}", + "starred_url": "https://api.github.com/users/dkam/starred{/owner}{/repo}", + "subscriptions_url": "https://api.github.com/users/dkam/subscriptions", + "organizations_url": "https://api.github.com/users/dkam/orgs", + "repos_url": "https://api.github.com/users/dkam/repos", + "events_url": "https://api.github.com/users/dkam/events{/privacy}", + "received_events_url": "https://api.github.com/users/dkam/received_events", + "type": "User", + "user_view_type": "public", + "site_admin": false + }, + "version": "9774aa95cbf7c1fa7bf2ea27b2cb66898d0769c8", + "committed_at": "2025-01-25T00:08:48Z", + "change_status": { + "total": 1, + "additions": 1, + "deletions": 0 + }, + "url": "https://api.github.com/gists/a06926360de8edf108be8591368ce1fb/9774aa95cbf7c1fa7bf2ea27b2cb66898d0769c8" + }, + { + "user": { + "login": "dkam", + "id": 18232, + "node_id": "MDQ6VXNlcjE4MjMy", + "avatar_url": "https://avatars.githubusercontent.com/u/18232?v=4", + "gravatar_id": "", + "url": "https://api.github.com/users/dkam", + "html_url": "https://github.com/dkam", + "followers_url": "https://api.github.com/users/dkam/followers", + "following_url": "https://api.github.com/users/dkam/following{/other_user}", + "gists_url": "https://api.github.com/users/dkam/gists{/gist_id}", + "starred_url": "https://api.github.com/users/dkam/starred{/owner}{/repo}", + "subscriptions_url": "https://api.github.com/users/dkam/subscriptions", + "organizations_url": "https://api.github.com/users/dkam/orgs", + "repos_url": "https://api.github.com/users/dkam/repos", + "events_url": "https://api.github.com/users/dkam/events{/privacy}", + "received_events_url": "https://api.github.com/users/dkam/received_events", + "type": "User", + "user_view_type": "public", + "site_admin": false + }, + "version": "7f52a7373edd4cbf186162dc71444792eece75b3", + "committed_at": "2025-01-25T00:07:09Z", + "change_status": { + "total": 15, + "additions": 15, + "deletions": 0 + }, + "url": "https://api.github.com/gists/a06926360de8edf108be8591368ce1fb/7f52a7373edd4cbf186162dc71444792eece75b3" + } + ], + "truncated": false +} diff --git a/test/files/uniquify_array_packaged.rb b/test/files/uniquify_array_packaged.rb new file mode 100644 index 0000000..860880e --- /dev/null +++ b/test/files/uniquify_array_packaged.rb @@ -0,0 +1,18 @@ +# Make an array of hashes contain a unique value for key: +def uniquify_array(array, key) + array.group_by { |item| item[key] }.map do |_, items| + (items.length > 1) ? yield(items) : items.first + end +end + +# new_array = uniquify_array(array, :some_key) do |dupes| +# dupes.max_by { |item| item[:updated_at] } +# end + +# @PICOPACKAGE_START +# --- +# filename: uniquify_array.rb +# url: https://api.github.com/gists/525ac4177964549ee2f2ca7febd03eea +# version: 0.1 +# source_url: https://gist.github.com/dkam/525ac4177964549ee2f2ca7febd03eea +# @PICOPACKAGE_END diff --git a/test/test_cache.rb b/test/test_cache.rb new file mode 100644 index 0000000..2b74acc --- /dev/null +++ b/test/test_cache.rb @@ -0,0 +1,36 @@ +# frozen_string_literal: true + +require "test_helper" + +class TestCache < Minitest::Test + def with_cache + Dir.mktmpdir("picopackage-cache") { |root| yield Picopackage::Cache.new(root: root) } + end + + def test_stores_and_fetches_by_content + with_cache do |cache| + checksum = cache.store("hello\n") + assert_equal Picopackage::Payload.checksum("hello\n"), checksum + assert_equal "hello\n", cache.fetch(checksum) + end + end + + def test_storing_twice_is_harmless + with_cache do |cache| + a = cache.store("hello\n") + b = cache.store("hello\n") + assert_equal a, b + assert_equal "hello\n", cache.fetch(a) + end + end + + def test_unknown_and_malformed_checksums_return_nil + with_cache do |cache| + assert_nil cache.fetch(Picopackage::Payload.checksum("never stored")) + assert_nil cache.fetch(nil) + assert_nil cache.fetch("") + assert_nil cache.fetch("sha256:../../escape") + refute cache.include?(nil) + end + end +end diff --git a/test/test_helper.rb b/test/test_helper.rb index 168e2c4..d79a3ac 100644 --- a/test/test_helper.rb +++ b/test/test_helper.rb @@ -4,3 +4,29 @@ $LOAD_PATH.unshift File.expand_path("../lib", __dir__) require "picopackage" require "minitest/autorun" +require "tmpdir" + +module PicopackageTest + # Every test gets its own cache root: the merge base cache is global state and + # a suite that shared one would pass or fail depending on what ran before it. + def with_dirs + Dir.mktmpdir("picopackage-test") do |root| + upstream = File.join(root, "upstream") + project = File.join(root, "project") + [upstream, project].each { |dir| Dir.mkdir(dir) } + yield(upstream, project, Picopackage::Cache.new(root: File.join(root, "cache"))) + end + end + + # Stands in for a server: a real file, fetched through the real FileProvider, + # so provider metadata population is exercised rather than faked. + def publish(dir, name, content) + path = File.join(dir, name) + File.write(path, content) + path + end + + def upstream_package(path) + Picopackage::FileProvider.new(path).package + end +end diff --git a/test/test_merge.rb b/test/test_merge.rb new file mode 100644 index 0000000..aa45fe5 --- /dev/null +++ b/test/test_merge.rb @@ -0,0 +1,55 @@ +# frozen_string_literal: true + +require "test_helper" + +class TestMerge < Minitest::Test + BASE = "one\ntwo\nthree\n" + + def test_a_backend_is_available + assert Picopackage::Merge.available?, "expected git or diff3 on PATH" + end + + def test_non_overlapping_changes_merge_cleanly + result = Picopackage::Merge.three_way( + base: BASE, + ours: "ONE\ntwo\nthree\n", + theirs: "one\ntwo\nTHREE\n" + ) + + assert_predicate result, :clean? + assert_equal "ONE\ntwo\nTHREE\n", result.content + end + + def test_identical_changes_on_both_sides_merge_cleanly + changed = "one\nTWO\nthree\n" + result = Picopackage::Merge.three_way(base: BASE, ours: changed, theirs: changed) + + assert_predicate result, :clean? + assert_equal changed, result.content + end + + def test_overlapping_changes_conflict + result = Picopackage::Merge.three_way( + base: BASE, + ours: "one\nMINE\nthree\n", + theirs: "one\nTHEIRS\nthree\n" + ) + + assert_predicate result, :conflict? + assert_operator result.conflicts, :>, 0 + assert_includes result.content, "MINE" + assert_includes result.content, "THEIRS" + assert_includes result.content, "<<<<<<<" + end + + def test_labels_appear_in_conflict_markers + result = Picopackage::Merge.three_way( + base: BASE, + ours: "one\nMINE\nthree\n", + theirs: "one\nTHEIRS\nthree\n" + ) + + assert_includes result.content, "local" + assert_includes result.content, "upstream" + end +end diff --git a/test/test_package.rb b/test/test_package.rb new file mode 100644 index 0000000..1ff0d8d --- /dev/null +++ b/test/test_package.rb @@ -0,0 +1,106 @@ +# frozen_string_literal: true + +require "test_helper" + +class TestPackage < Minitest::Test + include PicopackageTest + + BARE = "def hello\n :world\nend\n" + + def test_it_has_a_version_number + refute_nil ::Picopackage::VERSION + end + + def test_a_bare_file_is_bare + assert Picopackage::Package.new(content: BARE).bare? + end + + def test_a_generated_package_parses_back_as_a_package + package = Picopackage::Package.new(content: BARE).init_metadata(filename: "hello.rb") + round_tripped = Picopackage::Package.new(content: package.generate_package) + + refute_predicate round_tripped, :bare? + assert_equal "hello.rb", round_tripped.filename + end + + def test_reads_metadata_from_a_packaged_file + package = Picopackage::Package.from_file("test/files/uniquify_array_packaged.rb") + assert_equal "uniquify_array.rb", package.filename + assert_equal "https://api.github.com/gists/525ac4177964549ee2f2ca7febd03eea", package.url + end + + # This was the bug that made every file look locally modified: the checksum + # written into a package hashed the un-normalised payload while verification + # hashed the normalised one, so nothing ever verified against itself. + def test_a_freshly_packaged_file_verifies_against_its_own_checksum + package = Picopackage::Package.new(content: BARE).init_metadata(filename: "hello.rb") + written = Picopackage::Package.new(content: package.generate_package) + + assert written.verify_payload, "a package should satisfy its own checksum" + refute_predicate written, :modified? + end + + def test_payload_digest_is_stable_across_packaging + bare = Picopackage::Package.new(content: BARE) + packaged = Picopackage::Package.new( + content: Picopackage::Package.new(content: BARE).init_metadata(filename: "hello.rb").generate_package + ) + + assert_equal bare.payload_digest, packaged.payload_digest, + "adding a metadata block must not change the payload digest" + end + + def test_editing_a_payload_is_detected + packaged = Picopackage::Package.new(content: BARE).init_metadata(filename: "hello.rb").generate_package + edited = Picopackage::Package.new(content: packaged.sub(":world", ":everyone")) + + assert_predicate edited, :modified? + end + + def test_unmodelled_metadata_keys_survive_a_rewrite + content = <<~RUBY + code_here + + # @PICOPACKAGE_START + # --- + # filename: thing.rb + # licence: MIT + # test_url: https://example.com/thing_test.rb + # @PICOPACKAGE_END + RUBY + + package = Picopackage::Package.new(content: content) + assert_equal "MIT", package.metadata.extra["licence"] + + rewritten = Picopackage::Package.new(content: package.generate_package) + assert_equal "MIT", rewritten.metadata.extra["licence"] + assert_equal "https://example.com/thing_test.rb", rewritten.metadata.extra["test_url"] + end + + # A checksum is derived, never preserved. An author who edits a packaged file + # by hand and re-runs the tool must get a block describing what is now there, + # not the value that happened to already be sitting in it. + def test_a_stale_recorded_checksum_is_corrected_on_write + content = "code\n\n# @PICOPACKAGE_START\n# ---\n# filename: thing.rb\n" \ + "# payload_checksum: sha256:#{"de" * 32}\n# @PICOPACKAGE_END\n" + + stale = Picopackage::Package.new(content: content) + refute stale.verify_payload, "precondition: the recorded checksum is wrong" + + rewritten = Picopackage::Package.new(content: stale.generate_package) + assert rewritten.verify_payload + refute_predicate rewritten, :modified? + end + + def test_a_block_of_only_unmodelled_keys_is_not_treated_as_bare + content = "code\n\n# @PICOPACKAGE_START\n# ---\n# licence: MIT\n# @PICOPACKAGE_END\n" + refute_predicate Picopackage::Package.new(content: content), :bare?, + "a block we don't fully understand is still a block" + end + + def test_malformed_front_matter_does_not_raise + package = Picopackage::Package.from_file("test/files/broken_front_matter_1.rb") + refute_nil package + assert_predicate package.metadata, :empty? + end +end diff --git a/test/test_picopackage.rb b/test/test_picopackage.rb deleted file mode 100644 index 57fc9fa..0000000 --- a/test/test_picopackage.rb +++ /dev/null @@ -1,19 +0,0 @@ -# frozen_string_literal: true - -require "test_helper" - -class TestPicopackage < Minitest::Test - def test_that_it_has_a_version_number - refute_nil ::Picopackage::VERSION - end - - def test_it_can_load_a_picopackage_file - sf = Picopackage::FileProvider.new(File.read("test/files/uniquify_array_packaged.rb")) - assert_equal "uniquify_array_packaged.rb", sf.metadata.filename - end - - def test_it_can_create_a_picopackage_from_bare_file - sf = Picopackage::FileProvider.new(File.read("test/files/uniquify_array_bare.rb")) - assert_equal "uniquify_array_bare.rb", sf.metadata.filename - end -end diff --git a/test/test_provider.rb b/test/test_provider.rb new file mode 100644 index 0000000..82f704d --- /dev/null +++ b/test/test_provider.rb @@ -0,0 +1,69 @@ +# frozen_string_literal: true + +require "test_helper" + +# Two captured responses for the same gist, one revision apart: v2 prepends a +# `# Comments` line and carries a later `updated_at`. Real API shapes, no +# network, and enough of a difference to drive an update through the resolver. +class TestGithubGistProvider < Minitest::Test + include PicopackageTest + + GIST_URL = "https://gist.github.com/dkam/a06926360de8edf108be8591368ce1fb" + + StubFetcher = Struct.new(:body) do + def fetch(_uri) = body + end + + def gist(revision) + Picopackage::GithubGistProvider.new( + GIST_URL, fetcher: StubFetcher.new(File.read("test/files/gist_#{revision}.json")) + ) + end + + def test_resolves_the_page_url_to_the_api + assert_equal "https://api.github.com/gists/a06926360de8edf108be8591368ce1fb", + gist("v1").url.to_s + end + + def test_extracts_the_file_from_the_json_envelope + package = gist("v1").package + + assert_equal "ipv6_in_sqlite.rb", package.filename + assert_includes package.payload, "def ipv6" + refute_includes package.payload, "raw_url", "the JSON envelope must not leak into the payload" + assert_predicate package, :bare?, "the gist carries no metadata block of its own" + end + + # The gist's own revision time, not the moment we happened to fetch it. + def test_uses_the_gists_updated_at_as_the_timestamp + assert_equal "2025-01-25T00:07:09Z", gist("v1").package.payload_timestamp + assert_equal "2025-01-25T00:08:48Z", gist("v2").package.payload_timestamp + end + + def test_a_new_revision_is_a_different_payload + refute_equal gist("v1").package.payload_digest, gist("v2").package.payload_digest + end + + # End to end against a real upstream shape: install v1, edit it, take v2. + def test_updating_across_revisions_keeps_local_edits + with_dirs do |_upstream, project, cache| + local_path = File.join(project, "ipv6_in_sqlite.rb") + assert_equal :installed, resolve(gist("v1").package, nil, local_path, cache).state + + File.write(local_path, File.read(local_path).sub("where(ipv6_high:", "where!(ipv6_high:")) + result = resolve(gist("v2").package, Picopackage::Package.from_file(local_path), local_path, cache) + + assert_equal :merged, result.state, result.message + merged = File.read(local_path) + assert_includes merged, "where!(ipv6_high:", "local edit should survive" + assert_includes merged, "# Comments", "the new revision should be applied" + refute_includes merged, "<<<<<<<" + end + end + + private + + def resolve(remote, local, path, cache) + Picopackage::Resolver.new(remote, local, path, cache: cache).resolve + end +end diff --git a/test/test_resolver.rb b/test/test_resolver.rb new file mode 100644 index 0000000..efbf6c0 --- /dev/null +++ b/test/test_resolver.rb @@ -0,0 +1,340 @@ +# frozen_string_literal: true + +require "test_helper" + +class TestResolver < Minitest::Test + include PicopackageTest + + V1 = <<~RUBY + module Poller + INTERVAL = 30 + + def self.run + fetch + end + end + RUBY + + # Upstream carrying its own block is optional, not required — but when it does, + # only some of the block is upstream's to assert. This one is stale, as any + # author who edits without re-running the tool will ship. + PACKAGED = V1 + <<~RUBY + # @PICOPACKAGE_START + # --- + # url: https://example.com/canonical/poller.rb + # filename: upstream_name.rb + # payload_version: 1.4.0 + # payload_checksum: sha256:#{"de" * 32} + # licence: MIT + # @PICOPACKAGE_END + RUBY + + def resolve(upstream_path, project, cache, force: false, filename: nil) + Picopackage::Fetch.fetch(upstream_path, project, force: force, filename: filename, cache: cache) + end + + def test_installs_when_nothing_is_there + with_dirs do |upstream, project, cache| + path = publish(upstream, "poller.rb", V1) + result = resolve(path, project, cache) + + assert_equal :installed, result.state + installed = Picopackage::Package.from_file(File.join(project, "poller.rb")) + refute_predicate installed, :bare? + assert_equal path, installed.url + assert installed.verify_payload + end + end + + # A bare upstream file is a valid picopackage source: nobody has to adopt the + # format for their code to be installable. Everything the block would have + # said gets derived from the fetch instead. + def test_installs_from_an_upstream_with_no_metadata_block + with_dirs do |upstream, project, cache| + path = publish(upstream, "poller.rb", V1) + assert_predicate upstream_package(path), :bare?, "precondition: upstream carries no block" + + assert_equal :installed, resolve(path, project, cache).state + installed = Picopackage::Package.from_file(File.join(project, "poller.rb")) + assert installed.verify_payload + assert_equal path, installed.url + end + end + + def test_publisher_claims_are_adopted_and_local_records_are_recomputed + with_dirs do |upstream, project, cache| + path = publish(upstream, "poller.rb", PACKAGED) + + assert_equal :installed, resolve(path, project, cache).state + + # Claims: upstream's, and worth more than anything we could infer. The + # declared filename beats the url's basename, so the file lands as the + # author named it rather than as whatever the mirror called it. + local_path = File.join(project, "upstream_name.rb") + assert_path_exists local_path + installed = Picopackage::Package.from_file(local_path) + assert_equal "https://example.com/canonical/poller.rb", installed.url, + "the canonical url should outlive the mirror we fetched from" + assert_equal "1.4.0", installed.payload_version + assert_equal "MIT", installed.metadata.extra["licence"] + + # Records: ours, describing this copy on this disk. + assert installed.verify_payload, "a freshly installed file must verify, whatever upstream claimed" + refute_predicate installed, :modified? + refute_predicate installed, :diverged? + assert_equal "upstream_name.rb", installed.filename, "filename records where it actually landed" + end + end + + def test_a_stale_upstream_checksum_does_not_break_a_later_merge + with_dirs do |upstream, project, cache| + path = publish(upstream, "poller.rb", PACKAGED) + resolve(path, project, cache) + local_path = File.join(project, "upstream_name.rb") + + File.write(local_path, File.read(local_path).sub("INTERVAL = 30", "INTERVAL = 5")) + File.write(path, PACKAGED.sub(" fetch\n", " fetch\n prune\n")) + + result = resolve(path, project, cache) + + assert_equal :merged, result.state, result.message + merged = File.read(local_path) + assert_includes merged, "INTERVAL = 5" + assert_includes merged, "prune" + assert_equal "MIT", Picopackage::Package.from_file(local_path).metadata.extra["licence"] + end + end + + def test_reinstalling_the_same_thing_is_a_no_op + with_dirs do |upstream, project, cache| + path = publish(upstream, "poller.rb", V1) + resolve(path, project, cache) + before = File.read(File.join(project, "poller.rb")) + + result = resolve(path, project, cache) + + assert_equal :current, result.state + assert_equal before, File.read(File.join(project, "poller.rb")), "an up-to-date file should not be rewritten" + end + end + + # The pleasant case: a file you pasted in by hand months ago gets its + # provenance attached without its content being touched. + def test_adopts_a_bare_local_file_with_identical_content + with_dirs do |upstream, project, cache| + path = publish(upstream, "poller.rb", V1) + File.write(File.join(project, "poller.rb"), V1) + + result = resolve(path, project, cache) + + assert_equal :adopted, result.state + adopted = Picopackage::Package.from_file(File.join(project, "poller.rb")) + refute_predicate adopted, :bare? + assert_equal Picopackage::Payload.normalize(V1), adopted.payload + end + end + + def test_fast_forwards_an_unmodified_package + with_dirs do |upstream, project, cache| + path = publish(upstream, "poller.rb", V1) + resolve(path, project, cache) + + File.write(path, V1.sub("INTERVAL = 30", "INTERVAL = 60")) + result = resolve(path, project, cache) + + assert_equal :updated, result.state + assert_includes File.read(File.join(project, "poller.rb")), "INTERVAL = 60" + end + end + + # The headline capability: local edits and upstream edits both survive. + def test_merges_local_edits_with_upstream_changes + with_dirs do |upstream, project, cache| + path = publish(upstream, "poller.rb", V1) + resolve(path, project, cache) + + local_path = File.join(project, "poller.rb") + File.write(local_path, File.read(local_path).sub("INTERVAL = 30", "INTERVAL = 5 # tuned for us")) + + File.write(path, V1.sub(" fetch\n", " fetch\n prune\n")) + result = resolve(path, project, cache) + + assert_equal :merged, result.state, result.message + merged = File.read(local_path) + assert_includes merged, "INTERVAL = 5 # tuned for us", "local edit should survive" + assert_includes merged, "prune", "upstream change should be applied" + refute_includes merged, "<<<<<<<" + + package = Picopackage::Package.from_file(local_path) + assert package.verify_payload, "a merged file should verify against its own recorded checksum" + refute_equal package.payload_checksum, package.base_checksum, + "base_checksum should still point at upstream, not at the merged result" + end + end + + # A merged file must remain updatable: the base pointer has to advance to the + # upstream payload, not to the merge result, or the next merge replays old + # upstream changes as conflicts. + def test_a_merged_file_can_be_updated_again + with_dirs do |upstream, project, cache| + path = publish(upstream, "poller.rb", V1) + resolve(path, project, cache) + local_path = File.join(project, "poller.rb") + + File.write(local_path, File.read(local_path).sub("INTERVAL = 30", "INTERVAL = 5")) + File.write(path, V1.sub(" fetch\n", " fetch\n prune\n")) + assert_equal :merged, resolve(path, project, cache).state + + File.write(path, V1.sub(" fetch\n", " fetch\n prune\n report\n")) + result = resolve(path, project, cache) + + assert_equal :merged, result.state, result.message + merged = File.read(local_path) + assert_includes merged, "INTERVAL = 5" + assert_includes merged, "report" + refute_includes merged, "<<<<<<<" + end + end + + # Editing locally while upstream stands still is the common case, and there is + # nothing to merge: upstream is still the ancestor we branched from. + def test_local_edits_with_no_upstream_change_are_left_alone + with_dirs do |upstream, project, cache| + path = publish(upstream, "poller.rb", V1) + resolve(path, project, cache) + local_path = File.join(project, "poller.rb") + + File.write(local_path, File.read(local_path).sub("INTERVAL = 30", "INTERVAL = 5")) + before = File.read(local_path) + + result = resolve(path, project, cache) + + assert_equal :current, result.state, result.message + assert_equal before, File.read(local_path), "an edited file with no upstream change must not be rewritten" + end + end + + # Repopulating the merge base from an unchanged upstream is the whole reason + # that branch still touches the cache. + def test_an_unchanged_upstream_restores_a_lost_merge_base + with_dirs do |upstream, project, cache| + path = publish(upstream, "poller.rb", V1) + resolve(path, project, cache) + local_path = File.join(project, "poller.rb") + File.write(local_path, File.read(local_path).sub("INTERVAL = 30", "INTERVAL = 5")) + + # A fresh machine, or a cleared cache: the ancestor is gone. + empty = Picopackage::Cache.new(root: File.join(Dir.mktmpdir, "empty")) + assert_equal :current, resolve(path, project, empty).state + + # Upstream now moves. The base recovered above makes this mergeable. + File.write(path, V1.sub(" fetch\n", " fetch\n prune\n")) + result = resolve(path, project, empty) + + assert_equal :merged, result.state, result.message + assert_includes File.read(local_path), "INTERVAL = 5" + assert_includes File.read(local_path), "prune" + end + end + + def test_conflicting_edits_leave_the_original_alone + with_dirs do |upstream, project, cache| + path = publish(upstream, "poller.rb", V1) + resolve(path, project, cache) + local_path = File.join(project, "poller.rb") + + File.write(local_path, File.read(local_path).sub("INTERVAL = 30", "INTERVAL = 5")) + before = File.read(local_path) + + File.write(path, V1.sub("INTERVAL = 30", "INTERVAL = 90")) + result = resolve(path, project, cache) + + assert_equal :conflict, result.state + assert_equal before, File.read(local_path), "the user's file must not be touched" + + markers = local_path + Picopackage::Resolver::MERGE_SUFFIX + assert_path_exists markers + assert_includes File.read(markers), "<<<<<<<" + end + end + + def test_force_discards_local_changes + with_dirs do |upstream, project, cache| + path = publish(upstream, "poller.rb", V1) + resolve(path, project, cache) + local_path = File.join(project, "poller.rb") + + File.write(local_path, File.read(local_path).sub("INTERVAL = 30", "INTERVAL = 5")) + File.write(path, V1.sub("INTERVAL = 30", "INTERVAL = 90")) + + result = resolve(path, project, cache, force: true) + + assert_equal :updated, result.state + assert_includes File.read(local_path), "INTERVAL = 90" + refute_includes File.read(local_path), "INTERVAL = 5" + end + end + + def test_an_unrelated_bare_file_with_the_same_name_is_refused + with_dirs do |upstream, project, cache| + path = publish(upstream, "poller.rb", V1) + File.write(File.join(project, "poller.rb"), "# something else entirely\n") + + result = resolve(path, project, cache) + + assert_equal :conflict, result.state + assert_equal "# something else entirely\n", File.read(File.join(project, "poller.rb")) + end + end + + def test_a_modified_file_with_no_cached_ancestor_is_refused + with_dirs do |upstream, project, cache| + path = publish(upstream, "poller.rb", V1) + resolve(path, project, cache) + local_path = File.join(project, "poller.rb") + File.write(local_path, File.read(local_path).sub("INTERVAL = 30", "INTERVAL = 5")) + File.write(path, V1.sub(" fetch\n", " fetch\n prune\n")) + + empty_cache = Picopackage::Cache.new(root: File.join(Dir.mktmpdir, "empty")) + result = resolve(path, project, empty_cache) + + assert_equal :conflict, result.state + assert_match(/cache/, result.message) + end + end + + # Renaming a package locally is allowed; updating it must not install a second + # copy under upstream's preferred name. + def test_update_writes_back_to_the_local_filename + with_dirs do |upstream, project, cache| + path = publish(upstream, "poller.rb", V1) + resolve(path, project, cache) + FileUtils.mv(File.join(project, "poller.rb"), File.join(project, "my_poller.rb")) + + File.write(path, V1.sub("INTERVAL = 30", "INTERVAL = 60")) + result = resolve(path, project, cache, filename: "my_poller.rb") + + assert_equal :updated, result.state + refute_path_exists File.join(project, "poller.rb"), "should not have created a second copy" + assert_includes File.read(File.join(project, "my_poller.rb")), "INTERVAL = 60" + assert_equal "my_poller.rb", Picopackage::Package.from_file(File.join(project, "my_poller.rb")).filename + end + end + + def test_timestamps_never_override_the_checksum + with_dirs do |upstream, project, cache| + path = publish(upstream, "poller.rb", V1) + resolve(path, project, cache) + local_path = File.join(project, "poller.rb") + before = File.read(local_path) + + # Same bytes, much newer mtime — the old timestamp-driven logic called + # this an update and rewrote the file. + FileUtils.touch(path, mtime: Time.now + 86_400) + result = resolve(path, project, cache) + + assert_equal :current, result.state + assert_equal before, File.read(local_path) + end + end +end