Version 0.3.0: resolver/merge/cache refactor, signing design notes
- Split the monolith into package/provider/fetch/merge/cache modules with the Resolver deciding install/adopt/update/merge/conflict outcomes - Three-way merges via git merge-file/diff3 against a content-addressed merge-base cache; conflicts go to a .picopackage-merge sibling - Tests for package, provider, resolver, merge, and cache - notes.md: update UX and signing design — diff-by-default, SSH signature identity pinning (TOFU), key changes as a hard stop, exit-code contract - Add CLAUDE.md; remove the pre-refactor exe/pppkg monolith and scratch files Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EiyJC73Mz8xZyCTvCEY8qn
This commit is contained in:
Dan Milne
co-authored by
Claude Fable 5
Claude Fable 5
parent
e0cd0f0d7a
commit
6dd57e84f1
+10
@@ -6,3 +6,13 @@
|
||||
/pkg/
|
||||
/spec/reports/
|
||||
/tmp/
|
||||
*.gem
|
||||
|
||||
# Editors and OS
|
||||
*.swp
|
||||
*.swo
|
||||
*~
|
||||
.DS_Store
|
||||
|
||||
# Scratch: picopackages installed while trying the tool out on this repo
|
||||
*.bkup
|
||||
|
||||
@@ -0,0 +1,9 @@
|
||||
# standardrb reads this, not .rubocop.yml — keep the two in step.
|
||||
ruby_version: 3.1
|
||||
|
||||
ignore:
|
||||
# Fixtures are deliberately malformed or copied verbatim from elsewhere;
|
||||
# linting them defeats the point of having them.
|
||||
- "test/files/**/*"
|
||||
# The pre-refactor monolith, kept only for reference.
|
||||
- "exe/pppkg"
|
||||
@@ -1,5 +1,55 @@
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
|
||||
- Three-way merge on update, so a locally edited picopackage can still take
|
||||
upstream changes. Conflicts are written to `<file>.picopackage-merge` and the
|
||||
original file is never touched.
|
||||
- `base_checksum` metadata: the upstream payload a file was last reconciled
|
||||
with. Doubles as the key into a content-addressed merge-base cache under
|
||||
`$XDG_CACHE_HOME/picopackage` (`PICOPACKAGE_CACHE` overrides).
|
||||
- Unrecognised metadata keys are preserved across rewrites instead of being
|
||||
dropped.
|
||||
- An upstream metadata block is explicitly optional. Any plain file is a valid
|
||||
source; a block lets a publisher assert a canonical `url`, `filename`,
|
||||
`payload_version` and licence, which are adopted as given.
|
||||
- `ppkg verify`, `ppkg package --url`, `ppkg scan` output with urls, and
|
||||
`ppkg help`.
|
||||
|
||||
### Changed
|
||||
|
||||
- Resolution is decided by checksum, never by mtime. A file whose payload
|
||||
matches upstream is up to date however the timestamps compare.
|
||||
- Merges run on the payload only; the metadata block is regenerated afterwards
|
||||
rather than merged.
|
||||
- A locally edited file whose upstream hasn't moved reports `current` instead of
|
||||
running a no-op merge. It also restores the merge base to the cache, so a
|
||||
cleared cache recovers on the next `update` rather than at the next conflict.
|
||||
- Fetching uses `net/http` directly instead of `URI.open`, which treats a
|
||||
leading `|` as a command to run. Redirects (max 5, `http`/`https` only),
|
||||
`If-None-Match`, streamed size caps and typed errors come with it.
|
||||
- `update` writes back to the local filename rather than upstream's preferred
|
||||
one, so a renamed package doesn't install a second copy.
|
||||
- Timestamps are ISO 8601 from every provider.
|
||||
|
||||
### Fixed
|
||||
|
||||
- Checksums are computed over the normalised payload everywhere. Written
|
||||
packages previously failed their own verification, so every file looked
|
||||
locally modified.
|
||||
- Provider selection no longer aborts when a speculative provider guesses wrong,
|
||||
so plain URLs reach `DefaultProvider`. A 404 now reports as a 404 rather than
|
||||
"no provider could handle this".
|
||||
- `Scanner` returns usable results and survives binary, unreadable and oversized
|
||||
files.
|
||||
- `ppkg verify` reads the checksum it actually writes; `ppkg package` saves.
|
||||
- A metadata block containing only unmodelled keys is no longer treated as a
|
||||
bare file and overwritten.
|
||||
- Checksums are derived at write time, never adopted from an input. Installing
|
||||
from an upstream whose block carried a stale `payload_checksum` produced a
|
||||
file that failed its own verification and reported as locally modified before
|
||||
anyone had edited it.
|
||||
|
||||
## [0.2.0] - 2025-01-21
|
||||
|
||||
- Rename to from Picop to Picopackage
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## What this is
|
||||
|
||||
A Ruby gem (`picopackage`, CLI `ppkg`) for installing and updating single-file packages: ordinary source files that carry their own provenance in a commented YAML block at the end (`# @PICOPACKAGE_START` … `# @PICOPACKAGE_END`). See README.md for the format and notes.md for the resolution rules and open design questions.
|
||||
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
bin/setup # install dependencies
|
||||
rake # tests + StandardRB (the default task)
|
||||
rake test # tests only
|
||||
rake test N=/pattern/ # run tests whose names match a pattern
|
||||
ruby -Itest test/test_merge.rb # run a single test file
|
||||
bundle exec standardrb # lint (config in .standard.yml, not .rubocop.yml)
|
||||
bundle exec standardrb --fix
|
||||
```
|
||||
|
||||
Releasing: bump `lib/picopackage/version.rb`, then `bundle exec rake release`.
|
||||
|
||||
## Architecture
|
||||
|
||||
Everything lives under `lib/picopackage/`; `lib/picopackage.rb` just requires the pieces and defines the shared error classes.
|
||||
|
||||
The flow of an `install`/`update` is: **CLI → Fetch.fetch → Provider → Resolver → (Merge, Cache) → Package#save**.
|
||||
|
||||
- **package.rb** — the data model. `Package` wraps a file's content and splits it into `Payload` (the code, normalized to `rstrip + "\n\n"`) and `Metadata` (a Struct parsed from the YAML block). `Metadata#extra` round-trips unknown keys so old tools can't corrupt packages written by newer ones. Key distinction: `modified?` (payload differs from its recorded `payload_checksum` — a human edited it) vs `diverged?` (payload differs from `base_checksum`, the upstream ancestor — true after a merge even when `modified?` is false); `diverged?` is what decides whether a fast-forward is safe.
|
||||
- **provider.rb** — resolves a URL to a fetch strategy. `Provider.for` walks `PROVIDERS` (FileProvider, GithubGistProvider, OpenGistProvider, DefaultProvider); `handles_url?` returns true/false/`:maybe`, and `:maybe` providers prove themselves by fetching (they fetch in their constructor). Upstream metadata *claims* (url, version, licence) are adopted with `||=`; *records* (checksums) are always recomputed from the payload actually received.
|
||||
- **fetch.rb** — two things: the HTTP fetcher (`Fetch`: 1 MB cap, 10s timeout, ≤5 redirects to http/https only, ETag support; deliberately `Net::HTTP`, never `URI.open`) and the `Resolver`, which contains all install/update decision logic and returns a `Result` with a state (`installed/adopted/current/updated/merged/conflict`) — deciding what to do about a conflict is the CLI's job.
|
||||
- **merge.rb** — three-way merge shelled out to `git merge-file` or `diff3`; no Ruby reimplementation.
|
||||
- **cache.rb** — content-addressed store of upstream payloads under `$XDG_CACHE_HOME/picopackage` (override: `PICOPACKAGE_CACHE`), keyed by checksum. This is where merge bases come from; a cache miss means the merge is refused, not guessed.
|
||||
- **cli.rb** — argument parsing and reporting only; commands are `install update package verify inspect scan`.
|
||||
|
||||
## Invariants (violating these is a bug, not a style choice)
|
||||
|
||||
1. **The checksum decides, never the timestamp.** mtime is not evidence about content; equal payload checksums settle the content question regardless of timestamps.
|
||||
2. **Merges run on the payload alone; the metadata block is derived, never merged.** It is regenerated on every write — `Package#generate_package` is the single choke point where `payload_checksum` is computed, so a stale checksum can never survive into a written file.
|
||||
3. **Conflicts are non-destructive.** The user's file is never touched; conflict markers go to a sibling `<file>.picopackage-merge`.
|
||||
4. A bare local file whose payload differs from upstream is an *unrelated file*, not a modified package — refuse, don't merge.
|
||||
|
||||
## Tests
|
||||
|
||||
Minitest, in `test/test_*.rb`. `test/test_helper.rb` provides `PicopackageTest#with_dirs`, which gives every test its own upstream dir, project dir, and cache root (the cache is global state; never share one across tests). Fake an upstream by writing a real file and fetching it through the real `FileProvider` (`publish` + `upstream_package` helpers) rather than stubbing. Fixtures in `test/files/` are deliberately malformed or copied verbatim and are excluded from linting.
|
||||
+3
-11
@@ -1,9 +1,8 @@
|
||||
PATH
|
||||
remote: .
|
||||
specs:
|
||||
picopackage (0.2.1)
|
||||
picopackage (0.3.0)
|
||||
digest
|
||||
open-uri (~> 0.5)
|
||||
yaml (~> 0.4)
|
||||
|
||||
GEM
|
||||
@@ -23,11 +22,7 @@ GEM
|
||||
json (2.9.1)
|
||||
language_server-protocol (3.17.0.3)
|
||||
lint_roller (1.1.0)
|
||||
minitest (5.25.4)
|
||||
open-uri (0.5.0)
|
||||
stringio
|
||||
time
|
||||
uri
|
||||
minitest (5.27.0)
|
||||
parallel (1.26.3)
|
||||
parser (3.3.7.0)
|
||||
ast (~> 2.4.1)
|
||||
@@ -75,12 +70,9 @@ GEM
|
||||
lint_roller (~> 1.1)
|
||||
rubocop-performance (~> 1.23.0)
|
||||
stringio (3.1.2)
|
||||
time (0.4.1)
|
||||
date
|
||||
unicode-display_width (3.1.4)
|
||||
unicode-emoji (~> 4.0, >= 4.0.4)
|
||||
unicode-emoji (4.0.4)
|
||||
uri (1.0.2)
|
||||
unicode-emoji (4.2.0)
|
||||
yaml (0.4.0)
|
||||
|
||||
PLATFORMS
|
||||
|
||||
@@ -1,6 +1,11 @@
|
||||
# Picopackage
|
||||
|
||||
A command line tool for installing and managing [Picopackages](https://picopackage.org).
|
||||
A command line tool for installing and updating [picopackages](https://picopackage.org):
|
||||
single files that carry their own provenance, so you can install one into your
|
||||
project, edit it, and still pull upstream fixes later.
|
||||
|
||||
For code that's too small to be a gem but copied often enough that pasting it
|
||||
between projects hurts.
|
||||
|
||||
## Installation
|
||||
|
||||
@@ -10,18 +15,121 @@ gem install picopackage
|
||||
|
||||
## Usage
|
||||
|
||||
`picopackage install <url|filepath>`
|
||||
```bash
|
||||
ppkg install https://gist.github.com/you/abc123 # fetch into .
|
||||
ppkg install https://example.com/retry.rb lib/ # fetch into lib/
|
||||
ppkg update lib/retry.rb # re-fetch from its recorded url
|
||||
ppkg package lib/mine.rb --url https://… # turn a local file into a picopackage
|
||||
ppkg verify lib/retry.rb # check it against its checksum
|
||||
ppkg scan . # list picopackages in a tree
|
||||
ppkg inspect lib/retry.rb # print its metadata
|
||||
```
|
||||
|
||||
## What a picopackage looks like
|
||||
|
||||
An ordinary source file with a commented YAML block at the end:
|
||||
|
||||
```ruby
|
||||
def retry_with_backoff(attempts: 3)
|
||||
# …
|
||||
end
|
||||
|
||||
# @PICOPACKAGE_START
|
||||
# ---
|
||||
# url: https://gist.github.com/you/abc123
|
||||
# filename: retry.rb
|
||||
# payload_timestamp: '2026-08-12T14:06:38Z'
|
||||
# payload_checksum: sha256:7b856b1b…
|
||||
# base_checksum: sha256:7b856b1b…
|
||||
# @PICOPACKAGE_END
|
||||
```
|
||||
|
||||
Everything above the block is the **payload**. The block is regenerated on every
|
||||
write. Metadata keys the tool doesn't recognise are preserved as-is, so optional
|
||||
fields survive a tool that predates them.
|
||||
|
||||
## Upstream doesn't need a block
|
||||
|
||||
A plain file with no metadata at all is a valid picopackage source. You can
|
||||
install someone's gist, blog snippet or raw file without them adopting anything
|
||||
— everything the block would have said is derived from the fetch.
|
||||
|
||||
When upstream *does* carry a block, it's opting in to extra guarantees, and only
|
||||
half of it is upstream's to assert:
|
||||
|
||||
| | Fields | Who decides |
|
||||
| --- | --- | --- |
|
||||
| **Claims** | `url`, `filename`, `payload_version`, `licence` | upstream — adopted as given |
|
||||
| **Records** | `payload_checksum`, `base_checksum` | your machine — always recomputed |
|
||||
|
||||
A claim is something only the author knows: the canonical URL that outlives the
|
||||
mirror you fetched from, the version, the licence. A record describes one copy on
|
||||
one disk, so upstream can't speak for yours. In particular an upstream
|
||||
`payload_checksum` is never trusted — a checksum stored beside the content it
|
||||
hashes proves nothing, and an author who edits without re-running the tool ships
|
||||
a stale one. Adopting it would install a file that failed its own verification
|
||||
and reported as locally modified before you touched it.
|
||||
|
||||
## Updating a file you've edited
|
||||
|
||||
The point of a picopackage is that it's *your* file — so editing it has to be
|
||||
allowed, and updates have to cope with that. `ppkg update` does a three-way
|
||||
merge against the version you originally installed:
|
||||
|
||||
- **Unchanged locally** → fast-forwarded to upstream.
|
||||
- **Edited locally, upstream unchanged** → nothing happens. No merge is
|
||||
attempted; upstream is still the version you branched from.
|
||||
- **Edited locally, no overlap with upstream's changes** → merged, keeping both.
|
||||
- **Edited locally, overlapping changes** → refused. Your file is left exactly
|
||||
as it is and a copy with conflict markers is written to
|
||||
`<file>.picopackage-merge` for you to work from.
|
||||
- `--force` discards local changes and takes upstream wholesale.
|
||||
|
||||
Two rules decide every case:
|
||||
|
||||
1. **The checksum decides, never the timestamp.** An mtime says nothing about
|
||||
content — a file re-uploaded unchanged gets a fresh one.
|
||||
2. **Merges run on the payload alone.** The metadata block is regenerated
|
||||
afterwards, so it never conflicts.
|
||||
|
||||
The merge needs the common ancestor, which is kept in a content-addressed cache
|
||||
under `$XDG_CACHE_HOME/picopackage` (override with `PICOPACKAGE_CACHE`). A file
|
||||
installed on another machine, or one whose cache has been cleared, has no
|
||||
ancestor available — `ppkg` says so and refuses rather than guessing.
|
||||
|
||||
`git merge-file` is used when available, `diff3` otherwise.
|
||||
|
||||
## Sources
|
||||
|
||||
| Source | Handling |
|
||||
| --- | --- |
|
||||
| GitHub Gist | Resolved through the Gist API |
|
||||
| Opengist | Resolved via its `.json` endpoint |
|
||||
| Any URL | Fetched directly |
|
||||
| Local path | Read from disk |
|
||||
|
||||
Responses are capped at 1 MB and time out after 10s. Redirects are followed up
|
||||
to 5 times and only to `http`/`https`.
|
||||
|
||||
## What it deliberately doesn't do
|
||||
|
||||
- **No dependencies.** Each file stands alone. There is no solver, no version
|
||||
ranges, no transitive graph.
|
||||
- **No registry.** The file's `url` is its identity; no central index exists.
|
||||
- **No author verification.** A checksum stored beside the content it hashes
|
||||
proves nothing about who wrote it — its job is detecting *local* modification.
|
||||
Signing (SSH signatures, TOFU-pinned — see notes.md) is designed, not
|
||||
implemented. Until then, the security model is that a picopackage is one
|
||||
file and you can read it.
|
||||
|
||||
## Development
|
||||
|
||||
After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
|
||||
`bin/setup` installs dependencies, `rake test` runs the tests, `rake` runs tests
|
||||
and [Standard](https://github.com/standardrb/standard). `bin/console` gives you a
|
||||
prompt.
|
||||
|
||||
To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).
|
||||
To release: update `lib/picopackage/version.rb`, then `bundle exec rake release`.
|
||||
|
||||
## Contributing
|
||||
## Licence
|
||||
|
||||
Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/picop.
|
||||
|
||||
## License
|
||||
|
||||
The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
|
||||
MIT.
|
||||
|
||||
+6
-4
@@ -1,18 +1,20 @@
|
||||
# frozen_string_literal: true
|
||||
|
||||
require_relative "picopackage/version"
|
||||
require_relative "picopackage/package"
|
||||
require_relative "picopackage/cache"
|
||||
require_relative "picopackage/merge"
|
||||
require_relative "picopackage/fetch"
|
||||
require_relative "picopackage/provider"
|
||||
require_relative "picopackage/package"
|
||||
require_relative "picopackage/scanner"
|
||||
require_relative "picopackage/cli"
|
||||
|
||||
module Picopackage
|
||||
class Error < StandardError; end
|
||||
|
||||
class FileTooLargeError < StandardError; end
|
||||
class FileTooLargeError < Error; end
|
||||
|
||||
class FetchError < StandardError; end
|
||||
class FetchError < Error; end
|
||||
|
||||
class LocalModificationError < StandardError; end
|
||||
class LocalModificationError < Error; end
|
||||
end
|
||||
|
||||
@@ -0,0 +1,60 @@
|
||||
# frozen_string_literal: true
|
||||
|
||||
require "fileutils"
|
||||
|
||||
module Picopackage
|
||||
# Content-addressed store of upstream payloads, kept so that updates have a
|
||||
# merge base to work from.
|
||||
#
|
||||
# A three-way merge needs the common ancestor: the payload exactly as upstream
|
||||
# last sent it. Metadata already records that payload's checksum in
|
||||
# `base_checksum`, so the checksum doubles as the cache key and a package
|
||||
# carries the pointer to its own ancestor.
|
||||
class Cache
|
||||
def self.default_root
|
||||
ENV["PICOPACKAGE_CACHE"] ||
|
||||
File.join(ENV["XDG_CACHE_HOME"] || File.join(Dir.home, ".cache"), "picopackage")
|
||||
end
|
||||
|
||||
attr_reader :root
|
||||
|
||||
def initialize(root: self.class.default_root)
|
||||
@root = root
|
||||
end
|
||||
|
||||
# Returns the checksum the payload was stored under, for recording as a
|
||||
# `base_checksum`.
|
||||
def store(payload)
|
||||
Payload.checksum(payload).tap do |checksum|
|
||||
path = path_for(checksum)
|
||||
next if File.exist?(path)
|
||||
|
||||
FileUtils.mkdir_p(File.dirname(path))
|
||||
# Written via a sibling tempfile and renamed so a concurrent reader
|
||||
# never sees a half-written ancestor.
|
||||
tmp = "#{path}.#{Process.pid}.tmp"
|
||||
File.write(tmp, payload)
|
||||
File.rename(tmp, path)
|
||||
end
|
||||
end
|
||||
|
||||
def fetch(checksum)
|
||||
path = path_for(checksum)
|
||||
File.read(path) if path && File.exist?(path)
|
||||
end
|
||||
|
||||
def include?(checksum) = !fetch(checksum).nil?
|
||||
|
||||
private
|
||||
|
||||
# "sha256:abcdef…" => <root>/sha256/ab/cdef…
|
||||
def path_for(checksum)
|
||||
return nil if checksum.nil? || checksum.empty?
|
||||
|
||||
algorithm, digest = checksum.split(":", 2)
|
||||
return nil if digest.nil? || digest.length < 3 || !digest.match?(/\A[a-f0-9]+\z/i)
|
||||
|
||||
File.join(root, algorithm, digest[0, 2], digest[2..])
|
||||
end
|
||||
end
|
||||
end
|
||||
+110
-104
@@ -1,140 +1,146 @@
|
||||
# frozen_string_literal: true
|
||||
|
||||
require "optparse"
|
||||
|
||||
module Picopackage
|
||||
class CLI
|
||||
COMMANDS = %w[install update package verify inspect scan].freeze
|
||||
|
||||
def self.run(argv = ARGV)
|
||||
command = argv.shift
|
||||
|
||||
case command
|
||||
when "scan"
|
||||
options = {}
|
||||
OptionParser.new do |opts|
|
||||
opts.banner = "Usage: ppkg scan [options] DIRECTORY"
|
||||
# opts.on('-v', '--verbose', 'Run verbosely') { |v| options[:verbose] = v }
|
||||
end.parse!(argv)
|
||||
|
||||
dir = argv.first || "."
|
||||
Picopackage::Scanner.scan(dir).each { |f| puts f.file_path }
|
||||
|
||||
when "init"
|
||||
OptionParser.new do |opts|
|
||||
opts.banner = "Usage: ppkg init FILE"
|
||||
end.parse!(argv)
|
||||
|
||||
file = argv.first
|
||||
Picopackage::Package.from_file(file).init_metadata
|
||||
|
||||
when "checksum"
|
||||
OptionParser.new do |opts|
|
||||
opts.banner = "Usage: ppkg checksum FILE"
|
||||
end.parse!(argv)
|
||||
file = argv.first
|
||||
puts Picopackage::Package.from_file(file).checksum
|
||||
|
||||
when "verify"
|
||||
OptionParser.new do |opts|
|
||||
opts.banner = "Usage: ppkg sign FILE"
|
||||
end.parse!(argv)
|
||||
|
||||
path = argv.first
|
||||
source = Package.from_file(path)
|
||||
|
||||
if source.metadata["content_checksum"].nil?
|
||||
puts "⚠️ No checksum found in #{path}"
|
||||
puts "Run 'ppkg sign #{path}' to add one"
|
||||
exit 1
|
||||
end
|
||||
|
||||
unless source.verify_payload
|
||||
puts "❌ Checksum verification failed for #{path}"
|
||||
puts "Expected: #{source.metadata["content_checksum"]}"
|
||||
puts "Got: #{source.checksum}"
|
||||
exit 1
|
||||
end
|
||||
|
||||
puts "✅ #{path} verified successfully"
|
||||
|
||||
when "inspect"
|
||||
OptionParser.new do |opts|
|
||||
opts.banner = "Usage: ppkg inspect FILE|DIRECTORY"
|
||||
end.parse!(argv)
|
||||
|
||||
path = argv.first
|
||||
Picopackage::Package.from_file(path).inspect_metadata
|
||||
|
||||
when "fetch"
|
||||
when "install"
|
||||
options = {force: false}
|
||||
OptionParser.new do |opts|
|
||||
opts.banner = "Usage: ppkg fetch [options] URI [PATH]"
|
||||
opts.on("-f", "--force", "Force fetch") { |f| options[:force] = f }
|
||||
opts.banner = "Usage: ppkg install [options] URL [DIRECTORY]"
|
||||
opts.on("-f", "--force", "Overwrite local changes") { options[:force] = true }
|
||||
end.parse!(argv)
|
||||
|
||||
url = argv.shift
|
||||
path = argv.shift || "." # use '.' if no path provided
|
||||
destination = argv.shift || "."
|
||||
abort "Error: URL is required" if url.nil?
|
||||
|
||||
if url.nil?
|
||||
puts "Error: URI is required"
|
||||
exit 1
|
||||
end
|
||||
|
||||
begin
|
||||
Fetch.fetch(url, path, force: options[:force])
|
||||
rescue LocalModificationError => e
|
||||
puts "Error: #{e.message}"
|
||||
rescue => e
|
||||
puts "Error: #{e.message}"
|
||||
exit 1
|
||||
end
|
||||
report Fetch.fetch(url, destination, force: options[:force])
|
||||
|
||||
when "update"
|
||||
options = {force: false}
|
||||
OptionParser.new do |opts|
|
||||
opts.banner = "Usage: ppkg update [options] FILE"
|
||||
opts.on("-f", "--force", "Force update") { |f| options[:force] = f }
|
||||
opts.on("-f", "--force", "Discard local changes") { options[:force] = true }
|
||||
end.parse!(argv)
|
||||
|
||||
file = argv.first
|
||||
package = Package.from_file(file)
|
||||
begin
|
||||
Fetch.fetch(package.url, File.dirname(file), force: options[:force])
|
||||
rescue LocalModificationError => e
|
||||
puts "Error: #{e.message}"
|
||||
rescue => e
|
||||
puts "Error: #{e.message}"
|
||||
path = argv.shift
|
||||
abort "Error: FILE is required" if path.nil?
|
||||
|
||||
package = load_package(path)
|
||||
abort "Error: #{path} has no url in its metadata, so there is nothing to update from" if package.url.nil? || package.url.empty?
|
||||
|
||||
# Updates write back to the file the user named, not to whatever
|
||||
# filename upstream would prefer — renaming a package locally is
|
||||
# allowed and shouldn't silently install a second copy.
|
||||
report Fetch.fetch(package.url, File.dirname(path), force: options[:force], filename: File.basename(path))
|
||||
|
||||
when "package", "init"
|
||||
options = {}
|
||||
OptionParser.new do |opts|
|
||||
opts.banner = "Usage: ppkg package [--url URL] FILE"
|
||||
opts.on("--url URL", "Where updates should be fetched from") { |url| options[:url] = url }
|
||||
end.parse!(argv)
|
||||
|
||||
path = argv.shift
|
||||
abort "Error: FILE is required" if path.nil?
|
||||
|
||||
package = load_package(path)
|
||||
was_bare = package.bare?
|
||||
package.init_metadata(filename: File.basename(path), url: options[:url])
|
||||
package.save(path)
|
||||
Cache.new.store(package.payload)
|
||||
|
||||
puts was_bare ? "Packaged #{path}" : "Updated metadata in #{path}"
|
||||
warn "Note: no url recorded — 'ppkg update' won't work until one is set with --url" if package.url.nil?
|
||||
|
||||
when "verify"
|
||||
OptionParser.new { |opts| opts.banner = "Usage: ppkg verify FILE" }.parse!(argv)
|
||||
|
||||
path = argv.shift
|
||||
abort "Error: FILE is required" if path.nil?
|
||||
|
||||
package = load_package(path)
|
||||
|
||||
if package.payload_checksum.nil?
|
||||
puts "⚠️ No checksum found in #{path}"
|
||||
puts "Run 'ppkg package #{path}' to add one"
|
||||
exit 1
|
||||
end
|
||||
|
||||
unless package.verify_payload
|
||||
puts "❌ #{path} does not match its recorded checksum"
|
||||
puts "Recorded: #{package.payload_checksum}"
|
||||
puts "Actual: #{package.payload_digest}"
|
||||
exit 1
|
||||
end
|
||||
|
||||
puts "✅ #{path} matches its recorded checksum"
|
||||
|
||||
when "inspect"
|
||||
OptionParser.new { |opts| opts.banner = "Usage: ppkg inspect FILE" }.parse!(argv)
|
||||
|
||||
path = argv.shift
|
||||
abort "Error: FILE is required" if path.nil?
|
||||
load_package(path).inspect_metadata
|
||||
|
||||
when "scan"
|
||||
OptionParser.new { |opts| opts.banner = "Usage: ppkg scan [DIRECTORY]" }.parse!(argv)
|
||||
|
||||
found = Scanner.scan(argv.shift || ".")
|
||||
found.each { |entry| puts [entry.path, entry.package.url].compact.join("\t") }
|
||||
puts "No picopackages found" if found.empty?
|
||||
|
||||
when "--version", "-v", "version"
|
||||
puts Picopackage::VERSION
|
||||
|
||||
when nil, "help", "--help", "-h"
|
||||
usage
|
||||
exit(command.nil? ? 1 : 0)
|
||||
|
||||
else
|
||||
puts "Unknown command: #{command}"
|
||||
puts "Available commands: fetch, update, scan, sign, inspect"
|
||||
warn "Unknown command: #{command}"
|
||||
usage
|
||||
exit 1
|
||||
end
|
||||
rescue OptionParser::InvalidOption => e
|
||||
puts e.message
|
||||
exit 1
|
||||
rescue => e
|
||||
puts "Error: #{e.message}"
|
||||
puts e.backtrace if ENV["DEBUG"]
|
||||
rescue OptionParser::ParseError => e
|
||||
abort e.message
|
||||
rescue FetchError, Fetch::Error, ArgumentError => e
|
||||
warn "Error: #{e.message}"
|
||||
warn e.backtrace if ENV["DEBUG"]
|
||||
exit 1
|
||||
end
|
||||
|
||||
def self.determine_script_source
|
||||
# Get the full path of the currently executing script
|
||||
current_path = File.expand_path($0)
|
||||
def self.usage
|
||||
puts <<~TEXT
|
||||
Usage: ppkg <command> [options]
|
||||
|
||||
# Check if script is in GEM_PATH
|
||||
gem_paths = Gem.path.map { |p| File.expand_path(p) }
|
||||
Commands:
|
||||
install URL [DIR] Fetch a picopackage into DIR (default: .)
|
||||
update FILE Re-fetch FILE from the url in its metadata
|
||||
package FILE Add a metadata block to a local file
|
||||
verify FILE Check FILE against its recorded checksum
|
||||
inspect FILE Print FILE's metadata as JSON
|
||||
scan [DIR] List picopackages under DIR
|
||||
|
||||
is_gem = gem_paths.any? { |path| current_path.start_with?(path) }
|
||||
Run 'ppkg <command> --help' for command options.
|
||||
TEXT
|
||||
end
|
||||
|
||||
if is_gem
|
||||
# Running from gem installation
|
||||
gem_name = File.basename(File.dirname(File.dirname(current_path)))
|
||||
version = File.basename(File.dirname(current_path))
|
||||
{source: :gem, path: current_path, gem_name: gem_name, version: version}
|
||||
else
|
||||
# Running from local installation
|
||||
{source: :local, path: current_path}
|
||||
def self.load_package(path)
|
||||
abort "Error: no such file: #{path}" unless File.file?(path)
|
||||
Package.from_file(path) or abort "Error: could not read #{path}"
|
||||
end
|
||||
|
||||
def self.report(result)
|
||||
puts result.message
|
||||
exit 1 if result.conflict?
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
+205
-95
@@ -1,168 +1,278 @@
|
||||
# frozen_string_literal: true
|
||||
|
||||
require "net/http"
|
||||
require "openssl"
|
||||
require "fileutils"
|
||||
require "tempfile"
|
||||
require "json"
|
||||
require "debug"
|
||||
|
||||
module Picopackage
|
||||
class Fetch
|
||||
class Error < StandardError; end
|
||||
class HTTPError < Error; end
|
||||
class FileTooLargeError < Error; end
|
||||
class NotModifiedError < Error; end # Add this
|
||||
class TooManyRedirectsError < Error; end # Add this
|
||||
|
||||
MAX_REDIRECTS = 5 # This constant is used but not defined
|
||||
class HTTPError < Error; end
|
||||
|
||||
class FileTooLargeError < Error; end
|
||||
|
||||
class NotModifiedError < Error; end
|
||||
|
||||
class TooManyRedirectsError < Error; end
|
||||
|
||||
MAX_REDIRECTS = 5
|
||||
USER_AGENT = "picopackage/#{Picopackage::VERSION}"
|
||||
SCHEMES = %w[http https].freeze
|
||||
|
||||
def initialize(max_size: 1024 * 1024, timeout: 10)
|
||||
@max_size = max_size
|
||||
@timeout = timeout
|
||||
end
|
||||
|
||||
def fetch(uri)
|
||||
def fetch(uri, etag: nil)
|
||||
uri = URI.parse(uri.to_s) unless uri.is_a?(URI)
|
||||
case uri.scheme
|
||||
when "http", "https" then fetch_http(uri)
|
||||
when "http", "https" then fetch_http(uri, etag: etag)
|
||||
when "file" then fetch_file(uri)
|
||||
else
|
||||
raise Error, "Unsupported scheme: #{uri.scheme}"
|
||||
raise Error, "Unsupported scheme: #{uri.scheme.inspect}"
|
||||
end
|
||||
end
|
||||
|
||||
def self.fetch(url, destination, force: false)
|
||||
# Fetches a URL and reconciles it with whatever is already on disk.
|
||||
# Returns a Resolver::Result; deciding what to do about a conflict is the
|
||||
# caller's business, not ours.
|
||||
def self.fetch(url, destination, force: false, filename: nil, cache: Cache.new)
|
||||
raise ArgumentError, "Destination directory does not exist: #{destination}" unless Dir.exist?(destination)
|
||||
|
||||
provider = Provider.for(url)
|
||||
package = provider.package
|
||||
file_path = File.join(destination, package.filename)
|
||||
raise FetchError, "No provider could handle #{url}" if provider.nil?
|
||||
|
||||
local_package = File.exist?(file_path) ? FileProvider.new(file_path).package : nil
|
||||
remote = provider.package
|
||||
target = filename || remote.filename
|
||||
raise FetchError, "Could not determine a filename for #{url}" if target.nil?
|
||||
|
||||
resolver = Resolver.new(package, local_package, file_path, force: force).resolve
|
||||
file_path = File.join(destination, File.basename(target))
|
||||
local = Package.from_file(file_path)
|
||||
|
||||
case resolver[:state]
|
||||
when :kept, :updated
|
||||
puts resolver[:message]
|
||||
when :conflict
|
||||
raise LocalModificationError, resolver[:message]
|
||||
end
|
||||
provider.package
|
||||
Resolver.new(remote, local, file_path, force: force, cache: cache).resolve
|
||||
end
|
||||
|
||||
private
|
||||
|
||||
def fetch_http(uri, etag = nil)
|
||||
Net::HTTP.start(uri.host, uri.port, connection_options(uri)) do |http|
|
||||
request = Net::HTTP::Get.new(uri.request_uri)
|
||||
# Net::HTTP rather than URI.open, deliberately. This method's whole job is to
|
||||
# dereference a URL a user handed us, and URI.open treats a leading "|" as a
|
||||
# command to run. The scheme check in #fetch already blocks that, but a
|
||||
# remote-code-fetching tool should not be one refactor away from shelling
|
||||
# out, and doing it by hand is also what makes redirect and 304 handling
|
||||
# possible.
|
||||
def fetch_http(uri, etag: nil, redirects: MAX_REDIRECTS)
|
||||
raise TooManyRedirectsError, "More than #{MAX_REDIRECTS} redirects from #{uri}" if redirects.negative?
|
||||
|
||||
Net::HTTP.start(uri.host, uri.port, use_ssl: uri.scheme == "https",
|
||||
open_timeout: @timeout, read_timeout: @timeout) do |http|
|
||||
request = Net::HTTP::Get.new(uri)
|
||||
request["User-Agent"] = USER_AGENT
|
||||
request["Accept"] = "*/*"
|
||||
request["If-None-Match"] = etag if etag
|
||||
|
||||
response = http.request(request)
|
||||
handle_response(response, uri)
|
||||
end
|
||||
end
|
||||
|
||||
def fetch_file(uri)
|
||||
File.read(uri.path)
|
||||
end
|
||||
|
||||
def connection_options(uri)
|
||||
{
|
||||
use_ssl: uri.scheme == "https",
|
||||
read_timeout: @timeout,
|
||||
open_timeout: @timeout
|
||||
}
|
||||
end
|
||||
|
||||
def handle_response(response, uri)
|
||||
http.request(request) do |response|
|
||||
case response
|
||||
when Net::HTTPSuccess
|
||||
{
|
||||
body: read_body(response),
|
||||
etag: response["ETag"]
|
||||
}
|
||||
when Net::HTTPNotModified
|
||||
raise NotModifiedError.new("Resource not modified", etag: response["ETag"])
|
||||
raise NotModifiedError, "#{uri} is unchanged"
|
||||
when Net::HTTPRedirection
|
||||
handle_redirect(response, uri)
|
||||
return fetch_http(redirect_target(uri, response), etag: etag, redirects: redirects - 1)
|
||||
when Net::HTTPSuccess
|
||||
return read_capped(response)
|
||||
else
|
||||
raise HTTPError, "HTTP #{response.code}: #{response.message}"
|
||||
raise HTTPError, "#{response.code} #{response.message} for #{uri}"
|
||||
end
|
||||
end
|
||||
end
|
||||
rescue Net::OpenTimeout, Net::ReadTimeout
|
||||
raise HTTPError, "Timed out after #{@timeout}s fetching #{uri}"
|
||||
rescue SocketError, SystemCallError, OpenSSL::SSL::SSLError => e
|
||||
raise HTTPError, "Could not fetch #{uri}: #{e.message}"
|
||||
end
|
||||
|
||||
def handle_redirect(response, uri, redirect_count = 0)
|
||||
raise TooManyRedirectsError if redirect_count >= MAX_REDIRECTS
|
||||
def redirect_target(uri, response)
|
||||
location = response["location"]
|
||||
new_uri = URI(location)
|
||||
# Handle both relative paths and full URLs
|
||||
new_uri = uri.merge(location) if new_uri.relative?
|
||||
fetch(new_uri, redirect_count: redirect_count + 1)
|
||||
raise HTTPError, "Redirect from #{uri} with no Location header" if location.nil?
|
||||
|
||||
URI.join(uri, location).tap do |target|
|
||||
raise Error, "Refusing to follow a redirect to #{target.scheme.inspect}" unless SCHEMES.include?(target.scheme)
|
||||
end
|
||||
end
|
||||
|
||||
def read_body(response)
|
||||
# Trusts Content-Length only to fail early — the streamed tally is what
|
||||
# actually enforces the cap, since the header can lie or be absent.
|
||||
def read_capped(response)
|
||||
declared = response["content-length"]&.to_i
|
||||
raise FileTooLargeError, "#{declared} bytes exceeds the #{@max_size} byte limit" if declared&.positive? && declared > @max_size
|
||||
|
||||
buffer = String.new(capacity: @max_size)
|
||||
response.read_body do |chunk|
|
||||
raise FileTooLargeError, "Response would exceed #{@max_size} bytes" if buffer.bytesize + chunk.bytesize > @max_size
|
||||
raise FileTooLargeError, "Response exceeds the #{@max_size} byte limit" if buffer.bytesize + chunk.bytesize > @max_size
|
||||
|
||||
buffer << chunk
|
||||
end
|
||||
buffer
|
||||
end
|
||||
|
||||
def fetch_file(uri) = File.read(uri.path)
|
||||
end
|
||||
|
||||
##
|
||||
# Decides what happens when an upstream payload meets a local file.
|
||||
#
|
||||
# Two rules drive every branch, and they are worth stating because they are
|
||||
# what make the table in notes.md collapse to something small:
|
||||
#
|
||||
# 1. The checksum decides, never the timestamp. mtime is not evidence about
|
||||
# content — a file re-uploaded unchanged gets a fresh one — so equal
|
||||
# checksums end the content question outright, whatever the clocks say.
|
||||
# 2. Merges run on the payload alone. The metadata block is derived, so
|
||||
# merging it would conflict on every update; it is regenerated after.
|
||||
#
|
||||
# States:
|
||||
# - kept: local file was converted to a picopackage and kept
|
||||
# - updated: local file was updated with remote picopackage
|
||||
# - conflict: local and remote files differ - manually resolve or use -f to force
|
||||
# - installed: no local file existed, upstream was written
|
||||
# - adopted: a bare local file gained metadata; its content already matched
|
||||
# - current: nothing to do
|
||||
# - updated: local payload replaced with upstream
|
||||
# - merged: local edits preserved, upstream changes applied on top
|
||||
# - conflict: needs a human; nothing was written to the original file
|
||||
class Resolver
|
||||
attr_reader :remote, :local, :local_path, :force
|
||||
def initialize(remote_package, local_package, local_path, force: false)
|
||||
STATES = %i[installed adopted current updated merged conflict].freeze
|
||||
|
||||
Result = Struct.new(:state, :message, :path, keyword_init: true) do
|
||||
def conflict? = state == :conflict
|
||||
|
||||
def written? = %i[installed adopted updated merged].include?(state)
|
||||
end
|
||||
|
||||
MERGE_SUFFIX = ".picopackage-merge"
|
||||
|
||||
attr_reader :remote, :local, :local_path, :force, :cache
|
||||
|
||||
def initialize(remote_package, local_package, local_path, force: false, cache: Cache.new)
|
||||
@remote = remote_package
|
||||
@local = local_package
|
||||
@local_path = local_path
|
||||
@force = force
|
||||
@same_checksum = @remote.payload_checksum == @local&.payload_checksum
|
||||
@cache = cache
|
||||
end
|
||||
|
||||
STATES = %i[kept updated conflict].freeze
|
||||
|
||||
def resolve
|
||||
validate_state_hash(
|
||||
if @force
|
||||
@remote.save(local_path)
|
||||
{state: :updated, message: "Force mode: overwrote local file with remote package"}
|
||||
elsif @local.nil?
|
||||
@remote.save(local_path)
|
||||
{state: :kept, message: "Saved Package as new file"}
|
||||
elsif @remote.payload_version != @local.payload_version
|
||||
{state: :conflict, message: "Version conflict. Local: #{@local.payload_version}, Remote: #{@remote.payload_version}"}
|
||||
elsif @remote.payload_timestamp_as_time > @local.payload_timestamp_as_time
|
||||
@remote.save(local_path)
|
||||
{state: :updated, message: "Updated to newer version"}
|
||||
elsif !@same_checksum
|
||||
handle_checksum_mismatch
|
||||
elsif @local.was_bare_file
|
||||
debugger
|
||||
@local.save(local_path)
|
||||
{state: :kept, message: "Packaged existing file as Picopackage"}
|
||||
validate(
|
||||
if local.nil?
|
||||
write(:installed, "Installed #{File.basename(local_path)}")
|
||||
elsif force
|
||||
write(:updated, "Forced: overwrote #{File.basename(local_path)} with upstream")
|
||||
elsif same_payload?
|
||||
reconcile_metadata
|
||||
elsif local.bare?
|
||||
# No metadata, no ancestor, and a different payload: this is an
|
||||
# unrelated file that happens to share a name, not a modified
|
||||
# picopackage. There is no shared history to merge along.
|
||||
conflict("#{File.basename(local_path)} already exists, is not a picopackage, and differs from upstream. " \
|
||||
"Move it aside or use --force to overwrite it.")
|
||||
elsif !local.diverged?
|
||||
write(:updated, "Updated #{File.basename(local_path)} to upstream")
|
||||
elsif upstream_unchanged?
|
||||
keep_local_changes
|
||||
else
|
||||
{state: :kept, message: "Local file is up to date"}
|
||||
merge
|
||||
end
|
||||
)
|
||||
end
|
||||
|
||||
private
|
||||
|
||||
def validate_state_hash(hash)
|
||||
raise "Invalid state" unless STATES.include?(hash[:state])
|
||||
raise "Missing message" unless hash[:message].is_a?(String)
|
||||
hash
|
||||
def same_payload? = remote.payload_digest == local.payload_digest
|
||||
|
||||
# Same content on both sides, so the only open question is whose metadata
|
||||
# to keep. A bare local file gets adopted; a package is already fine.
|
||||
def reconcile_metadata
|
||||
cache.store(remote.payload)
|
||||
|
||||
if local.bare?
|
||||
write(:adopted, "Adopted existing #{File.basename(local_path)} as a picopackage")
|
||||
else
|
||||
Result.new(state: :current, message: "#{File.basename(local_path)} is up to date", path: local_path)
|
||||
end
|
||||
end
|
||||
|
||||
def handle_checksum_mismatch
|
||||
if @force
|
||||
@remote.save(local_path) # In force mode, remote wins
|
||||
{state: :updated, message: "Overwrote local file with remote package"}
|
||||
# Reached only when the payloads differ and the local one has diverged, so
|
||||
# the difference is entirely ours: upstream still holds the ancestor we
|
||||
# branched from.
|
||||
def upstream_unchanged? = remote.payload_digest == local.ancestor_checksum
|
||||
|
||||
# Nothing upstream to apply. Merging would be a correct no-op — base and
|
||||
# theirs are the same text, so the result is ours — but it spawns a merge
|
||||
# tool to prove it and reports `:merged` when nothing was merged.
|
||||
#
|
||||
# Storing the payload is not redundant: this is the one moment we are
|
||||
# holding the ancestor itself, so a cache that has been cleared gets
|
||||
# repopulated and the *next* update can still merge.
|
||||
def keep_local_changes
|
||||
cache.store(remote.payload)
|
||||
Result.new(state: :current,
|
||||
message: "#{File.basename(local_path)} is up to date; your local changes are untouched",
|
||||
path: local_path)
|
||||
end
|
||||
|
||||
def merge
|
||||
return conflict("#{File.basename(local_path)} has local changes and no merge tool (git or diff3) is available.") unless Merge.available?
|
||||
|
||||
base = cache.fetch(local.ancestor_checksum)
|
||||
if base.nil?
|
||||
return conflict("#{File.basename(local_path)} has local changes and its original upstream version " \
|
||||
"isn't in the cache, so there is nothing to merge against. " \
|
||||
"Diff it against upstream by hand, or use --force to discard your changes.")
|
||||
end
|
||||
|
||||
upstream_digest = remote.payload_digest
|
||||
cache.store(remote.payload)
|
||||
result = Merge.three_way(base: base, ours: local.payload, theirs: remote.payload)
|
||||
|
||||
case result.status
|
||||
when :clean
|
||||
remote.payload = result.content
|
||||
remote.base_checksum = upstream_digest
|
||||
remote.save(local_path)
|
||||
Result.new(state: :merged, message: "Merged upstream changes into #{File.basename(local_path)}, keeping your edits", path: local_path)
|
||||
when :conflict
|
||||
write_conflict_file(result)
|
||||
else
|
||||
{state: :conflict, message: "Files differ. Use --force to convert both to packages"}
|
||||
conflict("Merge tool failed on #{File.basename(local_path)}.")
|
||||
end
|
||||
end
|
||||
|
||||
# The original is never touched. A tool with no index and no `reset` has no
|
||||
# business writing conflict markers into a file the user may not have
|
||||
# committed, so the markers go to a sibling and the user drives.
|
||||
def write_conflict_file(result)
|
||||
path = local_path + MERGE_SUFFIX
|
||||
File.write(path, result.content)
|
||||
conflict("#{File.basename(local_path)} has local changes that conflict with upstream " \
|
||||
"(#{result.conflicts} #{(result.conflicts == 1) ? "conflict" : "conflicts"}). " \
|
||||
"Resolved-with-markers copy written to #{File.basename(path)}; your file is untouched.")
|
||||
end
|
||||
|
||||
def write(state, message)
|
||||
# Metadata describes *this* file, so the filename follows the local name
|
||||
# even when it differs from upstream's. `url` is what points upstream.
|
||||
remote.filename = File.basename(local_path)
|
||||
remote.base_checksum = remote.payload_digest
|
||||
cache.store(remote.payload)
|
||||
path = remote.save(local_path)
|
||||
Result.new(state: state, message: message, path: path)
|
||||
end
|
||||
|
||||
def conflict(message) = Result.new(state: :conflict, message: message, path: local_path)
|
||||
|
||||
def validate(result)
|
||||
raise "Invalid state: #{result.state.inspect}" unless STATES.include?(result.state)
|
||||
raise "Missing message" unless result.message.is_a?(String)
|
||||
|
||||
result
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
@@ -0,0 +1,94 @@
|
||||
# frozen_string_literal: true
|
||||
|
||||
require "tempfile"
|
||||
|
||||
module Picopackage
|
||||
# Three-way merge of payloads, delegated to `git merge-file` or POSIX `diff3`.
|
||||
#
|
||||
# Reimplementing diff3 in Ruby would be a few hundred lines of the most
|
||||
# bug-prone code in the project, to produce output people already know how to
|
||||
# read. Both external tools take their arguments in the same order — ours,
|
||||
# base, theirs — and both report a clean merge with status 0 and conflicts
|
||||
# with a positive status, so the two backends differ only in the command line.
|
||||
module Merge
|
||||
Result = Struct.new(:status, :content, :conflicts, keyword_init: true) do
|
||||
def clean? = status == :clean
|
||||
|
||||
def conflict? = status == :conflict
|
||||
|
||||
def unavailable? = status == :unavailable
|
||||
end
|
||||
|
||||
LABELS = {ours: "local", base: "ancestor", theirs: "upstream"}.freeze
|
||||
|
||||
module_function
|
||||
|
||||
# Returns :git, :diff3, or nil.
|
||||
def backend
|
||||
return @backend if defined?(@backend)
|
||||
|
||||
@backend = if command?("git")
|
||||
:git
|
||||
elsif command?("diff3")
|
||||
:diff3
|
||||
end
|
||||
end
|
||||
|
||||
def available? = !backend.nil?
|
||||
|
||||
def command?(name)
|
||||
ENV.fetch("PATH", "").split(File::PATH_SEPARATOR).any? do |dir|
|
||||
File.executable?(File.join(dir, name)) && !File.directory?(File.join(dir, name))
|
||||
end
|
||||
end
|
||||
|
||||
def three_way(base:, ours:, theirs:, labels: LABELS)
|
||||
return Result.new(status: :unavailable, content: nil, conflicts: 0) unless available?
|
||||
|
||||
with_temp_files(ours, base, theirs) do |ours_path, base_path, theirs_path|
|
||||
merged, status = run(backend, ours_path, base_path, theirs_path, labels)
|
||||
|
||||
# Negative (255 as unsigned) means the tool itself failed rather than
|
||||
# finding conflicts — don't report that as a mergeable conflict.
|
||||
if status.nil? || status > 128
|
||||
Result.new(status: :unavailable, content: nil, conflicts: 0)
|
||||
elsif status.zero?
|
||||
Result.new(status: :clean, content: merged, conflicts: 0)
|
||||
else
|
||||
Result.new(status: :conflict, content: merged, conflicts: status)
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
def run(backend, ours_path, base_path, theirs_path, labels)
|
||||
argv = case backend
|
||||
when :git
|
||||
["git", "merge-file", "-p", "--diff3",
|
||||
"-L", labels[:ours], "-L", labels[:base], "-L", labels[:theirs],
|
||||
ours_path, base_path, theirs_path]
|
||||
when :diff3
|
||||
["diff3", "-m",
|
||||
"-L", labels[:ours], "-L", labels[:base], "-L", labels[:theirs],
|
||||
ours_path, base_path, theirs_path]
|
||||
end
|
||||
|
||||
output = IO.popen(argv, err: File::NULL, &:read)
|
||||
[output, $?&.exitstatus]
|
||||
rescue SystemCallError
|
||||
[nil, nil]
|
||||
end
|
||||
|
||||
def with_temp_files(ours, base, theirs)
|
||||
files = {ours: ours, base: base, theirs: theirs}.map do |name, content|
|
||||
file = Tempfile.new(["picopackage-#{name}", ".txt"])
|
||||
file.binmode
|
||||
file.write(content)
|
||||
file.flush
|
||||
file
|
||||
end
|
||||
yield(*files.map(&:path))
|
||||
ensure
|
||||
files&.each { |f| f.close! }
|
||||
end
|
||||
end
|
||||
end
|
||||
+137
-67
@@ -1,33 +1,48 @@
|
||||
# frozen_string_literal: true
|
||||
|
||||
require "yaml"
|
||||
require "json"
|
||||
require "digest"
|
||||
require "time"
|
||||
require "forwardable"
|
||||
|
||||
module Picopackage
|
||||
METADATA_PATTERN = /^\n*#\s*@PICOPACKAGE_START\n(.*?)^\s*#\s*@PICOPACKAGE_END\s*$/m
|
||||
|
||||
class Metadata < Struct.new(:url, :filename, :payload_version, :payload_timestamp, :payload_checksum, :etag, keyword_init: true)
|
||||
# the #from_file method will create a new instance of Metadata from a file path, rather than read a package's metadata
|
||||
def self.from_file(file_path, content: nil)
|
||||
new(content: File.read(file_path))
|
||||
end
|
||||
|
||||
def self.from_url_response(url, response)
|
||||
end
|
||||
|
||||
class Metadata < Struct.new(:url, :filename, :payload_version, :payload_timestamp,
|
||||
:payload_checksum, :base_checksum, :etag, keyword_init: true)
|
||||
def self.from_content(content)
|
||||
return new unless content =~ METADATA_PATTERN
|
||||
return new unless (match = METADATA_PATTERN.match(content))
|
||||
|
||||
yaml_content = $1.each_line.map { |line| line.sub(/^\s*#\s?/, "").rstrip }.join("\n")
|
||||
parsed = YAML.safe_load(uncomment(match[1]))
|
||||
return new unless parsed.is_a?(Hash)
|
||||
|
||||
# Load and transform in one chain
|
||||
@metadata = new(**YAML.safe_load(yaml_content)
|
||||
.slice(*Metadata.members.map(&:to_s))
|
||||
.transform_keys(&:to_sym))
|
||||
rescue
|
||||
new # Return empty hash on any YAML/transformation errors
|
||||
known, unknown = parsed.partition { |key, _| members.include?(key.to_s.to_sym) }
|
||||
new(**known.to_h.transform_keys(&:to_sym)).tap { |metadata| metadata.extra = unknown.to_h }
|
||||
rescue Psych::Exception, TypeError, ArgumentError
|
||||
new
|
||||
end
|
||||
|
||||
def empty? = to_h.values.all?(&:nil?)
|
||||
def self.from_file(path) = from_content(File.read(path))
|
||||
|
||||
def self.uncomment(block)
|
||||
block.each_line.map { |line| line.sub(/^\s*#\s?/, "").rstrip }.join("\n")
|
||||
end
|
||||
|
||||
# Keys the struct doesn't model. The spec has a long tail of optional fields
|
||||
# — licence, test_url, signature, parent_revision — and a tool that dropped
|
||||
# the ones it didn't recognise would quietly destroy them the first time it
|
||||
# rewrote a file. Round-tripping them costs nothing and means old tools
|
||||
# can't corrupt packages written by newer ones.
|
||||
attr_writer :extra
|
||||
|
||||
def extra = @extra ||= {}
|
||||
|
||||
def empty? = to_h.values.all?(&:nil?) && extra.empty?
|
||||
|
||||
# Modelled members in declaration order, then unmodelled keys sorted, so a
|
||||
# rewrite that changes one field produces a one-line diff.
|
||||
def to_yaml_hash = to_h.compact.transform_keys(&:to_s).merge(extra.sort.to_h)
|
||||
end
|
||||
|
||||
class Payload
|
||||
@@ -35,94 +50,149 @@ module Picopackage
|
||||
|
||||
def self.normalize(payload) = payload.rstrip + "\n\n"
|
||||
|
||||
def self.normalized_from_content(content) = Payload.from_content(content).then { Payload.normalize(_1) }
|
||||
def self.normalized_from_content(content) = normalize(from_content(content))
|
||||
|
||||
def self.from_file(file_path) = normalized_from_content(File.read(file_path))
|
||||
def self.from_file(path) = normalized_from_content(File.read(path))
|
||||
|
||||
def self.checksum(payload) = "sha256:#{Digest::SHA256.hexdigest(payload)}"
|
||||
|
||||
def self.checksum_from_content(content) = checksum(from_content(content))
|
||||
# Always hashes the *normalised* payload. Hashing the raw payload here while
|
||||
# verification hashed the normalised one meant freshly written packages
|
||||
# failed their own checksum, so every file looked locally modified.
|
||||
def self.checksum_from_content(content) = checksum(normalized_from_content(content))
|
||||
end
|
||||
|
||||
class Package
|
||||
extend Forwardable
|
||||
attr_reader :content, :payload, :metadata, :was_bare_file
|
||||
|
||||
attr_reader :content, :payload, :metadata
|
||||
|
||||
def_delegators :@metadata,
|
||||
:url, :url=,
|
||||
:filename, :filename=,
|
||||
:payload_version, :payload_version=,
|
||||
:payload_timestamp, :payload_timestamp=,
|
||||
:payload_checksum, :payload_checksum=
|
||||
:payload_checksum, :payload_checksum=,
|
||||
:base_checksum, :base_checksum=,
|
||||
:etag, :etag=
|
||||
|
||||
def self.from_file(file_path)
|
||||
if File.exist?(file_path)
|
||||
new(content: File.read(file_path))
|
||||
end
|
||||
new(content: File.read(file_path)) if File.exist?(file_path)
|
||||
end
|
||||
|
||||
def initialize(content:)
|
||||
@content = content
|
||||
@payload = Payload.normalized_from_content(@content)
|
||||
@metadata = Metadata.from_content(@content)
|
||||
|
||||
if is_bare_file?
|
||||
@was_bare_file = true
|
||||
init_metadata
|
||||
else
|
||||
@was_bare_file = false
|
||||
end
|
||||
@had_metadata_block = METADATA_PATTERN.match?(content)
|
||||
@payload = Payload.normalized_from_content(content)
|
||||
@metadata = Metadata.from_content(content)
|
||||
end
|
||||
|
||||
def is_bare_file? = @metadata.empty?
|
||||
# True when the file arrived with no metadata block at all: a hand-copied
|
||||
# file, or an upstream that was never packaged. Deliberately not "its
|
||||
# metadata parsed to nothing" — a block holding only optional keys we don't
|
||||
# model would answer yes to that and get clobbered as though it were bare.
|
||||
def bare? = !@had_metadata_block
|
||||
|
||||
def init_metadata
|
||||
@metadata.url ||= url
|
||||
@metadata.filename ||= filename
|
||||
@metadata.payload_checksum ||= Payload.checksum_from_content(content)
|
||||
@metadata.payload_timestamp ||= payload_timestamp
|
||||
end
|
||||
# What the payload hashes to right now.
|
||||
def payload_digest = Payload.checksum(payload)
|
||||
|
||||
def save(path, filename = nil)
|
||||
path = File.join(path, filename || @metadata.filename) if File.directory?(path)
|
||||
# Local edits break the recorded checksum, and that divergence is the only
|
||||
# modification signal available — mtime says nothing about content.
|
||||
#
|
||||
# Note the asymmetry with #verify_payload when no checksum is recorded: we
|
||||
# have no *evidence of modification*, but we also cannot *verify*. Both
|
||||
# answers are false, and they mean different things.
|
||||
def modified?
|
||||
return false unless recorded_checksum?
|
||||
|
||||
File.write(path, generate_package)
|
||||
payload_digest != payload_checksum
|
||||
end
|
||||
|
||||
def verify_payload
|
||||
return false if metadata.payload_checksum.nil? || metadata.payload_checksum&.empty?
|
||||
Payload.checksum(payload) == metadata.payload_checksum
|
||||
return false unless recorded_checksum?
|
||||
|
||||
payload_digest == payload_checksum
|
||||
end
|
||||
|
||||
# The upstream version this file was last reconciled with. Packages written
|
||||
# before `base_checksum` existed, and hand-written metadata, have only
|
||||
# `payload_checksum` — which meant exactly this before merges did.
|
||||
def ancestor_checksum = base_checksum || payload_checksum
|
||||
|
||||
# Whether the payload carries changes relative to upstream — from a human
|
||||
# edit *or* from an earlier merge.
|
||||
#
|
||||
# This, not #modified?, is what decides whether upstream may simply
|
||||
# overwrite the file. A merged file is unmodified (we wrote it, so its
|
||||
# recorded checksum matches) while still holding local edits that a
|
||||
# fast-forward would silently discard.
|
||||
def diverged?
|
||||
return false if ancestor_checksum.nil? || ancestor_checksum.empty?
|
||||
|
||||
payload_digest != ancestor_checksum
|
||||
end
|
||||
|
||||
# Replaces the payload, e.g. with the result of a merge, and re-points the
|
||||
# recorded checksum at it so the file reads as unmodified until a human
|
||||
# touches it. `base_checksum` is left alone: it tracks the upstream ancestor,
|
||||
# which is not what we just wrote.
|
||||
def payload=(new_payload)
|
||||
@payload = Payload.normalize(new_payload)
|
||||
@content = @payload
|
||||
@metadata.payload_checksum = Payload.checksum(@payload)
|
||||
end
|
||||
|
||||
# Fills in what's missing. `payload_checksum` is deliberately absent: it is
|
||||
# derived at write time by #generate_package rather than carried over from
|
||||
# an input, so a stale value can never survive into a file we write.
|
||||
def init_metadata(filename: nil, url: nil)
|
||||
@metadata.filename ||= filename
|
||||
@metadata.url ||= url
|
||||
@metadata.payload_timestamp ||= Time.now.utc.iso8601
|
||||
@metadata.base_checksum ||= payload_digest
|
||||
self
|
||||
end
|
||||
|
||||
def payload_timestamp_as_time
|
||||
@metadata&.payload_timestamp ? Time.parse(@metadata.payload_timestamp) : nil
|
||||
payload_timestamp ? Time.parse(payload_timestamp.to_s) : nil
|
||||
rescue ArgumentError
|
||||
nil
|
||||
end
|
||||
|
||||
def modified? = !verify_payload
|
||||
# Returns the path written to.
|
||||
def save(path, filename = nil)
|
||||
path = File.join(path, filename || @metadata.filename) if File.directory?(path)
|
||||
File.write(path, generate_package)
|
||||
path
|
||||
end
|
||||
|
||||
def inspect_metadata = puts JSON.pretty_generate(@metadata.to_h)
|
||||
def inspect_metadata = puts(JSON.pretty_generate(@metadata.to_yaml_hash))
|
||||
|
||||
def generate_package
|
||||
@metadata.url = url.to_s unless url.nil?
|
||||
# The single choke point for the rule that a checksum is *derived*, never
|
||||
# adopted or preserved. Every write goes through here, so every file we
|
||||
# produce verifies against itself no matter which path built the package —
|
||||
# including one built from an upstream block whose author edited the file
|
||||
# without re-running the tool.
|
||||
@metadata.payload_checksum = payload_digest
|
||||
block = generate_metadata
|
||||
|
||||
if METADATA_PATTERN.match?(@content)
|
||||
@content.sub(METADATA_PATTERN, "\n#{block}")
|
||||
else
|
||||
[@content.rstrip, "\n#{block}"].join("\n")
|
||||
end
|
||||
end
|
||||
|
||||
private
|
||||
|
||||
def generate_package
|
||||
@metadata.url = url.to_s
|
||||
metadata_block = generate_metadata
|
||||
if METADATA_PATTERN.match?(content)
|
||||
content.sub(METADATA_PATTERN, "\n#{metadata_block}")
|
||||
else
|
||||
[content.rstrip, "\n#{metadata_block}"].join("\n")
|
||||
end
|
||||
end
|
||||
def recorded_checksum? = !(payload_checksum.nil? || payload_checksum.empty?)
|
||||
|
||||
# This will need a comment style one day, to work with other languages
|
||||
# This will need a comment style one day, to work with other languages.
|
||||
def generate_metadata
|
||||
yaml_content = @metadata.to_h.transform_keys(&:to_s).to_yaml.strip
|
||||
[
|
||||
"# @PICOPACKAGE_START",
|
||||
yaml_content.lines.map { |line| "# #{line}" }.join,
|
||||
"# @PICOPACKAGE_END",
|
||||
""
|
||||
].join("\n")
|
||||
commented = @metadata.to_yaml_hash.to_yaml.strip.each_line.map { |line| "# #{line.chomp}".rstrip }
|
||||
["# @PICOPACKAGE_START", *commented, "# @PICOPACKAGE_END", ""].join("\n")
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
+80
-37
@@ -1,3 +1,5 @@
|
||||
# frozen_string_literal: true
|
||||
|
||||
require "time"
|
||||
require "pathname"
|
||||
|
||||
@@ -6,34 +8,49 @@ module Picopackage
|
||||
def self.for(url)
|
||||
PROVIDERS.each do |provider|
|
||||
case provider.handles_url?(url)
|
||||
when false
|
||||
when false, nil
|
||||
next
|
||||
when true
|
||||
return provider.new(url)
|
||||
when :maybe
|
||||
# A `:maybe` provider proves itself by fetching, and providers fetch in
|
||||
# their constructor — so a speculative provider that guesses wrong
|
||||
# (OpenGistProvider appends ".json" to every URL it sees) raises here.
|
||||
# That has to fall through to the next candidate rather than abort the
|
||||
# whole resolution, or no plain URL would ever reach DefaultProvider.
|
||||
begin
|
||||
instance = provider.new(url)
|
||||
return instance if instance.handles_body?
|
||||
rescue FetchError, Fetch::Error, URI::Error
|
||||
next
|
||||
end
|
||||
end
|
||||
nil # Return nil if no provider found
|
||||
end
|
||||
nil
|
||||
end
|
||||
end
|
||||
|
||||
# Base class for fetching content from a URL
|
||||
# The variable `body` will contain the package_data retrieved from the URL
|
||||
# The variable `package_data` will contain both and payload + metadata - this would be writen to a file.
|
||||
# The variable `payload` will contain the payload extracted from `package_data`
|
||||
# The variable `metadata` will contain the metadata extracted from `package_data`
|
||||
|
||||
# Job of the Provider class is to fetch the body from the URL, and then extract the package_data
|
||||
# and the filename from the body. The Package class will then take the body and split it into payload and metadata
|
||||
|
||||
# Base class for fetching content from a URL.
|
||||
#
|
||||
# A provider's job is to fetch the body, and to pull the package content and a
|
||||
# filename out of it. Splitting content into payload and metadata is the
|
||||
# Package class's job, not a provider's.
|
||||
class DefaultProvider
|
||||
MAX_SIZE = 1024 * 1024
|
||||
TIMEOUT = 10
|
||||
|
||||
attr_reader :url, :package
|
||||
|
||||
def self.handles_url?(url) = :maybe
|
||||
# Not `:maybe`. This is the last resort in PROVIDERS, so if it declined on a
|
||||
# failed fetch the loop would fall off the end and report "no provider could
|
||||
# handle this" for what is really a 404. Claiming any URL it understands the
|
||||
# scheme of lets the actual HTTP error reach the user.
|
||||
def self.handles_url?(url)
|
||||
scheme = URI(url.to_s).scheme
|
||||
%w[http https file].include?(scheme)
|
||||
rescue URI::InvalidURIError
|
||||
false
|
||||
end
|
||||
|
||||
def initialize(url, fetcher: Fetch.new(max_size: MAX_SIZE, timeout: TIMEOUT))
|
||||
@url = transform_url(url)
|
||||
@@ -42,7 +59,7 @@ module Picopackage
|
||||
populate_metadata
|
||||
end
|
||||
|
||||
def transform_url(url) = URI(url)
|
||||
def transform_url(url) = URI(url.to_s)
|
||||
|
||||
def body
|
||||
@body ||= @fetcher.fetch(@url)
|
||||
@@ -56,78 +73,104 @@ module Picopackage
|
||||
raise FetchError, "Failed to parse JSON response"
|
||||
end
|
||||
|
||||
def payload_timestamp = Time.now.httpdate
|
||||
# ISO 8601 throughout: sortable, unambiguous, and the same shape from every
|
||||
# provider, which httpdate-here-iso-there was not.
|
||||
def payload_timestamp = Time.now.utc.iso8601
|
||||
|
||||
def handles_body?
|
||||
true
|
||||
rescue FileTooLargeError, Net::HTTPError, RuntimeError
|
||||
!body.nil? && !body.empty?
|
||||
rescue FetchError, FileTooLargeError
|
||||
false
|
||||
end
|
||||
|
||||
# Implement in subclass - this come from the `body`.
|
||||
# Spliting content into payload and metadata is the job of the Package class
|
||||
# Implement in a subclass when the content is wrapped in something (JSON,
|
||||
# HTML). Splitting payload from metadata is the Package class's job.
|
||||
def content = body
|
||||
|
||||
# Implement in subclass - this should return the filename extracted from the body - if it exists, but not from the metadata
|
||||
def filename = File.basename @url
|
||||
# Implement in a subclass when the body carries a filename. Not to be
|
||||
# confused with the filename in a package's metadata.
|
||||
def filename
|
||||
candidate = File.basename(@url.path.to_s)
|
||||
candidate.empty? ? nil : candidate
|
||||
end
|
||||
|
||||
# Upstream may or may not carry a metadata block, and the two halves of one
|
||||
# are not equally trustworthy:
|
||||
#
|
||||
# - *Claims* are upstream's to make — the canonical url (which outlives the
|
||||
# mirror we happened to fetch from), payload_version, licence, signature.
|
||||
# Those are adopted with `||=`.
|
||||
# - *Records* describe a particular copy on a particular disk, so upstream
|
||||
# cannot speak for ours. Checksums are recomputed from the payload we
|
||||
# actually received. Adopting a stale payload_checksum installed a file
|
||||
# that failed its own verification and read as locally modified before
|
||||
# anyone had touched it.
|
||||
#
|
||||
# The checksum covers the extracted payload, not the raw body: hashing the
|
||||
# body meant an upstream carrying a metadata block hashed its own metadata,
|
||||
# so it never matched the same payload fetched from anywhere else.
|
||||
def populate_metadata
|
||||
@package.filename ||= filename
|
||||
@package.url ||= @url
|
||||
@package.url ||= @url.to_s
|
||||
@package.payload_timestamp ||= payload_timestamp
|
||||
@package.payload_checksum ||= Payload.checksum(content)
|
||||
@package.payload_checksum = @package.payload_digest
|
||||
@package.base_checksum = @package.payload_checksum
|
||||
end
|
||||
end
|
||||
|
||||
class GithubGistProvider < DefaultProvider
|
||||
def self.handles_url?(url) = url.match?(%r{gist\.github\.com})
|
||||
def self.handles_url?(url) = url.to_s.match?(%r{gist\.github\.com})
|
||||
|
||||
def content = json_body["files"].values.first["content"]
|
||||
|
||||
def filename = json_body["files"].values.first["filename"]
|
||||
|
||||
def transform_url(url)
|
||||
gist_id = url[/gist\.github\.com\/[^\/]+\/([a-f0-9]+)/, 1]
|
||||
"https://api.github.com/gists/#{gist_id}"
|
||||
gist_id = url.to_s[%r{gist\.github\.com/[^/]+/([a-f0-9]+)}, 1]
|
||||
raise FetchError, "Could not extract a gist id from #{url}" if gist_id.nil?
|
||||
|
||||
URI("https://api.github.com/gists/#{gist_id}")
|
||||
end
|
||||
|
||||
def payload_timestamp
|
||||
Time.parse(json_body["created_at"])
|
||||
rescue ArgumentError
|
||||
nil
|
||||
Time.parse(json_body["updated_at"] || json_body["created_at"]).utc.iso8601
|
||||
rescue ArgumentError, TypeError
|
||||
super
|
||||
end
|
||||
end
|
||||
|
||||
class OpenGistProvider < DefaultProvider
|
||||
def handles_url?(url) = :maybe
|
||||
# Speculative: it appends ".json" to anything and sees what comes back, so it
|
||||
# must not inherit DefaultProvider's unconditional claim.
|
||||
def self.handles_url?(url) = :maybe
|
||||
|
||||
def transform_url(url) = "#{url}.json"
|
||||
def transform_url(url) = URI("#{url}.json")
|
||||
|
||||
def content = json_body.dig("files", 0, "content")
|
||||
|
||||
def filename = json_body.dig("files", 0, "filename")
|
||||
|
||||
# Only claims the URL if the JSON actually looks like a gist listing.
|
||||
def handles_body?
|
||||
content && filename
|
||||
rescue FileTooLargeError, Net::HTTPError, RuntimeError
|
||||
!content.nil? && !filename.nil?
|
||||
rescue FetchError, FileTooLargeError
|
||||
false
|
||||
end
|
||||
# If we successfully fetch the body, and the body contains content and a filename, then we can handle the body
|
||||
end
|
||||
|
||||
class FileProvider < DefaultProvider
|
||||
def self.handles_url?(url) = File.exist?(url)
|
||||
def self.handles_url?(url) = !url.to_s.empty? && File.exist?(url.to_s)
|
||||
|
||||
def transform_url(url) = Pathname(url)
|
||||
def transform_url(url) = Pathname(url.to_s)
|
||||
|
||||
def content = url.read
|
||||
|
||||
def filename = url.basename.to_s
|
||||
|
||||
def payload_timestamp
|
||||
url.mtime.httpdate
|
||||
url.mtime.utc.iso8601
|
||||
rescue Errno::ENOENT
|
||||
nil
|
||||
super
|
||||
end
|
||||
end
|
||||
|
||||
|
||||
@@ -1,11 +1,31 @@
|
||||
# frozen_string_literal: true
|
||||
|
||||
module Picopackage
|
||||
module Scanner
|
||||
Found = Struct.new(:path, :package)
|
||||
|
||||
# Scanning walks whole trees, so it has to survive binaries, unreadable
|
||||
# files and anything large enough that reading it to look for a comment
|
||||
# would be silly.
|
||||
MAX_SCAN_SIZE = 1024 * 1024
|
||||
|
||||
def self.scan(directory, pattern: "**/*")
|
||||
Dir.glob(File.join(directory, pattern)).select do |file|
|
||||
next unless File.file?(file)
|
||||
content = File.read(file)
|
||||
content.match?(Package::METADATA_PATTERN)
|
||||
end.map { |file| Package.new(file) }
|
||||
Dir.glob(File.join(directory, pattern)).sort.filter_map do |path|
|
||||
content = readable_text(path)
|
||||
next unless content&.match?(METADATA_PATTERN)
|
||||
|
||||
Found.new(path, Package.new(content: content))
|
||||
end
|
||||
end
|
||||
|
||||
def self.readable_text(path)
|
||||
return nil unless File.file?(path)
|
||||
return nil if File.size(path) > MAX_SCAN_SIZE
|
||||
|
||||
content = File.read(path, encoding: "UTF-8")
|
||||
content.valid_encoding? ? content : nil
|
||||
rescue SystemCallError, ArgumentError
|
||||
nil
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# frozen_string_literal: true
|
||||
|
||||
module Picopackage
|
||||
VERSION = "0.2.1"
|
||||
VERSION = "0.3.0"
|
||||
end
|
||||
|
||||
@@ -1,25 +1,195 @@
|
||||
## Package Installation Flow
|
||||
## Resolution rules
|
||||
|
||||
1. **Fetch URL**
|
||||
└─> Download content
|
||||
└─> Build Package Instance (Metadata & Payload)
|
||||
└─> Check local file status
|
||||
Two principles decide every case below:
|
||||
|
||||
2. **Local File Check**
|
||||
├─> If file doesn't exist:
|
||||
│ └─> Save file
|
||||
│
|
||||
└─> If file exists:
|
||||
└─> Compare versions
|
||||
├─> If older/same:
|
||||
│ └─> "Package already installed"
|
||||
│
|
||||
└─> If newer:
|
||||
└─> Check local modifications
|
||||
├─> If modified:
|
||||
│ └─> "Local modifications detected"
|
||||
│ └─> "Use 'update <file_path>'"
|
||||
│
|
||||
└─> If unmodified:
|
||||
└─> "Update available"
|
||||
└─> "Use 'update <file_path> -f' to force update"
|
||||
1. **The checksum decides, never the timestamp.** mtime is unreliable (a raw
|
||||
file re-uploaded unchanged gets a new mtime; a gist mirrored elsewhere gets
|
||||
an arbitrary one) and it is not evidence about content. If the payload
|
||||
checksums agree, the payload is settled — there is nothing to resolve no
|
||||
matter what the timestamps say. This is why every `????` in the old table
|
||||
collapsed: all of those rows assumed equal checksums, so all of them are
|
||||
metadata questions, not content questions.
|
||||
|
||||
2. **Merge the payload, regenerate the metadata.** The metadata block is
|
||||
derived, not authored, so it never takes part in a merge — merging it would
|
||||
conflict on every single update. Three-way merges run on the payload alone
|
||||
and the block is rewritten afterwards.
|
||||
|
||||
### Terms
|
||||
|
||||
- **payload** — the file with its metadata block removed, then normalised.
|
||||
- **bare** — a file with no metadata block. A hand-copied file, or an upstream
|
||||
that has never been packaged.
|
||||
- **`payload_checksum`** — checksum of the payload as it sits in this file.
|
||||
Local edits break it, which is how modification is detected.
|
||||
- **`base_checksum`** — checksum of the upstream payload this file was last
|
||||
reconciled with. Equal to `payload_checksum` on a clean install; they diverge
|
||||
after a merge or a local edit. This is the merge base pointer and the cache
|
||||
key.
|
||||
|
||||
### Same payload checksum
|
||||
|
||||
The content is identical, so the only question is which metadata to keep.
|
||||
|
||||
| Remote | Local | Action | State |
|
||||
|---------|---------|-------------------------------------------|-------------|
|
||||
| Bare | missing | Write payload + generated metadata | `installed` |
|
||||
| Package | missing | Write payload + upstream metadata | `installed` |
|
||||
| Bare | Bare | Adopt: attach generated metadata | `adopted` |
|
||||
| Package | Bare | Adopt: attach upstream metadata | `adopted` |
|
||||
| Bare | Package | Nothing to do | `current` |
|
||||
| Package | Package | Rewrite only if the metadata itself changed | `current` |
|
||||
|
||||
mtime appears nowhere in this table. That is the point of principle 1.
|
||||
|
||||
### Different payload checksum
|
||||
|
||||
| Local | Modified | Base available | Action | State |
|
||||
|--------------------|----------|----------------|-------------------------------------|-------------|
|
||||
| missing | — | — | Write it | `installed` |
|
||||
| Package | no | — | Fast-forward to upstream | `updated` |
|
||||
| Package | yes | yes | Three-way merge, clean | `merged` |
|
||||
| Package | yes | yes | Three-way merge, conflicted | `conflict` |
|
||||
| Package | yes | no | Refuse — no ancestor to merge from | `conflict` |
|
||||
| Bare | n/a | never | Refuse — unrelated file, same name | `conflict` |
|
||||
| any | any | any | `--force` overwrites unconditionally | `updated` |
|
||||
|
||||
A bare local file whose payload differs is not a modified picopackage — it is an
|
||||
unrelated file that happens to share a name. There is no ancestor and no claim
|
||||
that the two share a history, so merging would be guesswork. Refuse and say so.
|
||||
|
||||
### Conflicts are non-destructive
|
||||
|
||||
`git`/`diff3` conflict markers are written to `<file>.picopackage-merge`, beside
|
||||
the original. The original is never touched. A package manager with no `reset`
|
||||
and no index has no business writing conflict markers into a file the user may
|
||||
not have committed.
|
||||
|
||||
## Merge base cache
|
||||
|
||||
Three-way merge needs the common ancestor — the payload exactly as upstream last
|
||||
sent it. It is content-addressed under `$XDG_CACHE_HOME/picopackage/sha256/ab/cdef…`
|
||||
and written on every save, so a file installed by `ppkg` carries the pointer to
|
||||
its own ancestor in `base_checksum`.
|
||||
|
||||
Cache misses are expected and survivable: a file installed on another machine, a
|
||||
cleared cache, a package predating this feature. The fallback is to refuse the
|
||||
merge and say why, which is honest. A future `source_url` pinned to an immutable
|
||||
revision would let us refetch the ancestor instead of refusing — see below.
|
||||
|
||||
## Update UX and signing (designed 2026-08-13, not implemented)
|
||||
|
||||
The security question splits in three, and today's answers are: **first
|
||||
install** — pure trust in the URL, same as copy+paste, mitigated by "it's one
|
||||
file and you can read it"; **update** — worse than copy+paste, because `ppkg
|
||||
update` fetches whatever the mutable url serves and writes it with less
|
||||
friction than a human re-pasting; **transport** — plain `http` is currently
|
||||
allowed. The checksum contributes nothing adversarial: it lives inside the
|
||||
file it hashes and is regenerated on every write, so it is integrity against
|
||||
yourself, not against an attacker. The design below closes the update gap.
|
||||
|
||||
### Diff by default
|
||||
|
||||
`ppkg update` shows the diff and asks before writing. The diff shown is
|
||||
**ancestor → upstream** — the untrusted delta. Local edits are not news to the
|
||||
user, and the Resolver already holds all three payloads at that point, so this
|
||||
is nearly free. We are not a diff tool: shell out (`git diff --no-index`, else
|
||||
`diff -u`), respect `$PAGER` — the same philosophy as merge.rb. `-y`/`--yes`
|
||||
skips the prompt for scripts; it never covers an identity change (below).
|
||||
|
||||
### Identity: SSH signatures, TOFU-pinned
|
||||
|
||||
SSH rather than Sigstore. A full Sigstore bundle is 4–10KB of JSON — routinely
|
||||
larger than the payload it certifies — and drags a dependency tree into a
|
||||
zero-dependency gem. An armored ed25519 ssh signature is ~0.5KB, embeds in the
|
||||
metadata block, and the authors this format serves (gists, blogs) already hold
|
||||
ssh keys, with `github.com/<user>.keys` as a verification channel separate
|
||||
from the file itself. What Sigstore would add — identity that survives key
|
||||
loss, a transparency log — is real but not a different security class at this
|
||||
scale. It stays a possible opt-in upgrade, not the price of entry.
|
||||
|
||||
- The signature covers the **normalized payload + the claims** (url, filename,
|
||||
payload_version). Signing the payload alone leaves `url` unsigned, so a
|
||||
tampered first install could silently redirect every future update. The
|
||||
claims/records split in provider.rb is the seam: sign the claims, keep
|
||||
deriving the records.
|
||||
- TOFU: the key fingerprint is pinned at install, or the first time a
|
||||
signature appears on an already-installed file. Unsigned packages keep
|
||||
working; signing is opt-in per file.
|
||||
|
||||
### Key changes are a hard stop
|
||||
|
||||
The known_hosts model: banner, refuse, distinct exit code — not a y/n prompt
|
||||
someone can fat-finger through. The only door is `--accept-key <fingerprint>`;
|
||||
making the human transcribe the new fingerprint is what proves they looked.
|
||||
|
||||
Routine updates and identity changes are different signals and get different
|
||||
ceremonies. "Upstream changed, here's the diff" is routine: default-proceed,
|
||||
informational. "The signing identity changed" is how account compromise
|
||||
presents: default-refuse. Flattening both into one always-maximum-caution
|
||||
ceremony is how alarm fatigue happens.
|
||||
|
||||
### No rotation chains
|
||||
|
||||
Old-key-signs-new-key succession (signify/TUF style) was considered and
|
||||
dropped. If every key change is loud and manual anyway, a valid rotation proof
|
||||
can only *downgrade* the alarm — and a stolen key holds everything needed to
|
||||
mint that proof, so chains sharpen theft (a thief can rotate the author out of
|
||||
their own package) while buying convenience only for planned rotation. At this
|
||||
scale, "key changed, human reads the diff carefully, re-pins" *is* the
|
||||
rotation ceremony, and it covers loss and theft with the same motion.
|
||||
|
||||
### Exit codes are the contract
|
||||
|
||||
Callers — scripts and reviewing agents — build policy on exit codes, not on
|
||||
prose. An agent's policy may auto-accept a clean diff under a continuous key;
|
||||
an identity change must be unrepresentable in that path — no flag an agent
|
||||
routinely passes gets through it.
|
||||
|
||||
| Code | Meaning |
|
||||
|------|------------------------------------------------------------|
|
||||
| 0 | written (installed/adopted/updated/merged) or already current |
|
||||
| 1 | conflict — original untouched, as ever |
|
||||
| 2 | declined — user said no to the diff |
|
||||
| 3 | identity change — signing key differs from the pin |
|
||||
|
||||
### Review is the content check; the signature is the identity check
|
||||
|
||||
A signature answers "same author?" and says nothing about whether the code is
|
||||
safe. Review answers "does this change look sane?" and says nothing about who
|
||||
made it. They are complements, and single files invert the economics that make
|
||||
library review hopeless: the whole unit — old file, new file, three-way diff,
|
||||
the project it lives in — fits in one reading (or one context window), and
|
||||
with no install hooks, build steps, or dependencies, what you review is
|
||||
exactly what runs.
|
||||
|
||||
The gem's job is to surface facts, not to judge: emit the diff plus the
|
||||
machine-readable facts (`--json` — key changed, size delta, e.g. "adds
|
||||
net/http") and exit accordingly. Any LLM stays out of the gem. Note for
|
||||
whoever builds the reviewing agent: the payload is adversarial input —
|
||||
prompt injection in comments, aimed at the reviewer, is an expected attack.
|
||||
|
||||
### Order of work
|
||||
|
||||
1. Refuse plain `http`, including via redirect — cheap, orthogonal.
|
||||
2. `source_url` pinning (see open question below) — makes the update diff a
|
||||
pointer change and recovers merge bases on cache miss.
|
||||
3. Diff-by-default + the exit-code contract.
|
||||
4. SSH signing and pinning.
|
||||
|
||||
## Open questions
|
||||
|
||||
- **Immutable source pinning.** `url` is mutable ("whatever is at this address
|
||||
today"), which means builds aren't reproducible and "upstream improved" is
|
||||
indistinguishable from "upstream was compromised". Wants splitting into a
|
||||
pinned `source_url` (gist revision SHA, commit SHA) and a mutable `update_url`
|
||||
used only for checking. This also recovers merge bases on a cache miss.
|
||||
- **Multi-file packages.** `Jmap::Client` + `Jmap::Http` are one unit of reuse
|
||||
across two files. Either the spec grows a flat file-set, or single-file
|
||||
packages accept that they define several classes in one file and lose Zeitwerk
|
||||
file-per-class layout. Unresolved, and the first real consumer hits it.
|
||||
- **Comment styles.** `#` covers Ruby/Python/shell/YAML/Perl. Needs a table for
|
||||
`//`, `--`, `<!-- -->`, `/* */`, `;` before the format is honestly
|
||||
language-agnostic.
|
||||
- **Unknown metadata keys** are preserved verbatim on rewrite (see
|
||||
`Metadata#extra`), so optional spec fields survive a tool that predates them.
|
||||
|
||||
+1
-1
@@ -33,7 +33,7 @@ Gem::Specification.new do |spec|
|
||||
spec.executables = spec.files.grep(%r{\Aexe/}) { |f| File.basename(f) }
|
||||
spec.require_paths = ["lib"]
|
||||
|
||||
spec.add_dependency "open-uri", "~> 0.5"
|
||||
# Fetching is done with net/http directly — see the note in Fetch#fetch_http.
|
||||
spec.add_dependency "yaml", "~> 0.4"
|
||||
spec.add_dependency "digest"
|
||||
spec.add_development_dependency "debug"
|
||||
|
||||
@@ -0,0 +1,88 @@
|
||||
{
|
||||
"url": "https://api.github.com/gists/a06926360de8edf108be8591368ce1fb",
|
||||
"forks_url": "https://api.github.com/gists/a06926360de8edf108be8591368ce1fb/forks",
|
||||
"commits_url": "https://api.github.com/gists/a06926360de8edf108be8591368ce1fb/commits",
|
||||
"id": "a06926360de8edf108be8591368ce1fb",
|
||||
"node_id": "G_kwDNRzjaACBhMDY5MjYzNjBkZThlZGYxMDhiZTg1OTEzNjhjZTFmYg",
|
||||
"git_pull_url": "https://gist.github.com/a06926360de8edf108be8591368ce1fb.git",
|
||||
"git_push_url": "https://gist.github.com/a06926360de8edf108be8591368ce1fb.git",
|
||||
"html_url": "https://gist.github.com/dkam/a06926360de8edf108be8591368ce1fb",
|
||||
"files": {
|
||||
"ipv6_in_sqlite.rb": {
|
||||
"filename": "ipv6_in_sqlite.rb",
|
||||
"type": "application/x-ruby",
|
||||
"language": "Ruby",
|
||||
"raw_url": "https://gist.githubusercontent.com/dkam/a06926360de8edf108be8591368ce1fb/raw/13e47fb5d3c981a407997fc954b163fe50a37b45/ipv6_in_sqlite.rb",
|
||||
"size": 373,
|
||||
"truncated": false,
|
||||
"content": "class IpAddress < ApplicationRecord\n def ipv6\n (ipv6_high << 64) | ipv6_low\n end\n\n def ipv6=(addr)\n self.ipv6_high = addr >> 64\n self.ipv6_low = addr & 0xFFFFFFFFFFFFFFFF\n end\n\n def self.ipv6_range(start_ip, end_ip)\n where(ipv6_high: start_ip >> 64..end_ip >> 64)\n .where(ipv6_low: start_ip & 0xFFFFFFFFFFFFFFFF..end_ip & 0xFFFFFFFFFFFFFFFF)\n end\nend",
|
||||
"encoding": "utf-8"
|
||||
}
|
||||
},
|
||||
"public": false,
|
||||
"created_at": "2025-01-25T00:07:09Z",
|
||||
"updated_at": "2025-01-25T00:07:09Z",
|
||||
"description": "Store IPv6 address in SQLite, via Active Record",
|
||||
"comments": 0,
|
||||
"user": null,
|
||||
"comments_enabled": true,
|
||||
"comments_url": "https://api.github.com/gists/a06926360de8edf108be8591368ce1fb/comments",
|
||||
"owner": {
|
||||
"login": "dkam",
|
||||
"id": 18232,
|
||||
"node_id": "MDQ6VXNlcjE4MjMy",
|
||||
"avatar_url": "https://avatars.githubusercontent.com/u/18232?v=4",
|
||||
"gravatar_id": "",
|
||||
"url": "https://api.github.com/users/dkam",
|
||||
"html_url": "https://github.com/dkam",
|
||||
"followers_url": "https://api.github.com/users/dkam/followers",
|
||||
"following_url": "https://api.github.com/users/dkam/following{/other_user}",
|
||||
"gists_url": "https://api.github.com/users/dkam/gists{/gist_id}",
|
||||
"starred_url": "https://api.github.com/users/dkam/starred{/owner}{/repo}",
|
||||
"subscriptions_url": "https://api.github.com/users/dkam/subscriptions",
|
||||
"organizations_url": "https://api.github.com/users/dkam/orgs",
|
||||
"repos_url": "https://api.github.com/users/dkam/repos",
|
||||
"events_url": "https://api.github.com/users/dkam/events{/privacy}",
|
||||
"received_events_url": "https://api.github.com/users/dkam/received_events",
|
||||
"type": "User",
|
||||
"user_view_type": "public",
|
||||
"site_admin": false
|
||||
},
|
||||
"forks": [
|
||||
|
||||
],
|
||||
"history": [
|
||||
{
|
||||
"user": {
|
||||
"login": "dkam",
|
||||
"id": 18232,
|
||||
"node_id": "MDQ6VXNlcjE4MjMy",
|
||||
"avatar_url": "https://avatars.githubusercontent.com/u/18232?v=4",
|
||||
"gravatar_id": "",
|
||||
"url": "https://api.github.com/users/dkam",
|
||||
"html_url": "https://github.com/dkam",
|
||||
"followers_url": "https://api.github.com/users/dkam/followers",
|
||||
"following_url": "https://api.github.com/users/dkam/following{/other_user}",
|
||||
"gists_url": "https://api.github.com/users/dkam/gists{/gist_id}",
|
||||
"starred_url": "https://api.github.com/users/dkam/starred{/owner}{/repo}",
|
||||
"subscriptions_url": "https://api.github.com/users/dkam/subscriptions",
|
||||
"organizations_url": "https://api.github.com/users/dkam/orgs",
|
||||
"repos_url": "https://api.github.com/users/dkam/repos",
|
||||
"events_url": "https://api.github.com/users/dkam/events{/privacy}",
|
||||
"received_events_url": "https://api.github.com/users/dkam/received_events",
|
||||
"type": "User",
|
||||
"user_view_type": "public",
|
||||
"site_admin": false
|
||||
},
|
||||
"version": "7f52a7373edd4cbf186162dc71444792eece75b3",
|
||||
"committed_at": "2025-01-25T00:07:09Z",
|
||||
"change_status": {
|
||||
"total": 15,
|
||||
"additions": 15,
|
||||
"deletions": 0
|
||||
},
|
||||
"url": "https://api.github.com/gists/a06926360de8edf108be8591368ce1fb/7f52a7373edd4cbf186162dc71444792eece75b3"
|
||||
}
|
||||
],
|
||||
"truncated": false
|
||||
}
|
||||
@@ -0,0 +1,119 @@
|
||||
{
|
||||
"url": "https://api.github.com/gists/a06926360de8edf108be8591368ce1fb",
|
||||
"forks_url": "https://api.github.com/gists/a06926360de8edf108be8591368ce1fb/forks",
|
||||
"commits_url": "https://api.github.com/gists/a06926360de8edf108be8591368ce1fb/commits",
|
||||
"id": "a06926360de8edf108be8591368ce1fb",
|
||||
"node_id": "G_kwDNRzjaACBhMDY5MjYzNjBkZThlZGYxMDhiZTg1OTEzNjhjZTFmYg",
|
||||
"git_pull_url": "https://gist.github.com/a06926360de8edf108be8591368ce1fb.git",
|
||||
"git_push_url": "https://gist.github.com/a06926360de8edf108be8591368ce1fb.git",
|
||||
"html_url": "https://gist.github.com/dkam/a06926360de8edf108be8591368ce1fb",
|
||||
"files": {
|
||||
"ipv6_in_sqlite.rb": {
|
||||
"filename": "ipv6_in_sqlite.rb",
|
||||
"type": "application/x-ruby",
|
||||
"language": "Ruby",
|
||||
"raw_url": "https://gist.githubusercontent.com/dkam/a06926360de8edf108be8591368ce1fb/raw/458a32b55c57e42d6e541f29317e9445a87d4e43/ipv6_in_sqlite.rb",
|
||||
"size": 384,
|
||||
"truncated": false,
|
||||
"content": "# Comments\nclass IpAddress < ApplicationRecord\n def ipv6\n (ipv6_high << 64) | ipv6_low\n end\n\n def ipv6=(addr)\n self.ipv6_high = addr >> 64\n self.ipv6_low = addr & 0xFFFFFFFFFFFFFFFF\n end\n\n def self.ipv6_range(start_ip, end_ip)\n where(ipv6_high: start_ip >> 64..end_ip >> 64)\n .where(ipv6_low: start_ip & 0xFFFFFFFFFFFFFFFF..end_ip & 0xFFFFFFFFFFFFFFFF)\n end\nend",
|
||||
"encoding": "utf-8"
|
||||
}
|
||||
},
|
||||
"public": false,
|
||||
"created_at": "2025-01-25T00:07:09Z",
|
||||
"updated_at": "2025-01-25T00:08:48Z",
|
||||
"description": "Store IPv6 address in SQLite, via Active Record",
|
||||
"comments": 0,
|
||||
"user": null,
|
||||
"comments_enabled": true,
|
||||
"comments_url": "https://api.github.com/gists/a06926360de8edf108be8591368ce1fb/comments",
|
||||
"owner": {
|
||||
"login": "dkam",
|
||||
"id": 18232,
|
||||
"node_id": "MDQ6VXNlcjE4MjMy",
|
||||
"avatar_url": "https://avatars.githubusercontent.com/u/18232?v=4",
|
||||
"gravatar_id": "",
|
||||
"url": "https://api.github.com/users/dkam",
|
||||
"html_url": "https://github.com/dkam",
|
||||
"followers_url": "https://api.github.com/users/dkam/followers",
|
||||
"following_url": "https://api.github.com/users/dkam/following{/other_user}",
|
||||
"gists_url": "https://api.github.com/users/dkam/gists{/gist_id}",
|
||||
"starred_url": "https://api.github.com/users/dkam/starred{/owner}{/repo}",
|
||||
"subscriptions_url": "https://api.github.com/users/dkam/subscriptions",
|
||||
"organizations_url": "https://api.github.com/users/dkam/orgs",
|
||||
"repos_url": "https://api.github.com/users/dkam/repos",
|
||||
"events_url": "https://api.github.com/users/dkam/events{/privacy}",
|
||||
"received_events_url": "https://api.github.com/users/dkam/received_events",
|
||||
"type": "User",
|
||||
"user_view_type": "public",
|
||||
"site_admin": false
|
||||
},
|
||||
"forks": [
|
||||
|
||||
],
|
||||
"history": [
|
||||
{
|
||||
"user": {
|
||||
"login": "dkam",
|
||||
"id": 18232,
|
||||
"node_id": "MDQ6VXNlcjE4MjMy",
|
||||
"avatar_url": "https://avatars.githubusercontent.com/u/18232?v=4",
|
||||
"gravatar_id": "",
|
||||
"url": "https://api.github.com/users/dkam",
|
||||
"html_url": "https://github.com/dkam",
|
||||
"followers_url": "https://api.github.com/users/dkam/followers",
|
||||
"following_url": "https://api.github.com/users/dkam/following{/other_user}",
|
||||
"gists_url": "https://api.github.com/users/dkam/gists{/gist_id}",
|
||||
"starred_url": "https://api.github.com/users/dkam/starred{/owner}{/repo}",
|
||||
"subscriptions_url": "https://api.github.com/users/dkam/subscriptions",
|
||||
"organizations_url": "https://api.github.com/users/dkam/orgs",
|
||||
"repos_url": "https://api.github.com/users/dkam/repos",
|
||||
"events_url": "https://api.github.com/users/dkam/events{/privacy}",
|
||||
"received_events_url": "https://api.github.com/users/dkam/received_events",
|
||||
"type": "User",
|
||||
"user_view_type": "public",
|
||||
"site_admin": false
|
||||
},
|
||||
"version": "9774aa95cbf7c1fa7bf2ea27b2cb66898d0769c8",
|
||||
"committed_at": "2025-01-25T00:08:48Z",
|
||||
"change_status": {
|
||||
"total": 1,
|
||||
"additions": 1,
|
||||
"deletions": 0
|
||||
},
|
||||
"url": "https://api.github.com/gists/a06926360de8edf108be8591368ce1fb/9774aa95cbf7c1fa7bf2ea27b2cb66898d0769c8"
|
||||
},
|
||||
{
|
||||
"user": {
|
||||
"login": "dkam",
|
||||
"id": 18232,
|
||||
"node_id": "MDQ6VXNlcjE4MjMy",
|
||||
"avatar_url": "https://avatars.githubusercontent.com/u/18232?v=4",
|
||||
"gravatar_id": "",
|
||||
"url": "https://api.github.com/users/dkam",
|
||||
"html_url": "https://github.com/dkam",
|
||||
"followers_url": "https://api.github.com/users/dkam/followers",
|
||||
"following_url": "https://api.github.com/users/dkam/following{/other_user}",
|
||||
"gists_url": "https://api.github.com/users/dkam/gists{/gist_id}",
|
||||
"starred_url": "https://api.github.com/users/dkam/starred{/owner}{/repo}",
|
||||
"subscriptions_url": "https://api.github.com/users/dkam/subscriptions",
|
||||
"organizations_url": "https://api.github.com/users/dkam/orgs",
|
||||
"repos_url": "https://api.github.com/users/dkam/repos",
|
||||
"events_url": "https://api.github.com/users/dkam/events{/privacy}",
|
||||
"received_events_url": "https://api.github.com/users/dkam/received_events",
|
||||
"type": "User",
|
||||
"user_view_type": "public",
|
||||
"site_admin": false
|
||||
},
|
||||
"version": "7f52a7373edd4cbf186162dc71444792eece75b3",
|
||||
"committed_at": "2025-01-25T00:07:09Z",
|
||||
"change_status": {
|
||||
"total": 15,
|
||||
"additions": 15,
|
||||
"deletions": 0
|
||||
},
|
||||
"url": "https://api.github.com/gists/a06926360de8edf108be8591368ce1fb/7f52a7373edd4cbf186162dc71444792eece75b3"
|
||||
}
|
||||
],
|
||||
"truncated": false
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
# Make an array of hashes contain a unique value for key:
|
||||
def uniquify_array(array, key)
|
||||
array.group_by { |item| item[key] }.map do |_, items|
|
||||
(items.length > 1) ? yield(items) : items.first
|
||||
end
|
||||
end
|
||||
|
||||
# new_array = uniquify_array(array, :some_key) do |dupes|
|
||||
# dupes.max_by { |item| item[:updated_at] }
|
||||
# end
|
||||
|
||||
# @PICOPACKAGE_START
|
||||
# ---
|
||||
# filename: uniquify_array.rb
|
||||
# url: https://api.github.com/gists/525ac4177964549ee2f2ca7febd03eea
|
||||
# version: 0.1
|
||||
# source_url: https://gist.github.com/dkam/525ac4177964549ee2f2ca7febd03eea
|
||||
# @PICOPACKAGE_END
|
||||
@@ -0,0 +1,36 @@
|
||||
# frozen_string_literal: true
|
||||
|
||||
require "test_helper"
|
||||
|
||||
class TestCache < Minitest::Test
|
||||
def with_cache
|
||||
Dir.mktmpdir("picopackage-cache") { |root| yield Picopackage::Cache.new(root: root) }
|
||||
end
|
||||
|
||||
def test_stores_and_fetches_by_content
|
||||
with_cache do |cache|
|
||||
checksum = cache.store("hello\n")
|
||||
assert_equal Picopackage::Payload.checksum("hello\n"), checksum
|
||||
assert_equal "hello\n", cache.fetch(checksum)
|
||||
end
|
||||
end
|
||||
|
||||
def test_storing_twice_is_harmless
|
||||
with_cache do |cache|
|
||||
a = cache.store("hello\n")
|
||||
b = cache.store("hello\n")
|
||||
assert_equal a, b
|
||||
assert_equal "hello\n", cache.fetch(a)
|
||||
end
|
||||
end
|
||||
|
||||
def test_unknown_and_malformed_checksums_return_nil
|
||||
with_cache do |cache|
|
||||
assert_nil cache.fetch(Picopackage::Payload.checksum("never stored"))
|
||||
assert_nil cache.fetch(nil)
|
||||
assert_nil cache.fetch("")
|
||||
assert_nil cache.fetch("sha256:../../escape")
|
||||
refute cache.include?(nil)
|
||||
end
|
||||
end
|
||||
end
|
||||
@@ -4,3 +4,29 @@ $LOAD_PATH.unshift File.expand_path("../lib", __dir__)
|
||||
require "picopackage"
|
||||
|
||||
require "minitest/autorun"
|
||||
require "tmpdir"
|
||||
|
||||
module PicopackageTest
|
||||
# Every test gets its own cache root: the merge base cache is global state and
|
||||
# a suite that shared one would pass or fail depending on what ran before it.
|
||||
def with_dirs
|
||||
Dir.mktmpdir("picopackage-test") do |root|
|
||||
upstream = File.join(root, "upstream")
|
||||
project = File.join(root, "project")
|
||||
[upstream, project].each { |dir| Dir.mkdir(dir) }
|
||||
yield(upstream, project, Picopackage::Cache.new(root: File.join(root, "cache")))
|
||||
end
|
||||
end
|
||||
|
||||
# Stands in for a server: a real file, fetched through the real FileProvider,
|
||||
# so provider metadata population is exercised rather than faked.
|
||||
def publish(dir, name, content)
|
||||
path = File.join(dir, name)
|
||||
File.write(path, content)
|
||||
path
|
||||
end
|
||||
|
||||
def upstream_package(path)
|
||||
Picopackage::FileProvider.new(path).package
|
||||
end
|
||||
end
|
||||
|
||||
@@ -0,0 +1,55 @@
|
||||
# frozen_string_literal: true
|
||||
|
||||
require "test_helper"
|
||||
|
||||
class TestMerge < Minitest::Test
|
||||
BASE = "one\ntwo\nthree\n"
|
||||
|
||||
def test_a_backend_is_available
|
||||
assert Picopackage::Merge.available?, "expected git or diff3 on PATH"
|
||||
end
|
||||
|
||||
def test_non_overlapping_changes_merge_cleanly
|
||||
result = Picopackage::Merge.three_way(
|
||||
base: BASE,
|
||||
ours: "ONE\ntwo\nthree\n",
|
||||
theirs: "one\ntwo\nTHREE\n"
|
||||
)
|
||||
|
||||
assert_predicate result, :clean?
|
||||
assert_equal "ONE\ntwo\nTHREE\n", result.content
|
||||
end
|
||||
|
||||
def test_identical_changes_on_both_sides_merge_cleanly
|
||||
changed = "one\nTWO\nthree\n"
|
||||
result = Picopackage::Merge.three_way(base: BASE, ours: changed, theirs: changed)
|
||||
|
||||
assert_predicate result, :clean?
|
||||
assert_equal changed, result.content
|
||||
end
|
||||
|
||||
def test_overlapping_changes_conflict
|
||||
result = Picopackage::Merge.three_way(
|
||||
base: BASE,
|
||||
ours: "one\nMINE\nthree\n",
|
||||
theirs: "one\nTHEIRS\nthree\n"
|
||||
)
|
||||
|
||||
assert_predicate result, :conflict?
|
||||
assert_operator result.conflicts, :>, 0
|
||||
assert_includes result.content, "MINE"
|
||||
assert_includes result.content, "THEIRS"
|
||||
assert_includes result.content, "<<<<<<<"
|
||||
end
|
||||
|
||||
def test_labels_appear_in_conflict_markers
|
||||
result = Picopackage::Merge.three_way(
|
||||
base: BASE,
|
||||
ours: "one\nMINE\nthree\n",
|
||||
theirs: "one\nTHEIRS\nthree\n"
|
||||
)
|
||||
|
||||
assert_includes result.content, "local"
|
||||
assert_includes result.content, "upstream"
|
||||
end
|
||||
end
|
||||
@@ -0,0 +1,106 @@
|
||||
# frozen_string_literal: true
|
||||
|
||||
require "test_helper"
|
||||
|
||||
class TestPackage < Minitest::Test
|
||||
include PicopackageTest
|
||||
|
||||
BARE = "def hello\n :world\nend\n"
|
||||
|
||||
def test_it_has_a_version_number
|
||||
refute_nil ::Picopackage::VERSION
|
||||
end
|
||||
|
||||
def test_a_bare_file_is_bare
|
||||
assert Picopackage::Package.new(content: BARE).bare?
|
||||
end
|
||||
|
||||
def test_a_generated_package_parses_back_as_a_package
|
||||
package = Picopackage::Package.new(content: BARE).init_metadata(filename: "hello.rb")
|
||||
round_tripped = Picopackage::Package.new(content: package.generate_package)
|
||||
|
||||
refute_predicate round_tripped, :bare?
|
||||
assert_equal "hello.rb", round_tripped.filename
|
||||
end
|
||||
|
||||
def test_reads_metadata_from_a_packaged_file
|
||||
package = Picopackage::Package.from_file("test/files/uniquify_array_packaged.rb")
|
||||
assert_equal "uniquify_array.rb", package.filename
|
||||
assert_equal "https://api.github.com/gists/525ac4177964549ee2f2ca7febd03eea", package.url
|
||||
end
|
||||
|
||||
# This was the bug that made every file look locally modified: the checksum
|
||||
# written into a package hashed the un-normalised payload while verification
|
||||
# hashed the normalised one, so nothing ever verified against itself.
|
||||
def test_a_freshly_packaged_file_verifies_against_its_own_checksum
|
||||
package = Picopackage::Package.new(content: BARE).init_metadata(filename: "hello.rb")
|
||||
written = Picopackage::Package.new(content: package.generate_package)
|
||||
|
||||
assert written.verify_payload, "a package should satisfy its own checksum"
|
||||
refute_predicate written, :modified?
|
||||
end
|
||||
|
||||
def test_payload_digest_is_stable_across_packaging
|
||||
bare = Picopackage::Package.new(content: BARE)
|
||||
packaged = Picopackage::Package.new(
|
||||
content: Picopackage::Package.new(content: BARE).init_metadata(filename: "hello.rb").generate_package
|
||||
)
|
||||
|
||||
assert_equal bare.payload_digest, packaged.payload_digest,
|
||||
"adding a metadata block must not change the payload digest"
|
||||
end
|
||||
|
||||
def test_editing_a_payload_is_detected
|
||||
packaged = Picopackage::Package.new(content: BARE).init_metadata(filename: "hello.rb").generate_package
|
||||
edited = Picopackage::Package.new(content: packaged.sub(":world", ":everyone"))
|
||||
|
||||
assert_predicate edited, :modified?
|
||||
end
|
||||
|
||||
def test_unmodelled_metadata_keys_survive_a_rewrite
|
||||
content = <<~RUBY
|
||||
code_here
|
||||
|
||||
# @PICOPACKAGE_START
|
||||
# ---
|
||||
# filename: thing.rb
|
||||
# licence: MIT
|
||||
# test_url: https://example.com/thing_test.rb
|
||||
# @PICOPACKAGE_END
|
||||
RUBY
|
||||
|
||||
package = Picopackage::Package.new(content: content)
|
||||
assert_equal "MIT", package.metadata.extra["licence"]
|
||||
|
||||
rewritten = Picopackage::Package.new(content: package.generate_package)
|
||||
assert_equal "MIT", rewritten.metadata.extra["licence"]
|
||||
assert_equal "https://example.com/thing_test.rb", rewritten.metadata.extra["test_url"]
|
||||
end
|
||||
|
||||
# A checksum is derived, never preserved. An author who edits a packaged file
|
||||
# by hand and re-runs the tool must get a block describing what is now there,
|
||||
# not the value that happened to already be sitting in it.
|
||||
def test_a_stale_recorded_checksum_is_corrected_on_write
|
||||
content = "code\n\n# @PICOPACKAGE_START\n# ---\n# filename: thing.rb\n" \
|
||||
"# payload_checksum: sha256:#{"de" * 32}\n# @PICOPACKAGE_END\n"
|
||||
|
||||
stale = Picopackage::Package.new(content: content)
|
||||
refute stale.verify_payload, "precondition: the recorded checksum is wrong"
|
||||
|
||||
rewritten = Picopackage::Package.new(content: stale.generate_package)
|
||||
assert rewritten.verify_payload
|
||||
refute_predicate rewritten, :modified?
|
||||
end
|
||||
|
||||
def test_a_block_of_only_unmodelled_keys_is_not_treated_as_bare
|
||||
content = "code\n\n# @PICOPACKAGE_START\n# ---\n# licence: MIT\n# @PICOPACKAGE_END\n"
|
||||
refute_predicate Picopackage::Package.new(content: content), :bare?,
|
||||
"a block we don't fully understand is still a block"
|
||||
end
|
||||
|
||||
def test_malformed_front_matter_does_not_raise
|
||||
package = Picopackage::Package.from_file("test/files/broken_front_matter_1.rb")
|
||||
refute_nil package
|
||||
assert_predicate package.metadata, :empty?
|
||||
end
|
||||
end
|
||||
@@ -1,19 +0,0 @@
|
||||
# frozen_string_literal: true
|
||||
|
||||
require "test_helper"
|
||||
|
||||
class TestPicopackage < Minitest::Test
|
||||
def test_that_it_has_a_version_number
|
||||
refute_nil ::Picopackage::VERSION
|
||||
end
|
||||
|
||||
def test_it_can_load_a_picopackage_file
|
||||
sf = Picopackage::FileProvider.new(File.read("test/files/uniquify_array_packaged.rb"))
|
||||
assert_equal "uniquify_array_packaged.rb", sf.metadata.filename
|
||||
end
|
||||
|
||||
def test_it_can_create_a_picopackage_from_bare_file
|
||||
sf = Picopackage::FileProvider.new(File.read("test/files/uniquify_array_bare.rb"))
|
||||
assert_equal "uniquify_array_bare.rb", sf.metadata.filename
|
||||
end
|
||||
end
|
||||
@@ -0,0 +1,69 @@
|
||||
# frozen_string_literal: true
|
||||
|
||||
require "test_helper"
|
||||
|
||||
# Two captured responses for the same gist, one revision apart: v2 prepends a
|
||||
# `# Comments` line and carries a later `updated_at`. Real API shapes, no
|
||||
# network, and enough of a difference to drive an update through the resolver.
|
||||
class TestGithubGistProvider < Minitest::Test
|
||||
include PicopackageTest
|
||||
|
||||
GIST_URL = "https://gist.github.com/dkam/a06926360de8edf108be8591368ce1fb"
|
||||
|
||||
StubFetcher = Struct.new(:body) do
|
||||
def fetch(_uri) = body
|
||||
end
|
||||
|
||||
def gist(revision)
|
||||
Picopackage::GithubGistProvider.new(
|
||||
GIST_URL, fetcher: StubFetcher.new(File.read("test/files/gist_#{revision}.json"))
|
||||
)
|
||||
end
|
||||
|
||||
def test_resolves_the_page_url_to_the_api
|
||||
assert_equal "https://api.github.com/gists/a06926360de8edf108be8591368ce1fb",
|
||||
gist("v1").url.to_s
|
||||
end
|
||||
|
||||
def test_extracts_the_file_from_the_json_envelope
|
||||
package = gist("v1").package
|
||||
|
||||
assert_equal "ipv6_in_sqlite.rb", package.filename
|
||||
assert_includes package.payload, "def ipv6"
|
||||
refute_includes package.payload, "raw_url", "the JSON envelope must not leak into the payload"
|
||||
assert_predicate package, :bare?, "the gist carries no metadata block of its own"
|
||||
end
|
||||
|
||||
# The gist's own revision time, not the moment we happened to fetch it.
|
||||
def test_uses_the_gists_updated_at_as_the_timestamp
|
||||
assert_equal "2025-01-25T00:07:09Z", gist("v1").package.payload_timestamp
|
||||
assert_equal "2025-01-25T00:08:48Z", gist("v2").package.payload_timestamp
|
||||
end
|
||||
|
||||
def test_a_new_revision_is_a_different_payload
|
||||
refute_equal gist("v1").package.payload_digest, gist("v2").package.payload_digest
|
||||
end
|
||||
|
||||
# End to end against a real upstream shape: install v1, edit it, take v2.
|
||||
def test_updating_across_revisions_keeps_local_edits
|
||||
with_dirs do |_upstream, project, cache|
|
||||
local_path = File.join(project, "ipv6_in_sqlite.rb")
|
||||
assert_equal :installed, resolve(gist("v1").package, nil, local_path, cache).state
|
||||
|
||||
File.write(local_path, File.read(local_path).sub("where(ipv6_high:", "where!(ipv6_high:"))
|
||||
result = resolve(gist("v2").package, Picopackage::Package.from_file(local_path), local_path, cache)
|
||||
|
||||
assert_equal :merged, result.state, result.message
|
||||
merged = File.read(local_path)
|
||||
assert_includes merged, "where!(ipv6_high:", "local edit should survive"
|
||||
assert_includes merged, "# Comments", "the new revision should be applied"
|
||||
refute_includes merged, "<<<<<<<"
|
||||
end
|
||||
end
|
||||
|
||||
private
|
||||
|
||||
def resolve(remote, local, path, cache)
|
||||
Picopackage::Resolver.new(remote, local, path, cache: cache).resolve
|
||||
end
|
||||
end
|
||||
@@ -0,0 +1,340 @@
|
||||
# frozen_string_literal: true
|
||||
|
||||
require "test_helper"
|
||||
|
||||
class TestResolver < Minitest::Test
|
||||
include PicopackageTest
|
||||
|
||||
V1 = <<~RUBY
|
||||
module Poller
|
||||
INTERVAL = 30
|
||||
|
||||
def self.run
|
||||
fetch
|
||||
end
|
||||
end
|
||||
RUBY
|
||||
|
||||
# Upstream carrying its own block is optional, not required — but when it does,
|
||||
# only some of the block is upstream's to assert. This one is stale, as any
|
||||
# author who edits without re-running the tool will ship.
|
||||
PACKAGED = V1 + <<~RUBY
|
||||
# @PICOPACKAGE_START
|
||||
# ---
|
||||
# url: https://example.com/canonical/poller.rb
|
||||
# filename: upstream_name.rb
|
||||
# payload_version: 1.4.0
|
||||
# payload_checksum: sha256:#{"de" * 32}
|
||||
# licence: MIT
|
||||
# @PICOPACKAGE_END
|
||||
RUBY
|
||||
|
||||
def resolve(upstream_path, project, cache, force: false, filename: nil)
|
||||
Picopackage::Fetch.fetch(upstream_path, project, force: force, filename: filename, cache: cache)
|
||||
end
|
||||
|
||||
def test_installs_when_nothing_is_there
|
||||
with_dirs do |upstream, project, cache|
|
||||
path = publish(upstream, "poller.rb", V1)
|
||||
result = resolve(path, project, cache)
|
||||
|
||||
assert_equal :installed, result.state
|
||||
installed = Picopackage::Package.from_file(File.join(project, "poller.rb"))
|
||||
refute_predicate installed, :bare?
|
||||
assert_equal path, installed.url
|
||||
assert installed.verify_payload
|
||||
end
|
||||
end
|
||||
|
||||
# A bare upstream file is a valid picopackage source: nobody has to adopt the
|
||||
# format for their code to be installable. Everything the block would have
|
||||
# said gets derived from the fetch instead.
|
||||
def test_installs_from_an_upstream_with_no_metadata_block
|
||||
with_dirs do |upstream, project, cache|
|
||||
path = publish(upstream, "poller.rb", V1)
|
||||
assert_predicate upstream_package(path), :bare?, "precondition: upstream carries no block"
|
||||
|
||||
assert_equal :installed, resolve(path, project, cache).state
|
||||
installed = Picopackage::Package.from_file(File.join(project, "poller.rb"))
|
||||
assert installed.verify_payload
|
||||
assert_equal path, installed.url
|
||||
end
|
||||
end
|
||||
|
||||
def test_publisher_claims_are_adopted_and_local_records_are_recomputed
|
||||
with_dirs do |upstream, project, cache|
|
||||
path = publish(upstream, "poller.rb", PACKAGED)
|
||||
|
||||
assert_equal :installed, resolve(path, project, cache).state
|
||||
|
||||
# Claims: upstream's, and worth more than anything we could infer. The
|
||||
# declared filename beats the url's basename, so the file lands as the
|
||||
# author named it rather than as whatever the mirror called it.
|
||||
local_path = File.join(project, "upstream_name.rb")
|
||||
assert_path_exists local_path
|
||||
installed = Picopackage::Package.from_file(local_path)
|
||||
assert_equal "https://example.com/canonical/poller.rb", installed.url,
|
||||
"the canonical url should outlive the mirror we fetched from"
|
||||
assert_equal "1.4.0", installed.payload_version
|
||||
assert_equal "MIT", installed.metadata.extra["licence"]
|
||||
|
||||
# Records: ours, describing this copy on this disk.
|
||||
assert installed.verify_payload, "a freshly installed file must verify, whatever upstream claimed"
|
||||
refute_predicate installed, :modified?
|
||||
refute_predicate installed, :diverged?
|
||||
assert_equal "upstream_name.rb", installed.filename, "filename records where it actually landed"
|
||||
end
|
||||
end
|
||||
|
||||
def test_a_stale_upstream_checksum_does_not_break_a_later_merge
|
||||
with_dirs do |upstream, project, cache|
|
||||
path = publish(upstream, "poller.rb", PACKAGED)
|
||||
resolve(path, project, cache)
|
||||
local_path = File.join(project, "upstream_name.rb")
|
||||
|
||||
File.write(local_path, File.read(local_path).sub("INTERVAL = 30", "INTERVAL = 5"))
|
||||
File.write(path, PACKAGED.sub(" fetch\n", " fetch\n prune\n"))
|
||||
|
||||
result = resolve(path, project, cache)
|
||||
|
||||
assert_equal :merged, result.state, result.message
|
||||
merged = File.read(local_path)
|
||||
assert_includes merged, "INTERVAL = 5"
|
||||
assert_includes merged, "prune"
|
||||
assert_equal "MIT", Picopackage::Package.from_file(local_path).metadata.extra["licence"]
|
||||
end
|
||||
end
|
||||
|
||||
def test_reinstalling_the_same_thing_is_a_no_op
|
||||
with_dirs do |upstream, project, cache|
|
||||
path = publish(upstream, "poller.rb", V1)
|
||||
resolve(path, project, cache)
|
||||
before = File.read(File.join(project, "poller.rb"))
|
||||
|
||||
result = resolve(path, project, cache)
|
||||
|
||||
assert_equal :current, result.state
|
||||
assert_equal before, File.read(File.join(project, "poller.rb")), "an up-to-date file should not be rewritten"
|
||||
end
|
||||
end
|
||||
|
||||
# The pleasant case: a file you pasted in by hand months ago gets its
|
||||
# provenance attached without its content being touched.
|
||||
def test_adopts_a_bare_local_file_with_identical_content
|
||||
with_dirs do |upstream, project, cache|
|
||||
path = publish(upstream, "poller.rb", V1)
|
||||
File.write(File.join(project, "poller.rb"), V1)
|
||||
|
||||
result = resolve(path, project, cache)
|
||||
|
||||
assert_equal :adopted, result.state
|
||||
adopted = Picopackage::Package.from_file(File.join(project, "poller.rb"))
|
||||
refute_predicate adopted, :bare?
|
||||
assert_equal Picopackage::Payload.normalize(V1), adopted.payload
|
||||
end
|
||||
end
|
||||
|
||||
def test_fast_forwards_an_unmodified_package
|
||||
with_dirs do |upstream, project, cache|
|
||||
path = publish(upstream, "poller.rb", V1)
|
||||
resolve(path, project, cache)
|
||||
|
||||
File.write(path, V1.sub("INTERVAL = 30", "INTERVAL = 60"))
|
||||
result = resolve(path, project, cache)
|
||||
|
||||
assert_equal :updated, result.state
|
||||
assert_includes File.read(File.join(project, "poller.rb")), "INTERVAL = 60"
|
||||
end
|
||||
end
|
||||
|
||||
# The headline capability: local edits and upstream edits both survive.
|
||||
def test_merges_local_edits_with_upstream_changes
|
||||
with_dirs do |upstream, project, cache|
|
||||
path = publish(upstream, "poller.rb", V1)
|
||||
resolve(path, project, cache)
|
||||
|
||||
local_path = File.join(project, "poller.rb")
|
||||
File.write(local_path, File.read(local_path).sub("INTERVAL = 30", "INTERVAL = 5 # tuned for us"))
|
||||
|
||||
File.write(path, V1.sub(" fetch\n", " fetch\n prune\n"))
|
||||
result = resolve(path, project, cache)
|
||||
|
||||
assert_equal :merged, result.state, result.message
|
||||
merged = File.read(local_path)
|
||||
assert_includes merged, "INTERVAL = 5 # tuned for us", "local edit should survive"
|
||||
assert_includes merged, "prune", "upstream change should be applied"
|
||||
refute_includes merged, "<<<<<<<"
|
||||
|
||||
package = Picopackage::Package.from_file(local_path)
|
||||
assert package.verify_payload, "a merged file should verify against its own recorded checksum"
|
||||
refute_equal package.payload_checksum, package.base_checksum,
|
||||
"base_checksum should still point at upstream, not at the merged result"
|
||||
end
|
||||
end
|
||||
|
||||
# A merged file must remain updatable: the base pointer has to advance to the
|
||||
# upstream payload, not to the merge result, or the next merge replays old
|
||||
# upstream changes as conflicts.
|
||||
def test_a_merged_file_can_be_updated_again
|
||||
with_dirs do |upstream, project, cache|
|
||||
path = publish(upstream, "poller.rb", V1)
|
||||
resolve(path, project, cache)
|
||||
local_path = File.join(project, "poller.rb")
|
||||
|
||||
File.write(local_path, File.read(local_path).sub("INTERVAL = 30", "INTERVAL = 5"))
|
||||
File.write(path, V1.sub(" fetch\n", " fetch\n prune\n"))
|
||||
assert_equal :merged, resolve(path, project, cache).state
|
||||
|
||||
File.write(path, V1.sub(" fetch\n", " fetch\n prune\n report\n"))
|
||||
result = resolve(path, project, cache)
|
||||
|
||||
assert_equal :merged, result.state, result.message
|
||||
merged = File.read(local_path)
|
||||
assert_includes merged, "INTERVAL = 5"
|
||||
assert_includes merged, "report"
|
||||
refute_includes merged, "<<<<<<<"
|
||||
end
|
||||
end
|
||||
|
||||
# Editing locally while upstream stands still is the common case, and there is
|
||||
# nothing to merge: upstream is still the ancestor we branched from.
|
||||
def test_local_edits_with_no_upstream_change_are_left_alone
|
||||
with_dirs do |upstream, project, cache|
|
||||
path = publish(upstream, "poller.rb", V1)
|
||||
resolve(path, project, cache)
|
||||
local_path = File.join(project, "poller.rb")
|
||||
|
||||
File.write(local_path, File.read(local_path).sub("INTERVAL = 30", "INTERVAL = 5"))
|
||||
before = File.read(local_path)
|
||||
|
||||
result = resolve(path, project, cache)
|
||||
|
||||
assert_equal :current, result.state, result.message
|
||||
assert_equal before, File.read(local_path), "an edited file with no upstream change must not be rewritten"
|
||||
end
|
||||
end
|
||||
|
||||
# Repopulating the merge base from an unchanged upstream is the whole reason
|
||||
# that branch still touches the cache.
|
||||
def test_an_unchanged_upstream_restores_a_lost_merge_base
|
||||
with_dirs do |upstream, project, cache|
|
||||
path = publish(upstream, "poller.rb", V1)
|
||||
resolve(path, project, cache)
|
||||
local_path = File.join(project, "poller.rb")
|
||||
File.write(local_path, File.read(local_path).sub("INTERVAL = 30", "INTERVAL = 5"))
|
||||
|
||||
# A fresh machine, or a cleared cache: the ancestor is gone.
|
||||
empty = Picopackage::Cache.new(root: File.join(Dir.mktmpdir, "empty"))
|
||||
assert_equal :current, resolve(path, project, empty).state
|
||||
|
||||
# Upstream now moves. The base recovered above makes this mergeable.
|
||||
File.write(path, V1.sub(" fetch\n", " fetch\n prune\n"))
|
||||
result = resolve(path, project, empty)
|
||||
|
||||
assert_equal :merged, result.state, result.message
|
||||
assert_includes File.read(local_path), "INTERVAL = 5"
|
||||
assert_includes File.read(local_path), "prune"
|
||||
end
|
||||
end
|
||||
|
||||
def test_conflicting_edits_leave_the_original_alone
|
||||
with_dirs do |upstream, project, cache|
|
||||
path = publish(upstream, "poller.rb", V1)
|
||||
resolve(path, project, cache)
|
||||
local_path = File.join(project, "poller.rb")
|
||||
|
||||
File.write(local_path, File.read(local_path).sub("INTERVAL = 30", "INTERVAL = 5"))
|
||||
before = File.read(local_path)
|
||||
|
||||
File.write(path, V1.sub("INTERVAL = 30", "INTERVAL = 90"))
|
||||
result = resolve(path, project, cache)
|
||||
|
||||
assert_equal :conflict, result.state
|
||||
assert_equal before, File.read(local_path), "the user's file must not be touched"
|
||||
|
||||
markers = local_path + Picopackage::Resolver::MERGE_SUFFIX
|
||||
assert_path_exists markers
|
||||
assert_includes File.read(markers), "<<<<<<<"
|
||||
end
|
||||
end
|
||||
|
||||
def test_force_discards_local_changes
|
||||
with_dirs do |upstream, project, cache|
|
||||
path = publish(upstream, "poller.rb", V1)
|
||||
resolve(path, project, cache)
|
||||
local_path = File.join(project, "poller.rb")
|
||||
|
||||
File.write(local_path, File.read(local_path).sub("INTERVAL = 30", "INTERVAL = 5"))
|
||||
File.write(path, V1.sub("INTERVAL = 30", "INTERVAL = 90"))
|
||||
|
||||
result = resolve(path, project, cache, force: true)
|
||||
|
||||
assert_equal :updated, result.state
|
||||
assert_includes File.read(local_path), "INTERVAL = 90"
|
||||
refute_includes File.read(local_path), "INTERVAL = 5"
|
||||
end
|
||||
end
|
||||
|
||||
def test_an_unrelated_bare_file_with_the_same_name_is_refused
|
||||
with_dirs do |upstream, project, cache|
|
||||
path = publish(upstream, "poller.rb", V1)
|
||||
File.write(File.join(project, "poller.rb"), "# something else entirely\n")
|
||||
|
||||
result = resolve(path, project, cache)
|
||||
|
||||
assert_equal :conflict, result.state
|
||||
assert_equal "# something else entirely\n", File.read(File.join(project, "poller.rb"))
|
||||
end
|
||||
end
|
||||
|
||||
def test_a_modified_file_with_no_cached_ancestor_is_refused
|
||||
with_dirs do |upstream, project, cache|
|
||||
path = publish(upstream, "poller.rb", V1)
|
||||
resolve(path, project, cache)
|
||||
local_path = File.join(project, "poller.rb")
|
||||
File.write(local_path, File.read(local_path).sub("INTERVAL = 30", "INTERVAL = 5"))
|
||||
File.write(path, V1.sub(" fetch\n", " fetch\n prune\n"))
|
||||
|
||||
empty_cache = Picopackage::Cache.new(root: File.join(Dir.mktmpdir, "empty"))
|
||||
result = resolve(path, project, empty_cache)
|
||||
|
||||
assert_equal :conflict, result.state
|
||||
assert_match(/cache/, result.message)
|
||||
end
|
||||
end
|
||||
|
||||
# Renaming a package locally is allowed; updating it must not install a second
|
||||
# copy under upstream's preferred name.
|
||||
def test_update_writes_back_to_the_local_filename
|
||||
with_dirs do |upstream, project, cache|
|
||||
path = publish(upstream, "poller.rb", V1)
|
||||
resolve(path, project, cache)
|
||||
FileUtils.mv(File.join(project, "poller.rb"), File.join(project, "my_poller.rb"))
|
||||
|
||||
File.write(path, V1.sub("INTERVAL = 30", "INTERVAL = 60"))
|
||||
result = resolve(path, project, cache, filename: "my_poller.rb")
|
||||
|
||||
assert_equal :updated, result.state
|
||||
refute_path_exists File.join(project, "poller.rb"), "should not have created a second copy"
|
||||
assert_includes File.read(File.join(project, "my_poller.rb")), "INTERVAL = 60"
|
||||
assert_equal "my_poller.rb", Picopackage::Package.from_file(File.join(project, "my_poller.rb")).filename
|
||||
end
|
||||
end
|
||||
|
||||
def test_timestamps_never_override_the_checksum
|
||||
with_dirs do |upstream, project, cache|
|
||||
path = publish(upstream, "poller.rb", V1)
|
||||
resolve(path, project, cache)
|
||||
local_path = File.join(project, "poller.rb")
|
||||
before = File.read(local_path)
|
||||
|
||||
# Same bytes, much newer mtime — the old timestamp-driven logic called
|
||||
# this an update and rewrote the file.
|
||||
FileUtils.touch(path, mtime: Time.now + 86_400)
|
||||
result = resolve(path, project, cache)
|
||||
|
||||
assert_equal :current, result.state
|
||||
assert_equal before, File.read(local_path)
|
||||
end
|
||||
end
|
||||
end
|
||||
Reference in New Issue
Block a user