You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Emery Hemingway 1743234f96 Version 0.2.1 9 months ago
doc Diagram 1 year ago
extras/picard_plugin Defer set commit 1 year ago
genode Tup refactor 1 year ago
src Split ingest to standalone utility 1 year ago
tests Use Tiger tree hashes 1 year ago
.gitignore Picard plugin 1 year ago
.underconstruction.gif 1 year ago 1 year ago Add Tox-based Spry interpreter 1 year ago
blobsets.nimble Version 0.2.1 9 months ago
default.nix Use Tiger tree hashes 1 year ago

A post-UNIX storage layer


This repository hosts a proof-of-concept storage system and synthetic file- system frontend for the Genode OS framework. The concept of the file has been replaced with the blob, a finite and linear stream of bytes. Blobs are identified by 256 bit hash digests using BLAKE2B in tree mode.

A blobset is flat map of key hashes to blob hashes. Key hashes are 64 bits so the practical limit to the number of set members is 2^32. Sets are implemented as lazy-loaded hash array mapped tries with a 64 node fan-out. A store contains both data blobs and blobs of set metadata and is the public API to both blobs and sets.

Two stores are provided, one that stores blobs to a UNIX file-system backend and the other as a HTTP client. A primitive HTTP server is provided to serve the UNIX backend to the client. A Lisp-like REPL is provided to build and explore sets and a Genode file-system server is provided to serve sets as file-systems.

Exposing a set as a UNIX file-system is trivial because set keys are hash digests of arbitrary byte-strings, so names containing the / character may be mapped in the set.

Storing music

A plugin for the Picard tagger is available at _extras/picardplugin. This plugin ingests audio files into a rolling set and indexes them by MusicBrainz recording UUID. A simple playlist generator for playing these files may be found at