blobsets/README.md

31 lines
1.6 KiB
Markdown
Raw Normal View History

2019-03-19 11:55:10 +01:00
# A post-UNIX storage layer
![UNDER CONSTRUCTION](.underconstruction.gif)
2019-03-19 11:55:10 +01:00
This repository hosts a proof-of-concept storage system and synthetic file-
system frontend for the Genode OS framework. The concept of the *file* has been
replaced with the *blob*, a finite and linear stream of bytes. Blobs are
identified by 256 bit hash digests using BLAKE2B in tree mode.
A *blobset* is flat map of key hashes to blob hashes. Key hashes are 64 bits so
the practical limit to the number of set members is 2^32. Sets are implemented as
lazy-loaded [hash array mapped tries](https://en.wikipedia.org/wiki/Hash_array_mapped_trie)
with a 64 node fan-out. A *store* contains both data blobs and blobs of set
metadata and is the public API to both blobs and sets.
Two stores are provided, one that stores blobs to a UNIX file-system backend and
the other as a HTTP client. A primitive HTTP server is provided to serve the
UNIX backend to the client. A Lisp-like REPL is provided to build and explore
sets and a Genode file-system server is provided to serve sets as file-systems.
Exposing a set as a UNIX file-system is trivial because set keys are hash
digests of arbitrary byte-strings, so names containing the */* character may be
mapped in the set.
## Storing music
A plugin for the [Picard](https://picard.musicbrainz.org/) tagger is available
at _extras/picard_plugin_. This plugin ingests audio files into a rolling set
and indexes them by [MusicBrainz recording UUID](https://musicbrainz.org/doc/Recording).
A simple playlist generator for playing these files may be found at
https://github.com/ehmry/musicbrainz_playlist_generator.