ehmry/blobsets
ehmry
/
blobsets
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
90 Commits 1 Branch 0 Tags 618 KiB
Nim 83.1%
C++ 9.6%
Python 5.7%
Nix 0.9%
C 0.7%
Go to file
Ehmry - b905b45525 Use Tiger tree hashes 
Switch from BLAKE2B-256 tree hashing to the Tiger tree hash scheme
used by the ADC protocol. Hashes are now 192-bit (96-bit security)
and the tree leaf nodes 1 KiB.
 		2019-06-17 09:26:25 +02:00
doc Diagram 2018-09-11 11:17:49 +02:00
extras/picard_plugin Defer set commit 2019-02-18 23:39:57 +01:00
genode Packaging cleanup 2019-05-25 15:10:35 +02:00
src Use Tiger tree hashes 2019-06-17 09:26:25 +02:00
tests Use Tiger tree hashes 2019-06-17 09:26:25 +02:00
.gitignore Picard plugin 2019-02-16 23:07:59 +01:00
README.md Update README 2019-03-19 11:59:32 +01:00
blobs.sy Add Tox-based Spry interpreter 2019-06-05 19:41:36 +02:00
blobsets.nimble Use Tiger tree hashes 2019-06-17 09:26:25 +02:00
default.nix Use Tiger tree hashes 2019-06-17 09:26:25 +02:00

README.md

A post-UNIX storage layer

This repository hosts a proof-of-concept storage system and synthetic file- system frontend for the Genode OS framework. The concept of the file has been replaced with the blob, a finite and linear stream of bytes. Blobs are identified by 256 bit hash digests using BLAKE2B in tree mode.

A blobset is flat map of key hashes to blob hashes. Key hashes are 64 bits so the practical limit to the number of set members is 2^32. Sets are implemented as lazy-loaded hash array mapped tries with a 64 node fan-out. A store contains both data blobs and blobs of set metadata and is the public API to both blobs and sets.

Two stores are provided, one that stores blobs to a UNIX file-system backend and the other as a HTTP client. A primitive HTTP server is provided to serve the UNIX backend to the client. A Lisp-like REPL is provided to build and explore sets and a Genode file-system server is provided to serve sets as file-systems.

Exposing a set as a UNIX file-system is trivial because set keys are hash digests of arbitrary byte-strings, so names containing the / character may be mapped in the set.

Storing music

A plugin for the Picard tagger is available at extras/picard_plugin. This plugin ingests audio files into a rolling set and indexes them by MusicBrainz recording UUID. A simple playlist generator for playing these files may be found at https://github.com/ehmry/musicbrainz_playlist_generator.