aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorpukkamustard <pukkamustard@posteo.net>2020-11-23 08:42:52 +0100
committerpukkamustard <pukkamustard@posteo.net>2020-11-23 08:42:52 +0100
commitbcfaec61621479974f0aa8f76d86f610968a4e3b (patch)
treeae1ff9bc7cfedf5874ca34292cf38784a7c95a32
parent08babc22fb54ab20c38d88394f7cf5b6e2bd3abd (diff)
eris.adoc: minor fixes
-rw-r--r--doc/eris.adoc87
-rw-r--r--public/index.html200
2 files changed, 156 insertions, 131 deletions
diff --git a/doc/eris.adoc b/doc/eris.adoc
index 6feb7cd..b49e0e4 100644
--- a/doc/eris.adoc
+++ b/doc/eris.adoc
@@ -7,7 +7,7 @@ pukkamustard <pukkamustard@posteo.net>
:sectanchors:
[abstract]
-The Encoding for Robust Immutable Storage (ERIS) is an encoding of arbitrary content into a set of uniformly sized, encrypted and content-addressed blocks as well as a short identifier - the _read capability_. The content can be reassembled from the encrypted blocks only with the read capability. The encoding is defined independent of any storage or transport layer. Together with content-addressable storage, ERIS can be used as a building block for robust and decentralized applications.
+This document describes the Encoding for Robust Immutable Storage (ERIS). ERIS is an encoding of arbitrary content into a set of uniformly sized, encrypted and content-addressed blocks as well as a short identifier (a URN). The content can be reassembled from the encrypted blocks only with this identifier. The encoding is defined independent of any storage and transport layer or any specific application. We illustrate how ERIS can be used as a building block for robust and decentralized applications.
== Introduction
@@ -17,16 +17,14 @@ Availability can be increased by caching content on multiple peers. However most
An alternative to identifying content by its location is to identify content by its content itself. This is called content-addressing. The hash of some content is computed and used as an unique identifier for the content.
-Content-addressed content is much easier to cache as the content is completely decoupled from any physical location. It is much easier to ensure availability of content-addressed content than it is for location-addressed content.
-
-Authenticity of content is automatically ensured with content-addressing (when using a cryptographic hash) as the identifier of the content can be computed and be checked to match the requested identifier.
+Caching content-addressed content and making it available redundantly is much easier as the content is completely decoupled from any physical location. Authenticity of content is automatically ensured with content-addressing (when using a cryptographic hash) as the identifier of the content can be computed to check that the content matches the requested identifier.
However, naive content-addressing has certain drawbacks:
-- Large content is stored as a large blob. In order to optimize storage and network operations it is better to split up content into smaller uniformly sized blocks and reassemble blocks when needed.
+- Large content is stored as a large chunk of data. In order to optimize storage and network operations it is better to split up content into smaller uniformly sized blocks and reassemble blocks when needed.
- Confidentiality: Content is readable by all peers involved in transporting, caching and storing content.
-ERIS is an encoding that addresses these issues by splitting blocks into small uniformly sized blocks and encrypting blocks.
+ERIS addresses these issues by splitting content into small uniformly sized and encrypted blocks.
=== Objectives
@@ -56,20 +54,20 @@ ERIS is inspired and based on the encoding used in the file-sharing application
ERIS differs from ECRS in following points:
Cryptographic primitives :: ECRS itself does not specify any cryptographic primitives but the GNUNet implementation uses the SHA-512 hash and AES cipher. ERIS uses the Blake2b-256 cryptographic hash <<RFC7693>> and the ChaCha20 stream cipher <<RFC8439>>. This improves performance, storage efficiency (as hash references are smaller) and allows a convergence secret to be used (via Blake2b keyed hashing; see <<_convergence_secret>>).
-Block size :: ECRS uses a fixed block size of 32 KiB. This is inefficient when encoding small content. ERIS allows a block size of 1 KiB or 32 KiB, allowing efficient encoding of small and large content (see <<_block_size>>).
-URN :: ECRS does not specify an URN for referring to encoded content (this is specified as part of the GNUNet file-sharing application). ERIS specifies an URN for encoded content regardless of encoding application or storage and transport layer.
+Block size :: ECRS uses a fixed block size of 32 KiB. This can be very inefficient when encoding many small pieces of content. ERIS allows a block size of 1 KiB or 32 KiB, allowing efficient encoding of small and large content (see <<_block_size>>).
+URN :: ECRS does not specify an URN for referring to encoded content (this is specified as part of the GNUNet file-sharing application). ERIS specifies an URN for encoded content regardless of encoding application or storage and transport layer (see <<_urn>>).
Namespaces :: ECRS defines two mechanisms for grouping and discovering encoded content (SBlock and KBlock). ERIS does not specify any such mechanisms (see <<_namespaces>>).
Other related projects include Tahoe-LAFS and Freenet. The reader is referred to the ECRS paper <<ECRS>> for an in-depth explanation and comparison of related projects.
+ERIS is being developed in close collaboration with the https://datashards.net/[Datashards] initiative.
+
=== Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 <<RFC2119>>.
We use binary prefixes for multiples of bytes, i.e: 1024 bytes is 1 kibibyte (KiB), 1024 kibibytes is 1 mebibyte (MiB) and 1024 mebibytes is 1 gigibytes (GiB).
-TODO a glossary of terms used.
-
== Specification of ERIS
=== Cryptographic Primitives
@@ -80,7 +78,7 @@ The cryptographic primitives used by ERIS are a cryptographic hash funciton, a s
Blake2b <<RFC7693>> with output size of 256 bit (32 byte). We use the keying feature and refer to the key used for keying Blake2b as the _hashing key_.
-Provides the functions `Blake2b-256(INPUT,HASHING-KEY)` for keyed hashing and `Blake2b-256(INPUT)` for unkeyed hashing.
+Provides the functions `Blake2b-256(INPUT, HASHING-KEY)` for keyed hashing and `Blake2b-256(INPUT)` for unkeyed hashing.
==== Symmetric Key Cipher
ChaCha20 (IETF variant) <<RFC8439>>. Provides `ChaCha20(INPUT, KEY)`, where `INPUT` is an arbirtarty length byte sequence and `KEY` is the 256 bit encryption key. The output is the encrypted byte sequence.
@@ -96,7 +94,7 @@ We use a byte padding scheme to ensure that input content size is a multiple of
`PAD(INPUT,BLOCK-SIZE)` :: For `INPUT` of size `n` adds a mandatory byte valued `0x80` (hexadecimal) to `INPUT` followed by `m < BLOCK-SIZE - 1` bytes valued `0x00` such that `n + m + 1` is a multiple of `BLOCK-SIZE`.
`UNPAD(INPUT,BLOCK-SIZE)` :: Starts reading bytes from the end of `INPUT` until a `0x80` is read and then returns bytes of `INPUT` before the `0x80`. Throws an error if a value other than `0x00` is read before reading `0x80` or if no `0x80` is read after reading `BLOCK-SIZE - 1` bytes from the end.
-This is the padding algorithm implemented in https://libsodium.gitbook.io/doc/padding[libsodium]footnote:[This padding algorithm is apparently also specified in ISO/IEC 7816-4. However, the speicifcation is not openly available. Fuck you ISO.].
+This is the padding algorithm implemented in https://libsodium.gitbook.io/doc/padding[libsodium]footnote:[This padding algorithm is apparently also specified in ISO/IEC 7816-4. However, the speicifcation is not openly available. So, fuck you ISO.].
=== Block Size
@@ -121,7 +119,7 @@ Using the hash of the content as key is called _convergent encryption_.
Because the hash of the content is deterministically computed from the content, the key will be the same when the same content is encoded twice. This results in de-duplication of content. Convergent encryption suffers from two known attacks: The Confirmation Of A File Attack and The Learn-The-Remaining-Information Attack <<Zooko2008>>. A defense against both attacks is to use a _convergence secret_. This results in different encoding of the same content with different convergence secret.
-If no convergence secret is specified a null convergence secret is used (32 bytes of zeroes).
+If no convergence secret is specified a null convergence secret MUST be used (32 bytes of zeroes).
The convergence secret is implemented as the keying feature of the Blake2 cryptographic hash <<RFC7693>>.
@@ -147,11 +145,11 @@ An encoding of a content that is split into eight blocks is depicted in <<figure
.Encoding of content as tree. Solid edges are concatenations of reference-key pairs as described in <<_collect_reference_key_pairs_in_nodes>>. Dotted edges are encryption and computation of reference-key pairs as described in <<_encrypt_block_and_compute_reference_key_pair>>.
image::eris-merkle-tree.svg[Merkle Tree,opts=inline]
-The block-size, the level of the root reference and the root reference-key pair itself are the necessary pieces of information required to decode content. The tuple consisting of block size, level, root reference and key is called the _read capability_.
+The block size, the level of the root reference and the root reference-key pair itself are the necessary pieces of information required to decode content. The tuple consisting of block size, level, root reference and key is called the _read capability_.
The encrypted blocks and the read capability are the outputs of the encoding process.
-A pseudo-code implementation of the encoding process:
+A pseudo-code implementation of the encoding process is provided in the following. Note that the pseudo-code implementation is naive and given for illustration purposes only. It is RECOMMENDED that imlementations use a streaming encoding process (as described in <<_streaming>>) which allows encoding of content larger than the available memory.
[source,pseudocode]
----
@@ -186,6 +184,7 @@ ERIS-Encode(CONTENT, CONVERGENCE-SECRET, BLOCK-SIZE):
The sub-process `Split-Content` and `Collect-RK-Pairs` are explained in the following sections.
+
==== Splitting Input Content into Blocks
Input content is split into blocks of size at most block size such that only the last content block may be smaller than block size.
@@ -317,7 +316,9 @@ Where the block-storage can be accessed as follows:
A streaming decoding procedure can be implemented where the content can be output block wise and does not need to be kept in memory for unpadding. For an example, see https://gitlab.com/openengiadina/eris/-/raw/main/eris/decode.scm[the reference Guile implementation].
-Random access is possible by only decoding selected sub-trees.
+==== Random Access
+
+A decoder that allows random access to the encoded content can be implemented by decoding selected sub-trees.
=== Binary Encoding of Read Capability
@@ -336,7 +337,7 @@ We specify an binary encoding of the read-capability 66 bytes:
The initial field (block size) also encodes the ERIS version. Future versions of ERIS MUST use different codes to encode block sizes.
-TODO using 1 byte to encode level limits size of content that can be encoded. Add a comment on that.
+Note that using a single byte to encode the level limits the size of content that can be encoded with ERIS. However, the size of the largest encodable content is approximately 1e300 TiB, which seems to be sufficient for any conceivable practical applications (including an index of all atoms in the universe).
=== URN
@@ -352,20 +353,6 @@ For example the ERIS URN of the UTF-8 encoded string "Hello world!" (with block
=== Namespaces
-== Implementations
-
-A list of known implementations that satisify the test vectors:
-
-|===
-| Name | Programming language | License | Notes | Homepage
-
-| `guile-eris` | Guile | GPL-3.0-or-later | Reference implementation | https://gitlab.com/openengiadina/eris/
-| `elixir-eris` | Elixir | GPL-3.0-or-later | | https://gitlab.com/openengiadina/elixir-eris/
-|===
-
-== Acknowledgments
-
-[appendix]
== Test Vectors
=== Machine Readable
@@ -412,7 +399,6 @@ Implementations MUST verify that the content encodes to the URN given the specif
=== Large content
-
In order to verify implementations that encode content by streaming (see <<_streaming>>) URNs of large contents that are generated in a specified way are provided:
|===
@@ -424,7 +410,21 @@ In order to verify implementations that encode content by streaming (see <<_stre
Content is the ChaCha20 stream using a null nonce and the key which is the Blake2b hash of the UTF-8 encoded test name (e.g. `KEY := Blake2b-256("100MiB (block size 1KiB)")`). The ChaCha20 stream can be computed by encoding a null byte sequence (e.g. `CHACHA20_STREAM := ChaCha20(NULL, KEY)`).
-[appendix]
+== Implementations
+
+A list of known implementations that satisify the test vectors:
+
+|===
+| Name | Programming language | License | Notes | Homepage
+
+| `guile-eris` | Guile | GPL-3.0-or-later | Reference implementation | https://gitlab.com/openengiadina/eris/
+| `elixir-eris` | Elixir | GPL-3.0-or-later | | https://gitlab.com/openengiadina/elixir-eris/
+|===
+
+== Acknowledgments
+
+
+:sectnums!:
== Changelog
[discrete]
@@ -438,21 +438,28 @@ Initial version.
Major update of encoding that removes the _verification capability_ - ability to verify integrity of content without reading content.
-[appendix]
== Copyright
This work is licensed under a http://creativecommons.org/licenses/by-sa/4.0/[Creative Commons Attribution-ShareAlike 4.0 International License].
-[bibliography]
== References
-- [[[content-addressable-rdf]]] openEngiadina. https://openengiadina.net/papers/content-addressable-rdf.html[Content-addressable RDF]. 2020
-- [[[rdf-signify]]] openEngiadina. https://openengiadina.net/papers/rdf-signify.html[RDF Signify]. 2020
-- [[[Polleres2020]]] Polleres, Kamdar, Fernández, Javier David, Tudorache & Musen. https://epub.wu.ac.at/6371/1/IPM_workingpaper_02_2018.pdf[A more decentralized vision for Linked Data]. 2020
-- [[[ECRS]]] Grothoff, Grothoff, Horozov, & Lindgren. https://grothoff.org/christian/ecrs.pdf[An encoding for censorship-resistant sharing]. 2003
+[bibliography]
+=== Normative References
+
- [[[RFC2119]]] S. Bradner. https://tools.ietf.org/html/rfc2119[Key words for use in RFCs to Indicate Requirement Levels]. 1997
- [[[RFC4648]]] S. Josefsson. https://tools.ietf.org/html/rfc4648[The Base16, Base32, and Base64 Data Encodings]. 2006
-- [[[RFC7049]]] C. Bormann & P. Hoffman. https://tools.ietf.org/html/rfc7049[Concise Binary Object Representation (CBOR)]. 2013
- [[[RFC7693]]] M-J. Saarinen & J-P. Aumasson. https://tools.ietf.org/html/rfc7693[The BLAKE2 Cryptographic Hash and Message Authentication Code (MAC)]. 2015
- [[[RFC8439]]] Nir & Langley. https://tools.ietf.org/html/rfc8439[ChaCha20 and Poly1305 for IETF Protocols]. 2018
+- [[[RFC8141]]] Saint-Andre, Filament & Klensin, https://tools.ietf.org/html/rfc8141[Uniform Resource Names (URNs)]. 2017
+
+[bibliography]
+=== Informative References
+
+- [[[Polleres2020]]] Polleres, Kamdar, Fernández, Javier David, Tudorache & Musen. https://epub.wu.ac.at/6371/1/IPM_workingpaper_02_2018.pdf[A more decentralized vision for Linked Data]. 2020
+- [[[ECRS]]] Grothoff, Grothoff, Horozov, & Lindgren. https://grothoff.org/christian/ecrs.pdf[An encoding for censorship-resistant sharing]. 2003
- [[[Zooko2008]]] Zooko Wilcox-O'Hearn. https://tahoe-lafs.org/hacktahoelafs/drew_perttula.html[Drew Perttula and Attacks on Convergent Encryption]. 2008
+
+- [[[content-addressable-rdf]]] openEngiadina. https://openengiadina.net/papers/content-addressable-rdf.html[Content-addressable RDF]. 2020
+- [[[rdf-signify]]] openEngiadina. https://openengiadina.net/papers/rdf-signify.html[RDF Signify]. 2020
+- [[[RFC7049]]] C. Bormann & P. Hoffman. https://tools.ietf.org/html/rfc7049[Concise Binary Object Representation (CBOR)]. 2013
diff --git a/public/index.html b/public/index.html
index f2441ba..6581f54 100644
--- a/public/index.html
+++ b/public/index.html
@@ -472,17 +472,22 @@ body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-b
<li><a href="#_namespaces">3.2. Namespaces</a></li>
</ul>
</li>
-<li><a href="#_implementations">4. Implementations</a></li>
-<li><a href="#_acknowledgments">5. Acknowledgments</a></li>
-<li><a href="#_test_vectors">Appendix A: Test Vectors</a>
+<li><a href="#_test_vectors">4. Test Vectors</a>
<ul class="sectlevel2">
-<li><a href="#_machine_readable">A.1. Machine Readable</a></li>
-<li><a href="#_large_content">A.2. Large content</a></li>
+<li><a href="#_machine_readable">4.1. Machine Readable</a></li>
+<li><a href="#_large_content">4.2. Large content</a></li>
+</ul>
+</li>
+<li><a href="#_implementations">5. Implementations</a></li>
+<li><a href="#_acknowledgments">6. Acknowledgments</a></li>
+<li><a href="#_changelog">Changelog</a></li>
+<li><a href="#_copyright">Copyright</a></li>
+<li><a href="#_references">References</a>
+<ul class="sectlevel2">
+<li><a href="#_normative_references">Normative References</a></li>
+<li><a href="#_informative_references">Informative References</a></li>
</ul>
</li>
-<li><a href="#_changelog">Appendix B: Changelog</a></li>
-<li><a href="#_copyright">Appendix C: Copyright</a></li>
-<li><a href="#_references">References</a></li>
</ul>
</div>
</div>
@@ -491,7 +496,7 @@ body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-b
<div class="sectionbody">
<div class="quoteblock abstract">
<blockquote>
-The Encoding for Robust Immutable Storage (ERIS) is an encoding of arbitrary content into a set of uniformly sized, encrypted and content-addressed blocks as well as a short identifier - the <em>read capability</em>. The content can be reassembled from the encrypted blocks only with the read capability. The encoding is defined independent of any storage or transport layer. Together with content-addressable storage, ERIS can be used as a building block for robust and decentralized applications.
+This document describes the Encoding for Robust Immutable Storage (ERIS). ERIS is an encoding of arbitrary content into a set of uniformly sized, encrypted and content-addressed blocks as well as a short identifier (a URN). The content can be reassembled from the encrypted blocks only with this identifier. The encoding is defined independent of any storage and transport layer or any specific application. We illustrate how ERIS can be used as a building block for robust and decentralized applications.
</blockquote>
</div>
</div>
@@ -509,10 +514,7 @@ The Encoding for Robust Immutable Storage (ERIS) is an encoding of arbitrary con
<p>An alternative to identifying content by its location is to identify content by its content itself. This is called content-addressing. The hash of some content is computed and used as an unique identifier for the content.</p>
</div>
<div class="paragraph">
-<p>Content-addressed content is much easier to cache as the content is completely decoupled from any physical location. It is much easier to ensure availability of content-addressed content than it is for location-addressed content.</p>
-</div>
-<div class="paragraph">
-<p>Authenticity of content is automatically ensured with content-addressing (when using a cryptographic hash) as the identifier of the content can be computed and be checked to match the requested identifier.</p>
+<p>Caching content-addressed content and making it available redundantly is much easier as the content is completely decoupled from any physical location. Authenticity of content is automatically ensured with content-addressing (when using a cryptographic hash) as the identifier of the content can be computed to check that the content matches the requested identifier.</p>
</div>
<div class="paragraph">
<p>However, naive content-addressing has certain drawbacks:</p>
@@ -520,7 +522,7 @@ The Encoding for Robust Immutable Storage (ERIS) is an encoding of arbitrary con
<div class="ulist">
<ul>
<li>
-<p>Large content is stored as a large blob. In order to optimize storage and network operations it is better to split up content into smaller uniformly sized blocks and reassemble blocks when needed.</p>
+<p>Large content is stored as a large chunk of data. In order to optimize storage and network operations it is better to split up content into smaller uniformly sized blocks and reassemble blocks when needed.</p>
</li>
<li>
<p>Confidentiality: Content is readable by all peers involved in transporting, caching and storing content.</p>
@@ -528,7 +530,7 @@ The Encoding for Robust Immutable Storage (ERIS) is an encoding of arbitrary con
</ul>
</div>
<div class="paragraph">
-<p>ERIS is an encoding that addresses these issues by splitting blocks into small uniformly sized blocks and encrypting blocks.</p>
+<p>ERIS addresses these issues by splitting content into small uniformly sized and encrypted blocks.</p>
</div>
<div class="sect2">
<h3 id="_objectives"><a class="anchor" href="#_objectives"></a>1.1. Objectives</h3>
@@ -591,11 +593,11 @@ The Encoding for Robust Immutable Storage (ERIS) is an encoding of arbitrary con
</dd>
<dt class="hdlist1">Block size </dt>
<dd>
-<p>ECRS uses a fixed block size of 32 KiB. This is inefficient when encoding small content. ERIS allows a block size of 1 KiB or 32 KiB, allowing efficient encoding of small and large content (see <a href="#_block_size">Section 2.2</a>).</p>
+<p>ECRS uses a fixed block size of 32 KiB. This can be very inefficient when encoding many small pieces of content. ERIS allows a block size of 1 KiB or 32 KiB, allowing efficient encoding of small and large content (see <a href="#_block_size">Section 2.2</a>).</p>
</dd>
<dt class="hdlist1">URN </dt>
<dd>
-<p>ECRS does not specify an URN for referring to encoded content (this is specified as part of the GNUNet file-sharing application). ERIS specifies an URN for encoded content regardless of encoding application or storage and transport layer.</p>
+<p>ECRS does not specify an URN for referring to encoded content (this is specified as part of the GNUNet file-sharing application). ERIS specifies an URN for encoded content regardless of encoding application or storage and transport layer (see <a href="#_urn">Section 2.7</a>).</p>
</dd>
<dt class="hdlist1">Namespaces </dt>
<dd>
@@ -606,6 +608,9 @@ The Encoding for Robust Immutable Storage (ERIS) is an encoding of arbitrary con
<div class="paragraph">
<p>Other related projects include Tahoe-LAFS and Freenet. The reader is referred to the ECRS paper <a href="#ECRS">[ECRS]</a> for an in-depth explanation and comparison of related projects.</p>
</div>
+<div class="paragraph">
+<p>ERIS is being developed in close collaboration with the <a href="https://datashards.net/">Datashards</a> initiative.</p>
+</div>
</div>
<div class="sect2">
<h3 id="_terminology"><a class="anchor" href="#_terminology"></a>1.4. Terminology</h3>
@@ -615,9 +620,6 @@ The Encoding for Robust Immutable Storage (ERIS) is an encoding of arbitrary con
<div class="paragraph">
<p>We use binary prefixes for multiples of bytes, i.e: 1024 bytes is 1 kibibyte (KiB), 1024 kibibytes is 1 mebibyte (MiB) and 1024 mebibytes is 1 gigibytes (GiB).</p>
</div>
-<div class="paragraph">
-<p>TODO a glossary of terms used.</p>
-</div>
</div>
</div>
</div>
@@ -635,7 +637,7 @@ The Encoding for Robust Immutable Storage (ERIS) is an encoding of arbitrary con
<p>Blake2b <a href="#RFC7693">[RFC7693]</a> with output size of 256 bit (32 byte). We use the keying feature and refer to the key used for keying Blake2b as the <em>hashing key</em>.</p>
</div>
<div class="paragraph">
-<p>Provides the functions <code>Blake2b-256(INPUT,HASHING-KEY)</code> for keyed hashing and <code>Blake2b-256(INPUT)</code> for unkeyed hashing.</p>
+<p>Provides the functions <code>Blake2b-256(INPUT, HASHING-KEY)</code> for keyed hashing and <code>Blake2b-256(INPUT)</code> for unkeyed hashing.</p>
</div>
</div>
<div class="sect3">
@@ -713,7 +715,7 @@ The Encoding for Robust Immutable Storage (ERIS) is an encoding of arbitrary con
<p>Because the hash of the content is deterministically computed from the content, the key will be the same when the same content is encoded twice. This results in de-duplication of content. Convergent encryption suffers from two known attacks: The Confirmation Of A File Attack and The Learn-The-Remaining-Information Attack <a href="#Zooko2008">[Zooko2008]</a>. A defense against both attacks is to use a <em>convergence secret</em>. This results in different encoding of the same content with different convergence secret.</p>
</div>
<div class="paragraph">
-<p>If no convergence secret is specified a null convergence secret is used (32 bytes of zeroes).</p>
+<p>If no convergence secret is specified a null convergence secret MUST be used (32 bytes of zeroes).</p>
</div>
<div class="paragraph">
<p>The convergence secret is implemented as the keying feature of the Blake2 cryptographic hash <a href="#RFC7693">[RFC7693]</a>.</p>
@@ -1172,13 +1174,13 @@ The Encoding for Robust Immutable Storage (ERIS) is an encoding of arbitrary con
<div class="title">Figure 1. Encoding of content as tree. Solid edges are concatenations of reference-key pairs as described in <a href="#_collect_reference_key_pairs_in_nodes">Section 2.4.3</a>. Dotted edges are encryption and computation of reference-key pairs as described in <a href="#_encrypt_block_and_compute_reference_key_pair">Section 2.4.2</a>.</div>
</div>
<div class="paragraph">
-<p>The block-size, the level of the root reference and the root reference-key pair itself are the necessary pieces of information required to decode content. The tuple consisting of block size, level, root reference and key is called the <em>read capability</em>.</p>
+<p>The block size, the level of the root reference and the root reference-key pair itself are the necessary pieces of information required to decode content. The tuple consisting of block size, level, root reference and key is called the <em>read capability</em>.</p>
</div>
<div class="paragraph">
<p>The encrypted blocks and the read capability are the outputs of the encoding process.</p>
</div>
<div class="paragraph">
-<p>A pseudo-code implementation of the encoding process:</p>
+<p>A pseudo-code implementation of the encoding process is provided in the following. Note that the pseudo-code implementation is naive and given for illustration purposes only. It is RECOMMENDED that imlementations use a streaming encoding process (as described in <a href="#_streaming">Section 2.4.4</a>) which allows encoding of content larger than the available memory.</p>
</div>
<div class="listingblock">
<div class="content">
@@ -1380,8 +1382,11 @@ ERIS-Decode(BLOCK-SIZE, LEVEL, ROOT-REFERENCE, ROOT-KEY):
<div class="paragraph">
<p>A streaming decoding procedure can be implemented where the content can be output block wise and does not need to be kept in memory for unpadding. For an example, see <a href="https://gitlab.com/openengiadina/eris/-/raw/main/eris/decode.scm">the reference Guile implementation</a>.</p>
</div>
+<div class="sect3">
+<h4 id="_random_access"><a class="anchor" href="#_random_access"></a>2.5.1. Random Access</h4>
<div class="paragraph">
-<p>Random access is possible by only decoding selected sub-trees.</p>
+<p>A decoder that allows random access to the encoded content can be implemented by decoding selected sub-trees.</p>
+</div>
</div>
</div>
<div class="sect2">
@@ -1432,7 +1437,7 @@ ERIS-Decode(BLOCK-SIZE, LEVEL, ROOT-REFERENCE, ROOT-KEY):
<p>The initial field (block size) also encodes the ERIS version. Future versions of ERIS MUST use different codes to encode block sizes.</p>
</div>
<div class="paragraph">
-<p>TODO using 1 byte to encode level limits size of content that can be encoded. Add a comment on that.</p>
+<p>Note that using a single byte to encode the level limits the size of content that can be encoded with ERIS. However, the size of the largest encodable content is approximately 1e300 TiB, which seems to be sufficient for any conceivable practical applications (including an index of all atoms in the universe).</p>
</div>
</div>
<div class="sect2">
@@ -1463,58 +1468,10 @@ ERIS-Decode(BLOCK-SIZE, LEVEL, ROOT-REFERENCE, ROOT-KEY):
</div>
</div>
<div class="sect1">
-<h2 id="_implementations"><a class="anchor" href="#_implementations"></a>4. Implementations</h2>
-<div class="sectionbody">
-<div class="paragraph">
-<p>A list of known implementations that satisify the test vectors:</p>
-</div>
-<table class="tableblock frame-all grid-all stretch">
-<colgroup>
-<col style="width: 20%;">
-<col style="width: 20%;">
-<col style="width: 20%;">
-<col style="width: 20%;">
-<col style="width: 20%;">
-</colgroup>
-<thead>
-<tr>
-<th class="tableblock halign-left valign-top">Name</th>
-<th class="tableblock halign-left valign-top">Programming language</th>
-<th class="tableblock halign-left valign-top">License</th>
-<th class="tableblock halign-left valign-top">Notes</th>
-<th class="tableblock halign-left valign-top">Homepage</th>
-</tr>
-</thead>
-<tbody>
-<tr>
-<td class="tableblock halign-left valign-top"><p class="tableblock"><code>guile-eris</code></p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">Guile</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">GPL-3.0-or-later</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">Reference implementation</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock"><a href="https://gitlab.com/openengiadina/eris/" class="bare">https://gitlab.com/openengiadina/eris/</a></p></td>
-</tr>
-<tr>
-<td class="tableblock halign-left valign-top"><p class="tableblock"><code>elixir-eris</code></p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">Elixir</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">GPL-3.0-or-later</p></td>
-<td class="tableblock halign-left valign-top"></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock"><a href="https://gitlab.com/openengiadina/elixir-eris/" class="bare">https://gitlab.com/openengiadina/elixir-eris/</a></p></td>
-</tr>
-</tbody>
-</table>
-</div>
-</div>
-<div class="sect1">
-<h2 id="_acknowledgments"><a class="anchor" href="#_acknowledgments"></a>5. Acknowledgments</h2>
-<div class="sectionbody">
-
-</div>
-</div>
-<div class="sect1">
-<h2 id="_test_vectors"><a class="anchor" href="#_test_vectors"></a>Appendix A: Test Vectors</h2>
+<h2 id="_test_vectors"><a class="anchor" href="#_test_vectors"></a>4. Test Vectors</h2>
<div class="sectionbody">
<div class="sect2">
-<h3 id="_machine_readable"><a class="anchor" href="#_machine_readable"></a>A.1. Machine Readable</h3>
+<h3 id="_machine_readable"><a class="anchor" href="#_machine_readable"></a>4.1. Machine Readable</h3>
<div class="paragraph">
<p>A set of test vectors are provided in the <a href="https://gitlab.com/openengiadina/eris/-/tree/main/test-vectors">ERIS repository</a>. Implementations of the ERIS encoding MUST be able to satisfy the test vectors.</p>
</div>
@@ -1591,7 +1548,7 @@ ERIS-Decode(BLOCK-SIZE, LEVEL, ROOT-REFERENCE, ROOT-KEY):
</div>
</div>
<div class="sect2">
-<h3 id="_large_content"><a class="anchor" href="#_large_content"></a>A.2. Large content</h3>
+<h3 id="_large_content"><a class="anchor" href="#_large_content"></a>4.2. Large content</h3>
<div class="paragraph">
<p>In order to verify implementations that encode content by streaming (see <a href="#_streaming">Section 2.4.4</a>) URNs of large contents that are generated in a specified way are provided:</p>
</div>
@@ -1641,7 +1598,55 @@ ERIS-Decode(BLOCK-SIZE, LEVEL, ROOT-REFERENCE, ROOT-KEY):
</div>
</div>
<div class="sect1">
-<h2 id="_changelog"><a class="anchor" href="#_changelog"></a>Appendix B: Changelog</h2>
+<h2 id="_implementations"><a class="anchor" href="#_implementations"></a>5. Implementations</h2>
+<div class="sectionbody">
+<div class="paragraph">
+<p>A list of known implementations that satisify the test vectors:</p>
+</div>
+<table class="tableblock frame-all grid-all stretch">
+<colgroup>
+<col style="width: 20%;">
+<col style="width: 20%;">
+<col style="width: 20%;">
+<col style="width: 20%;">
+<col style="width: 20%;">
+</colgroup>
+<thead>
+<tr>
+<th class="tableblock halign-left valign-top">Name</th>
+<th class="tableblock halign-left valign-top">Programming language</th>
+<th class="tableblock halign-left valign-top">License</th>
+<th class="tableblock halign-left valign-top">Notes</th>
+<th class="tableblock halign-left valign-top">Homepage</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td class="tableblock halign-left valign-top"><p class="tableblock"><code>guile-eris</code></p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Guile</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">GPL-3.0-or-later</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Reference implementation</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock"><a href="https://gitlab.com/openengiadina/eris/" class="bare">https://gitlab.com/openengiadina/eris/</a></p></td>
+</tr>
+<tr>
+<td class="tableblock halign-left valign-top"><p class="tableblock"><code>elixir-eris</code></p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Elixir</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">GPL-3.0-or-later</p></td>
+<td class="tableblock halign-left valign-top"></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock"><a href="https://gitlab.com/openengiadina/elixir-eris/" class="bare">https://gitlab.com/openengiadina/elixir-eris/</a></p></td>
+</tr>
+</tbody>
+</table>
+</div>
+</div>
+<div class="sect1">
+<h2 id="_acknowledgments"><a class="anchor" href="#_acknowledgments"></a>6. Acknowledgments</h2>
+<div class="sectionbody">
+
+</div>
+</div>
+<div class="sect1">
+<h2 id="_changelog"><a class="anchor" href="#_changelog"></a>Changelog</h2>
<div class="sectionbody">
<h3 id="_v0_1_0_11_june_2020" class="discrete"><a href="eris-v0.1.html">v0.1.0 (11. June 2020)</a></h3>
<div class="paragraph">
@@ -1654,7 +1659,7 @@ ERIS-Decode(BLOCK-SIZE, LEVEL, ROOT-REFERENCE, ROOT-KEY):
</div>
</div>
<div class="sect1">
-<h2 id="_copyright"><a class="anchor" href="#_copyright"></a>Appendix C: Copyright</h2>
+<h2 id="_copyright"><a class="anchor" href="#_copyright"></a>Copyright</h2>
<div class="sectionbody">
<div class="paragraph">
<p>This work is licensed under a <a href="http://creativecommons.org/licenses/by-sa/4.0/">Creative Commons Attribution-ShareAlike 4.0 International License</a>.</p>
@@ -1664,52 +1669,65 @@ ERIS-Decode(BLOCK-SIZE, LEVEL, ROOT-REFERENCE, ROOT-KEY):
<div class="sect1">
<h2 id="_references"><a class="anchor" href="#_references"></a>References</h2>
<div class="sectionbody">
+<div class="sect2">
+<h3 id="_normative_references"><a class="anchor" href="#_normative_references"></a>Normative References</h3>
<div class="ulist bibliography">
<ul class="bibliography">
<li>
-<p><a id="content-addressable-rdf"></a>[content-addressable-rdf] openEngiadina. <a href="https://openengiadina.net/papers/content-addressable-rdf.html">Content-addressable RDF</a>. 2020</p>
+<p><a id="RFC2119"></a>[RFC2119] S. Bradner. <a href="https://tools.ietf.org/html/rfc2119">Key words for use in RFCs to Indicate Requirement Levels</a>. 1997</p>
</li>
<li>
-<p><a id="rdf-signify"></a>[rdf-signify] openEngiadina. <a href="https://openengiadina.net/papers/rdf-signify.html">RDF Signify</a>. 2020</p>
+<p><a id="RFC4648"></a>[RFC4648] S. Josefsson. <a href="https://tools.ietf.org/html/rfc4648">The Base16, Base32, and Base64 Data Encodings</a>. 2006</p>
</li>
<li>
-<p><a id="Polleres2020"></a>[Polleres2020] Polleres, Kamdar, Fernández, Javier David, Tudorache &amp; Musen. <a href="https://epub.wu.ac.at/6371/1/IPM_workingpaper_02_2018.pdf">A more decentralized vision for Linked Data</a>. 2020</p>
+<p><a id="RFC7693"></a>[RFC7693] M-J. Saarinen &amp; J-P. Aumasson. <a href="https://tools.ietf.org/html/rfc7693">The BLAKE2 Cryptographic Hash and Message Authentication Code (MAC)</a>. 2015</p>
</li>
<li>
-<p><a id="ECRS"></a>[ECRS] Grothoff, Grothoff, Horozov, &amp; Lindgren. <a href="https://grothoff.org/christian/ecrs.pdf">An encoding for censorship-resistant sharing</a>. 2003</p>
+<p><a id="RFC8439"></a>[RFC8439] Nir &amp; Langley. <a href="https://tools.ietf.org/html/rfc8439">ChaCha20 and Poly1305 for IETF Protocols</a>. 2018</p>
</li>
<li>
-<p><a id="RFC2119"></a>[RFC2119] S. Bradner. <a href="https://tools.ietf.org/html/rfc2119">Key words for use in RFCs to Indicate Requirement Levels</a>. 1997</p>
+<p><a id="RFC8141"></a>[RFC8141] Saint-Andre, Filament &amp; Klensin, <a href="https://tools.ietf.org/html/rfc8141">Uniform Resource Names (URNs)</a>. 2017</p>
</li>
+</ul>
+</div>
+</div>
+<div class="sect2">
+<h3 id="_informative_references"><a class="anchor" href="#_informative_references"></a>Informative References</h3>
+<div class="ulist bibliography">
+<ul class="bibliography">
<li>
-<p><a id="RFC4648"></a>[RFC4648] S. Josefsson. <a href="https://tools.ietf.org/html/rfc4648">The Base16, Base32, and Base64 Data Encodings</a>. 2006</p>
+<p><a id="Polleres2020"></a>[Polleres2020] Polleres, Kamdar, Fernández, Javier David, Tudorache &amp; Musen. <a href="https://epub.wu.ac.at/6371/1/IPM_workingpaper_02_2018.pdf">A more decentralized vision for Linked Data</a>. 2020</p>
</li>
<li>
-<p><a id="RFC7049"></a>[RFC7049] C. Bormann &amp; P. Hoffman. <a href="https://tools.ietf.org/html/rfc7049">Concise Binary Object Representation (CBOR)</a>. 2013</p>
+<p><a id="ECRS"></a>[ECRS] Grothoff, Grothoff, Horozov, &amp; Lindgren. <a href="https://grothoff.org/christian/ecrs.pdf">An encoding for censorship-resistant sharing</a>. 2003</p>
</li>
<li>
-<p><a id="RFC7693"></a>[RFC7693] M-J. Saarinen &amp; J-P. Aumasson. <a href="https://tools.ietf.org/html/rfc7693">The BLAKE2 Cryptographic Hash and Message Authentication Code (MAC)</a>. 2015</p>
+<p><a id="Zooko2008"></a>[Zooko2008] Zooko Wilcox-O&#8217;Hearn. <a href="https://tahoe-lafs.org/hacktahoelafs/drew_perttula.html">Drew Perttula and Attacks on Convergent Encryption</a>. 2008</p>
</li>
<li>
-<p><a id="RFC8439"></a>[RFC8439] Nir &amp; Langley. <a href="https://tools.ietf.org/html/rfc8439">ChaCha20 and Poly1305 for IETF Protocols</a>. 2018</p>
+<p><a id="content-addressable-rdf"></a>[content-addressable-rdf] openEngiadina. <a href="https://openengiadina.net/papers/content-addressable-rdf.html">Content-addressable RDF</a>. 2020</p>
</li>
<li>
-<p><a id="Zooko2008"></a>[Zooko2008] Zooko Wilcox-O&#8217;Hearn. <a href="https://tahoe-lafs.org/hacktahoelafs/drew_perttula.html">Drew Perttula and Attacks on Convergent Encryption</a>. 2008</p>
+<p><a id="rdf-signify"></a>[rdf-signify] openEngiadina. <a href="https://openengiadina.net/papers/rdf-signify.html">RDF Signify</a>. 2020</p>
+</li>
+<li>
+<p><a id="RFC7049"></a>[RFC7049] C. Bormann &amp; P. Hoffman. <a href="https://tools.ietf.org/html/rfc7049">Concise Binary Object Representation (CBOR)</a>. 2013</p>
</li>
</ul>
</div>
</div>
</div>
</div>
+</div>
<div id="footnotes">
<hr>
<div class="footnote" id="_footnotedef_1">
-<a href="#_footnoteref_1">1</a>. This padding algorithm is apparently also specified in ISO/IEC 7816-4. However, the speicifcation is not openly available. Fuck you ISO.
+<a href="#_footnoteref_1">1</a>. This padding algorithm is apparently also specified in ISO/IEC 7816-4. However, the speicifcation is not openly available. So, fuck you ISO.
</div>
</div>
<div id="footer">
<div id="footer-text">
-Last updated 2020-11-19 20:26:15 +0100
+Last updated 2020-11-23 08:23:24 +0100
</div>
</div>
</body>