add the spec folder

8 files changed, 630 insertions, 0 deletions

diff --git a/spec/Zot6/Changelog.md b/spec/Zot6/Changelog.md
new file mode 100644
index 000000000..301ad48fa
--- /dev/null
+++ b/spec/Zot6/Changelog.md
@@ -0,0 +1,16 @@
+### Changes
+
+2018-09-14
+Remove 'request' message type. To request missing conversation contents fetch the message 'id' attribute with an Accept header of 'application/x-zot+json'. An OrderedCollection of the containing conversation will be returned. The fetch should be signed by the requestor and permissions checked by the target server.  
+
+2018-08-28
+Moved linked identity paragraph which was incorrectly placed in the 'nomadic considerations' text block.
+
+2018-08-17
+Clarify that ActivityStreams objects are assumed to have a default @context of "https://www.w3.org/ns/activitystreams" in the absence of a specific @context declaration.
+
+2018-08-16
+Added reference to encrypted HTTP Signatures, added pointer to reference implementation.
+
+2018-08-01
+Removed the "mail" message type and documented the "sync" message.
\ No newline at end of file
diff --git a/spec/Zot6/Content.md b/spec/Zot6/Content.md
new file mode 100644
index 000000000..a3179ff16
--- /dev/null
+++ b/spec/Zot6/Content.md
@@ -0,0 +1,15 @@
+## Content
+
+Some Zot applications/implementations are able to support complex content structures including embedded apps, identity-aware content, and authenticated links. Not all implementations are required to support, or may be able to support these types of content. Additionally, the data serialisation formats supported by the implementation may affect the ability to fully represent rich identity-aware content.
+
+### ActivityStreams
+
+An implementation which only supports ActivityStreams2 will receive "message" content as type 'Article', and which contains a generic HTML rendering of the source content in the "content" or "contentMap" elements. This generic rendering is suitable for any observer and some HTML links may be inaccessible due to permission and authentication requirements.
+
+The source for the rendered HTML is available in the "source" element. Implementations which wish to support identity-aware content and authenticated links should use this source content (especially if it is type "text/x-zot-bbcode") to dynamically generate an HTML rendering that is specific to the current observer. 
+
+The exact feature set supported by the implementation and local security filtering MAY result in rendered content that is empty. Implementations MAY choose not to display rendered content that is empty or MAY indicate to the observer that the content could not be rendered sucessfully. Implementations SHOULD provide a menu option to 'view source' and provide the ability to access the original content if the rendering results in empty content and is a mimetype that is determined to be safe to display in source format.
+
+### Zot
+
+The same considerations apply to content which uses the 'zot' serialisation. In this case, the content source is the "body" element and is of type "mimetype". The observer-neutral HTML rendering will be provided in the "html" element.
\ No newline at end of file
diff --git a/spec/Zot6/Discovery.md b/spec/Zot6/Discovery.md
new file mode 100644
index 000000000..a5f3f01c8
--- /dev/null
+++ b/spec/Zot6/Discovery.md
@@ -0,0 +1,112 @@
+### Discovery
+
+Channel discovery is accomplished primarily through webfinger (RFC7033). You will be looking for an entry with 
+
+````
+rel: http://purl.org/zot/protocol/6.0
+type: application/x-zot+json
+href: (discovery rhef)
+````
+
+Load the url from the href using and HTTP Accept header of
+
+````
+Accept: application/x-zot+json;
+````
+
+This will provide a channel discovery document. It also includes a site discovery section.
+You may load the site discovery packet separately by accessing the domain top level with 
+
+````
+Accept: application/x-zot+json;
+````
+
+Sample channel discovery document
+
+````
+{
+    "id": "XSWtrP_U65k9ZXeBxoYbcBgy6cVteo3yLwmLy4ppkjXqAryky0pYBq8YWnj7rApoTSdgy-QeciQG7Yhz7QYt4g",
+    "id_sig": "sha256.yG_Biu8pTpV0DlKPzoZHi0PbpM3okDYB7v5z2UuB___6J85gSOiOdes1tLwSmFkjbYMZbL2oksIe2tmD32lJWxpycSWJlDNbK8oggAtMx1sfVwyZOX_O0QBde2SxWCp0EIrRTRacIyKBzJhPRxCsGkc0uWin6XesXVZuYEVCxESr0KMT35Y79keOXGjJGv822C-Z2Nb4vphpbpftllGjxXOV70PxTNF0uZTWeVSmv2O0FGhkqBeBvBZU0FaWdYZqZZbd2AN_bto-8P95KMw6Fdfl2NIeL6vpD3xSu59Qhztl8L5npU13S3yvywzSvNg8DVgpNqcRmMiebaspfcjttCEAKtB2H-uiPkeuvDUk_iMXGtSUulcsNt1VFtSTnLEG371O6kj3dsczCV4QrpKBdIWNF3_41xHhrLi4Pug5JQg_wncyBSXu6Uj9pkCiD-JPVfI0ViCPccJcCKB-kXpP2EQIoPMhjV5x3bruI0TFLxrJqKWuoY6m8KUYrlGRdewaPYJ7pOY2NSNeLb9z6PO3UHT0bnr3DLyNxypxiUo5Pg4BxnHeuVKmiTxULF06KSwLmPDGsscrBSX1wIbHP6rhcmh0vDP0af2ixluLJbcLbptI2d137tFDVT4lTWBZ8PRNPWi1rfSl_x-dzevF8Dd3vi0iWd7D-aK89rqmRfUKsWk",
+    "aliases": [
+        "acct:zapper@zap.macgirvin.com",
+        "https://zap.macgirvin.com/channel/zapper"
+    ],
+    "primary_location": {
+        "address": "zapper@zap.macgirvin.com",
+        "url": "https://zap.macgirvin.com/channel/zapper",
+        "connections_url": "https://zap.macgirvin.com/poco/zapper",
+        "follow_url": "https://zap.macgirvin.com/follow?f=&url=%s"
+    },
+    "public_key": "-----BEGIN PUBLIC KEY-----\nMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA154VVJChRzsdm1Aba4su\nMLhhnuTELsQCIuDGB07lxTlHmJeeD9eImLXzQPDTNlLcCYVQkKH1uPhfhwnBFITu\noG7hr9QklhFZxfe9IFiAfr7w1+IRFXCYEwfe/eAaYDoprNqtMxonniOiVvnAdNs+\nRi1/Bfasd9d4BRBSj2KwuAeTke7gvHlACRKmauhFmhfG2fzj53AL2ixsraBzdccR\nFseICC99eldgWk2Jg16J1Euh51HV56jUWoz4ZYb1Kxri8zf55R2GiQJCHvSCLlgl\noFaiHLPS+yBIlCkNZ+47ee6DL7ePJ6kip6+ukAP5/1vOM0ahTPANoqaWmg19RsOu\n2q0LPyotWyLX1JX13rtLC7qFlSST31gxY082G8QIJfEbWgYoci0g2XOCDjAZ3JjI\neK/+tZwLzrV5l5Tcorf35Kbc/lHbMSm+wuyc6eQV14tj542nUftZAgpkPOImHsRm\nLqwiG6uxpfIW0TFuUse45F/faYbV4pbcaGm7Hwp/MbXl8vNiE0LpV9eqZu5ryVjZ\nT+Vt32ISWuTeyLqB1Wb/lWP9lGsMFDtsiZ5Wst1W8omrAgYfYo4RxOr14TUiJg3T\nmgTg10fBrcTC210Kl0lKNkHGi0qK8LaB2QwuWy7FKgx5I7yhgWKSWLVFhIOlca/X\nFo0mxCf8NPqOrhxRdS6UeucCAwEAAQ==\n-----END PUBLIC KEY-----\n",
+    "name": "Zapper",
+    "name_updated": "2018-06-04 03:22:19",
+    "username": "zapper",
+    "photo": {
+        "url": "https://zap.macgirvin.com/photo/profile/l/2",
+        "type": "image/png",
+        "updated": "2018-06-04 03:22:19"
+    },
+    "channel_role": "social",
+    "searchable": true,
+    "adult_content": false,
+    "public_forum": false,
+    "profile": {
+        "description": "",
+        "birthday": "",
+        "gender": "",
+        "marital": "",
+        "sexual": "",
+        "locale": "",
+        "region": "",
+        "postcode": "",
+        "country": "",
+        "about": "",
+        "homepage": "",
+        "hometown": ""
+    },
+    "permissions": "view_stream,view_profile,view_contacts,view_storage,view_pages,view_wiki",
+    "permissions_for": "",
+    "locations": [
+        {
+            "host": "zap.macgirvin.com",
+            "address": "zapper@zap.macgirvin.com",
+            "id_url": "https://zap.macgirvin.com/channel/zapper",
+            "primary": true,
+            "url": "https://zap.macgirvin.com",
+            "url_sig": "sha256.qBKZU6tReyUkVcNgGldRfdINiPoBneN9wWc-RHN7CFj8z9GgRW26LDUgmWL6kNoobYvHO6VIdZLxJb6CGdTLs7pjYGMZeTxpHHTgo3uHdBBIJdWPAwyEoppKGR3qT3S5iYWW9P0dsMtGjQ_q2VdaiqguoG8Z3lnTWikT7ujPI4NXZP2R0PVzEmaefN4SXqTO22XhXO-SuK4EOHylGcusQCfO6hXji9KItfwH1rnPx588YNRQ9WvBkV95ArZYSELRoFuJfHWh4ABqqAwQ4BqTO4-Pv1LiN1bWoNwVTki79Lx2GhQlw-_7HcHtVpqW_TQ04G4iPXvWHLzKfErfbGnQ575sbsF1gc2MYOINCofOmTq8eU_HOWaGK8D10HxpCVMMZXK37i8b6QEk3wpCoGiStGe5nsytVepZwNhsdnmW5msyO2ew_jZo3t_lP7U3oRvJHyJ7JpZBZg4E-MkLa-00KtiVGIosCesmFbZ042OwwTH3iJeSgz-yxrOsg3xdCj3e7rx4E93ra91OdN-mL-x8K4iYjcPQ6UpjInx2qsc_B8qwiw7L4jqJan7SYRDlxSiAb3yd1NFqL5gEMBg7RkS9s9a92hU3R6_K6CPpP9fb4mzRAzxpgRMpzhmBKnzlWF_Uz6c0urQpJLqJxfV4OzFThyxuqi3UyPg_R59DY3JfDUI",
+            "site_id": "gcwJ1OzIZbwtfgDcBYVYhwlUmjaxsgPyJezd-F2IS1F3IrlVsyOesNpm3hvoWemIBxoHmgIlMYKkhFeYihsqBQ",
+            "callback": "https://zap.macgirvin.com/zot",
+            "sitekey": "-----BEGIN PUBLIC KEY-----\nMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAuOkQslfq9EjZJLVniWP1\n3anzfdASnlgUKUYK1zyxy/HbwxAXl0GupYpJEVNIpkrGtTUMWY7ppxH7y/EAiSTJ\nsMIFIy1AnHgS/ecx6N/tH6rzZ68jD8yJQxjZBUk7MfhfYOK8KUti/qmp859Pr3cA\n1K1woCtSLRx2HNzHED8LDTUCGSwneHA2m7Ffc1MNfII8Ia/VtoF7pBwOixayws2N\nlY5syuDqOO3LtMJnDMBRN5WbtTw5jobyaoK6o4+Kg7Kln0nymloD3knFFsPvIdJd\nl493ItBi6k5QR0PV8NtElZMWRy8gZjzI5c4yukXku0WK1lVJBpjl4/LfjulWSMaR\nzj+YZKTjw7G56EEB2drNUVn/ZYLYwvMn6Bv/8lYu4VwL873UbupITNGX/Wh7+EU3\ntJqvXvdh8scIAKR3+sS/SrNI6OMn34HKewaX8iMf6NgW5lrskr9dhOYuZVr63huc\nhXxdGv+6b5+ARYEOpTn5QQd89WSL4Vui+1VaO4FARt9oRiC9s3sd0swyTxFqJSY4\nYJcpuVMUKY54jOGpsT0+1DlYFZf+lOk7pRpuYY1Vv/AhWCpkt6Uamf5d11rnVikA\nuPFgFqaObFenM1u1EKF1xrNaQqy3NbhOb0yRatVPcnAwOlesHbM7tgKmyZSopJTW\nJ2ug8isS+vNI0q+4IwET5FMCAwEAAQ==\n-----END PUBLIC KEY-----\n",
+            "deleted": false
+        }
+    ],
+    "site": {
+        "url": "https://zap.macgirvin.com",
+        "site_sig": "sha256.OWCNR-OocRwk2Y4RwSHtE-y_bcWPkXYPQetvRO-8VjO_b0eqwKDjvB52WPKKl43UQRHRn0CN6Xc487zPY7bVqIaLcsE23R0JOacl9IO3_9k-RbeH6H8KV8E_GynM-6cucPI1Dh7s3rgcqk-GXwoRaFaraurYDHvoGEVHOa1jHpv75lT2COCi1sDGhDR3-KPJbug61y58CUu3bJj2VRAqBaoiMz5TwbUIY9Sb22d205X_UzoIF_TlPDMoZv-Mbrkcxn9kgIfatgVyKGKKyoAnvyJeFzjHm1xCY4sZtt4C_em0a5wpcVPl31KbI5BodKn910ChErHXMCedBPeYWhRA0a-9Y_vYGonun3jXqJZ33WxzG9P1Gllp4bhxK6tm9X1iRpUnB7j8g8RHSH4PukQKSl2ErZ2vPLdHMIkczX9YEhhCbeZIvcX6T_5s82Ua75rlVktJGHsh8yLw3iqCdWljpCVhTWpEK0NmhJB6TcadE9qlRN9Gun7keEV4Ov6Dl5O7I-0ssoWbhv7lHU6JcjhAuf2TDLod_Izka32ZZk_8s8ZmqFzEEG6g6pRyuzvqk4XNK6cTL6dvBGIua5D0bRTBn4XTELx8u4B3yK7_MArr_m5Z5KfXOm_ngGMCN-lIZuKhxAQpCVDD5jcHmnCzjcDiR5m5LvOfvBSxwtpgHZUr3AU",
+        "post": "https://zap.macgirvin.com/zot",
+        "openWebAuth": "https://zap.macgirvin.com/owa",
+        "authRedirect": "https://zap.macgirvin.com/magic",
+        "sitekey": "-----BEGIN PUBLIC KEY-----\nMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAuOkQslfq9EjZJLVniWP1\n3anzfdASnlgUKUYK1zyxy/HbwxAXl0GupYpJEVNIpkrGtTUMWY7ppxH7y/EAiSTJ\nsMIFIy1AnHgS/ecx6N/tH6rzZ68jD8yJQxjZBUk7MfhfYOK8KUti/qmp859Pr3cA\n1K1woCtSLRx2HNzHED8LDTUCGSwneHA2m7Ffc1MNfII8Ia/VtoF7pBwOixayws2N\nlY5syuDqOO3LtMJnDMBRN5WbtTw5jobyaoK6o4+Kg7Kln0nymloD3knFFsPvIdJd\nl493ItBi6k5QR0PV8NtElZMWRy8gZjzI5c4yukXku0WK1lVJBpjl4/LfjulWSMaR\nzj+YZKTjw7G56EEB2drNUVn/ZYLYwvMn6Bv/8lYu4VwL873UbupITNGX/Wh7+EU3\ntJqvXvdh8scIAKR3+sS/SrNI6OMn34HKewaX8iMf6NgW5lrskr9dhOYuZVr63huc\nhXxdGv+6b5+ARYEOpTn5QQd89WSL4Vui+1VaO4FARt9oRiC9s3sd0swyTxFqJSY4\nYJcpuVMUKY54jOGpsT0+1DlYFZf+lOk7pRpuYY1Vv/AhWCpkt6Uamf5d11rnVikA\nuPFgFqaObFenM1u1EKF1xrNaQqy3NbhOb0yRatVPcnAwOlesHbM7tgKmyZSopJTW\nJ2ug8isS+vNI0q+4IwET5FMCAwEAAQ==\n-----END PUBLIC KEY-----\n",
+        "directory_mode": "normal",
+        "encryption": [
+            "aes256ctr.oaep",
+            "camellia256cfb.oaep",
+            "cast5cfb.oaep"
+        ],
+        "zot": "6.1",
+        "register_policy": "closed",
+        "access_policy": "private",
+        "accounts": 1,
+        "channels": 3,
+        "admin": "mike@macgirvin.com",
+        "plugins": [],
+        "sitehash": "c89e5a2b5059d04cc05078899c2083d4b89c190e6d6b247300256bfc66a930b3",
+        "sitename": "Zap Development",
+        "sellpage": "",
+        "location": "",
+        "realm": "RED_GLOBAL",
+        "project": "zap",
+        "version": "6.6"
+    }
+}
+````
\ No newline at end of file
diff --git a/spec/Zot6/Encryption+Signatures.md b/spec/Zot6/Encryption+Signatures.md
new file mode 100644
index 000000000..b1071fcb4
--- /dev/null
+++ b/spec/Zot6/Encryption+Signatures.md
@@ -0,0 +1,142 @@
+### Encryption
+
+Sites provide in their site discovery document an array containing 0 or more encryption algorithms which it is willing to accept in order of preference. Sites sending encrypted documents MUST use this list to determine the most suitable algorithm to both parties. If a suitable algorithm cannot be negotiated, the site MAY fall back to plaintext (unencrypted data) but if the communications channel is not secured with SSL the sending site MUST NOT use plaintext and a receiving site MAY ignore or reject the communication if it contains private or sensitive information.
+
+If the receiving site does not support the provided algorithm, it MUST return error 400.
+
+Encrypted information is encapsulated into a JSON array/object with the following components:
+
+````
+'encrypted' => true
+'key'       => The encryption key, base64urlencoded
+'iv'        => The encryption initialisation vector, base64urlencoded
+'alg'       => The encryption algorithm used
+'data'      => The encrypted payload, base64urlencoded
+````
+
+The 'encrypted' boolean flag indicates this is a cryptographic structure and requires decryption to extract the information. 'alg' is required. Other elements may be change as necessary to support different mechanisms/algorithms. For instance some mechanisms may require an 'hmac' field. The elements shown support a wide variety of standard encryption algorithms. 
+
+The key and iv are psuedo-random byte sequences, encrypted with the RSA public key of the recipient prior to base64urlencoding. The recipient key used in most cases (by default) will be the remote site public key. In certain circumstances (where specified) the RSA public key will be that of the target channel or recipient.
+
+Both 'key' and 'iv' MAY be padded to 255 chars. The amount of padding necessary is dependent on the encryption algorithm. The incoming site MUST strip additional padding on both parameters to limit the maximum length supported by the chosen algorithm. For example, the aes256cbc algorithm (not recommended) uses a key length of 32 bytes and an iv of 16 bytes.
+
+Algorithm names for common algorithms are the lowercase algorithm names used by the openssl library with punctuation removed. The openssl 'aes-256-ctr' algorithm for example is designated as 'aes256ctr'.
+
+Uncommon algorithms which are unsupported by openssl may be used, but the exact algorithm names are undefined by this document. 
+
+
+### Signatures
+
+Identity provenance is provided using HTTP Signatures (draft-cavage-http-signatures-10 is the relevant specification currently). If using encrypted transport, the HTTP Signature MAY be encrypted using the same negotiated algorithm as is used in the message for envelope protection. See [HTTP Signatures](spec/HTTPSignatures/Home). If a site uses negotiated encryption as described in the preceding section, it MUST be capable of decrypting the HTTP Signatures. 
+
+In several places of the communications where verification is associated with a third party which is not the sender of the relevant HTTP packet, signed data/objects are specified/required. Two signature methods may be used, depending on whether the signed data is a single value or a JSON object. The method used for single values is referred to here as SimpleSignatures. The object signature method used is traditionally known as salmon magic signatures, using the JSON serialisation.
+
+#### Simple Signatures
+
+A data value is signed with an appropriate RSA method and hash algorithm; for instance 'sha256' which indicates an RSA keypair signature using the designated RSA private key  and the 'sha256' hash algorithm. The result is base64url encoded, prepended with the algorithm name and a period (0x2e) and sent as an additional data item where specified. 
+
+````
+"foo_sig": "sha256.EvGSD2vi8qYcveHnb-rrlok07qnCXjn8YSeCDDXlbhILSabgvNsPpbe..."
+````
+
+To verify, the element content is separated at the first period to extract the algorithm and signature. The signature is base64urldecoded and verified using the appropriate method and hash algorithm using the designated RSA public key (in the case of RSA signatures). The appropriate key to use is defined elsewhere in the Zot protocol depending on the context of the signature.
+
+Implementations MUST support RSA-SHA256 signatures. They MAY support additional signature methods. 
+
+#### Salmon "magic envelope" Signatures with JSON serialisation.
+
+````
+{
+  "signed": true,
+  "data": "PD94bWwgdmVyc2lvbj0nMS4wJyBlbmNvZGl...",
+  "data_type": "application/x-zot+json",
+  "encoding": "base64url",
+  "alg": "RSA-SHA256",
+  "sigs": [
+    {
+    "value": "EvGSD2vi8qYcveHnb-rrlok07qnCXjn8YSeCDDXlbhILSabgvNsPpbe...",
+    "key_id": "4k8ikoyC2Xh+8BiIeQ+ob7Hcd2J7/Vj3uM61dy9iRMI"
+    }
+  ]
+}
+````
+
+The boolean "signed" element is not defined in the magic envelope specification. This is a boolean flag which indicates the current element is a signed object and requires verification and unpacking to retrieve the actual element content.  
+
+The signed data is retrieved by unpacking the magic signature 'data' component. Unpacking is accomplished by stripping all whitespace characters (0x0d 0x0a 0x20 and 0x09) and applying base64 "url" decoding.
+
+The key_id is the base64urlencoded identifier of the signer, which when applied to the Zot 'Discovery' process will result in locating the public key. This is typically the server base url or the channel "home" url. Webfinger identifiers (acct:user@domain) MAY also be used if the resultant webfinger document contains a discoverable public key (salmon-public-key or Webid key). 
+
+Verification is performed by using the magic envelope verification method. First remove all whitespace characters from the 'data' value. Append the following fields together separated with a period (0x2e). 
+
+data . data_type . encoding . algorithm
+
+This is the "signed data". Verification will take place using the RSA verify function using the signed data, the base64urldecoded sigs.value,  algorithm specified in 'alg' and the public key found by performing Zot discovery on the base64urldecoded sigs.key_id. 
+
+When performing Zot discovery for keys, it is important to verify that the principal returned in the discovery response matches the principal in the key_id and that the discovery response is likewise signed and validated.  
+
+Some historical variants of magic signatures emit base64 url encoding with or without padding.
+
+In this specification, encoding MUST be the string "base64url", indicating the url safe base64 encoding as described in RFC4648, and without any trailing padding using equals (=) characters
+
+The unpacked data (once verified) may contain a single value or a compound object. If it contains a single value, this becomes the value of the enclosing element. Otherwise it is merged back as a compound object. 
+
+Example: source document
+
+````
+{ 
+    "guid": {
+      "signed": true,
+      "data": "PD94bWwgdmVyc2lvbj0nMS4wJyBlbmNvZGl...",
+      "data_type": "application/x-zot+json",
+      "encoding": "base64url",
+      "alg": "RSA-SHA256",
+      "sigs": [
+        {
+        "value": "EvGSD2vi8qYcveHnb-rrlok07qnCXjn8YSeCDDXlbhILSabgvNsPpbe...",
+        "key_id": "4k8ikoyC2Xh+8BiIeQ+ob7Hcd2J7/Vj3uM61dy9iRMI"
+        }
+      ]
+    },
+    "address": "foo@bar"
+}
+````
+
+Decoding the data parameter (and assuming successful signature verification) results in
+
+````
+"abc12345"
+````
+
+Merging with the original document produces
+
+````
+{
+    "guid": "abc12345",
+    "address": "foo@bar"
+}
+````
+
+Example using a signed object containing multiple elements:
+
+
+Decoding the data parameter (and assuming successful signature verification) results in
+
+````
+{
+    "guid": "abc12345",
+    "name": "Barbara Jenkins"
+}
+````
+
+Merging this with the original document produces
+
+````
+{
+    "guid": {
+        "guid": "abc12345",
+        "name": "Barbara Jenkins"
+    },
+    "address": "foo@bar"
+}
+````
\ No newline at end of file
diff --git a/spec/Zot6/Home.md b/spec/Zot6/Home.md
new file mode 100644
index 000000000..8b563dc7b
--- /dev/null
+++ b/spec/Zot6/Home.md
@@ -0,0 +1,92 @@
+## Zot/6 aka Zot 2018
+
+This document describes version 6 of the Zot protocol. This is a living document, last modified on 2018-09-14.
+
+Zot is a **WebMTA** which provides a decentralised identity and communications protocol using HTTPS/JSON. 
+
+Earlier revisions of the zot protocol dealt with the creation of nomadic identities and cross-domain authentication to enable a decentralised network with features rivaling large centralised providers.
+
+Zot/6 builds on those concepts and streamlines many of the interactions, applying lessons learned over decades of building decentralised systems.
+
+A reference implementation (a social media app) is provided at https://framagit.org/zot/zap ; the reference implementation is not yet 100% spec compliant or complete but may be used for basic compatibility testing and development purposes. 
+
+### Differences from earlier Zot versions
+
+1. Streamlined communications using direct (push) transfer. Earlier versions used a 'notify/pickup' delivery model.
+2. The authentication component (Magic-Auth) has been spun off into a separate and independent specification ['OpenWebAuth'](spec/OpenWebAuth/Home.md).
+3. Inclusion of ActivityStreams (JSON-LD) as a supported (primary) serialisation.
+4. Dropping the requirements for implementations to support secondary serialisations.
+5. Moving service discovery to "Accept-header" based service endpoints; where different representations can be selected by modification of the HTTPS request Accept: header. 
+6. Separation of the "portable-id" from the signature algorithm used.
+
+### Differences between Zot and other WebMTA services
+
+Zot is architecturally different from other HTTPS-based "social communications" protocols such as OStatus, ActivityPub, and Diaspora. The primary differences are:
+
+1. Support for nomadic identity where an identity is not permanently bound to a DNS server.
+2. MUAs built on top of Zot are able to use and express detailed cross-domain permissions.
+3. Encryption negotation for additional message protection in addition to HTTPS
+4. Zot does not define an absolute payload format for content. Implementations MUST support ActivityStreams. Additional message types and serialisation formats MAY provide vendor specific enhancements.   
+5. Federation between other WebMTA protocols is not hampered by unnecessary restrictions on 3rd party communications. Messages from incompatible systems may be relayed to other sites which do not support the 3rd party protocol.
+6. Detailed delivery reporting is provided for auditing and troubleshooting; which is critical in a distributed communications service.
+
+## Fundamental Concepts
+
+### Identity
+
+In Zot/6 decentralised identity is a core attribute of the system. This is somewhat different than (and often incompatible with) typical decentralised or P2P communications projects where messaging is the key component and identity is merely an atribution on a message.
+
+An identity consists of two primary components.
+
+1. An identity "claim" which is essentially a text string
+2. An RSA based key with which to verify that identity
+
+These will be described in detail later. A "valid" identity is an identity which has been verified using public key cryptographic techniques. Until verified, an identity is "claimed" (unverified). An "invalid" identity is an identity which failed verification or has been revoked. 
+
+In Zot/6 claimed (unverified) identities are allowed to exist, and MAY be allowed permissions to perform operations. This allows cross communication to occur between Zot/6 and other communications systems with different concepts of identity and different methods of validation.
+
+An implementation MAY choose to block unverified identities. This may significantly reduce the ability to interact with foreign systems, protocols, and networks.   
+
+Invalid identities MUST NOT be allowed any permissions. By definition they are forgeries or have been revoked. 
+
+### Channels
+
+Channels are what would typically be referred to as "users" on other systems, although this notion is abstracted in Zot/6 and can take on other meanings. A channel is represented as an identity. 
+
+### Location and Location Independence
+
+A location is an identity and follows all the rules associated with an identity. The identity claim is typically the fully qualified "base" URL of the web service providing Zot/6 services, lowercased, with trailing slashes removed. International domain names are converted to "punycode".
+
+A "location independent identity" is a valid channel identity paired with a valid location, with the additional property that the channel/location pair has been validated using the key of the channel identity. In layman's terms, the user signs his/her website with their own key.
+
+The location independent model allows mobility of a channel to different or even multiple locations; which may all be valid simultaneously. This is referred to elsewhere as a "nomadic" identity. 
+
+For the benefit of those that are new to this concept, this means in basic terms that a channel can appear at any location at any time as long as the location independent identity is valid.
+
+### Linked identities
+
+One or more channels/identities can be linked. In simple terms, this allows us to define a persistent token which refers to an identity whose claim string or public key (or both) have changed or is being merged from another system. In order to link identities, in addition to the requirement that both identities MUST validate, each identity MUST sign the other linked identity and all of these signatures MUST be valid.
+
+In the case of providing controlled access to website resources, all linked identities SHOULD be folded or reduced to a common identifier so that an attempt to access a protected resource by any linked identity will succeed for a resource that has been made available to any of its linked identities.
+
+### Nomadic Considerations
+
+The location independent properties of Zot/6 place additional requirements on communications systems. For outgoing messages, delivery SHOULD be attempted to every linked and nomadic location associated with the identity being delivered to. The service MAY remove a valid linked or nomadic location from the list of delivery targets if the location is marked "dead" or "unreachable". A location MAY be marked "dead" or "unreachable" for a number of reasons; generally by a failure to communicate for more than 30 days, or by manual means if the site administrator has reason to believe the location has been shut down permanently. A site SHOULD retain the location record and automatically remove the "dead" or "unreachable" designation if valid communications are initiated from that location in the future.   
+
+
+## Transport and Protocol Basics
+
+Communications with Zot/6 take place primarily with https JSON requests. Requests are signed by the "sender" using HTTP-Signatures (draft-cavage-http-signatures-xx). If a payload is present, the signed headers MUST include a Digest header using SHA-256 or SHA-512 as specified in RFC5843. 
+
+For HTTP requests which do not carry a payload any headers may be signed, but at least one header MUST be signed. 
+
+Signed requests MAY be rejected if
+
+- Digest verification fails
+- The request has a JSON payload and the content is not signed
+- public-key retrieval fails
+
+
+Public keys are retrieved by utilising Zot/6 Discovery on the signature data's 'keyId' property and retrieving the public key from that document. Zot Discovery is described in a later chapter.
+
+Receivers verifying these requests SHOULD verify that the signer has authority over the data provided; either by virtue of being the author or a relay agent (sender).
\ No newline at end of file
diff --git a/spec/Zot6/Message Types.md b/spec/Zot6/Message Types.md
new file mode 100644
index 000000000..e4e0f1c22
--- /dev/null
+++ b/spec/Zot6/Message Types.md
@@ -0,0 +1,111 @@
+## Message Types
+
+### purge
+
+This message type has no payload or encoding. If the message has recipients, it is considered an "unfriend" action. The sender's relationship with the recipients is severed. The exact actions which are undertaken as a result are implementation specific. Generally, the sender's permissions to the receiver are revoked. Permissions granted to the sender by the receiver MAY be left intact. 
+
+If the message has no recipients, it is considered a notification that the sender identity no longer exists. Receiving sites MUST mark the channel as unavailable and discontinue further communications. They SHOULD destroy all public content attributed to the sender and MAY remove the connection and private content from connected channels. 
+
+The response to this message is 
+
+````
+{
+  'success': true
+}
+````
+
+or
+
+````
+{
+  'success': false,
+  'message': 'optional error message or reason'
+}
+````
+
+Servers SHOULD only provide a single recipient when using a targetted message as the single return response may be ambiguous.
+
+
+### refresh
+
+This message has no payload or encoding. It is a message to the receiving server that important channel information has changed. The receiving server MUST process a zot discovery operation and update any locally stored information which has changed. If the message has recipients, this action should be accomplished by the receiver using a signed discovery fetch, signed by the recipient. Response is the same as for a purge message. 
+
+A targetted refresh message (e.g. containing recipients) is commonly used to indicate a change in permissions granted to the recipient by the sender. If no permissions have previously been associated with this sender, it is considered to be a "friend request", meaning some permissions are available which may not have been available before. The receiving channel SHOULD store the updated permissions and SHOULD add the sender to the receiver's known connections. It MAY notify the recipient that a new friend exists and put the connection into a 'pending' state until the request has been reviewed/accepted/rejected by the recipient. 
+
+### rekey
+
+The rekey message is sent with no recipients. The message indicates the sender has changed their public key. The key change MUST be signed with both the old and new private keys and these signatures MUST validate or the operation MUST fail.
+The 'update' boolean flag (if present) indicates the old portable\_id associated with that key is to be changed, and the old portable\_id discarded. If 'update' is false, a new portable\_id is generated and the old and new identities are linked. This means that both portable\_id's are valid nomadic identifiers for the same channel. Response is the same as for the purge message. 
+
+### activity
+
+Message encoding: activitystreams
+
+This message type is used for normal communications. If recipients are specified, the message is private. If no recipients are specified, the message is public. The recipient list MAY be filtered and contain only the recipients which are known to be available on the receiving website. A message MUST NOT be sent by the sender if the message is private and the recipient list for a particular receiving site is empty.
+
+
+````
+{
+    "type": "Create",
+    "id": "https://example.org/item/e8a20a21dd7e0d8d2207a5df62d2168d65db7db08f1a91ca",
+    "published": "2018-06-25T04:14:08Z",
+    "actor": "https://example.org/channel/zapper",
+    "object": {
+        "type": "Article",
+        "id": "https://example.org/item/e8a20a21dd7e0d8d2207a5df62d2168d65db7db08f1a91ca",
+        "published": "2018-06-25T04:14:08Z",
+        "content": "just another Zot6 message",
+        "actor": "https://example.org/channel/zapper",
+    },
+}
+````
+
+Upon delivery, a delivery report is generated and returned to the sending site.
+
+````
+{
+    "success": true,
+    "delivery_report": [
+        {
+            "location": "https://zap.macgirvin.com",
+            "sender": "T5ni0wUAlYmyqlibiTQiS54PqLKXCL7XAJhOSKeJMqEXeKn46AkDPdCJZ4JUA05Vlhux25OLTkBPyV7L60JmyQ",
+            "recipient": "xxWsqvZp3w-sr3FXrmb6wxmKZx6khMLjBCOafPdRT1lWzYmCPHeaDDBD9KwOqpOAt4lezIFQbyaLt9I3H54M9Q",
+            "name": "System",
+            "message_id": "https://zap.macgirvin.com/item/ebaa483a2e8331a21a68b9d9e4a72a079c260162d2e37edc",
+            "status": "posted",
+            "date": "2018-06-26 05:19:48"
+        },
+        {
+            "location": "https://zap.macgirvin.com",
+            "sender": "T5ni0wUAlYmyqlibiTQiS54PqLKXCL7XAJhOSKeJMqEXeKn46AkDPdCJZ4JUA05Vlhux25OLTkBPyV7L60JmyQ",
+            "recipient": "wXDz7WR51QHwAORaVR8-0wff06LBtBWvhd_zfDWTYEzaqaPfJ_fsK7nRaM4aVeKPmZklUAgtqs09zUzitwNT2w",
+            "name": "Zapper",
+            "message_id": "https://zap.macgirvin.com/item/ebaa483a2e8331a21a68b9d9e4a72a079c260162d2e37edc",
+            "status": "posted",
+            "date": "2018-06-26 05:19:48"
+        },
+        {
+            "location": "https://zap.macgirvin.com",
+            "sender": "T5ni0wUAlYmyqlibiTQiS54PqLKXCL7XAJhOSKeJMqEXeKn46AkDPdCJZ4JUA05Vlhux25OLTkBPyV7L60JmyQ",
+            "recipient": "T5ni0wUAlYmyqlibiTQiS54PqLKXCL7XAJhOSKeJMqEXeKn46AkDPdCJZ4JUA05Vlhux25OLTkBPyV7L60JmyQ",
+            "name": "Bopper",
+            "message_id": "https://zap.macgirvin.com/item/ebaa483a2e8331a21a68b9d9e4a72a079c260162d2e37edc",
+            "status": "update ignored",
+            "date": "2018-06-26 05:19:48"
+        }
+    ]
+}
+````
+
+### response
+
+The response message is used to send a comment or reply "upstream" to the sender. The sender will then redeliver the message to all downstream recipients. It is the same as the activity message type except that it MUST have one and only one recipient - the sender of the activity that this activity references. This entire activity SHOULD be signed by the sender of the response and encapsulated as a json salmon magic envelope as described in [[Encryption/Signatures]]. 
+
+
+### sync
+
+Sync messages are used between nomadic clones to synchronise changed data structures. These messages are sent to nomadic instances of the sender as private messages and SHOULD be encrypted with additional encryption beyond HTTPS transport encryption as the messages may contain private keys.
+
+Implementations MAY provide sync messages and they MAY attempt to co-exist with sync packets created by other implementations. If their data synchronisation needs cannot be mapped to the sync structures provided by others, they MUST provide a unique encoding type (recommend the name of the implementation using only the letters [a-z] of the US-ASCII character set). If a site receives a sync message with an unknown encoding and data format it MUST be ignored. 
+
+Basic sync information includes any local changes to personal settings, profile settings, and changes in the social graph. Implementations MAY support syncing of all available information including uploaded files and photos, events, and other application specific data structures.
\ No newline at end of file
diff --git a/spec/Zot6/Messages.md b/spec/Zot6/Messages.md
new file mode 100644
index 000000000..744f0a4c0
--- /dev/null
+++ b/spec/Zot6/Messages.md
@@ -0,0 +1,135 @@
+### Messages
+
+Zot6 is primarily a transport and identification format. The semantics of message content are in many respects outside the scope of this document. Sites/servers MUST provide in their site discovery document what standardised message formats are acceptable.
+
+The message formats of primary concern to us are 
+
+1. ActivityStreams (ActivityStreams JSON-LD) (format='activitystreams')
+2. Zot (format='zot')
+
+When using ActivityStreams JSON-LD, a default @context of "https://www.w3.org/ns/activitystreams" is assumed. Activity objects only need to specify or provide a @context declaration if there are differences from the default. 
+
+### Delivery
+
+Delivery is a POST of envelope+data to the zot endpoint. HTTP Signatures are used to validate the sender. 
+
+Delivery reports for private messages SHOULD be encrypted and MUST include results for host blocking.
+
+Here is a sample activity...
+
+````
+{
+    "type": "activity",
+    "encoding": "activitystreams",
+    "sender": "wXDz7WR51QHwAORaVR8-0wff06LBtBWvhd_zfDWTYEzaqaPfJ_fsK7nRaM4aVeKPmZklUAgtqs09zUzitwNT2w",
+    "site_id": "gcwJ1OzIZbwtfgDcBYVYhwlUmjaxsgPyJezd-F2IS1F3IrlVsyOesNpm3hvoWemIBxoHmgIlMYKkhFeYihsqBQ",
+    "recipients": {
+         "xxWsqvZp3w-sr3FXrmb6wxmKZx6khMLjBCOafPdRT1lWzYmCPHeaDDBD9KwOqpOAt4lezIFQbyaLt9I3H54M9Q",
+         "T5ni0wUAlYmyqlibiTQiS54PqLKXCL7XAJhOSKeJMqEXeKn46AkDPdCJZ4JUA05Vlhux25OLTkBPyV7L60JmyQ",
+    },
+    "version": "6.0",
+    "data": {
+        "type": "Create",
+        "id": "https://example.org/item/e8a20a21dd7e0d8d2207a5df62d2168d65db7db08f1a91ca",
+        "published": "2018-06-25T04:14:08Z",
+        "actor": "https://example.org/channel/zapper",
+        "object": {
+            "type": "Article",
+            "id": "https://example.org/item/e8a20a21dd7e0d8d2207a5df62d2168d65db7db08f1a91ca",
+            "published": "2018-06-25T04:14:08Z",
+            "content": "just another Zot6 message",
+            "actor": "https://example.org/channel/zapper",
+        },
+    },
+}
+````
+
+The sender\_id is the portable\_id of the sender. The site\_id is the portable\_id of the sender's website.
+Recipients is a simple array of portable_id's of message recipients. This list MAY be filtered and contain only the recipients which are known to be available on the receiving website. 
+
+Version is provided to resolve potential protocol version differences over time. Data contains the actual ActivityStreams (in this case) payload.
+
+
+Receiving sites MUST verify the HTTP Signature provided and MUST reject (error 400) posts with no signature or invalid signatures. Discovery is used during the verification process and generates a locally stored portable\_id. If the portable\_id does not match the verified signer's portable\_id, the message MUST be rejected (error 400). 
+
+If the recipients field is non-existent or empty, the post is considered public and may be delivered to any channel following the sender. If the recipient field has contents, the message MUST NOT be delivered to anybody except the listed recipients.
+
+Upon successful delivery, a delivery report is generated and returned to the sending site.
+
+````
+ {
+    "success": true,
+    "delivery_report": [
+        {
+            "location": "https://zap.macgirvin.com",
+            "sender": "T5ni0wUAlYmyqlibiTQiS54PqLKXCL7XAJhOSKeJMqEXeKn46AkDPdCJZ4JUA05Vlhux25OLTkBPyV7L60JmyQ",
+            "recipient": "xxWsqvZp3w-sr3FXrmb6wxmKZx6khMLjBCOafPdRT1lWzYmCPHeaDDBD9KwOqpOAt4lezIFQbyaLt9I3H54M9Q",
+            "name": "System ",
+            "message_id": "https://zap.macgirvin.com/item/ebaa483a2e8331a21a68b9d9e4a72a079c260162d2e37edc",
+            "status": "posted",
+            "date": "2018-06-26 05:19:48"
+        },
+        {
+            "location": "https://zap.macgirvin.com",
+            "sender": "T5ni0wUAlYmyqlibiTQiS54PqLKXCL7XAJhOSKeJMqEXeKn46AkDPdCJZ4JUA05Vlhux25OLTkBPyV7L60JmyQ",
+            "recipient": "wXDz7WR51QHwAORaVR8-0wff06LBtBWvhd_zfDWTYEzaqaPfJ_fsK7nRaM4aVeKPmZklUAgtqs09zUzitwNT2w",
+            "name": "Zapper ",
+            "message_id": "https://zap.macgirvin.com/item/ebaa483a2e8331a21a68b9d9e4a72a079c260162d2e37edc",
+            "status": "posted",
+            "date": "2018-06-26 05:19:48"
+        },
+        {
+            "location": "https://zap.macgirvin.com",
+            "sender": "T5ni0wUAlYmyqlibiTQiS54PqLKXCL7XAJhOSKeJMqEXeKn46AkDPdCJZ4JUA05Vlhux25OLTkBPyV7L60JmyQ",
+            "recipient": "T5ni0wUAlYmyqlibiTQiS54PqLKXCL7XAJhOSKeJMqEXeKn46AkDPdCJZ4JUA05Vlhux25OLTkBPyV7L60JmyQ",
+            "name": "Bopper ",
+            "message_id": "https://zap.macgirvin.com/item/ebaa483a2e8331a21a68b9d9e4a72a079c260162d2e37edc",
+            "status": "update ignored",
+            "date": "2018-06-26 05:19:48"
+        }
+    ]
+}
+````
+
+### Followups
+
+Followups to any post (replies, likes, reactions, etc.) MUST be sent as a private activity (single recipient) to the sender of the original with a message type 'response'. This is referred to as an "upstream delivery". 
+
+Additionally these activities MUST provide an 'inReplyTo' element set to the id of the activity that is the object of the response. Implementations SHOULD support multi-level likes. Servers MAY support multi-level comments. 
+
+
+The original sender MUST resend the followups to all of the original message recipients using message type 'activity'. This is referred to as a "downstream delivery". 
+
+This single-source mechanism ensures the original sender's privacy settings are respected and conversations are kept intact for all recipients of the original message. 
+
+### Multi-level support
+
+If multi-level comments are not supported, the receiver MUST re-parent the multi-level comment to the original conversation head, flattening the conversation to one level. The receiving server SHOULD store the correct target id, even if the comment is re-parented. 
+
+If multi-level likes are not supported, the incoming "like" activity for a non-parent conversation node MAY be dropped or rejected, rather than being re-parented to a potentially unrelated activity than was intended.   
+
+
+### Portable IDs
+
+Portable IDs are intended to provide location-independent identifiers across the network. A portable ID can only be trusted if it has been verified or if it has been generated on this site. Verification is performed during the [[Discovery]] process. 
+
+The portable ID uses an obtained public\_key. In order to ensure that the portable ID calculation is portable, the calculation MUST be performed on a PKCS#8 public\_key. If the source key is provided in another format (such as a salmon [modulus/exponent] key or PKCS#1), the key MUST be converted to PKCS#8 format prior to calculating the portable ID.   
+
+Here are the steps for generating a portable\_id.
+
+1. Via [[Discovery]], obtain the zot-info packet for a network URI. The network URI will often be presented as the signer keyID in a signed message, but may also be provided as an acct: URI submitted by site users.
+2. The zot-info packet contains everything necessary to verify an identity claim.
+3. With the public\_key, verify the id\_sig against the id (claimed identity).
+4. With the public\_key, verify the location->url\_sig against the location->url **for the discovery site**.
+5. With site->sitekey, verify the site->site\_sig against the site->url.
+6. Concatenate the id and the public\_key. Perform a whirlpool hash function on this concatenated string and base64_url encode the result. 
+7. This is the channel portable\_id.
+8. Concatenate the url and site->sitekey. Perform a whirlpool hash function on this concatenated string and base64_url encode the result. 
+9. This is the portable site\_id. Check that it matches the location->site\_id in the discovery packet. 
+10. If any verification step fails, discard the results and return an error. 
+
+This is generally considered an expensive calculation; hence the results should be stored as a channel/location pair. 
+
+When receiving a Zot message, use the stored results (if available) for the signer's keyID when checking the HTTP Signature. If the HTTP Signature validates, and the keyID matches the location->id\_url for this location, and the sender portable\_id matches the calculated/stored portable\_id for this channel/location pair, the nomadic sender has been validated.
+
+If no stored results are available for the keyID, perform [[Discovery]] as described.
\ No newline at end of file
diff --git a/spec/Zot6/Nomadic Identity.md b/spec/Zot6/Nomadic Identity.md
new file mode 100644
index 000000000..188707f62
--- /dev/null
+++ b/spec/Zot6/Nomadic Identity.md
@@ -0,0 +1,7 @@
+### Nomadic Identity
+
+One of the fundamental differences between Zot and other messaging systems/protocols is the support for nomadic identity. This simply means that your identity (who you are) is independent from the server you are posting from (where you are).
+
+As a consequence, a person can have any number of active locations. Implementations which support nomadic identity MUST send a copy of all communications destined for that identity to all known active locations. If sites receive a communication from the given identity from any location, they MUST validate that the identity has authorised this location and (if verification is successful) deliver it appropriately. They SHOULD store the newly verified location and MAY subsequently used the stored information rather than re-validating.
+
+When an identity modifies its location information, it MUST send a 'refresh' packet to all known sites where they maintain connections and which are capable of nomadic operation. The 'refresh' message informs the other site to perform network discovery and update all stored information related to the identity which may have changed.
\ No newline at end of file