Reorganized and revised Zot Protocol content. Deleted some of the old pages that have been refactored.

10 files changed, 368 insertions, 479 deletions

diff --git a/doc/developer/zot_protocol.bb b/doc/developer/zot_protocol.bb
index 007672c70..159024612 100644
--- a/doc/developer/zot_protocol.bb
+++ b/doc/developer/zot_protocol.bb
@@ -1,6 +1,14 @@
 [h3] What is Zot?[/h3]
 Zot is the revolutionary protocol that powers $Projectname, providing [b]communications[/b], [b]identity management[/b], and [b]access control[/b] across a fully [b]decentralised[/b] network of independent websites, often called "the grid". The resulting platform is a robust system that supports privacy and security while enabling the kind of rich web services typically seen only in centralized, proprietary solutions.
 
+Consider this typical scenario: 
+
+Jaquelina wishes to share photos with Roberto from her blog at [b]jaquelina.org[/b], but to nobody else. Roberto maintains his own family hub at [b]roberto.net[/b] on a completely independent server. Zot allows Jaquelina to publish her photos using an [i]access control list (ACL)[/i] that includes only Roberto. That means that while Roberto can see the photos when he visits her blog, his brother Marco cannot, and neither can any of his other family members who have accounts on [b]roberto.net[/b].
+
+The magic in this scenario comes from the fact that Roberto never logged in to Jaquelina's website. Instead, he had to login only once using his password on his [i]own[/i] website at [b]roberto.net[/b]. When Roberto visits [b]jaquelina.org[/b], her hub seamlessly authenticates him by remotely querying his server in the background. 
+
+It is not uncommon for servers to have technical problems or become inaccessible for a variety of reasons. Zot provides robustness for Roberto's online activities by allowing him to have [i]clones[/i] of his online identity, or [i]channel[/i], on multiple independent hubs. Imagine that Roberto's server crashes for some reason and he cannot log in there. He simply logs in to one of his clones at [b]gadfly.com[/b], a site operated by his friend Peter. Once authenticated at [b]gadfly.com[/b], Roberto can view Jaquelina's blog as before, without Jaquelina having to grant any additional access!
+
 [h4] Communications[/h4]
 Communications and social networking are an integral part of the grid. Any channel (and any services provided by that channel) can make full use of feature-rich social communications on a global scale. These communications may be public or private - and private communications comprise not only fully encrypted transport, but also encrypted storage to help protect against accidental snooping and disclosure by rogue system administrators and internet service providers. 
 
@@ -32,6 +40,366 @@ This type of control is trivial on large corporate providers because they own th
 
 Access can be granted or denied for any resource, to any channel, or any group of channels; anywhere within the grid. Others can access your content if you permit them to do so, and they do not even need to have an account on your hub. 
 
+[h3]Technical Introduction[/h3]
+Zot is a JSON-based web framework for implementing secure decentralised communications and services. In order to provide this functionality, Zot creates a decentralised globally unique identifier for each hub on the network. This global identifier is not linked inextricably to DNS, providing the requisite mobility. Many existing decentralised communications frameworks provide the communication aspect, but do not provide remote access control and authentication. Additionally most of these are based on 'webfinger', which still binds identity to domain names and cannot support nomadic identity.
+
+The primary issues Zot addresses are 
+
+[list]
+[*] completely decentralised communications
+[*] independence from DNS-based identity
+[*] node mobility
+[*] seamless remote authentication
+[*] high performance
+[/list]
+We will rely on DNS-based user@host addresses as a "user-friendly" mechanism to let people know where you are, specifically on a named host with a given username, and communication will be carried out to DNS entities using TCP and the web.
+
+But the underlying protocol will provide an abstraction layer on top of this, so that a communications node (e.g. "identity") can move to an alternate DNS location and (to the best of our ability) gracefully recover from site re-locations and/or maintain pre-existing communication relationships. A side effect of this requirement is the ability to communicate from alternate/multiple DNS locations and service providers and yet maintain a single online identity.
+
+We will call this overlay network the "grid". Servers attached to this network are known as "hubs" and may support any number of individual identities. 
+
+An identity does not necessarily correspond to a person. It is just something which requires the ability to communicate within the grid. 
+
+The ability to recover will be accomplished by communication to the original location when creating a new or replacement identity, or as a fallback from a stored file describing the identity and their contacts in the case where the old location no longer responds. 
+
+At least on the short term, the mobility of existing content is not a top priority. This may or may not take place at a later stage. The most important things we want to keep are your identity and your friends.  
+
+Addresses which are shared amongst people are user@host, and which describe the [b]current[/b] local account credentials for a given identity. These are DNS based addresses and used as a seed to locate a given identity within the grid. The machine communications will bind this address to a globally unique ID. A single globally unique ID may be attached or bound to any number of DNS locations. Once an identity has been mapped or bound to a DNS location, communications will consist of just knowing the globally unique address, and what DNS (url) is being used currently (in order to call back and verify/complete the current communication). These concepts will be specified in better detail. 
+
+In order for an identity to persist across locations, one must be able to provide or recover 
+
+[list]
+[*] the globally unique ID for that identity
+[*] the private key assigned to that identity
+[*] (if the original server no longer exists) an address book of contacts for that identity.
+[/list]
+This information will be exportable from the original server via API, and/or downloadable to disk or thumb-drive. 
+
+We may also attempt to recover with even less information, but doing so is prone to account hijacking and will require that your contacts confirm the change. 
+
+In order to implement high performance communications, the data transfer format for all aspects of Zot is JSON. XML communications require way too much overhead.
+
+Bi-directional encryption is based on RSA 4096-bit keys expressed in DER/ASN.1 format using the PKCS#8 encoding variant, with AES-256-CBC used for block encryption of variable length or large items.
+
+Some aspects of well known "federation protocols" (webfinger, salmon, activitystreams, portablecontacts, etc.) may be used in zot, but we are not tied to them and will not be bound by them. $Projectname project is attempting some rather novel developments in decentralised communications and if there is any need to diverge from such "standard protocols" we will do so without question or hesitation. 
+
+In order to create a globally unique ID, we will base it on a whirlpool hash of the identity URL of the origination node and a psuedo-random number, which should provide us with a 256 bit ID with an extremely low probability of collision (256 bits represents approximately 115 quattuorviginitillion or 1.16 X 10^77 unique numbers). This will be represented in communications as a base64url-encoded string. We will not depend on probabilities however and the ID must also be attached to a public key with public key cryptography used to provide an assurance of identity which has not been copied or somehow collided in whirlpool hash space.
+
+Basing this ID on DNS provides a globally unique seed, which would not be the case if it was based completely on psuedo-random number generation.  
+
+We will refer to the encoded globally unique uid string as zot_uid 
+
+As there may be more than one DNS location attached/bound to a given zot_uid identity, delivery processes should deliver to all of them - as we do not know for sure which hub instance may be accessed at any given time. However we will designate one DNS location as "primary" and which will be the preferred location to view web profile information.
+
+We must also provide the ability to change the primary to a new location. A look-up of information on the current primary location may provide a "forwarding pointer" which will tell us to update our records and move everything to the new location. There is also the possibility that the primary location is defunct and no longer responding. In that case, a location which has already been mapped to this zot_uid can seize control, and declare itself to be primary. In both cases the primary designation is automatically approved and moved. A request can also be made from a primary location which requests that another location be removed. 
+
+In order to map a zot_uid to a second (or tertiary) location, we require a secure exchange which verifies that the new location is in possession of the private key for this zot_uid. Security of the private key is thus essential to avoid hijacking of identities.
+
+Communications will then require
+[list]
+[*] zot_uid (string)
+[*] uid_sig 
+[*] callback (current location Zot endpoint url)
+[*] callback_sig
+[*] spec (int)
+[/list]
+passed with every communique. The spec is a revision number of the applicable Zot spec so that communications can be maintained with hubs using older and perhaps incompatible protocol specifications. Communications are verified using a stored public key which was copied when the connection to this zot_uid was first established. 
+
+Key revocation and replacement must take place from the primary hub. The exact form for this is still being worked out, but will likely be a notification to all other bound hubs to "phone home" (to the primary hub) and request a copy of the new key. This communique should be verified using a site or hub key; as the original identity key may have been compromised and cannot be trusted.   
+
+In order to eliminate confusion, there should be exactly one canonical url for any hub, so that these can be indexed and referenced without ambiguity. 
+
+So as to avoid ambiguity of scheme, it is strongly encouraged that all addresses to be https with a "browser valid" cert and a single valid host component (either www.domain.ext or domain.ext, but not both), which is used in all communications. Multiple URLs may be provided locally, but only one unique URL should be used for all Zot communications within the grid.
+
+Test installation which do not connect to the public grid may use non-SSL, but all traffic flowing over public networks should be safe from session-hijacking, preferably with a "browser recognised" cert.
+
+Where possible, Zot encourages the use of "batching" to minimise network traffic between two hubs. This means that site 'A' can send multiple messages to site 'B' in a single transaction, and also consolidate deliveries of identical messages to multiple recipients on the same hub.
+
+Messages themselves may or may not be encrypted in transit, depending on the private nature of the messages. SSL (strongly encouraged) provides unconditional encryption of the data stream, however there is little point in encrypting public communications which have been designated as having unrestricted visibility. The encryption of data storage and so-called "end-to-end encryption" is outside the scope of zot. It is presumed that hub operators will take adequate safeguards to ensure the security of their data stores and these are functions of application and site integrity as opposed to protocol integrity. 
+
+
+[h4]Messages[/h4]
+
+Given the constraints listed previously, a Zot communique should therefore be a json array of individual messages. These can be mixed and combined within the same transmission.
+
+Each message then requires:
+[list]
+[*] type
+[*] (optional) recipient list
+[/list]
+Lack of a recipient list would indicate an unencrypted public or site level message. The recipient list would contain an array of zot_uid with an individual decryption key, and a common iv. The decryption key is encoded with the recipient identity's public key. The iv is encrypted with the sender's private key.
+
+All messages should be digitally signed by the sender.    
+
+The type can be one of (this list is extensible):
+[list]
+[*] post (or activity)
+[*] mail
+[*] identity
+[*] authenticate
+[/list]
+Identity messages have no recipients and notify the system social graph of an identity update, which could be a new or deleted identity, a new or deleted location, or a change in primary hub. The signature for these messages uses system keys as opposed to identity-specific keys.
+
+Posts include many different types of activities, such as top-level posts, likes/dislikes, comments, tagging activities, etc. These types are indicated within the message sturcture.
+
+authenticate messages result in mutual authentication and browser redirect to protected resources on the remote hub such as the ability to post wall-to-wall messages and view private photo albums and events, etc.
+
+[h4]Discovery[/h4]
+
+A well-known url is used to probe a hub for Zot capabilities and identity lookups, including the discovery of public keys, profile locations, profile photos, and primary hub location. 
+
+The location for this service is /.well-known/zot-info - and must be available at the root of the chosen domain.
+
+To perform a lookup, a POST request is made to the discovery location with the following parameters:
+
+Required:
+
+address  => an address on the target system such as "john"
+
+Optional
+
+target     => the Zot "guid" of the observer for evaluating permissions
+
+target_sig => an RSA signature (base64url encoded) of the guid
+
+key        => The public key needed to verify the signature
+
+token      => a string (possibly random) chosen by the requesting service. If provided, an entry in the discovered packet will be provided called 'signed_token' which consists of the base64url_encoded RSA signature of the concatenation of the string 'token.' and the provided token using the private key of the discovered channel. This can be verified using the provided 'key' entry, and provides assurance that the server is in possession of the private key for the discovered identity. After 2017-01-01 it is [b]required[/b] that a server provide a signed_token [i]if[/i] a token was provided in the request.  
+
+With no target provided, the permissions returned will be generic permissions 
+for unknown or unauthenticated observers
+
+Example of discovery packet for 'mike@zothub.com'
+[code nowrap]
+	{
+
+    "success": true,
+	"signed_token": "KBJrKTq1qrctNuxF3GwVh3GAGRqmgkirlXANPcJZAeWlvSt_9TMV097slR4AYnYCBEushbVqHEJ9Rb5wHTa0HzMbfRo8cRdl2yAirvvv5d98dtwHddQgX1jB0xEypXtmIYMdPGDLvhI1RNdIBhHkkrRcNreRzoy4xD--HM6m1W0-A8PJJJ9BcNxmGPcBtLzW08wzoP9trJ3M7DQ6Gkk6j7iwVsyApw1ZBaDvabGTdc_SFV-Iegtqw3rjzT_xXWsfzMlKBy-019MYn_KS-gu23YzjvGu5tS_zDfkQb8DMUlPLz5yyxM0yOMlUDtG2qQgIJAU2O0X6T5xDdJ6mtolNyhepg845PvFDEqBQGMIH1nc47CNumeudDi8IWymEALhjG_U8KAK7JVlQTJj2EKUb0au1g6fpiBFab5mmxCMtZEX3Jreyak5GOcFFz-WpxuXJD9TdSoIvaBfBFOoJnXkg2zE4RHXeQzZ2FotmrbBG5dm8B-_6byYGoHBc08ZsWze1K96JIeRnLpBaj6ifUDcVHxZMPcGHHT27dvU2PNbgLiBjlAsxhYqkhN5qOHN8XBcg2KRjcMBaI3V0YMxlzXz5MztmZq3fcB1p-ccIoIyMPMzSj3yMB7J9CEU2LYPSTHMdPkIeDE6GaCkQKviaQQJQde346tK_YjA2k7_SOBmvPYE",
+    "guid": "sebQ-IC4rmFn9d9iu17m4BXO-kHuNutWo2ySjeV2SIW1LzksUkss12xVo3m3fykYxN5HMcc7gUZVYv26asx-Pg",
+    "guid_sig": "Llenlbl4zHo6-g4sa63MlQmTP5dRCrsPmXHHFmoCHG63BLq5CUZJRLS1vRrrr_MNxr7zob_Ykt_m5xPKe5H0_i4pDj-UdP8dPZqH2fqhhx00kuYL4YUMJ8gRr5eO17vsZQ3XxTcyKewtgeW0j7ytwMp6-hFVUx_Cq08MrXas429ZrjzaEwgTfxGnbgeQYQ0R5EXpHpEmoERnZx77VaEahftmdjAUx9R4YKAp13pGYadJOX5xnLfqofHQD8DyRHWeMJ4G1OfWPSOlXfRayrV_jhnFlZjMU7vOdQwHoCMoR5TFsRsHuzd-qepbvo3pzvQZRWnTNu6oPucgbf94p13QbalYRpBXKOxdTXJrGdESNhGvhtaZnpT9c1QVqC46jdfP0LOX2xrVdbvvG2JMWFv7XJUVjLSk_yjzY6or2VD4V6ztYcjpCi9d_WoNHruoxro_br1YO3KatySxJs-LQ7SOkQI60FpysfbphNyvYMkotwUFI59G08IGKTMu3-GPnV1wp7NOQD1yzJbGGEGSEEysmEP0SO9vnN45kp3MiqbffBGc1r4_YM4e7DPmqOGM94qksOcLOJk1HNESw2dQYWxWQTBXPfOJT6jW9_crGLMEOsZ3Jcss0XS9KzBUA2p_9osvvhUKuKXbNztqH0oZIWlg37FEVsDs_hUwUJpv2Ar09k4",
+    "key": "-----BEGIN PUBLIC KEY-----\nMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA7QCwvuEIwCHjhjbpz3Oc\ntyei/Pz9nDksNbsc44Cm8jxYGMXsTPFXDZYCcCB5rcAhPPdZSlzaPkv4vPVcMIrw\n5cdX0tvbwa3rNTng6uFE7qkt15D3YCTkwF0Y9FVZiZ2Ko+G23QeBt9wqb9dlDN1d\nuPmu9BLYXIT/JXoBwf0vjIPFM9WBi5W/EHGaiuqw7lt0qI7zDGw77yO5yehKE4cu\n7dt3SakrXphL70LGiZh2XGoLg9Gmpz98t+gvPAUEotAJxIUqnoiTA8jlxoiQjeRK\nHlJkwMOGmRNPS33awPos0kcSxAywuBbh2X3aSqUMjcbE4cGJ++/13zoa6RUZRObC\nZnaLYJxqYBh13/N8SfH7d005hecDxWnoYXeYuuMeT3a2hV0J84ztkJX5OoxIwk7S\nWmvBq4+m66usn6LNL+p5IAcs93KbvOxxrjtQrzohBXc6+elfLVSQ1Rr9g5xbgpub\npSc+hvzbB6p0tleDRzwAy9X16NI4DYiTj4nkmVjigNo9v2VPnAle5zSam86eiYLO\nt2u9YRqysMLPKevNdj3CIvst+BaGGQONlQalRdIcq8Lin+BhuX+1TBgqyav4XD9K\nd+JHMb1aBk/rFLI9/f2S3BJ1XqpbjXz7AbYlaCwKiJ836+HS8PmLKxwVOnpLMbfH\nPYM8k83Lip4bEKIyAuf02qkCAwEAAQ==\n-----END PUBLIC KEY-----\n",
+    "name": "Mike Macgirvin",
+    "name_updated": "2012-12-06 04:59:13",
+    "address": "mike@zothub.com",
+    "photo_mimetype": "image/jpeg",
+    "photo": "https://zothub.com/photo/profile/l/1",
+    "photo_updated": "2012-12-06 05:06:11",
+    "url": "https://zothub.com/channel/mike",
+    "connections_url": "https://zothub.com/poco/mike",
+    "target": "",
+    "target_sig": "",
+    "searchable": false,
+    "permissions": {
+        "view_stream": true,
+        "view_profile": true,
+        "view_photos": true,
+        "view_contacts": true,
+        "view_storage": true,
+        "view_pages": true,
+        "send_stream": false,
+        "post_wall": false,
+        "post_comments": false,
+        "post_mail": false,
+        "post_photos": false,
+        "tag_deliver": false,
+        "chat": false,
+        "write_storage": false,
+        "write_pages": false,
+        "delegate": false
+    },
+    "profile": {
+        "description": "Freedom Fighter",
+        "birthday": "0000-05-14",
+        "next_birthday": "2013-05-14 00:00:00",
+        "gender": "Male",
+        "marital": "It's complicated",
+        "sexual": "Females",
+        "locale": "",
+        "region": "",
+        "postcode": "",
+        "country": "Australia"
+    },
+    "locations": [
+        {
+            "host": "zothub.com",
+            "address": "mike@zothub.com",
+            "primary": true,
+            "url": "https://zothub.com",
+            "url_sig": "eqkB_9Z8nduBYyyhaSQPPDN1AhSm5I4R0yfcFxPeFpuu17SYk7jKD7QzvmsyahM5Kq7vDW6VE8nx8kdFYpcNaurqw0_IKI2SWg15pGrhkZfrCnM-g6A6qbCv_gKCYqXvwpSMO8SMIO2mjQItbBrramSbWClUd2yO0ZAceq3Z_zhirCK1gNm6mGRJaDOCuuTQNb6D66TF80G8kGLklv0o8gBfxQTE12Gd0ThpUb5g6_1L3eDHcsArW_RWM2XnNPi_atGNyl9bS_eLI2TYq0fuxkEdcjjYx9Ka0-Ws-lXMGpTnynQNCaSFqy-Fe1aYF7X1JJVJIO01LX6cCs-kfSoz29ywnntj1I8ueYldLB6bUtu4t7eeo__4t2CUWd2PCZkY3PKcoOrrnm3TJP5_yVFV_VpjkcBCRj3skjoCwISPcGYrXDufJxfp6bayGKwgaCO6QoLPtqqjPGLFm-fbn8sVv3fYUDGilaR3sFNxdo9mQ3utxM291XE2Pd0jGgeUtpxZSRzBuhYeOybu9DPusID320QbgNcbEbEImO8DuGIxuVRartzEXQF4WSYRdraZzbOqCzmU0O55P836JAfrWjgxTQkXlYCic-DBk-iE75JeT72smCtZ4AOtoFWCjZAABCw42J7JELY9APixZXWriKtjy6JI0G9d3fs6r7SrXr1JMy0",
+            "callback": "https://zothub.com/post",
+            "sitekey": "-----BEGIN PUBLIC KEY-----\nMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA1IWXwd/BZuevq8jzNFoR\n3VkenduQH2RpR3Wy9n4+ZDpbrUKGJddUGm/zUeWEdKMVkgyllVA/xHdB7jdyKs1X\nuIet9mIdnzvhdLO/JFD5hgbNG2wpSBIUY6aSNeCFTzszqXmuSXMW5U0Ef5pCbzEA\nnhoCoGL1KAgPqyxnGKUlj7q2aDwC9IRNtAqNyFQL67oT91vOQxuMThjlDhbR/29Q\ncYR4i1RzyahgEPCnHCPkT2GbRrkAPjNZAdlnk9UesgP16o8QB3tE2j50TVrbVc/d\nYRbzC56QMPP9UgUsapNeSJBHji75Ip/E5Eg/kfJC/HEQgyCqjCGfb7XeUaeQ7lLO\nqc7CGuMP+Jqr/cE54/aSHg8boTwxkMp11Ykb+ng17fl57MHTM2RJ99qZ1KBkXezR\nuH1lyvjzeJPxEFr9rkUqc4GH74/AgfbgaFvQc8TS7ovEa5I/7Pg04m7vLSEYc6UF\nYJYxXKrzmZT2TDoKeJzaBBx5MFLhW19l68h9dQ8hJXIpTP0hJrpI+Sr6VUAEfFQC\ndIDRiFcgjz6j7T/x8anqh63/hpsyf2PMYph1+4/fxtSCWJdvf+9jCRM8F1IDfluX\n87gm+88KBNaklYpchmGIohbjivJyru41CsSLe0uinQFvA741W00w6JrcrOAX+hkS\nRQuK1dDVwGKoIY85KtTUiMcCAwEAAQ==\n-----END PUBLIC KEY-----\n"
+        }
+    ],
+    "site": {
+        "url": "https://zothub.com",
+        "directory_mode": "primary",
+        "directory_url": "https://zothub.com/dirsearch"
+    }
+
+	}
+[/code]
+
+
+Discovery returns a JSON array with the following components:
+
+'success' => ('1' or '')  Operation was successful if '1'. Otherwise an optional 'message' may be present indicating the source of error.
+
+'signed_token' => If a token parameter was provided in the request, it is prepended with the text 'token.' and then RSA signed with the channel private key and base64url encoded and returned as 'signed_token'.
+
+'guid' => the guid of the address on the target system
+
+'guid_sig' => the base64url encoded RSA signature of the guid, signed with the private key associated with that guid.
+
+'key' => the public key associated with that guid
+
+'name' => channel name
+
+'name_updated' => MySQL style timestamp '2012-01-01 00:00:00' when the name was last changed (UTC)
+
+'address' => "webbie" or user@host address associated with this channel
+
+'photo' => URL of a profile photo for this channel (ideally 175x175)
+
+'photo_mimetype' => content-type of the profile photo
+
+'photo_updated' => MySQL style timestamp when photo was last updated (UTC)  
+
+'url' => location of channel homepage
+
+'connections_url' => location of Portable Contacts (extended for zot) URL for this channel
+
+'target' => if a permissions target was specified, it is mirrored
+
+'target_sig' => if a permissions target was specified, the signature is mirrored.
+    
+'searchable' => ('1' or '') '1' indicates this entry can be searched in a directory
+
+[h5]Permissions[/h5]
+
+
+'permisssions' => extensible array of permissions appropriate to this target, values are '1' or ''
+
+  Permissions may include:
+[list]
+[*] view_stream
+[*] view_profile
+[*] view_photos
+[*] view_contacts
+[*] view_storage
+[*] view_pages
+[*] send_stream
+[*] post_wall
+[*] post_comments
+[*] post_mail
+[*] post_photos
+[*] tag_deliver
+[*] chat
+[*] write_storage
+[*] write_pages
+[*] delegate
+[/list]
+
+
+[h5]Profile[/h5]
+
+
+'profile' => array of important profile fields
+[list]
+[*] description
+[*] birthday YYYY-MM-DD , all fields are optional, any field (such as year) may be zero
+[*] next_birthday => MySQL datetime string representing the next upcoming birthday, converted from the channel's default timezone to UTC. 
+[*] gender (free form)
+[*] marital (marital status)
+[*] sexual (preference)
+[*] locale (city)
+[*] region (state)
+[*] postcode
+[*] country
+[/list]
+
+[h5]Locations[/h5]
+
+
+'locations' => array of registered locations (DNS locations) this channel may be visible or may be posting from
+
+Each location is an array of
+
+'host' => DNS hostname, e.g. example.com
+
+'address' => the webbie or user@host identifier associated with this location
+
+'primary' => ('1' or '') whether or not this is the primary location for this channel where files and web pages are generally found
+
+'url' => url of the root of this DNS location e.g. https://example.com
+
+'url_sig' => base64url encoded RSA signature of the URL, signed with the channel private key
+
+'callback' => Zot communications endpoint on this site, usually https://example.com/post
+
+'sitekey' => public key of this site/host
+
+
+[h5]Site[/h5]
+
+
+'site' => array providing the directory role of the site responding to this request
+
+'url' => url of this site e.g. https://example.com
+
+'directory_mode' => one of 'primary', 'secondary', 'normal', or 'standalone'
+
+'directory_url' => if this is a directory server or standalone, the URL for the directory search module
+
+
+
+[h3]Magic Auth[/h3]
+
+So-called "magic auth" takes place by a special exchange. On the remote computer, a redirection is made to the Zot endpoint with special GET parameters.
+
+Endpoint: https://example.com/post/name
+
+where 'name' is the left hand side of the channel webbie, for instance 'mike' where the webbie is 'mike@zothub.com'
+
+Additionally four parameters are supplied:
+[list]
+[*] auth => the webbie of the person requesting access
+[*] dest => the desired destination URL (urlencoded)
+[*] sec  => a random string which is also stored locally for use during the verification phase. 
+[*] version => the Zot revision
+[/list]
+When this packet is received, a Zot message is initiated to the auth identity:
+
+[code nowrap]
+    {
+      "type":"auth_check",
+      "sender":{
+        "guid":"kgVFf_1_SSbyqH-BNWjWuhAvJ2EhQBTUdw-Q1LwwssAntr8KTBgBSzNVzUm9_RwuDpxI6X8me_QQhZMf7RfjdA",
+        "guid_sig":"PT9-TApzpm7QtMxC63MjtdK2nUyxNI0tUoWlOYTFGke3kNdtxSzSvDV4uzq_7SSBtlrNnVMAFx2_1FDgyKawmqVtRPmT7QSXrKOL2oPzL8Hu_nnVVTs_0YOLQJJ0GYACOOK-R5874WuXLEept5-KYg0uShifsvhHnxnPIlDM9lWuZ1hSJTrk3NN9Ds6AKpyNRqf3DUdz81-Xvs8I2kj6y5vfFtm-FPKAqu77XP05r74vGaWbqb1r8zpWC7zxXakVVOHHC4plG6rLINjQzvdSFKCQb5R_xtGsPPfvuE24bv4fvN4ZG2ILvb6X4Dly37WW_HXBqBnUs24mngoTxFaPgNmz1nDQNYQu91-ekX4-BNaovjDx4tP379qIG3-NygHTjFoOMDVUvs-pOPi1kfaoMjmYF2mdZAmVYS2nNLWxbeUymkHXF8lT_iVsJSzyaRFJS1Iqn7zbvwH1iUBjD_pB9EmtNmnUraKrCU9eHES27xTwD-yaaH_GHNc1XwXNbhWJaPFAm35U8ki1Le4WbUVRluFx0qwVqlEF3ieGO84PMidrp51FPm83B_oGt80xpvf6P8Ht5WvVpytjMU8UG7-js8hAzWQeYiK05YTXk-78xg0AO6NoNe_RSRk05zYpF6KlA2yQ_My79rZBv9GFt4kUfIxNjd9OiV1wXdidO7Iaq_Q",
+        "url":"http:\/\/podunk.edu",
+        "url_sig":"T8Bp7j5DHHhQDCFcAHXfuhUfGk2P3inPbImwaXXF1xJd3TGgluoXyyKDx6WDm07x0hqbupoAoZB1qBP3_WfvWiJVAK4N1FD77EOYttUEHZ7L43xy5PCpojJQmkppGbPJc2jnTIc_F1vvGvw5fv8gBWZvPqTdb6LWF6FLrzwesZpi7j2rsioZ3wyUkqb5TDZaNNeWQrIEYXrEnWkRI_qTSOzx0dRTsGO6SpU1fPWuOOYMZG8Nh18nay0kLpxReuHCiCdxjXRVvk5k9rkcMbDBJcBovhiSioPKv_yJxcZVBATw3z3TTE95kGi4wxCEenxwhSpvouwa5b0hT7NS4Ay70QaxoKiLb3ZjhZaUUn4igCyZM0h6fllR5I6J_sAQxiMYD0v5ouIlb0u8YVMni93j3zlqMWdDUZ4WgTI7NNbo8ug9NQDHd92TPmSE1TytPTgya3tsFMzwyq0LZ0b-g-zSXWIES__jKQ7vAtIs9EwlPxqJXEDDniZ2AJ6biXRYgE2Kd6W_nmI7w31igwQTms3ecXe5ENI3ckEPUAq__llNnND7mxp5ZrdXzd5HHU9slXwDShYcW3yDeQLEwAVomTGSFpBrCX8W77n9hF3JClkWaeS4QcZ3xUtsSS81yLrp__ifFfQqx9_Be89WVyIOoF4oydr08EkZ8zwlAsbZLG7eLXY"
+      },
+      "recipients":{
+        {
+        "guid":"ZHSqb3yGar3TYV_o9S-JkD-6o_V4DhUcxtv0VeyX8Kj_ENHPI_M3SyAUucU835-mIayGMmTpqJz3ujPkcavVhA",
+        "guid_sig":"JsAAXigNghTkkbq8beGMJjj9LBKZn28hZ-pHSsoQuwYWvBJ2lSnfc4r9l--WgO6sucH-SR6RiBo78eWn1cZrh_cRMu3x3LLx4y-tjixg-oOOgtZakkBg4vmOhkKPkci0mFtzvUrpY4YHySqsWTuPwRx_vOlIYIGEY5bRXpnkNCoC8J4EJnRucDrgSARJvA8QQeQQL0H4mWEdGL7wlsZp_2VTC6nEMQ48Piu6Czu5ThvLggGPDbr7PEMUD2cZ0jE4SeaC040REYASq8IdXIEDMm6btSlGPuskNh3cD0AGzH2dMciFtWSjmMVuxBU59U1I6gHwcxYEV6BubWt_jQSfmA3EBiPhKLyu02cBMMiOvYIdJ3xmpGoMY1Cn__vhHnx_vEofFOIErb6nRzbD-pY49C28AOdBA5ffzLW3ss99d0A-6GxZmjsyYhgJu4tFUAa7JUl84tMbq28Tib0HW6qYo6QWw8K1HffxcTpHtwSL5Ifx0PAoGMJsGDZDD1y_r9a4vH5pjqmGrjL3RXJJUy-m4eLV5r7xMWXsxjqu3D8r04_dcw4hwwexpMT1Nwf8CTB0TKb8ElgeOpDFjYVgrqMYWP0XdhatcFtAJI7gsS-JtzsIwON9Kij66-VAkqy_z1IXI0ziyqV1yapSVYoUV1vMScRZ_nMqwiB5rEDx-XLfzko"
+        }
+      }
+      "callback":"\/post",
+      "version":1,
+      "secret":"1eaa6613699be6ebb2adcefa5379c61a3678aa0df89025470fac871431b70467",
+      "secret_sig":"eKV968b1sDkOVdSMi0tLRtOhQw4otA8yFKaVg6cA4I46_zlAQGbFptS-ODiZlSAqR7RiiZQv4E2uXCKar52dHo0vvNKbpOp_ezWYcwKRu1shvAlYytsflH5acnDWL-FKOOgz5zqLLZ6cKXFMoR1VJGG_Od-DKjSwajyV9uVzTry58Hz_w0W2pjxwQ-Xv11rab5R2O4kKSW77YzPG2R5E6Q7HN38FrLtyWD_ai3K9wlsFOpwdYC064dk66X7BZtcIbKtM6zKwMywcfJzvS5_0U5yc5GGbIY_lY6SViSfx9shOKyxkEKHfS29Ynk9ATYGnwO-jnlMqkJC7t149H-sI9hYWMkLuCzaeLP56k2B2TmtnYvE_vHNQjzVhTwuHCIRVr-p6nplQn_P3SkOpYqPi3k_tnnOLa2d3Wtga8ClEY90oLWFJC3j2UkBf_VEOBNcg-t5XO3T-j9O4Sbk96k1Qoalc-QlznbGx4bOVsGkRBBMiH4YUqiiWB_OkFHtdqv7dqGeC-u-B4u9IxzYst46vvmyA3O-Q4APSZ1RY8ITUH0jLTbh6EAV7Oki8pIbOg0t56p-8RlanOZqmFvR-grVSc7Ak1ZcD8NACmvidUpa1B7WEvRcOeffx9lype0bt5XenDnMyx6szevwxZIiM8qGM2lsSk4fu8HI9cW0mLywzZT0"
+    }
+[/code]
+
+auth_check messages MUST be encrypted with AES256CBC. This message is sent to the origination site, which checks the 'secret' to see if it is the same as the 'sec' which it passed originally. It also checks the secret_sig which is the secret signed by the destination channel's private key and base64url encoded. If everything checks out, a json packet is returned:
+[code nowrap]
+    { 
+      "success":1, 
+      "confirm":"q0Ysovd1uQRsur2xG9Tg6bC23ynzw0191SkVd7CJcYoaePy6e_v0vnmPg2xBUtIaHpx_aSuhgAkd3aVjPeaVBmts6aakT6a_yAEy7l2rBydntu2tvrHhoVqRNOmw0Q1tI6hwobk1BgK9Pm0lwOeAo8Q98BqIJxf47yO9pATa0wktOg6a7LMogC2zkkhwOV5oEqjJfeHeo27TiHr1e2WaphfCusjmk27V_FAYTzw05HvW4SPCx55EeeTJYIwDfQwjLfP4aKV-I8HQCINt-2yxJvzH7Izy9AW-7rYU0Il_gW5hrhIS5MTM12GBXLVs2Ij1CCLXIs4cO0x6e8KEIKwIjf7iAu60JPmnb_fx4QgBlF2HLw9vXMwZokor8yktESoGl1nvf5VV5GHWSIKAur3KPS2Tb0ekNh-tIk9u-xob4d9eIf6tge_d3aq1LcAtrDBDLk8AD0bho5zrVuTmZ9k-lBVPr_DRHSV_dlpu088j3ThaBsuV1olHK3vLFRhYCDIO0CqqK5IuhqtRNnRaqhlNN6fQUHpXk2SwHiJ2W36RCYMTnno6ezFk_tN-RA2ly-FomNZoC5FPA9gFwoJR7ZmVFDmUeK3bW-zYTA5vu15lpBPnt7Up_5rZKkr0WQVbhWJmylqOuwuNWbn3SrMQ8rYFZ23Tv300cOfKVgRBaePWQb4" 
+    }
+[code]
+'confirm' in this case is the base64url encoded RSA signature of the concatenation of 'secret' with the base64url encoded whirlpool hash of the source guid and guid_sig; signed with the source channel private key. This prevents a man-in-the-middle from inserting a rogue success packet. Upon receipt and successful verification of this packet, the destination site will redirect to the original destination URL and indicate a successful remote login. 
+
 [h3]Zot Structures[/h3]
 [h4]Zot Signatures[/h4]
 All signed data in Zot is accomplished by performing an RSA sign operation using the private key of the initiator. The binary result is then base64url encoded for transport.
diff --git a/doc/nomadic-identity.bb b/doc/nomadic-identity.bb
deleted file mode 100644
index a062e58c8..000000000
--- a/doc/nomadic-identity.bb
+++ /dev/null
@@ -1 +0,0 @@
-This still needs to be written.
diff --git a/doc/permissions.bb b/doc/permissions.bb
deleted file mode 100644
index 64690d608..000000000
--- a/doc/permissions.bb
+++ /dev/null
@@ -1,84 +0,0 @@
-[h1]Permissions[/h1]

-Permissions in $Projectname are more complete than you may be used to. This allows us to define more fine graded relationships than the black and white "this person is my friend, so they can do everything" or "this person is not my friend, so they can't do anything" permissions you may find elsewhere.

-

-[b]Permission Roles[/b]

-

-When you create a channel we allow you to select different 'roles' for that channel. These create an entire family of permissions and privacy settings that are appropriate for that role. Typical roles are "Social - mostly public", "Social - mostly private", "Forum - public" and many others. These bring a level of simplicity to managing permissions. Just choose a role and appropriate permissions are automatically applied. You can also choose 'Custom/Expert mode' and change any individual permission setting in any way you desire. 

-

-

-[b]Default Permission Limits[/b]

-

-There are a large number of individual permissions. These control everything from the ability to view your stream to the ability to chat with you. Every permission has a limit. The scope of these permissions varies from "Only me" to "Everybody on the internet" - though some scopes may not be available for some permissions. The limit applies to any published thing you create which has no privacy or access control. For example if you publish a photo and didn't select a specific audience with permission to view it, we apply the limit. These limits apply to everything within that permission rule, so you cannot apply a limit to one photo. The limit applies to all your photos. If all your photos are visible to everybody on the internet and you reduce the limit only to friends, [b]all[/b] of your photos will now be visible only to friends.

-

-[b]Access Control[/b]

- 

-Access Control is the preferred method of managing privacy in [i]most[/i] cases, rather than using permission limits. This creates lists of either connections or privacy groups (or both) and uses the access list to decide if a permission is allowed. An access list is attached to everything you publish. Unlike permission limits, if you change the access control list on a single photo, it doesn't affect any of your other photos. You can use privacy groups and a "default access control list" to create and automate the management of access control lists to provide any level of privacy you desire on anything you publish. 

-

-We highly recommend that you use the "typical social network" settings when you create your first channel, as it allows others to communicate with you and help you out if you have difficulty. You will find that these settings allow you as much privacy as you desire - when you desire it; but also allow you to communicate in public if you choose to. You are free to use much more private settings once you have learned your way around.

-

-

-[dl terms="l"]

-[*= The scopes of permissions are:]

-[dl terms="i"]

-  [*= Nobody Except Yourself ] This is self explanatory. Only you will be allowed access.

-  

-  [*= Only those you specifically allow ] By default, people you are not connected to, and all new contacts will   have this permission denied. You will be able to make exceptions for individual channels on their contact edit   screen.

-  

-  [*= Anybody in your address book ] Anybody you do not know will have this permission denied, but anybody you   accept as a contact will have this permission approved. This is the way most legacy platforms handle   permissions.

-  

-  [*= Anybody On This Hub ] Anybody with a channel on the same hub/website as you will have permission approved. Anybody who is registered at a different hub will have this permission denied.

-  

-  [*= Anybody in this network ] Anybody in $Projectname will have this permission approved. Even complete   strangers. However, anybody not logged in/authenticated will have this permission denied.

-  

-  [*= Anybody authenticated ] This is similar to "anybody in this network" except that it can include anybody   who can authenticate by any means - and therefore [i]may[/i] include visitors from other networks.

-  

-  [*=Guest Access Token] This allows you to share a file, folder, photo, album, or channel with a specific person or group of people. They don't need to be Hubzilla members. You can set an expiration for the Access Token.

-

-  [*= Anybody on the internet ] Completely public. This permission will be approved for anybody at all.

-[/dl]

-[*= The individual permissions are:]

-[dl terms="i"]

-  [*= Can view my "public" stream and posts. ] This permision determines who can view your channel "stream" that is, the non-private posts that appear on the "home" tab when you're logged in.

-

-  [*= Can view my "public" channel profile. ] This permission determines who can view your channel's profile. This refers to the "about" tab

-

-  [*= Can view my "public" photo albums. ] This permission determines who can view your photo albums. Individual photographs may still be posted to a more private audience.

-

-  [*= Can view my "public" address book. ] This permission determines who can view your contacts. These are the connections displayed in the "View connections" section.

-

-  [*= Can view my "public" file storage. ] This permission determines who can view your public files stored in your cloud.

-

-  [*= Can view my "public" pages. ] This permission determines who can view your public web pages. 

-

-  [*= Can send me their channel stream and posts. ] This permission determines whose posts you will view. If your channel is a personal channel (ie, you as a person), you would probably want to set this to "anyone in my address book" at a minimum. A personal notes channel would probably want to choose "nobody except myself". Setting this to "Anybody in the network" will show you posts from complete strangers, which is a good form of discovery.

-

-  [*= Can post on my channel page ("wall"). ] This permission determines who can write to your wall when clicking through to your channel.

-

-  [*= Can comment on my posts. ] This permission determines who can comment on posts you create. Normally, you would want this to match your "can view my public stream and posts" permission

-

-  [*= Can send me private mail messages. ] This determines who can send you private messages (zotmail).

-

-  [*= Can post photos to my photo albums. ] This determines who can post photographs in your albums. This is very useful for forum-like channels where connections may not be connected to each other.

-

-  [*= Can forward to all my channel contacts via post tags. ] Using @- mentions will reproduce a copy of your post on the profile specified, as though you posted on the channel wall. This determines if people can post to your channel in this way.

-

-  [*= Can chat with me (when available). ] This determines who can join the public chat rooms created by your channel.

-

-  [*= Can write to my "public" file storage. ] This determines who can upload files to your public file storage, or 'cloud'.

-

-  [*= Can edit my "public" pages. ] This determines who can edit your webpages. This is useful for wikis or sites with multiple editors.

-

-  [*= Can administer my channel resources. ] This determines who can have full control of your channel. This should normally be set to "nobody except myself".

-[/dl][/dl]

-[i]Note:[/i]

-Plugins/addons may provide special permission settings, so you may be offered additional permission settings beyond what is described here.

-

-If you have set any of these permissions to "only those I specifically allow", you may specify indivudal permissions on the connnection edit screen.

-

-[b]Affinity[/b]

-

-The connection edit screen offers a slider to select a degree of friendship with the connnection (this tool is enabled through the "Extra Features" tab of your Settings page). Think of this as a measure of how much you like or dislike them. 1 is for people you like, whose posts you want to see all the time. 99 is for people you don't care for, and whose posts you might only wish to look at occasionally. Once you've assigned a value here, you can use the affinity tool on the matrix page to filter content based on this number.

-

-The slider on the matrix page has both a minimum and maximum value. Posts will only be shown from people who fall between this range. Affinity has no relation to permissions, and is only useful in conjunction with the affinity tool feature.

-

-#include doc/macros/main_footer.bb;

diff --git a/doc/registration.bb b/doc/registration.bb
deleted file mode 100644
index f656eeaa6..000000000
--- a/doc/registration.bb
+++ /dev/null
@@ -1,35 +0,0 @@
-[size=large][b]Registration[/b][/size]

-

-Not all $Projectname sites allow open registration. If registration is allowed, you will see a "Register" link immediately below the login prompts on the site home page. Following this link will take you to the site Registration page. On some sites it may redirect you to another site which allow registrations. As all $Projectname sites are linked, it does not matter where your account resides.

-

-[b]Your Email Address[/b]

-

-Please provide a valid email address. Your email address is never published. This address will be used to activate your account, to (optionally) send email notifications for incoming messages or items, [i]and to recover lost passwords[/i].

-

-[b]Password[/b]

-

-Enter a password of your choice, and repeat it in the second box to ensure it was typed correctly. As $Projectname offers a decentralised identity, your account can log you in to many other websites.

-

-[b]Terms Of Service[/b]

-

-Click the link to read the site's [zrl=[baseurl]/help/TermsOfService]Terms of Service[/zrl]. Once you've read them, tick the box in the register form to confirm.

-

-[b]Register[/b]

-

-Once you have provided the necessary details, click the 'Register' button. Some sites may require administrator approval before the registration is processed, and you will be alerted if this is the case. Please watch your email (including spam folders) for your registration approval.

-

-[b]Create a Channel[/b]

-

-Next, you will be presented with the "Add a channel" screen. Normally, your first channel will be one that represents you - so using your own name (or psuedonym) as the channel name is a good idea. The channel name should be thought of as a title, or brief description of your channel. The "choose a short nickname" box is similar to a "username" field. We will use whatever you enter here to create a channel address, which other people will use to connect to you, and you will use to log in to other sites. This looks like an email address, and takes the form nickname@siteyouregisteredat.xyz

-

-When your channel is created you will be taken straight to your settings page where you can define permissions, enable features, etc. All these things are covered in the appropriate section of the helpfiles.

-

-See Also

-[zrl=[baseurl]/help/accounts_profiles_channels_basics]The Basics about Identities within $Projectname[/zrl]

-[zrl=[baseurl]/help/accounts]Accounts[/zrl]

-[zrl=[baseurl]/help/profiles]Profiles[/zrl]

-[zrl=[baseurl]/help/permissions]Permissions[/zrl]

-[zrl=[baseurl]/help/remove_account]Remove Account[/zrl]

-

-#include doc/macros/main_footer.bb;

-

diff --git a/doc/remove_account.bb b/doc/remove_account.bb
deleted file mode 100644
index 704f0b94c..000000000
--- a/doc/remove_account.bb
+++ /dev/null
@@ -1,27 +0,0 @@
-[b]Remove Channel or Account[/b]

-

-[b]Remove Channel[/b]

-

-Go to the bottom of your channel settings page or visit the URL:

-

-	[baseurl]/removeme 

-

-You will need to confirm your password and the channel you are currently logged into will be removed. 

-

-This is irreversible.

-

-If you have identity clones on other hubs this only removes  by default the channel instance which exists on this hub.

-

-[b]Remove Account[/b]

-

-Go to the bottom of your account settings page or visit the URL:

-

-	[baseurl]/removeaccount

-	

-You will need to confirm your password and the account you are currently logged into will be removed. 

-

-This is irreversible.

-

-All your channels will be deleted. If you have identity clones on other hubs this only removes by default the channels instances which exists on this hub.

-

-#include doc/macros/main_footer.bb;

diff --git a/doc/roles.md b/doc/roles.md
deleted file mode 100644
index a7608ff90..000000000
--- a/doc/roles.md
+++ /dev/null
@@ -1,66 +0,0 @@
-Account Permission Roles
-========================
-
-
-##Social
-
-**Mostly Public**
-
-The channel is a typical social networking profile. By default posts and published items are public, but one can over-ride this when creating the item and restrict it. You are listed in the directory. Your online presence and connections are visible to others. 
-
-
-**Restricted**
-
-By default all posts and published items are sent to your 'Friends' privacy group and not made public. New friends are added to this privacy group. You can over-ride this and create a public post or published item if you desire. You are listed in the directory. Your online presence (for chat) and your connections (friends) are visible to your profile viewers.
-
-**Private**
-
-By default all posts and published items are sent to your 'Friends' privacy group. New friends are added to this privacy group. You can over-ride this and create a public post or public item if you desire. You are NOT listed in the directory. Only your connections can see your other connections. Your online presence is hidden. 
-
-
-##Forum
-
-**Mostly Public**
-
-The channel is a typical forum. By default posts and published items are public. Members may post by @mention+ or wall-to-wall post. Posting photos and other published items is blocked. The channel is visible in the directory. Members are added automatically.
-
-
-**Restricted**
-
-By default all posts and published items are sent to the channel's 'Friends' privacy group. New friends are added to this privacy group. Members may post by @mention+ or wall-to-wall post, but posts and replies may also be seen by other receipients of the top-level post who are not members. The channel is visible in the directory. Members must be manually added by the forum owner. 
-
-**Private**
-
-By default all posts and published items are sent to your 'Friends' privacy group. New friends are added to this privacy group. The owner can over-ride this and create a public post or public item if desired. Members cannot. You are NOT listed in the directory. Only your connections can see your other connections. Your online presence is hidden. Members must be manually added by the forum owner. Posting by @mention+ is disabled. Posts can only be made via wall-to-wall posts, and sent to members of the 'Friends' privacy group. They are not publicly visible.  
-
-
-##Feed
-
-
-**Public**
-
-Similiar to Social - Mostly Public, but tailored for RSS feed sources. Items may be freely republished and sourced. Online presence is meaningless, therefore hidden. New connections are automatically approved.  
-
-
-**Restricted**
-
-Not listed in directory. Online presence is meaningless, therefore hidden. Feed is published only to members of the 'Friends' privacy group. New connections are automatically added to this privacy group. Members must be manually approved by the channel owner. 
-
-
-##Special
-
-**Celebrity/Soapbox**
-
-Listed in directory. Communications are by default public. Online presence is hidden. No commenting or feedback of any form is allowed, though connections have the ability to "like" your profile. 
-
-
-**Group Repository**
-
-A public forum which allows members to post files/photos/webpages. 
-
-
-##Custom/Expert Mode
-
-Set all the privacy and permissions manually to suit your specific needs. 
-
-#include doc/macros/main_footer.bb;
diff --git a/doc/webpage-element-import.md b/doc/webpage-element-import.md
deleted file mode 100644
index 4330227c2..000000000
--- a/doc/webpage-element-import.md
+++ /dev/null
@@ -1,94 +0,0 @@
-## <a href="#webpage-element-import"></a>Webpage element import
-
-There are two methods of importing webpage elements: uploading a zip file or 
-referencing a local cloud files folder. Both methods require that the webpage 
-elements are specified using a specific folder structure. The import tool makes 
-it possible to import all the elements necessary to construct an entire website
-or set of websites. The goal is to accommodate external development of webpages 
-as well as tools to simplify and automate deployment on a hub.
-
-### Folder structure
-Element definitions must be stored in the repo root under folders called 
-
-	/pages/
-	/blocks/
-	/layouts/
-
-
-Each element of these types must be defined in an individual subfolder using two files: one JSON-formatted file for the metadata and one plain text file for the element content.
-
-### Page elements
-Page element metadata is specified in a JSON-formatted file called `page.json` with the following properties:
-
- * title
- * pagelink
- * mimetype
- * layout
- * contentfile
-
-**Example** 
-
-Files: 
-	
-	/pages/my-page/page.json
-	/pages/my-page/my-page.bbcode
-	
-Content of `page.json`:
-	
-	{
-		"title": "My Page",
-		"pagelink": "mypage",
-		"mimetype": "text/bbcode",
-		"layout": "my-layout",
-		"contentfile": "my-page.bbcode"
-	}
-	
-
-### Layout elements
-Layout element metadata is specified in a JSON-formatted file called `layout.json` with the following properties:
-
- * name
- * description
- * contentfile
-
-**Example** 
-
-Files:
-
-	/layouts/my-layout/layout.json
-	/layouts/my-layout/my-layout.bbcode
-	
-Content of `layout.json`:
-	
-	{
-		"name": "my-layout",
-		"description": "Layout for my project page",
-		"contentfile": "my-layout.bbcode"
-	}
-
-
-### Block elements
-Block element metadata is specified in a JSON-formatted file called `block.json` with the following properties:
-
- * name
- * title
- * mimetype
- * contentfile
-
-**Example** 
-
-Files:
-	
-	/blocks/my-block/block.json
-	/blocks/my-block/my-block.html
-
-Content of `block.json`:	
-
-	
-	{
-		"name": "my-block",
-		"title": "",
-		"mimetype": "text/html",
-		"contentfile": "my-block.html"
-	}
-	
\ No newline at end of file
diff --git a/doc/webpages.bb b/doc/webpages.bb
deleted file mode 100644
index 2b909dc63..000000000
--- a/doc/webpages.bb
+++ /dev/null
@@ -1,93 +0,0 @@
-[b]Creating Web Pages[/b]

-

-Hubzilla enables users to create static webpages.  To activate this feature, enable the web pages feature in your Additional Features section.

-

-Once enabled, a new tab will appear on your channel page labelled &quot;Webpages&quot;.  Clicking this link will take you to the webpage editor.

-Pages will be accessible at mydomain/page/username/pagelinktitle

-

-The &quot;page link title&quot; box allows a user to specify the &quot;pagelinktitle&quot; of this URL.  If no page link title is set, we will set one for you automatically, using the message ID of the item.  

-

-Beneath the page creation box, a list of existing pages will appear with an &quot;edit&quot; link.  Clicking this will take you to an editor, similar to that of the post editor, where you can make changes to your webpages.

-

-See also: 

-

-[zrl=[baseurl]/help/webpage-element-import]Webpage element import tool[/zrl]

-

-[b]Using Blocks[/b]

-

-Blocks can be parts of webpages. The basic HTML of a block looks like this

-[code]

-	<div>

-		Block Content

-	</div>

-

-[/code]

-

-If a block has text/html content type it can also contain menu elements. Sample content of

-[code]

-	<p>HTML block content</p> 

-	[menu]menuname[/menu]

-

-[/code]

-will produce HTML like this

-[code]

-	<div>

-		<p>HTML block content</p>

-		<div>

-			<ul>

-				<li><a href="#">Link 1</a></li>

-				<li><a href="#">Link 2</a></li>

-				<li><a href="#">Link 3</a></li>

-			</ul>

-		</div>

-	</div>

-

-[/code]

-

-Via the $content macro a block can also contain the actual webpage content. For this create a block with only

-[code]

-	$content

-

-[/code]as content.

-

-To make a block appear in the webpage it must be defined in the page layout inside a region.

-[code]

-	[region=aside]

-		[block]blockname[/block]

-	[/region]

-

-[/code]

-

-The block appearance can be manipulated in the page layout.

-

-Custom classes can be assigned

-[code]

-	[region=aside]

-		[block=myclass]blockname[/block]

-	[/region]

-

-[/code]

-will produce this HTML

-[code]

-	<div class="myclass">

-		Block Content

-	</div>

-

-[/code]

-

-Via the wrap variable a block can be stripped off its wrapping <div></div> tag

-[code]

-	[region=aside]

-		[block][var=wrap]none[/var]blockname[/block]

-	[/region]

-

-[/code]

-will produce this HTML

-[code]

-	Block Content

-

-[/code]

-

-

-#include doc/macros/main_footer.bb;

-

diff --git a/doc/webpages.md b/doc/webpages.md
deleted file mode 100644
index 05c16f2bb..000000000
--- a/doc/webpages.md
+++ /dev/null
@@ -1,17 +0,0 @@
-Creating Webpages
-=================
-
-Hubzilla enables users to create static webpages.  To activate this feature, enable the web pages feature in your Additional Features section.
-
-Once enabled, a new tab will appear on your channel page labelled "Webpages".  Clicking this link will take you to the webpage editor.  Here you can create a post using either BBCode or the rich text editor.
-
-Pages will be accessible at mydomain/page/username/pagelinktitle
-
-The "page link title" box allows a user to specify the "pagelinktitle" of this URL.  If no page link title is set, we will set one for you automatically, using the message ID of the item.  
-
-Beneath the page creation box, a list of existing pages will appear with an "edit" link.  Clicking this will take you to an editor, similar to that of the post editor, where you can make changes to your webpages.
-
-If you are the admin of a site, you can specify a channel whose webpages we will use at key points around the site.  Presently, the only place this is implemented is the home page.  If you specify the channel "admin" and then the channel called "admin" creates a webpage called "home", we will display it's content on your websites home page.  We expect this functionality to be extended to other areas in future.
-
-#include doc/webpage-element-import.md;
-#include doc/macros/main_footer.bb;
diff --git a/doc/zot_structures.md b/doc/zot_structures.md
deleted file mode 100644
index e8a65ff1f..000000000
--- a/doc/zot_structures.md
+++ /dev/null
@@ -1,62 +0,0 @@
-Zot Structures
-==============
-
-**Zot signatures**
-All signed data in zot is accomplished by performing an RSA sign operation using the private key of the initiator. The binary result is then base64url encoded for transport.
-
-**Zot encryption**
-
-Encryption is currently provided by AES256CBC, though additional algorithms MAY be supported. A 32-octet key and 16-octet initialisation vector are randomly generated. The desired data is then encrypted using these generated strings and the result base64url encoded. Then we build an array:
-
-    data: the base64url encoded encrypted data
-    alg: The chosen algorithm, in this case the string 'aes256cbc'.
-    key: The randomly generated key, RSA encrypted using the recipients public key, and the result base64url encoded
-    iv: The randomly generated IV, RDSA encrypted using the recipient's public key, and the result base64url encoded
-
-
-
-**The zot basic packet**
-
-Used for initiating a dialogue with another zot site. This packet MAY be encrypted. The presence of an array element 'iv' indicates encryption has been applied. When sending an 'auth_check' packet type, this packet MUST be encrypted, using the public key of the destination site (the site key, as opposed to a sender key).  
-
-
-    {
-      "type":"notify",
-      "sender":{
-        "guid":"kgVFf_1_SSbyqH-BNWjWuhAvJ2EhQBTUdw-Q1LwwssAntr8KTBgBSzNVzUm9_RwuDpxI6X8me_QQhZMf7RfjdA",
-        "guid_sig":"PT9-TApzpm7QtMxC63MjtdK2nUyxNI0tUoWlOYTFGke3kNdtxSzSvDV4uzq_7SSBtlrNnVMAFx2_1FDgyKawmqVtRPmT7QSXrKOL2oPzL8Hu_nnVVTs_0YOLQJJ0GYACOOK-R5874WuXLEept5-KYg0uShifsvhHnxnPIlDM9lWuZ1hSJTrk3NN9Ds6AKpyNRqf3DUdz81-Xvs8I2kj6y5vfFtm-FPKAqu77XP05r74vGaWbqb1r8zpWC7zxXakVVOHHC4plG6rLINjQzvdSFKCQb5R_xtGsPPfvuE24bv4fvN4ZG2ILvb6X4Dly37WW_HXBqBnUs24mngoTxFaPgNmz1nDQNYQu91-ekX4-BNaovjDx4tP379qIG3-NygHTjFoOMDVUvs-pOPi1kfaoMjmYF2mdZAmVYS2nNLWxbeUymkHXF8lT_iVsJSzyaRFJS1Iqn7zbvwH1iUBjD_pB9EmtNmnUraKrCU9eHES27xTwD-yaaH_GHNc1XwXNbhWJaPFAm35U8ki1Le4WbUVRluFx0qwVqlEF3ieGO84PMidrp51FPm83B_oGt80xpvf6P8Ht5WvVpytjMU8UG7-js8hAzWQeYiK05YTXk-78xg0AO6NoNe_RSRk05zYpF6KlA2yQ_My79rZBv9GFt4kUfIxNjd9OiV1wXdidO7Iaq_Q",
-        "url":"http:\/\/podunk.edu",
-        "url_sig":"T8Bp7j5DHHhQDCFcAHXfuhUfGk2P3inPbImwaXXF1xJd3TGgluoXyyKDx6WDm07x0hqbupoAoZB1qBP3_WfvWiJVAK4N1FD77EOYttUEHZ7L43xy5PCpojJQmkppGbPJc2jnTIc_F1vvGvw5fv8gBWZvPqTdb6LWF6FLrzwesZpi7j2rsioZ3wyUkqb5TDZaNNeWQrIEYXrEnWkRI_qTSOzx0dRTsGO6SpU1fPWuOOYMZG8Nh18nay0kLpxReuHCiCdxjXRVvk5k9rkcMbDBJcBovhiSioPKv_yJxcZVBATw3z3TTE95kGi4wxCEenxwhSpvouwa5b0hT7NS4Ay70QaxoKiLb3ZjhZaUUn4igCyZM0h6fllR5I6J_sAQxiMYD0v5ouIlb0u8YVMni93j3zlqMWdDUZ4WgTI7NNbo8ug9NQDHd92TPmSE1TytPTgya3tsFMzwyq0LZ0b-g-zSXWIES__jKQ7vAtIs9EwlPxqJXEDDniZ2AJ6biXRYgE2Kd6W_nmI7w31igwQTms3ecXe5ENI3ckEPUAq__llNnND7mxp5ZrdXzd5HHU9slXwDShYcW3yDeQLEwAVomTGSFpBrCX8W77n9hF3JClkWaeS4QcZ3xUtsSS81yLrp__ifFfQqx9_Be89WVyIOoF4oydr08EkZ8zwlAsbZLG7eLXY"
-      },
-      "callback":"\/post",
-      "version":1,
-      "secret":"1eaa6613699be6ebb2adcefa5379c61a3678aa0df89025470fac871431b70467",
-      "secret_sig":"0uShifsvhHnxnPIlDM9lWuZ1hSJTrk3NN9Ds6AKpyNRqf3DUdz81-Xvs8I2kj6y5vfFtm-FPKAqu77XP05r74vGaWbqb1r8zpWC7zxXakVVOHHC4plG6rLINjQzvdSFKCQb5R_xtGsPPfvuE24bv4fvN4ZG2ILvb6X4Dly37WW_HXBqBnUs24mngoTxFaPgNmz1nDQNYQu91-ekX4-BNaovjDx4tP379qIG3-NygHTjFoOMDVUvs-pOPi1kfaoMjmYF2mdZAmVYS2nNLWxbeUymkHXF8lT_iVsJSzyaRFJS1Iqn7zbvwH1iUBjD_pB9EmtNmnUraKrCU9eHES27xTwD-yaaH_GHNc1XwXNbhWJaPFAm35U8ki1Le4WbUVRluFx0qwVqlEF3ieGO84PMidrp51FPm83B_oGt80xpvf6P8Ht5WvVpytjMU8UG7-js8hAzWQeYiK05YTXk-78xg0AO6NoNe_RSRk05zYpF6KlA2yQ_My79rZBv9GFt4kUfIxNjd9OiV1wXdidO7Iaq_Q"
-    }
-
-
-Type: is the message type. One of 'notify', 'purge' refresh' 'force_refresh', 'auth_check', 'ping' or 'pickup'. The packet contents vary by message type. Here we will describe the 'notify' packet. 
-
-Sender is an array of four components. 
-
-The guid of the sender is typically a 64 character base64url encoded string. This is generated when an identity is created and an attempt is made that it be unique; though this isn't required.
-
-The guid_sig is the RSA signature of the guid, signed by the sender's private key.
-
-url is the base url of the location this post is originating from.
-
-url_sig is the RSA signature of url, signed by the sender's private key.
-
-
-These four elements provide a portable identity. We can contact the URL provided and download a zot info packet to obtain the public key of the sender, and use that to verify the sender guid and the posting URL signatures. 
-
-
-callback is a string to be appended onto the url which identifies the zot communications endpoint on this system. It is typically the string "/post".
-
-version is the zot protocol identifier, to allow future protocol revisions to co-exist. 
-
-secret is a 64-char string which is randomly generated by the sending site.
-
-secret_sig is the RSA signature of the secret, signed with the sender's private key. 
-
-#include doc/macros/main_footer.bb;