Tuvik's Corner

Indie web proposal: Webthread

Since the last few days, I've been working on a way for websites to reply to each other. The protocol I came up with is simple enough to be easy to implement, and could even work statically!

Webthread is a self-hosted indie web reply system based on a very simple protocol that is completely able to work without the need of server code! Yes, you can run this system completely by hand if you wish to!

The most basic implementation only requires you to host a json file and allows your web pages to be sent as replies to other people's Webthread instances!
More features become available if you start implementing more of the protocol, including showing off replies you received from other users, displaying a follow list, and all this can work with or without any server code!

Both a client (js) and a server (php) script implementing the protocol are available for your convenience. None are required if you want to implement the protocol by yourself. These are here to help bootstrap you into using the system!

The JS client script contains both an API to directly call the server, and a plug-and-play widget that will automatically connect to your webthread instance and fetch data.
All you need is to include the script on your pages declaring a web-thread <meta> tag, and either initialize and use the API, or simply initialize the default style and widget.

The PHP server script includes a microframework that implements all the features written in this first proposal.
All you need is to host the file as .php extension, configure the constants at the top of the file, and set up the relevant web-thread <meta> tag in your pages.

An interesting outcome this system allows is a forum-like structure, in which I decided to open a Webthread forum for that matter.

Here is the proposal (converted from markdown):

Webthread

Webthread is a simplistic communication protocol between websites, giving the possibility to reply to web pages using other web pages.

The concepts

Webthread introduces a handful of concepts that allows its functioning. Let's delve into those in theory first:

Webthread instance

Webthread by itself is not a software. It is a protocol that can be held by some software, or can even be run statically if the user has the patience to write the data down manually. The most basic webthread instance is a static JSON file at a location. This JSON contains the Webthread identity of this page. Each page can link to a different instance, thus also a different identity if you wish to.

Webthread identity

The identity is a JSON object holding various information about the user of this page, but it also holds most importantly a list of capabilities the webthread instance is providing. A list of these capabilities can be found in the protocol section of this paper.

Each capability defines one or more sets of endpoints that the webthread instance allows to use. As an example, the 'replies' capability states that the user can call /replies?target=[page_url] to obtain a list of replies for this page. Note that while there is a parameter provided here, it is needed only if you run a script. If your instance is being run statically, you will want to host one webthread instance per different page, in order for the /replies call to only provide the replies for your page. It is more cumbersome than using a scripted instance, but it works.

Replies

A reply is in its fundamentals a link indicating that a web page replied to another.

The replying page must backlink to the replied page before submitting a reply to it.

The replied page is not required to show the replying page if they do not wish to.

Follows and bans

Webthread proposes a simple concept of following and banning domains.

Assuming you are using webthread with a scripted instance, follows and bans are useful to auto manage replies that has been sent using the send capability.

Following someone means automatically approving their replies to be published without prior review.

Banning someone means automatically denying their replies without prior review.

Running Webthread statically

You can run Webthread statically through multiple levels of compatibility:

• You can provide an identity to be able to reply to other websites
• You can provide a follow list to show off who you trust
• You can provide replies to your pages

Providing an identity

This is the simplest form of Webthread: providing just the identity.

For that, you only need to create a json file and insert the identity data there. You will then add on the relevant pages the <meta> tag pointing to that file.

You are now ready to send replies to other web pages!

Providing more functions to your static instance

From this point on, you will want to host your web thread instance in a separate directory.

In this directory, put your identity file there and configure your web server to point to this file if the user types the url.
Imagine you created the /wt/ directory. You put your file index.json in there with your JSON identity data inside. You then configure your server to say that the index file of that directory is index.json.

If you are using apache as a web server, you can just insert a .htaccess in this directory and add the directive

DirectoryIndex index.json

Alternatively, you can put your JSON data into a index.html file (which is the default file name) and it will work exactly the same, it is just less "best practice".

Providing a following list

Inside your webthread directory, add a file following (with no extension), and fill it with the JSON data expected for the following capability (see protocol)

Providing replies to your pages

Same as for the following list, providing replies is just a matter of creating a replies file into your webthread directory, and setting up its content as the protocol expects. Be sure to check that your client script filter the result according to the page!

As for receiving replies while using a static method, you can definitely do it manually too! All you need is provide some way for others to contact you with their URLs, and then you can modify the files accordingly. (for instance, providing an email address or a contact form to do so).

The protocol (version 0.x)

Replyable web pages must implement a meta tag in their header:

<meta name="web-thread" content="[url]">

This URL must respond with some JSON data (called Webthread identity):

{
  "software": "Webthread",
  "version": "0.2",
  "domain":"alice.com",
  "username":"Alice",
  "avatar":"https://alice.com/avatar.jpg",
  "capabilities":["send", "replies", "following", "admin"]
}
  

Most of the file is facultative, except the capabilities key: it holds an array describing what the webthread capabilities are for your website/page. It is important for potential clients to know what they can do with your instance.

Let's first describe what the whole JSON is about before:

key	description	example
software	Name of the software used for Webthread	static, webthread, ...
version	Version number of the software	0.1,1.0,1.2.3, ...
domain	The domain that holds this instance.	example.com
username	The username you wish other websites display on your replies	John Doe
avatar	A picture you wish other websites display on your replies	any absolute URL to a picture
capabilities	A list of what your instance of Webthread is capable of	See below

The capabilities list can hold a set of multiple values which will be described in depth in the next parts:

Sending replies with send

Webthread instances showing the send capability allows users to use POST [webthread_url]/send to send a reply to their page. The endpoint must accept POST requests with the following required parameters:

key	description
remote	URL of the page containing the reply
target	URL of the page the user wish to send their reply to

The remote URL must also host an instance of Webthread, with no expected capabilities. The endpoints may perform several checks in order to allow your reply to be inserted in the system.
In order to verify if your page has a Webthread instance, it'll check if you correctly set the web-thread meta tag, and will pull your username and avatar from there. Your page must also hold an absolute link to the target page, and your page must not exceed 10MB in size.

The target URL must be an existing page within the website you wish to reply to. The endpoints may perform checks to confirm the page has the right domain and does exist.

In case of success, the webthread instance shall respond with an HTTP code 200, or any 4XX and 5XX in case of failure. The instance can choose to display a message or not in such cases.

Your reply may appear with the status 'pending', meaning that the website owner needs to review and manually accept your reply in order to display it on their page. The webthread software may not accept more replies from you until this one has been accepted.

Fetching replies with replies

Webthread instances showing the replies capability allows users to use GET [webthread_url]/replies to obtain replies from their page. The endpoint must accept or ignore the following parameter:

key	description
target	URL of the page the user wish to obtain replies from

The target URL must be an existing page within the website you wish to reply to. The endpoints may perform checks to confirm the page has the right domain and does exist. If target is empty or not provided, the endpoint must return all the replies across all pages. In such cases, the client will need to manually perform the filtering.

The endpoint will respond with a list of replies attached to the relevant page:

[
  {
    "id":"69b597bf7b8238.09984645",
    "username":"Bob",
    "avatar":"https://bob.com/avatar.jpg",
    "title":"Response to Alice - Bob's website",
    "target": "https://alice.com/my-cool-article.html",
    "url":"https://alice.com/response-to-alice.html",
    "domain":"example.com",
    "status":"published"
  }
]
  

Most of these keys are necessary for the protocol to work properly.

key	description
id	Unique identifier for the reply. Must be unique across pages
username	Username of the user that sent the reply
avatar	Avatar of the user that sent the reply. Can be nullable
title	Title of the replying page. Can be any other text
target	URL of the replied page
url	URL of the replying page
domain	domain of the replying page
status	Status of the reply. The protocol defines 'pending' and 'published'

Fetch who you are following with following

Webthread instances showing the following capability allows users to use GET [webthread_url]/following to obtain replies from their page.

The endpoint must respond with a list of users in the following format:

[
  {
    "username": "Bob",
    "avatar": "https://bob.com/avatar.jpg",
    "domain": "bob.com"
  }
]
  

All of these keys are mandatory:

key	description
username	Username of the user that is followed
avatar	Avatar of the user that is followed. Can be nullable
domain	domain of the followed user

Manage your replies with admin

Webthread instances showing the admin capability allows users to use POST [webthread_url]/manage to manage replies on their pages.

The endpoint must accept POST requests with the following required parameters:

key	description
target	URL of the page the user wish to send their reply to
action	Action to take on the reply. Can be 'follow', 'approve', 'deny' or 'ban'
reply	Reply ID of the reply you wish to manage

The endpoint will change the status of the provided reply id on the provided page, to either publish it in the case of 'follow' or 'approve', or discard the reply entirely in the case of 'deny' or 'ban'.
'follow' means 'approve' but also automatically approve every reply from that domain
'ban' means 'deny' but also automatically deny every reply from that domain.

This endpoint may ask for a password or some sort of authentication mechanism in order to not let anybody manage your own replies.

In case of success, the webthread instance shall respond with an HTTP code 200, or any 4XX and 5XX in case of failure. The instance can choose to display a message or not in such cases.