Setting up AnyType self-hosted sync
Setting up AnyType self-hosted sync and prepare it for automation.
Introduction
List most others, i used obsidian for years, but it lagged behind in terms of functionalities, although plenty of plugins are made for it and almost everything can be done, the SelfHosted Sync (opens in a new tab) messed up my files a few times and i had enough, without proper sync it wasn’t enough for me and i didn’t want to use official obsidian sync (opens in a new tab) either.
In my research i found AnyType, at first glance it was difference since i didn’t have file structure hierarchy, but soon i picked up the idea and went with it.
However, i didn’t want my files to be stored on AnyType servers either, but they offer self-hosted solutions as well and that’s what this post is all about.
Setting up Anytype self-hosted sync
First of all, we need to generate a file named networks.yml which is required for self-hosted connections, it’s described in details here (opens in a new tab).
I used a repo made by AnyType (currently only supports docker-compose).
Lets start by cloning the repo (opens in a new tab):
git clone https://github.com/anyproto/any-sync-dockercompose.git
cd any-sync-dockercomposeFind server public IP address:
curl -4 ifconfig.ioLets prepare the .env file:
cp .env.example .envAdd the your public ip address(es) to .env.override: (if it didn’t work add it to .env file)
echo 'EXTERNAL_LISTEN_HOSTS="{SERVER_PUBLIC_IP_SEP_BY_COMMA}"' >> .env.overrideStart the build process:
make startHow to backup and restore
If you want to backup the sync-server, just copy the ./storage folder which is created in the same folder you’re running your docker-compose, make sure to copy your .env file as well.
How to connect
The settings are available in onboarding screen, complete guide is described in AnyType docs (opens in a new tab).
Setting up API for the bot
As always, i want to create bots to automate things, like creating daily notes at midnight, or automatically set status of the tasks which is marked as done to completed.
Here is how it’s gonna work
flowchart BT
%% define nodes
SYNC@{label: "Sync Server", shape: "database"}
BOT@{label: "API BOT (Cli Client)"}
CLIENT@{label: "Mobile/Desktop Client"}
%% connections
CLIENT <--> SYNC
BOT <--> SYNCSince AnyType is offline first for data protection, the API it exposes is only available in localhost, so we needed a way to access it from a server.
Fortunately AnyType team introduced a cli client (opens in a new tab) that connects to self-hosted sync server and acts as an API, the cli will expose endpoints running on localhost (the VPS or k8s pod in my case), we can add an http server to expose them on the internet (0.0.0.0/0).
To better understand what the bot does, it’s just another client like the mobile app, as it happens it’s running in the terminal and that’s what why it’s useful for your purpose.
Running the CLI client
Lets install the AnyType CLI client:
/usr/bin/env bash -c "$(curl -fsSL https://raw.githubusercontent.com/anyproto/anytype-cli/HEAD/install.sh)"Copy your network.yml and put it under ~/.config/anytype/network.yml.
If you want to listen on your public IP instead of the default localhost, add --listen-address:
anytype service install --listen-address 0.0.0.0:31012
anytype service startNow you have the client ready, it’s just like you installed desktop or mobile app, however you need an account to log-in with right? so we will create a dedicated account for the bot and invite it into your space through the self-hosted sync we configured.
Lets create a new account for our bot:
anytype auth create [my-bot-name] --network-config ~/.config/anytype/network.ymlAfter running it you will receive BOT ACCOUNT KEY and Account Id, the Bot Account Key is base64 encoded of a hex code, which with account id will result in secret phrase, it’s the same when you create an account with onboarding screen.
So loosing that results in loosing the account you just created for the bot.
Now the bot have an account, and can write notes for itself, but the idea of creating the bot was to manage notes that we are creating, lets join it into our space.
Get an invite link from your Channel Settings > Members > Invite Link (make sure you add with editor access), then run this command in order to join the bot into your space:
anytype space join "<invite-link>"Lets confirm it really did join the space:
anytype space listNow you can create an API key, this is where the HTTP layer starts to take shape:
anytype auth apikey create "<my-bot-api-key>" --all-spaces --read-write
This will create an api key that you can use to interact with your space in AnyType, it’s the same as if you go into your profile in other clients and create an API key.
Now that you have created your api key, you can control your notes and types anyway you like, take a look at OpenAPI specification (opens in a new tab) to make your automation.
If you’re interested, i’ve made a repo to do all this with http exposing the api, take a look (opens in a new tab).
A note about minio back-end storage
As i was testing out, they are using minio as file storage, in k8s it should be centralized and i found out that there is a bug in their system, if you upload a file and delete the object in anytype, the file will remain in minio (creating memory leak), checking their github task board, it seems they are aware of this, hopefully in the future updates this will get fixed.
