Main API functions of the server
The PrivMX server module provides API methods to client programs allowing for implementation of the following functions:
- data storage – functions of saving and reading of data blocks and descriptors;
- handling of communication boxes and functions for receiving and reading of messages;
- storage and verification of public keys – PrivMX PKI functions are described separately in the last chapter of this document;
- creation of accounts, login, proxy, users’ profiles;
- administration – description of API function connected with the PrivMX server management is omitted in this document.
In this chapter we focus on the first two issues.
ECC keys – identifiers and access rights
The PrivMX architecture assumes that the IDs for newly created objects are given in most cases by the client program.
- The identifiers of the most important objects are (serialized) ECC public keys or the 160 bit bitcoin addresses counted from them. They are created based on ECC private keys randomly chosen by the client program;
- the server does NOT allow the client programs to enumerate objects that it stores.
- access to "normal", "unprotected" operations on the data requires the client program to have their identifier, i.e. the ECC public key. However the access to specific "protected" operations, i.e. modifying an object or removing it, requires to possess additionally an appropriate ECC private key;
Creating objects on the PrivMX server is always done only and exclusively at the wish of the client program. Due to the lack of possibility of enumeration of objects, the ECC keys pair (privkey, pubkey) randomly chosen by the client module becomes the only way of access to the object. Storage of this pair is the responsibility of the client program – PrivMX architecture does not specify the specific way to store these keys, but the standard client library contains some ready-to-use solutions (see further chapters).
PrivMX Server accepts and serves data like disk drivers in files systems – in form of blocks of maximum size of up to 128KB (this size can be configured).
The blocks are "raw data portions" with assigned identifiers – BIDs. Blocks are created by the client module, which sends them to the server where they are stored without modifying the content.
- The server does not encrypt the data – the client program is responsible for it.
- The client program determines the BID of the block so that it is equal to SHA256 of the content of the block sent.
In respect of the block design (BID of the block = hash of its contents):
- client and server modules can benefit from the natural control of the consistency of transmitted data – block identifier is also a checksum of data;
- it is not possible to create two blocks on the server for which the same data was sent;
- blocks are unmodifiable (immutable) – "change of block data" can only be implemented through creating and submitting of a new block – new data means a new BID.
Blocks do not exist independently – they can be created and read by the client module always in conjunction with the operation on the object, which uses these blocks – see descriptors and messages below. Each block can be used by several such objects. The PrivMX architecture assumes that blocks that are not assigned to any object should be automatically removed by the server.
Descriptors enable to store portions of data on the server larger than 128KB – descriptors "are grouping the blocks" and are access points to them for the client program. They are also facilities implementing a basic data access rights policy (in line with the previously mentioned manner of use of public and private ECC keys). Below we describe it more accurately.
The creation of descriptors (
API.descriptorCreate*) and their modification (
API.descriptorUpdate*) typically requires the transmission of one or more data blocks through single or multiple use of the corresponding API method. For this purpose, a "transfer sessions" mechanism is used – a temporary unique data transfer identifier is created and used in subsequent calls of the
- The client program (user logged in) calls the
API.descriptorCreateInitand obtains the TransferID.
- Then it sends the blocks using
API.blockCreate– giving the TransferID and BID and the data of each block. If the client program wants to use blocks that already exist on the server, then it uses another function –
- It randomly creates a new dpriv private key and generates the dpub public key for it.
- Finally, it creates a new descriptor (
API.descriptorCreateFinish) on the server. To this end it sends to it:
- DID (160 bit) – bitcoin address ("main" network) for dpub;
- TransferID, Blocks – TransferID used and BIDs list of blocks sent during the session;
- Extra – space for additional, arbitrary descriptor data to be used by the client application.
- dpub – public key of the descriptor;
- signature of the above data with the dpriv key – so that the server will know that the client has the appropriate private key, i.e. it is the owner of the descriptor.
Descriptors and blocks, combined with client-side data encryption (encryption of block data and Extra fields) enable for the creation of data-storing mechanisms that hide the contents of files and directory structure from the server – see the chapter about the client module.
Reading descriptors is possible for each client program that has the corresponding DIDs. Then through calling of
API.descriptorGet the client can obtain a full set of the above information (Blocks, Extra, dpub, etc.) and through subsequent calls of
API.blockGet the client can retrieve all data blocks. Calling of
API.blockGet requires to give BID and DID – it is not sufficient to know only the block ID in order to obtain access to its data.
Modification of the descriptor (
API.descriptorUpdate) is an operation similar to creating a descriptor – it requires the request to be signed with the dpriv key and it requires sending new blocks (or using the existing ones). The descriptor removal request (
API.descriptorDelete) also requires the signature by a private key. Removal of the descriptor does not cause automatic removal of blocks – this should happen only if they are not used by other descriptors or messages.
The mechanism of the operation of mailboxes and messages between PrivMX client programs (users) can generally be described as follows:
- to send the message, the sender’s client program must sign it with the sender key and establish a standard PrivMX TLS connection with the recipient server (this can be done using a proxy). It calls the
API.messagePutmethod on this server to drop a message into a specific mailbox. Boxes are identified by public keys.
- The recipient receives a message from their server when the client program checks the status of their mailboxes (
API.messageGet). Corresponding requests must be signed with private keys of the boxes.
The manner of receiving messages by a mailbox, defined in the PrivMX API terminology as "sink", is determined at the time of its creation. Then the client program, after randomly creating a new pair of keys (spriv, spub), must convey the following data to the
- SID, which is the serialized spub public key.
- WriteMode – the value "private", "public" or "anonymous" specifying who may send messages to this mailbox:
- private – only the owner of the private key of the mailbox may leave messages;
- public – any user of any PrivMX server may leave messages. The message signature is then verified by using PrivMX PKI.
- anonymous – the mailbox accepts messages signed with any key, i.e. by such one which is randomly created before sending.
- Extra – a field for any use by the client program.
- Signature of the above data executed with spriv key.
The WriteMode and Extra fields can later be read by the client program (
API.sinkGetInfo) and modified (
API.sinkUpdate) and the box as a whole can be removed by calling of
API.sinkDelete. All the operations listed here are available by default only in the "logged in" connection and must be signed with the appropriate private key.
Sending a message (i.e. creating and placing it in a specific target server’s mailbox using the
API.messsagePut method) is similar to creating a descriptor – it involves the creation of a "transfer session" and the sending of data blocks. Only messages with sizes of up to 1 MB can be sent without the use of blocks.
Assuming the client program has a pair of sender’s keys (cpriv, cpub):
- sending a message starts calling of
- SID of destination mailbox.
- SenderAddress – sender’s address in the format "user#server.net". This field is not used in case of anonymous boxes.
- SenderPubKey – cpub public key belonging to the sender.
- ExtraAuth – place for optional additional data to be used by the sender verification server. This field is especially useful when the box is running in anonymous mode – e.g. it receives messages from web forms using captcha.
- Signature of the entire request with cpriv key.
When the client is not rejected (see below), it receives a new TransferID that can be used to send message data blocks. It does this through the appropriate calls of
- Sending the message ends with the
API.messagePutFinishcall with parameters:
- Extra – field for any use by the client (max 1MB).
- TransferID, Blocks – transfer session confirmation – TransferID used and list of BIDs sent to the blocks
- Tags – list of any strings set by the sender. The server stores them with the message.
- Signature of the entire request done with cpriv key.
Rejecting a message may result from poor verification of the sender’s address or key (PrivMX PKI) or because of an optional ExtraAuth verification failure. The server does not check the data sent in the ExtraAuth field by default, but it can be done by server extensions – depending on the specified application.
The mailbox numbers the received messages and provides the number of the last message (lastNumber) through the already mentioned
API.sinkGetInfo. Reading the list of messages contained in the mailbox is available to the holder of the private key of the mailbox via a properly signed call of
API.sinkGetMessages. This method allows the client to obtain the message IDs (MIDs) from the given numbering range and those that have set specific tags.
The client program can read the message by calling of
API.messageGet using the appropriate MID and SID. It then obtains access to the SenderAddress, SenderPubKey, ExtraAnon, Blocks, Extra, Tags fields. The request to read the message must be signed with the private key of the mailbox. The
API.messageDelete method requires that the similar data and signature are given.
Public and private users’ data
In addition to the objects described in the previous chapters, the server also stores the data associated with specific users. Their scope depends on specific applications.
In default configuration, the PrivMX server, for each user, stores the following private data generated by the client program:
- login data – hashing parameters (salt, rounds number, hashing algorithm name used on the client side) and SRP verifier. The server does not store the user’s password.
- a small PrivData data record encrypted on the client side.
By using the PrivMX PKI mechanism, the server, by default, makes available publicly, i.e. for any client program, the following user-related data:
- SID of the default mailbox – associated with the user’s address;
- IdentityKeyPub, i.e. the public key of the user;
- User’s profile data – optional data such as full name, description, avatar, etc.
The meaning of the above data and ways of accessing them are described further in the course of the document.
This sub-chapter contains a variety of notes regarding configuration, extensions, and implementation of the PrivMX server.
Storing of blocks, descriptors, mailboxes, and messages can be accomplished in the best way by the use of simple and fast key-value, document data bases.
Removing blocks – the requirement to automatically remove blocks that are not assigned to any object (descriptor, messages) can be accomplished, for example, by cyclical running of a garbage collector. It is also possible to extend the API with the
API.blockDelete method and then the client program could store BIDs of blocks in (encrypted) Extra descriptor field.
There are plenty of options for configuring PrivMX servers for eligibility of connections – from blocking access to specific API methods within the framework of a standard connection, by limiting connections between servers, up to specifying specific IP addresses and users which can contact the server.
The default implementation of the server assumes that all mailbox API functions are available only in the logged-in connection. This means that it is possible to extend the API with mailbox methods "automatically" using the keys of the logged-in user. At that time, there would be no need to constantly sign requests and convey a public key.
Mailboxes number their messages and share their number – lastNumber. It is possible to implement additional "counters" that can optimize the process of checking boxes, reading and modifying messages.
Next: Standard PrivMX client library – Extended ECC keys, creating accounts, login and initialization of the client, files and directories, sending and receiving messages, data sharing.
TOC: PrivMX Architecture
PDF version: PrivMX Architecture – whitepaper (pdf)