removed PGP
parent
b15c0b7e00
commit
895b1919fd
|
@ -1,72 +1,49 @@
|
||||||
# Onionr Protocol Spec
|
# Onionr Protocol Spec v2
|
||||||
|
|
||||||
A social network/microblogging platform for Tor & I2P
|
A P2P platform for Tor & I2P
|
||||||
|
|
||||||
Draft Dec 25 2017
|
|
||||||
|
|
||||||
# Overview
|
# Overview
|
||||||
|
|
||||||
Onionr is an encrypted microblogging & mailing system designed in the spirit of Twitter.
|
Onionr is an encrypted microblogging & mailing system designed in the spirit of Twitter.
|
||||||
There are no central servers and all traffic is peer to peer by default (routed via Tor or I2P).
|
There are no central servers and all traffic is peer to peer by default (routed via Tor or I2P).
|
||||||
User IDs are simply Tor onion service/I2P host id + PGP fingerprint.
|
User IDs are simply Tor onion service/I2P host id + Ed25519 key fingerprint.
|
||||||
Clients consolidate feeds from peers into 1 “timeline” using RSS format.
|
Private blocks are only able to be read by the intended peer.
|
||||||
Private messages are only accessible by the intended peer based on the PGP id.
|
All traffic is over Tor/I2P, connecting only to Tor onion and I2P hidden services.
|
||||||
Onionr is not intended to be a replacement for Ricochet, OnionShare, or Briar.
|
|
||||||
All traffic is over onion/I2P because if only some was, then that would make that traffic inherently suspicious.
|
|
||||||
## Goals:
|
## Goals:
|
||||||
• Selective sharing of information with friends & public
|
• Selective sharing of information
|
||||||
• Secure & semi-anonymous direct messaging
|
• Secure & semi-anonymous direct messaging
|
||||||
• Forward secrecy
|
• Forward secrecy
|
||||||
• Defense in depth
|
• Defense in depth
|
||||||
• Data should be secure for years to come, quantum safe (though not necessarily every “layer”)
|
• Data should be secure for years to come
|
||||||
• Decentralization
|
• Decentralization
|
||||||
* Avoid browser-based exploits that plague similar software
|
* Avoid browser-based exploits that plague similar software
|
||||||
* Avoid timing attacks & unexpected metadata leaks
|
* Avoid timing attacks & unexpected metadata leaks
|
||||||
## Assumptions:
|
|
||||||
• Tor & I2P’s transport protocols & AES-256 are not broken, sha3-512 2nd preimage attacks will remain infeasible indefinitely
|
|
||||||
• All traffic is logged indefinitely by powerful adversaries
|
|
||||||
## Protocol
|
## Protocol
|
||||||
Clients MUST use HTTP(s) to communicate with one another to maintain compatibility cross platform. HTTPS is recommended, but HTTP is acceptable because Tor & I2P provide transport layer security.
|
|
||||||
|
Onionr nodes use HTTP (over Tor/I2P) to exchange keys, metadata, and blocks. Blocks are identified by their sha3_256 hash. Nodes sync a table of blocks hashes and attempt to download blocks they do not yet have from random peers.
|
||||||
|
|
||||||
## Connections
|
## Connections
|
||||||
When a node first comes online, it attempts to bootstrap using a default list provided by a client.
|
|
||||||
When two peers connect, they exchange PGP public keys and then generate a shared AES-SHA3-512 HMAC token. These keys are stored in a peer database until expiry.
|
|
||||||
HMAC tokens are regenerated either every X many communications with a peer or every X minutes. Every 10MB or every 2 hours is a recommended default.
|
|
||||||
All valid requests with HMAC should be recorded until used HMAC's expiry to prevent replay attacks.
|
|
||||||
Peer Types
|
|
||||||
* Friends:
|
|
||||||
* Encrypted ‘friends only’ posts to one another
|
|
||||||
* Usually less strict rate & storage limits
|
|
||||||
* OPTIONALLY sign one another’s keys. Users may not want to do this in order to avoid exposing their entire friends list.
|
|
||||||
• Strangers:
|
|
||||||
* Used for storage of encrypted or public information
|
|
||||||
* Can only read public posts
|
|
||||||
* Usually stricter rate & storage limits
|
|
||||||
## Data Storage/Delivery
|
|
||||||
|
|
||||||
Posts (public or friends only) are stored across the network.
|
When a node first comes online, it attempts to bootstrap using a default list provided by a client.
|
||||||
Private messages SHOULD be delivered directly if both peers are online, otherwise stored in the network.
|
When two peers connect, they exchange Ed25519 keys (if applicable) then Salsa20 keys.
|
||||||
Data SHOULD be stored in an entirely encrypted state when a client is offline, including metadata. Data SHOULD be stored in a minimal size with garbage data to ensure some level of plausible deniablity.
|
|
||||||
Data SHOULD be stored as long as the node’s user prefers and only erased once disk quota is reached due to new data.
|
|
||||||
Posts
|
|
||||||
Posts can contain text and images. All posts MUST be time stamped.
|
|
||||||
Images SHOULD not be displayed by non-friends by default, to prevent unwanted viewing of offensive material & to reduce attack surface.
|
|
||||||
All received posts must be verified to be stored and/or displayed to the user.
|
|
||||||
|
|
||||||
All data being transfered MUST be encrypted to the end node receiving the data, then the data MUST be encrypted the node(s) transporting/storing the data,
|
Salsa20 keys are regenerated either every X many communications with a peer or every X minutes.
|
||||||
|
|
||||||
Posts have two settings:
|
Every 100kb or every 2 hours is a recommended default.
|
||||||
• Friends only:
|
|
||||||
◦ Posts MUST be encrypted to all trusted peers via AES256-HMAC-SHA256 and PGP signed (signed before encryption) and time stamped to prevent replaying. A temporary RSA key for use in every post (or message) is exchanged every X many configured post (or message), for use in addition with PGP and the HMAC.
|
|
||||||
• Public:
|
|
||||||
◦ Posts MUST be PGP signed, and MUST NOT use any encryption.
|
|
||||||
## Private Messages
|
|
||||||
|
|
||||||
Private messages are messages that can have attached images. They MUST be encrypted via AES256-HMAC-SHA256 and PGP signed (signed before encryption) and time stamped to prevent replaying. A temporary EdDSA key for use in every message is exchanged every X many configured messages (or posts), for use in addition with PGP and the HMAC.
|
All valid requests with HMAC should be recorded until used HMAC's expiry to prevent replay attacks.
|
||||||
When both peers are online messages SHOULD be dispatched directly between peers.
|
Peer Types
|
||||||
All messages must be verified prior to being displayed.
|
* Friends:
|
||||||
|
* Encrypted ‘friends only’ posts to one another
|
||||||
|
* Usually less strict rate & storage limits
|
||||||
|
* Strangers:
|
||||||
|
* Used for storage of encrypted or public information
|
||||||
|
* Can only read public posts
|
||||||
|
* Usually stricter rate & storage limits
|
||||||
|
|
||||||
Clients SHOULD allow configurable message padding.
|
|
||||||
## Spam mitigation
|
## Spam mitigation
|
||||||
|
|
||||||
To send or receive data, a node can optionally request that the other node generate a hash that when in hexadecimal representation contains a random string at a random location in the string. Clients will configure what difficulty to request, and what difficulty is acceptable for themselves to perform. Difficulty should correlate with recent network & disk usage and data size. Friends can be configured to have less strict (to non existent) limits, separately from strangers. (proof of work).
|
To send or receive data, a node can optionally request that the other node generate a hash that when in hexadecimal representation contains a random string at a random location in the string. Clients will configure what difficulty to request, and what difficulty is acceptable for themselves to perform. Difficulty should correlate with recent network & disk usage and data size. Friends can be configured to have less strict (to non existent) limits, separately from strangers. (proof of work).
|
||||||
Rate limits can be strict, as Onionr is not intended to be an instant messaging application.
|
Rate limits can be strict, as Onionr is not intended to be an instant messaging application.
|
|
@ -20,7 +20,7 @@
|
||||||
import flask
|
import flask
|
||||||
from flask import request, Response, abort
|
from flask import request, Response, abort
|
||||||
from multiprocessing import Process
|
from multiprocessing import Process
|
||||||
import configparser, sys, random, threading, hmac, hashlib, base64, time, math, gnupg, os, logger
|
import configparser, sys, random, threading, hmac, hashlib, base64, time, math, os, logger
|
||||||
|
|
||||||
from core import Core
|
from core import Core
|
||||||
import onionrutils, onionrcrypto
|
import onionrutils, onionrcrypto
|
||||||
|
@ -140,8 +140,6 @@ class API:
|
||||||
resp = Response(self._utils.getBlockDBHash())
|
resp = Response(self._utils.getBlockDBHash())
|
||||||
elif action == 'getBlockHashes':
|
elif action == 'getBlockHashes':
|
||||||
resp = Response(self._core.getBlockList())
|
resp = Response(self._core.getBlockList())
|
||||||
elif action == 'getPGP':
|
|
||||||
resp = Response(self._utils.exportMyPubkey())
|
|
||||||
# setData should be something the communicator initiates, not this api
|
# setData should be something the communicator initiates, not this api
|
||||||
elif action == 'getData':
|
elif action == 'getData':
|
||||||
resp = self._core.getData(data)
|
resp = self._core.getData(data)
|
||||||
|
|
|
@ -40,13 +40,6 @@ class OnionrCommunicate:
|
||||||
|
|
||||||
self.peerData = {} # Session data for peers (recent reachability, speed, etc)
|
self.peerData = {} # Session data for peers (recent reachability, speed, etc)
|
||||||
|
|
||||||
# get our own PGP fingerprint
|
|
||||||
fingerprintFile = 'data/own-fingerprint.txt'
|
|
||||||
if not os.path.exists(fingerprintFile):
|
|
||||||
self._core.generateMainPGP(torID)
|
|
||||||
with open(fingerprintFile,'r') as f:
|
|
||||||
self.pgpOwnFingerprint = f.read()
|
|
||||||
logger.info('My PGP fingerprint is ' + logger.colors.underline + self.pgpOwnFingerprint + logger.colors.reset + logger.colors.fg.green + '.')
|
|
||||||
if os.path.exists(self._core.queueDB):
|
if os.path.exists(self._core.queueDB):
|
||||||
self._core.clearDaemonQueue()
|
self._core.clearDaemonQueue()
|
||||||
while True:
|
while True:
|
||||||
|
|
|
@ -17,7 +17,7 @@
|
||||||
You should have received a copy of the GNU General Public License
|
You should have received a copy of the GNU General Public License
|
||||||
along with this program. If not, see <https://www.gnu.org/licenses/>.
|
along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||||
'''
|
'''
|
||||||
import sqlite3, os, sys, time, math, gnupg, base64, tarfile, getpass, simplecrypt, hashlib, nacl, logger
|
import sqlite3, os, sys, time, math, base64, tarfile, getpass, simplecrypt, hashlib, nacl, logger
|
||||||
#from Crypto.Cipher import AES
|
#from Crypto.Cipher import AES
|
||||||
#from Crypto import Random
|
#from Crypto import Random
|
||||||
import netcontroller
|
import netcontroller
|
||||||
|
@ -38,10 +38,8 @@ class Core:
|
||||||
'''
|
'''
|
||||||
self.queueDB = 'data/queue.db'
|
self.queueDB = 'data/queue.db'
|
||||||
self.peerDB = 'data/peers.db'
|
self.peerDB = 'data/peers.db'
|
||||||
self.ownPGPID = ''
|
|
||||||
self.blockDB = 'data/blocks.db'
|
self.blockDB = 'data/blocks.db'
|
||||||
self.blockDataLocation = 'data/blocks/'
|
self.blockDataLocation = 'data/blocks/'
|
||||||
self.gpgHome = './data/pgp/'
|
|
||||||
self._utils = onionrutils.OnionrUtils(self)
|
self._utils = onionrutils.OnionrUtils(self)
|
||||||
self._crypto = onionrcrypto.OnionrCrypto(self)
|
self._crypto = onionrcrypto.OnionrCrypto(self)
|
||||||
|
|
||||||
|
@ -55,28 +53,6 @@ class Core:
|
||||||
|
|
||||||
return
|
return
|
||||||
|
|
||||||
def generateMainPGP(self, myID):
|
|
||||||
'''
|
|
||||||
Generate the main PGP key for our client. Should not be done often.
|
|
||||||
|
|
||||||
Uses own PGP home folder in the data/ directory
|
|
||||||
'''
|
|
||||||
gpg = gnupg.GPG(homedir=self.gpgHome)
|
|
||||||
input_data = gpg.gen_key_input(key_type="RSA", key_length=1024, name_real=myID, name_email='anon@onionr', testing=True)
|
|
||||||
key = gpg.gen_key(input_data)
|
|
||||||
logger.info("Generating PGP key, this will take some time..")
|
|
||||||
while key.status != "key created":
|
|
||||||
time.sleep(0.5)
|
|
||||||
print(key.status)
|
|
||||||
|
|
||||||
logger.info("Finished generating PGP key")
|
|
||||||
# Write the key
|
|
||||||
myFingerpintFile = open('data/own-fingerprint.txt', 'w')
|
|
||||||
myFingerpintFile.write(key.fingerprint)
|
|
||||||
myFingerpintFile.close()
|
|
||||||
|
|
||||||
return
|
|
||||||
|
|
||||||
def addPeer(self, peerID, name=''):
|
def addPeer(self, peerID, name=''):
|
||||||
'''
|
'''
|
||||||
Add a peer by their ID, with an optional name, to the peer database
|
Add a peer by their ID, with an optional name, to the peer database
|
||||||
|
@ -104,8 +80,7 @@ class Core:
|
||||||
c.execute('''CREATE TABLE peers(
|
c.execute('''CREATE TABLE peers(
|
||||||
ID text not null,
|
ID text not null,
|
||||||
name text,
|
name text,
|
||||||
pgpKey text,
|
pubkey text,
|
||||||
hmacKey text,
|
|
||||||
blockDBHash text,
|
blockDBHash text,
|
||||||
forwardKey text,
|
forwardKey text,
|
||||||
dateSeen not null,
|
dateSeen not null,
|
||||||
|
@ -335,7 +310,6 @@ class Core:
|
||||||
|
|
||||||
id text 0
|
id text 0
|
||||||
name text, 1
|
name text, 1
|
||||||
pgpKey text, 2
|
|
||||||
hmacKey text, 3
|
hmacKey text, 3
|
||||||
blockDBHash text, 4
|
blockDBHash text, 4
|
||||||
forwardKey text, 5
|
forwardKey text, 5
|
||||||
|
@ -346,7 +320,7 @@ class Core:
|
||||||
conn = sqlite3.connect(self.peerDB)
|
conn = sqlite3.connect(self.peerDB)
|
||||||
c = conn.cursor()
|
c = conn.cursor()
|
||||||
command = (peer,)
|
command = (peer,)
|
||||||
infoNumbers = {'id': 0, 'name': 1, 'pgpKey': 2, 'hmacKey': 3, 'blockDBHash': 4, 'forwardKey': 5, 'dateSeen': 6, 'bytesStored': 7, 'trust': 8}
|
infoNumbers = {'id': 0, 'name': 1, 'hmacKey': 3, 'blockDBHash': 4, 'forwardKey': 5, 'dateSeen': 6, 'bytesStored': 7, 'trust': 8}
|
||||||
info = infoNumbers[info]
|
info = infoNumbers[info]
|
||||||
iterCount = 0
|
iterCount = 0
|
||||||
retVal = ''
|
retVal = ''
|
||||||
|
@ -369,7 +343,7 @@ class Core:
|
||||||
c = conn.cursor()
|
c = conn.cursor()
|
||||||
command = (data, peer)
|
command = (data, peer)
|
||||||
# TODO: validate key on whitelist
|
# TODO: validate key on whitelist
|
||||||
if key not in ('id', 'text', 'name', 'pgpKey', 'hmacKey', 'blockDBHash', 'forwardKey', 'dateSeen', 'bytesStored', 'trust'):
|
if key not in ('id', 'name', 'pubkey', 'blockDBHash', 'forwardKey', 'dateSeen', 'bytesStored', 'trust'):
|
||||||
raise Exception("Got invalid database key when setting peer info")
|
raise Exception("Got invalid database key when setting peer info")
|
||||||
c.execute('UPDATE peers SET ' + key + ' = ? WHERE id=?', command)
|
c.execute('UPDATE peers SET ' + key + ' = ? WHERE id=?', command)
|
||||||
conn.commit()
|
conn.commit()
|
||||||
|
|
|
@ -17,7 +17,7 @@
|
||||||
You should have received a copy of the GNU General Public License
|
You should have received a copy of the GNU General Public License
|
||||||
along with this program. If not, see <https://www.gnu.org/licenses/>.
|
along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||||
'''
|
'''
|
||||||
import nacl, gnupg
|
import nacl
|
||||||
|
|
||||||
class OnionrCrypto:
|
class OnionrCrypto:
|
||||||
def __init__(self, coreInstance):
|
def __init__(self, coreInstance):
|
||||||
|
@ -29,15 +29,9 @@ class OnionrCrypto:
|
||||||
|
|
||||||
def symmetricPeerDecrypt(self, data, key):
|
def symmetricPeerDecrypt(self, data, key):
|
||||||
return
|
return
|
||||||
|
|
||||||
def rsaEncrypt(self, peer, data):
|
|
||||||
return
|
|
||||||
|
|
||||||
def verifyPGP(self, peer, signature):
|
|
||||||
'''Verify PGP signed data'''
|
|
||||||
gpg = gnupg.GPG(homedir=self._core.gpgHome)
|
|
||||||
|
|
||||||
def generateSymmetric():
|
def generateSymmetric():
|
||||||
return
|
return
|
||||||
|
|
||||||
def generateHMAC():
|
def generateHMAC():
|
||||||
return
|
return
|
|
@ -18,7 +18,7 @@
|
||||||
along with this program. If not, see <https://www.gnu.org/licenses/>.
|
along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||||
'''
|
'''
|
||||||
# Misc functions that do not fit in the main api, but are useful
|
# Misc functions that do not fit in the main api, but are useful
|
||||||
import getpass, sys, requests, configparser, os, socket, gnupg, hashlib, logger, sqlite3
|
import getpass, sys, requests, configparser, os, socket, hashlib, logger, sqlite3
|
||||||
if sys.version_info < (3, 6):
|
if sys.version_info < (3, 6):
|
||||||
try:
|
try:
|
||||||
import sha3
|
import sha3
|
||||||
|
@ -93,19 +93,6 @@ class OnionrUtils:
|
||||||
else:
|
else:
|
||||||
return True
|
return True
|
||||||
|
|
||||||
def exportMyPubkey(self):
|
|
||||||
'''
|
|
||||||
Export our PGP key if it exists
|
|
||||||
'''
|
|
||||||
if not os.path.exists(self.fingerprintFile):
|
|
||||||
raise Exception("No fingerprint found, cannot export our PGP key.")
|
|
||||||
gpg = gnupg.GPG(homedir='./data/pgp/')
|
|
||||||
with open(self.fingerprintFile,'r') as f:
|
|
||||||
fingerprint = f.read()
|
|
||||||
ascii_armored_public_keys = gpg.export_keys(fingerprint)
|
|
||||||
|
|
||||||
return ascii_armored_public_keys
|
|
||||||
|
|
||||||
def getBlockDBHash(self):
|
def getBlockDBHash(self):
|
||||||
'''
|
'''
|
||||||
Return a sha3_256 hash of the blocks DB
|
Return a sha3_256 hash of the blocks DB
|
||||||
|
@ -153,17 +140,6 @@ class OnionrUtils:
|
||||||
retVal = False
|
retVal = False
|
||||||
|
|
||||||
return retVal
|
return retVal
|
||||||
|
|
||||||
def getPeerPGPFingerprint(self, peer):
|
|
||||||
'''
|
|
||||||
Get peer's PGP fingerprint
|
|
||||||
'''
|
|
||||||
retData = ''
|
|
||||||
gpg = gnupg.GPG(homedir=self._core.gpgHome)
|
|
||||||
for i in gpg.list_keys():
|
|
||||||
if peer in i['uids'][0]:
|
|
||||||
retData = i['fingerprint']
|
|
||||||
return retData
|
|
||||||
|
|
||||||
def validateID(self, id):
|
def validateID(self, id):
|
||||||
'''
|
'''
|
||||||
|
|
|
@ -85,33 +85,6 @@ class OnionrTests(unittest.TestCase):
|
||||||
else:
|
else:
|
||||||
self.assertTrue(False)
|
self.assertTrue(False)
|
||||||
|
|
||||||
def testPGPGen(self):
|
|
||||||
logger.debug('--------------------------')
|
|
||||||
logger.info('Running PGP key generation test...')
|
|
||||||
if os.path.exists('data/pgp/'):
|
|
||||||
self.assertTrue(True)
|
|
||||||
else:
|
|
||||||
import core, netcontroller
|
|
||||||
myCore = core.Core()
|
|
||||||
net = netcontroller.NetController(1337)
|
|
||||||
net.startTor()
|
|
||||||
torID = open('data/hs/hostname').read()
|
|
||||||
myCore.generateMainPGP(torID)
|
|
||||||
if os.path.exists('data/pgp/'):
|
|
||||||
self.assertTrue(True)
|
|
||||||
|
|
||||||
def testHMACGen(self):
|
|
||||||
logger.debug('--------------------------')
|
|
||||||
logger.info('Running HMAC generation test...')
|
|
||||||
# Test if hmac key generation is working
|
|
||||||
import core
|
|
||||||
myCore = core.Core()
|
|
||||||
key = myCore.generateHMAC()
|
|
||||||
if len(key) > 10:
|
|
||||||
self.assertTrue(True)
|
|
||||||
else:
|
|
||||||
self.assertTrue(False)
|
|
||||||
|
|
||||||
def testQueue(self):
|
def testQueue(self):
|
||||||
logger.debug('--------------------------')
|
logger.debug('--------------------------')
|
||||||
logger.info('Running daemon queue test...')
|
logger.info('Running daemon queue test...')
|
||||||
|
|
|
@ -1,10 +1,6 @@
|
||||||
PyNaCl==1.2.1
|
PyNaCl==1.2.1
|
||||||
gnupg==2.3.1
|
requests==2.12.4
|
||||||
Flask==0.12.2
|
Flask==0.12.2
|
||||||
requests==2.18.4
|
|
||||||
urllib3==1.22
|
|
||||||
simple_crypt==4.1.7
|
simple_crypt==4.1.7
|
||||||
|
urllib3==1.19.1
|
||||||
sha3==0.2.1
|
sha3==0.2.1
|
||||||
pycrypto==2.6.1
|
|
||||||
pynacl==1.2.1
|
|
||||||
PySocks==1.6.8
|
|
||||||
|
|
Loading…
Reference in New Issue