Merge branch 'crypto' of github.com:beardog108/onionr into crypto
Kevin Froman
commit
b6cfe0154d
|
|
@ -1,71 +1,48 @@
|
|||
# Onionr Protocol Spec
|
||||
# Onionr Protocol Spec v2
|
||||
|
||||
A social network/microblogging platform for Tor & I2P
|
||||
|
||||
Draft Dec 25 2017
|
||||
A P2P platform for Tor & I2P
|
||||
|
||||
# Overview
|
||||
|
||||
Onionr is an encrypted microblogging & mailing system designed in the spirit of Twitter.
|
||||
There are no central servers and all traffic is peer to peer by default (routed via Tor or I2P).
|
||||
User IDs are simply Tor onion service/I2P host id + PGP fingerprint.
|
||||
Clients consolidate feeds from peers into 1 “timeline” using RSS format.
|
||||
Private messages are only accessible by the intended peer based on the PGP id.
|
||||
Onionr is not intended to be a replacement for Ricochet, OnionShare, or Briar.
|
||||
All traffic is over onion/I2P because if only some was, then that would make that traffic inherently suspicious.
|
||||
User IDs are simply Tor onion service/I2P host id + Ed25519 key fingerprint.
|
||||
Private blocks are only able to be read by the intended peer.
|
||||
All traffic is over Tor/I2P, connecting only to Tor onion and I2P hidden services.
|
||||
|
||||
## Goals:
|
||||
• Selective sharing of information with friends & public
|
||||
• Selective sharing of information
|
||||
• Secure & semi-anonymous direct messaging
|
||||
• Forward secrecy
|
||||
• Defense in depth
|
||||
• Data should be secure for years to come, quantum safe (though not necessarily every “layer”)
|
||||
• Data should be secure for years to come
|
||||
• Decentralization
|
||||
* Avoid browser-based exploits that plague similar software
|
||||
* Avoid timing attacks & unexpected metadata leaks
|
||||
## Assumptions:
|
||||
• Tor & I2P’s transport protocols & AES-256 are not broken, sha3-512 2nd preimage attacks will remain infeasible indefinitely
|
||||
• All traffic is logged indefinitely by powerful adversaries
|
||||
|
||||
## Protocol
|
||||
Clients MUST use HTTP(s) to communicate with one another to maintain compatibility cross platform. HTTPS is recommended, but HTTP is acceptable because Tor & I2P provide transport layer security.
|
||||
|
||||
Onionr nodes use HTTP (over Tor/I2P) to exchange keys, metadata, and blocks. Blocks are identified by their sha3_256 hash. Nodes sync a table of blocks hashes and attempt to download blocks they do not yet have from random peers.
|
||||
|
||||
## Connections
|
||||
When a node first comes online, it attempts to bootstrap using a default list provided by a client.
|
||||
When two peers connect, they exchange PGP public keys and then generate a shared AES-SHA3-512 HMAC token. These keys are stored in a peer database until expiry.
|
||||
HMAC tokens are regenerated either every X many communications with a peer or every X minutes. Every 10MB or every 2 hours is a recommended default.
|
||||
All valid requests with HMAC should be recorded until used HMAC's expiry to prevent replay attacks.
|
||||
Peer Types
|
||||
* Friends:
|
||||
* Encrypted ‘friends only’ posts to one another
|
||||
* Usually less strict rate & storage limits
|
||||
* OPTIONALLY sign one another’s keys. Users may not want to do this in order to avoid exposing their entire friends list.
|
||||
• Strangers:
|
||||
* Used for storage of encrypted or public information
|
||||
* Can only read public posts
|
||||
* Usually stricter rate & storage limits
|
||||
## Data Storage/Delivery
|
||||
|
||||
Posts (public or friends only) are stored across the network.
|
||||
Private messages SHOULD be delivered directly if both peers are online, otherwise stored in the network.
|
||||
Data SHOULD be stored in an entirely encrypted state when a client is offline, including metadata. Data SHOULD be stored in a minimal size with garbage data to ensure some level of plausible deniablity.
|
||||
Data SHOULD be stored as long as the node’s user prefers and only erased once disk quota is reached due to new data.
|
||||
Posts
|
||||
Posts can contain text and images. All posts MUST be time stamped.
|
||||
Images SHOULD not be displayed by non-friends by default, to prevent unwanted viewing of offensive material & to reduce attack surface.
|
||||
All received posts must be verified to be stored and/or displayed to the user.
|
||||
When a node first comes online, it attempts to bootstrap using a default list provided by a client.
|
||||
When two peers connect, they exchange Ed25519 keys (if applicable) then Salsa20 keys.
|
||||
|
||||
All data being transfered MUST be encrypted to the end node receiving the data, then the data MUST be encrypted the node(s) transporting/storing the data,
|
||||
Salsa20 keys are regenerated either every X many communications with a peer or every X minutes.
|
||||
|
||||
Posts have two settings:
|
||||
• Friends only:
|
||||
◦ Posts MUST be encrypted to all trusted peers via AES256-HMAC-SHA256 and PGP signed (signed before encryption) and time stamped to prevent replaying. A temporary RSA key for use in every post (or message) is exchanged every X many configured post (or message), for use in addition with PGP and the HMAC.
|
||||
• Public:
|
||||
◦ Posts MUST be PGP signed, and MUST NOT use any encryption.
|
||||
## Private Messages
|
||||
Every 100kb or every 2 hours is a recommended default.
|
||||
|
||||
Private messages are messages that can have attached images. They MUST be encrypted via AES256-HMAC-SHA256 and PGP signed (signed before encryption) and time stamped to prevent replaying. A temporary EdDSA key for use in every message is exchanged every X many configured messages (or posts), for use in addition with PGP and the HMAC.
|
||||
When both peers are online messages SHOULD be dispatched directly between peers.
|
||||
All messages must be verified prior to being displayed.
|
||||
All valid requests with HMAC should be recorded until used HMAC's expiry to prevent replay attacks.
|
||||
Peer Types
|
||||
* Friends:
|
||||
* Encrypted ‘friends only’ posts to one another
|
||||
* Usually less strict rate & storage limits
|
||||
* Strangers:
|
||||
* Used for storage of encrypted or public information
|
||||
* Can only read public posts
|
||||
* Usually stricter rate & storage limits
|
||||
|
||||
Clients SHOULD allow configurable message padding.
|
||||
## Spam mitigation
|
||||
|
||||
To send or receive data, a node can optionally request that the other node generate a hash that when in hexadecimal representation contains a random string at a random location in the string. Clients will configure what difficulty to request, and what difficulty is acceptable for themselves to perform. Difficulty should correlate with recent network & disk usage and data size. Friends can be configured to have less strict (to non existent) limits, separately from strangers. (proof of work).
|
||||
|
|
|
|||
|
|
@ -20,7 +20,7 @@
|
|||
import flask
|
||||
from flask import request, Response, abort
|
||||
from multiprocessing import Process
|
||||
import configparser, sys, random, threading, hmac, hashlib, base64, time, math, gnupg, os, logger
|
||||
import configparser, sys, random, threading, hmac, hashlib, base64, time, math, os, logger
|
||||
|
||||
from core import Core
|
||||
import onionrutils, onionrcrypto
|
||||
|
|
@ -140,8 +140,6 @@ class API:
|
|||
resp = Response(self._utils.getBlockDBHash())
|
||||
elif action == 'getBlockHashes':
|
||||
resp = Response(self._core.getBlockList())
|
||||
elif action == 'getPGP':
|
||||
resp = Response(self._utils.exportMyPubkey())
|
||||
# setData should be something the communicator initiates, not this api
|
||||
elif action == 'getData':
|
||||
resp = self._core.getData(data)
|
||||
|
|
|
|||
|
|
@ -40,13 +40,6 @@ class OnionrCommunicate:
|
|||
|
||||
self.peerData = {} # Session data for peers (recent reachability, speed, etc)
|
||||
|
||||
# get our own PGP fingerprint
|
||||
fingerprintFile = 'data/own-fingerprint.txt'
|
||||
if not os.path.exists(fingerprintFile):
|
||||
self._core.generateMainPGP(torID)
|
||||
with open(fingerprintFile,'r') as f:
|
||||
self.pgpOwnFingerprint = f.read()
|
||||
logger.info('My PGP fingerprint is ' + logger.colors.underline + self.pgpOwnFingerprint + logger.colors.reset + logger.colors.fg.green + '.')
|
||||
if os.path.exists(self._core.queueDB):
|
||||
self._core.clearDaemonQueue()
|
||||
while True:
|
||||
|
|
|
|||
|
|
@ -17,7 +17,7 @@
|
|||
You should have received a copy of the GNU General Public License
|
||||
along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
'''
|
||||
import sqlite3, os, sys, time, math, gnupg, base64, tarfile, getpass, simplecrypt, hashlib, nacl, logger
|
||||
import sqlite3, os, sys, time, math, base64, tarfile, getpass, simplecrypt, hashlib, nacl, logger
|
||||
#from Crypto.Cipher import AES
|
||||
#from Crypto import Random
|
||||
import netcontroller
|
||||
|
|
@ -38,10 +38,8 @@ class Core:
|
|||
'''
|
||||
self.queueDB = 'data/queue.db'
|
||||
self.peerDB = 'data/peers.db'
|
||||
self.ownPGPID = ''
|
||||
self.blockDB = 'data/blocks.db'
|
||||
self.blockDataLocation = 'data/blocks/'
|
||||
self.gpgHome = './data/pgp/'
|
||||
self._utils = onionrutils.OnionrUtils(self)
|
||||
self._crypto = onionrcrypto.OnionrCrypto(self)
|
||||
|
||||
|
|
@ -55,28 +53,6 @@ class Core:
|
|||
|
||||
return
|
||||
|
||||
def generateMainPGP(self, myID):
|
||||
'''
|
||||
Generate the main PGP key for our client. Should not be done often.
|
||||
|
||||
Uses own PGP home folder in the data/ directory
|
||||
'''
|
||||
gpg = gnupg.GPG(homedir=self.gpgHome)
|
||||
input_data = gpg.gen_key_input(key_type="RSA", key_length=1024, name_real=myID, name_email='anon@onionr', testing=True)
|
||||
key = gpg.gen_key(input_data)
|
||||
logger.info("Generating PGP key, this will take some time..")
|
||||
while key.status != "key created":
|
||||
time.sleep(0.5)
|
||||
print(key.status)
|
||||
|
||||
logger.info("Finished generating PGP key")
|
||||
# Write the key
|
||||
myFingerpintFile = open('data/own-fingerprint.txt', 'w')
|
||||
myFingerpintFile.write(key.fingerprint)
|
||||
myFingerpintFile.close()
|
||||
|
||||
return
|
||||
|
||||
def addPeer(self, peerID, name=''):
|
||||
'''
|
||||
Add a peer by their ID, with an optional name, to the peer database
|
||||
|
|
@ -104,8 +80,7 @@ class Core:
|
|||
c.execute('''CREATE TABLE peers(
|
||||
ID text not null,
|
||||
name text,
|
||||
pgpKey text,
|
||||
hmacKey text,
|
||||
pubkey text,
|
||||
blockDBHash text,
|
||||
forwardKey text,
|
||||
dateSeen not null,
|
||||
|
|
@ -335,7 +310,6 @@ class Core:
|
|||
|
||||
id text 0
|
||||
name text, 1
|
||||
pgpKey text, 2
|
||||
hmacKey text, 3
|
||||
blockDBHash text, 4
|
||||
forwardKey text, 5
|
||||
|
|
@ -346,7 +320,7 @@ class Core:
|
|||
conn = sqlite3.connect(self.peerDB)
|
||||
c = conn.cursor()
|
||||
command = (peer,)
|
||||
infoNumbers = {'id': 0, 'name': 1, 'pgpKey': 2, 'hmacKey': 3, 'blockDBHash': 4, 'forwardKey': 5, 'dateSeen': 6, 'bytesStored': 7, 'trust': 8}
|
||||
infoNumbers = {'id': 0, 'name': 1, 'hmacKey': 3, 'blockDBHash': 4, 'forwardKey': 5, 'dateSeen': 6, 'bytesStored': 7, 'trust': 8}
|
||||
info = infoNumbers[info]
|
||||
iterCount = 0
|
||||
retVal = ''
|
||||
|
|
@ -369,7 +343,7 @@ class Core:
|
|||
c = conn.cursor()
|
||||
command = (data, peer)
|
||||
# TODO: validate key on whitelist
|
||||
if key not in ('id', 'text', 'name', 'pgpKey', 'hmacKey', 'blockDBHash', 'forwardKey', 'dateSeen', 'bytesStored', 'trust'):
|
||||
if key not in ('id', 'name', 'pubkey', 'blockDBHash', 'forwardKey', 'dateSeen', 'bytesStored', 'trust'):
|
||||
raise Exception("Got invalid database key when setting peer info")
|
||||
c.execute('UPDATE peers SET ' + key + ' = ? WHERE id=?', command)
|
||||
conn.commit()
|
||||
|
|
|
|||
|
|
@ -17,7 +17,7 @@
|
|||
You should have received a copy of the GNU General Public License
|
||||
along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
'''
|
||||
import nacl, gnupg
|
||||
import nacl
|
||||
|
||||
class OnionrCrypto:
|
||||
def __init__(self, coreInstance):
|
||||
|
|
@ -30,14 +30,8 @@ class OnionrCrypto:
|
|||
def symmetricPeerDecrypt(self, data, key):
|
||||
return
|
||||
|
||||
def rsaEncrypt(self, peer, data):
|
||||
return
|
||||
|
||||
def verifyPGP(self, peer, signature):
|
||||
'''Verify PGP signed data'''
|
||||
gpg = gnupg.GPG(homedir=self._core.gpgHome)
|
||||
|
||||
def generateSymmetric():
|
||||
return
|
||||
|
||||
def generateHMAC():
|
||||
return
|
||||
|
|
@ -18,7 +18,7 @@
|
|||
along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
'''
|
||||
# Misc functions that do not fit in the main api, but are useful
|
||||
import getpass, sys, requests, configparser, os, socket, gnupg, hashlib, logger, sqlite3
|
||||
import getpass, sys, requests, configparser, os, socket, hashlib, logger, sqlite3
|
||||
if sys.version_info < (3, 6):
|
||||
try:
|
||||
import sha3
|
||||
|
|
@ -93,19 +93,6 @@ class OnionrUtils:
|
|||
else:
|
||||
return True
|
||||
|
||||
def exportMyPubkey(self):
|
||||
'''
|
||||
Export our PGP key if it exists
|
||||
'''
|
||||
if not os.path.exists(self.fingerprintFile):
|
||||
raise Exception("No fingerprint found, cannot export our PGP key.")
|
||||
gpg = gnupg.GPG(homedir='./data/pgp/')
|
||||
with open(self.fingerprintFile,'r') as f:
|
||||
fingerprint = f.read()
|
||||
ascii_armored_public_keys = gpg.export_keys(fingerprint)
|
||||
|
||||
return ascii_armored_public_keys
|
||||
|
||||
def getBlockDBHash(self):
|
||||
'''
|
||||
Return a sha3_256 hash of the blocks DB
|
||||
|
|
@ -154,17 +141,6 @@ class OnionrUtils:
|
|||
|
||||
return retVal
|
||||
|
||||
def getPeerPGPFingerprint(self, peer):
|
||||
'''
|
||||
Get peer's PGP fingerprint
|
||||
'''
|
||||
retData = ''
|
||||
gpg = gnupg.GPG(homedir=self._core.gpgHome)
|
||||
for i in gpg.list_keys():
|
||||
if peer in i['uids'][0]:
|
||||
retData = i['fingerprint']
|
||||
return retData
|
||||
|
||||
def validateID(self, id):
|
||||
'''
|
||||
Validate if a user ID is a valid tor or i2p hidden service
|
||||
|
|
|
|||
|
|
@ -85,33 +85,6 @@ class OnionrTests(unittest.TestCase):
|
|||
else:
|
||||
self.assertTrue(False)
|
||||
|
||||
def testPGPGen(self):
|
||||
logger.debug('--------------------------')
|
||||
logger.info('Running PGP key generation test...')
|
||||
if os.path.exists('data/pgp/'):
|
||||
self.assertTrue(True)
|
||||
else:
|
||||
import core, netcontroller
|
||||
myCore = core.Core()
|
||||
net = netcontroller.NetController(1337)
|
||||
net.startTor()
|
||||
torID = open('data/hs/hostname').read()
|
||||
myCore.generateMainPGP(torID)
|
||||
if os.path.exists('data/pgp/'):
|
||||
self.assertTrue(True)
|
||||
|
||||
def testHMACGen(self):
|
||||
logger.debug('--------------------------')
|
||||
logger.info('Running HMAC generation test...')
|
||||
# Test if hmac key generation is working
|
||||
import core
|
||||
myCore = core.Core()
|
||||
key = myCore.generateHMAC()
|
||||
if len(key) > 10:
|
||||
self.assertTrue(True)
|
||||
else:
|
||||
self.assertTrue(False)
|
||||
|
||||
def testQueue(self):
|
||||
logger.debug('--------------------------')
|
||||
logger.info('Running daemon queue test...')
|
||||
|
|
|
|||
|
|
@ -1,10 +1,7 @@
|
|||
PyNaCl==1.2.1
|
||||
gnupg==2.3.1
|
||||
requests==2.12.4
|
||||
Flask==0.12.2
|
||||
requests==2.18.4
|
||||
urllib3==1.22
|
||||
simple_crypt==4.1.7
|
||||
urllib3==1.19.1
|
||||
sha3==0.2.1
|
||||
pycrypto==2.6.1
|
||||
pynacl==1.2.1
|
||||
PySocks==1.6.8
|
||||
|
|
|
|||
Loading…
Reference in New Issue