Merge branch 'crypto' of github.com:beardog108/onionr into crypto

master
Kevin Froman 2018-02-17 13:04:38 -06:00
commit b6cfe0154d
8 changed files with 36 additions and 154 deletions

View File

@ -1,72 +1,49 @@
# Onionr Protocol Spec
# Onionr Protocol Spec v2
A social network/microblogging platform for Tor & I2P
Draft Dec 25 2017
A P2P platform for Tor & I2P
# Overview
Onionr is an encrypted microblogging & mailing system designed in the spirit of Twitter.
There are no central servers and all traffic is peer to peer by default (routed via Tor or I2P).
User IDs are simply Tor onion service/I2P host id + PGP fingerprint.
Clients consolidate feeds from peers into 1 “timeline” using RSS format.
Private messages are only accessible by the intended peer based on the PGP id.
Onionr is not intended to be a replacement for Ricochet, OnionShare, or Briar.
All traffic is over onion/I2P because if only some was, then that would make that traffic inherently suspicious.
User IDs are simply Tor onion service/I2P host id + Ed25519 key fingerprint.
Private blocks are only able to be read by the intended peer.
All traffic is over Tor/I2P, connecting only to Tor onion and I2P hidden services.
## Goals:
• Selective sharing of information with friends & public
• Selective sharing of information
• Secure & semi-anonymous direct messaging
• Forward secrecy
• Defense in depth
• Data should be secure for years to come, quantum safe (though not necessarily every “layer”)
• Data should be secure for years to come
• Decentralization
* Avoid browser-based exploits that plague similar software
* Avoid timing attacks & unexpected metadata leaks
## Assumptions:
• Tor & I2Ps transport protocols & AES-256 are not broken, sha3-512 2nd preimage attacks will remain infeasible indefinitely
• All traffic is logged indefinitely by powerful adversaries
## Protocol
Clients MUST use HTTP(s) to communicate with one another to maintain compatibility cross platform. HTTPS is recommended, but HTTP is acceptable because Tor & I2P provide transport layer security.
Onionr nodes use HTTP (over Tor/I2P) to exchange keys, metadata, and blocks. Blocks are identified by their sha3_256 hash. Nodes sync a table of blocks hashes and attempt to download blocks they do not yet have from random peers.
## Connections
When a node first comes online, it attempts to bootstrap using a default list provided by a client.
When two peers connect, they exchange PGP public keys and then generate a shared AES-SHA3-512 HMAC token. These keys are stored in a peer database until expiry.
HMAC tokens are regenerated either every X many communications with a peer or every X minutes. Every 10MB or every 2 hours is a recommended default.
All valid requests with HMAC should be recorded until used HMAC's expiry to prevent replay attacks.
Peer Types
* Friends:
* Encrypted friends only posts to one another
* Usually less strict rate & storage limits
* OPTIONALLY sign one anothers keys. Users may not want to do this in order to avoid exposing their entire friends list.
• Strangers:
* Used for storage of encrypted or public information
* Can only read public posts
* Usually stricter rate & storage limits
## Data Storage/Delivery
Posts (public or friends only) are stored across the network.
Private messages SHOULD be delivered directly if both peers are online, otherwise stored in the network.
Data SHOULD be stored in an entirely encrypted state when a client is offline, including metadata. Data SHOULD be stored in a minimal size with garbage data to ensure some level of plausible deniablity.
Data SHOULD be stored as long as the nodes user prefers and only erased once disk quota is reached due to new data.
Posts
Posts can contain text and images. All posts MUST be time stamped.
Images SHOULD not be displayed by non-friends by default, to prevent unwanted viewing of offensive material & to reduce attack surface.
All received posts must be verified to be stored and/or displayed to the user.
When a node first comes online, it attempts to bootstrap using a default list provided by a client.
When two peers connect, they exchange Ed25519 keys (if applicable) then Salsa20 keys.
All data being transfered MUST be encrypted to the end node receiving the data, then the data MUST be encrypted the node(s) transporting/storing the data,
Salsa20 keys are regenerated either every X many communications with a peer or every X minutes.
Posts have two settings:
• Friends only:
◦ Posts MUST be encrypted to all trusted peers via AES256-HMAC-SHA256 and PGP signed (signed before encryption) and time stamped to prevent replaying. A temporary RSA key for use in every post (or message) is exchanged every X many configured post (or message), for use in addition with PGP and the HMAC.
• Public:
◦ Posts MUST be PGP signed, and MUST NOT use any encryption.
## Private Messages
Every 100kb or every 2 hours is a recommended default.
Private messages are messages that can have attached images. They MUST be encrypted via AES256-HMAC-SHA256 and PGP signed (signed before encryption) and time stamped to prevent replaying. A temporary EdDSA key for use in every message is exchanged every X many configured messages (or posts), for use in addition with PGP and the HMAC.
When both peers are online messages SHOULD be dispatched directly between peers.
All messages must be verified prior to being displayed.
All valid requests with HMAC should be recorded until used HMAC's expiry to prevent replay attacks.
Peer Types
* Friends:
* Encrypted friends only posts to one another
* Usually less strict rate & storage limits
* Strangers:
* Used for storage of encrypted or public information
* Can only read public posts
* Usually stricter rate & storage limits
Clients SHOULD allow configurable message padding.
## Spam mitigation
To send or receive data, a node can optionally request that the other node generate a hash that when in hexadecimal representation contains a random string at a random location in the string. Clients will configure what difficulty to request, and what difficulty is acceptable for themselves to perform. Difficulty should correlate with recent network & disk usage and data size. Friends can be configured to have less strict (to non existent) limits, separately from strangers. (proof of work).
Rate limits can be strict, as Onionr is not intended to be an instant messaging application.
Rate limits can be strict, as Onionr is not intended to be an instant messaging application.

View File

@ -20,7 +20,7 @@
import flask
from flask import request, Response, abort
from multiprocessing import Process
import configparser, sys, random, threading, hmac, hashlib, base64, time, math, gnupg, os, logger
import configparser, sys, random, threading, hmac, hashlib, base64, time, math, os, logger
from core import Core
import onionrutils, onionrcrypto
@ -140,8 +140,6 @@ class API:
resp = Response(self._utils.getBlockDBHash())
elif action == 'getBlockHashes':
resp = Response(self._core.getBlockList())
elif action == 'getPGP':
resp = Response(self._utils.exportMyPubkey())
# setData should be something the communicator initiates, not this api
elif action == 'getData':
resp = self._core.getData(data)

View File

@ -40,13 +40,6 @@ class OnionrCommunicate:
self.peerData = {} # Session data for peers (recent reachability, speed, etc)
# get our own PGP fingerprint
fingerprintFile = 'data/own-fingerprint.txt'
if not os.path.exists(fingerprintFile):
self._core.generateMainPGP(torID)
with open(fingerprintFile,'r') as f:
self.pgpOwnFingerprint = f.read()
logger.info('My PGP fingerprint is ' + logger.colors.underline + self.pgpOwnFingerprint + logger.colors.reset + logger.colors.fg.green + '.')
if os.path.exists(self._core.queueDB):
self._core.clearDaemonQueue()
while True:

View File

@ -17,7 +17,7 @@
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
'''
import sqlite3, os, sys, time, math, gnupg, base64, tarfile, getpass, simplecrypt, hashlib, nacl, logger
import sqlite3, os, sys, time, math, base64, tarfile, getpass, simplecrypt, hashlib, nacl, logger
#from Crypto.Cipher import AES
#from Crypto import Random
import netcontroller
@ -38,10 +38,8 @@ class Core:
'''
self.queueDB = 'data/queue.db'
self.peerDB = 'data/peers.db'
self.ownPGPID = ''
self.blockDB = 'data/blocks.db'
self.blockDataLocation = 'data/blocks/'
self.gpgHome = './data/pgp/'
self._utils = onionrutils.OnionrUtils(self)
self._crypto = onionrcrypto.OnionrCrypto(self)
@ -55,28 +53,6 @@ class Core:
return
def generateMainPGP(self, myID):
'''
Generate the main PGP key for our client. Should not be done often.
Uses own PGP home folder in the data/ directory
'''
gpg = gnupg.GPG(homedir=self.gpgHome)
input_data = gpg.gen_key_input(key_type="RSA", key_length=1024, name_real=myID, name_email='anon@onionr', testing=True)
key = gpg.gen_key(input_data)
logger.info("Generating PGP key, this will take some time..")
while key.status != "key created":
time.sleep(0.5)
print(key.status)
logger.info("Finished generating PGP key")
# Write the key
myFingerpintFile = open('data/own-fingerprint.txt', 'w')
myFingerpintFile.write(key.fingerprint)
myFingerpintFile.close()
return
def addPeer(self, peerID, name=''):
'''
Add a peer by their ID, with an optional name, to the peer database
@ -104,8 +80,7 @@ class Core:
c.execute('''CREATE TABLE peers(
ID text not null,
name text,
pgpKey text,
hmacKey text,
pubkey text,
blockDBHash text,
forwardKey text,
dateSeen not null,
@ -335,7 +310,6 @@ class Core:
id text 0
name text, 1
pgpKey text, 2
hmacKey text, 3
blockDBHash text, 4
forwardKey text, 5
@ -346,7 +320,7 @@ class Core:
conn = sqlite3.connect(self.peerDB)
c = conn.cursor()
command = (peer,)
infoNumbers = {'id': 0, 'name': 1, 'pgpKey': 2, 'hmacKey': 3, 'blockDBHash': 4, 'forwardKey': 5, 'dateSeen': 6, 'bytesStored': 7, 'trust': 8}
infoNumbers = {'id': 0, 'name': 1, 'hmacKey': 3, 'blockDBHash': 4, 'forwardKey': 5, 'dateSeen': 6, 'bytesStored': 7, 'trust': 8}
info = infoNumbers[info]
iterCount = 0
retVal = ''
@ -369,7 +343,7 @@ class Core:
c = conn.cursor()
command = (data, peer)
# TODO: validate key on whitelist
if key not in ('id', 'text', 'name', 'pgpKey', 'hmacKey', 'blockDBHash', 'forwardKey', 'dateSeen', 'bytesStored', 'trust'):
if key not in ('id', 'name', 'pubkey', 'blockDBHash', 'forwardKey', 'dateSeen', 'bytesStored', 'trust'):
raise Exception("Got invalid database key when setting peer info")
c.execute('UPDATE peers SET ' + key + ' = ? WHERE id=?', command)
conn.commit()

View File

@ -17,7 +17,7 @@
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
'''
import nacl, gnupg
import nacl
class OnionrCrypto:
def __init__(self, coreInstance):
@ -29,15 +29,9 @@ class OnionrCrypto:
def symmetricPeerDecrypt(self, data, key):
return
def rsaEncrypt(self, peer, data):
return
def verifyPGP(self, peer, signature):
'''Verify PGP signed data'''
gpg = gnupg.GPG(homedir=self._core.gpgHome)
def generateSymmetric():
return
def generateHMAC():
return

View File

@ -18,7 +18,7 @@
along with this program. If not, see <https://www.gnu.org/licenses/>.
'''
# Misc functions that do not fit in the main api, but are useful
import getpass, sys, requests, configparser, os, socket, gnupg, hashlib, logger, sqlite3
import getpass, sys, requests, configparser, os, socket, hashlib, logger, sqlite3
if sys.version_info < (3, 6):
try:
import sha3
@ -93,19 +93,6 @@ class OnionrUtils:
else:
return True
def exportMyPubkey(self):
'''
Export our PGP key if it exists
'''
if not os.path.exists(self.fingerprintFile):
raise Exception("No fingerprint found, cannot export our PGP key.")
gpg = gnupg.GPG(homedir='./data/pgp/')
with open(self.fingerprintFile,'r') as f:
fingerprint = f.read()
ascii_armored_public_keys = gpg.export_keys(fingerprint)
return ascii_armored_public_keys
def getBlockDBHash(self):
'''
Return a sha3_256 hash of the blocks DB
@ -153,17 +140,6 @@ class OnionrUtils:
retVal = False
return retVal
def getPeerPGPFingerprint(self, peer):
'''
Get peer's PGP fingerprint
'''
retData = ''
gpg = gnupg.GPG(homedir=self._core.gpgHome)
for i in gpg.list_keys():
if peer in i['uids'][0]:
retData = i['fingerprint']
return retData
def validateID(self, id):
'''

View File

@ -85,33 +85,6 @@ class OnionrTests(unittest.TestCase):
else:
self.assertTrue(False)
def testPGPGen(self):
logger.debug('--------------------------')
logger.info('Running PGP key generation test...')
if os.path.exists('data/pgp/'):
self.assertTrue(True)
else:
import core, netcontroller
myCore = core.Core()
net = netcontroller.NetController(1337)
net.startTor()
torID = open('data/hs/hostname').read()
myCore.generateMainPGP(torID)
if os.path.exists('data/pgp/'):
self.assertTrue(True)
def testHMACGen(self):
logger.debug('--------------------------')
logger.info('Running HMAC generation test...')
# Test if hmac key generation is working
import core
myCore = core.Core()
key = myCore.generateHMAC()
if len(key) > 10:
self.assertTrue(True)
else:
self.assertTrue(False)
def testQueue(self):
logger.debug('--------------------------')
logger.info('Running daemon queue test...')

View File

@ -1,10 +1,7 @@
PyNaCl==1.2.1
gnupg==2.3.1
requests==2.12.4
Flask==0.12.2
requests==2.18.4
urllib3==1.22
simple_crypt==4.1.7
urllib3==1.19.1
sha3==0.2.1
pycrypto==2.6.1
pynacl==1.2.1
PySocks==1.6.8