Merge branch 'crypto'

.gitignore

@@ -6,3 +6,6 @@ onionr/*.pyc
 onionr/*.log
 onionr/data/hs/hostname
 onionr/data/*
+onionr/data-backup/*
+onionr/gnupg/*
+run.sh

.gitmodules

@@ -0,0 +1,3 @@
+[submodule "onionr/bitpeer"]
+	path = onionr/bitpeer
+	url = https://github.com/beardog108/bitpeer.py

.travis.yml

@@ -5,4 +5,4 @@ python:
 install:
         - sudo apt install gnupg tor
         - pip install -r requirements.txt
-script: ./test.sh
+script: make test

Makefile

@@ -0,0 +1,29 @@
+.DEFAULT_GOAL := setup
+
+setup:
+	sudo pip3 install -r requirements.txt
+
+install:
+	sudo rm -rf /usr/share/onionr/
+	sudo rm -f /usr/bin/onionr
+	sudo cp -rp ./onionr /usr/share/onionr
+	sudo sh -c "echo \"#!/bin/sh\ncd /usr/share/onionr/\n./onionr.py \\\"\\\$$@\\\"\" > /usr/bin/onionr"
+	sudo chmod +x /usr/bin/onionr
+	sudo chown -R `whoami` /usr/share/onionr/
+
+uninstall:
+	sudo rm -rf /usr/share/onionr
+	sudo rm -f /usr/bin/onionr
+
+test:
+	@rm -rf onionr/data-backup
+	@mv onionr/data onionr/data-backup | true > /dev/null 2>&1
+	-@cd onionr; ./tests.py
+	@rm -rf onionr/data
+	@mv onionr/data-backup onionr/data | true > /dev/null 2>&1
+
+reset:
+	rm -f onionr/data/blocks/*.dat | true > /dev/null 2>&1
+	rm -f onionr/data/peers.db | true > /dev/null 2>&1
+	rm -f onionr/data/blocks.db | true > /dev/null 2>&1
+	rm -f onionr/data/address.db | true > /dev/null 2>&1

docs/api.md

@@ -1,3 +1,9 @@
+BLOCK HEADERS (simple ID system to identify block type)
+-----------------------------------------------
+-crypt- (encrypted block)
+-bin- (binary file)
+-txt- (plaintext)
+
 HTTP API
 ------------------------------------------------
 /client/ (Private info, not publicly accessible)

docs/onionr-draft.md

@@ -1,71 +1,50 @@
-# Onionr Protocol Spec
+# Onionr Protocol Spec v2
 
-A social network/microblogging platform for Tor & I2P
+A P2P platform for Tor & I2P
-
-Draft Dec 25 2017
 
 # Overview
 
 Onionr is an encrypted microblogging & mailing system designed in the spirit of Twitter.
 There are no central servers and all traffic is peer to peer by default (routed via Tor or I2P).
-User IDs are simply Tor onion service/I2P host id + PGP fingerprint.
+User IDs are simply Tor onion service/I2P host id + Ed25519 key fingerprint.
-Clients consolidate feeds from peers into 1 “timeline” using RSS format.
+Private blocks are only able to be read by the intended peer.
-Private messages are only accessible by the intended peer based on the PGP id.
+All traffic is over Tor/I2P, connecting only to Tor onion and I2P hidden services.
-Onionr is not intended to be a replacement for Ricochet, OnionShare, or Briar.
+
-All traffic is over onion/I2P because if only some was, then that would make that traffic inherently suspicious.
 ## Goals:
-    • Selective sharing of information with friends & public
+    • Selective sharing of information
     • Secure & semi-anonymous direct messaging
     • Forward secrecy
     • Defense in depth
-    • Data should be secure for years to come, quantum safe (though not necessarily every “layer”)
+    • Data should be secure for years to come
     • Decentralization
     * Avoid browser-based exploits that plague similar software
     * Avoid timing attacks & unexpected metadata leaks
-## Assumptions:
+
-    • Tor & I2P’s transport protocols & AES-256 are not broken, sha3-512 2nd preimage attacks will remain infeasible indefinitely
-    • All traffic is logged indefinitely by powerful adversaries
 ## Protocol
-Clients MUST use HTTP(s) to communicate with one another to maintain compatibility cross platform. HTTPS is recommended, but HTTP is acceptable because Tor & I2P provide transport layer security.
+
+Onionr nodes use HTTP (over Tor/I2P) to exchange keys, metadata, and blocks. Blocks are identified by their sha3_256 hash. Nodes sync a table of blocks hashes and attempt to download blocks they do not yet have from random peers.
+
+Blocks may be encrypted using Curve25519.
+
 ## Connections
-    When a node first comes online, it attempts to bootstrap using a default list provided by a client.
+
-    When two peers connect, they exchange PGP public keys and then generate a shared AES-SHA3-512 HMAC token. These keys are stored in a peer database until expiry.
+When a node first comes online, it attempts to bootstrap using a default list provided by a client.
-    HMAC tokens are regenerated either every X many communications with a peer or every X minutes. Every 10MB or every 2 hours is a recommended default.
+When two peers connect, they exchange Ed25519 keys (if applicable) then Salsa20 keys.
-    All valid requests with HMAC should be recorded until used HMAC's expiry to prevent replay attacks.
+
-    Peer Types
+Salsa20 keys are regenerated either every X many communications with a peer or every X minutes. 
+
+Every 100kb or every 2 hours is a recommended default.
+
+All valid requests with HMAC should be recorded until used HMAC's expiry to prevent replay attacks.
+Peer Types
     * Friends:
         * Encrypted ‘friends only’ posts to one another
         * Usually less strict rate & storage limits
-            * OPTIONALLY sign one another’s keys. Users may not want to do this in order to avoid exposing their entire friends list.
+    * Strangers:
-        • Strangers:
         * Used for storage of encrypted or public information
         * Can only read public posts
         * Usually stricter rate & storage limits
-## Data Storage/Delivery
 
-    Posts (public or friends only) are stored across the network.
-    Private messages SHOULD be delivered directly if both peers are online, otherwise stored in the network.
-    Data SHOULD be stored in an entirely encrypted state when a client is offline, including metadata. Data SHOULD be stored in a minimal size with garbage data to ensure some level of plausible deniablity.
-    Data SHOULD be stored as long as the node’s user prefers and only erased once disk quota is reached due to new data.
-    Posts
-    Posts can contain text and images. All posts MUST be time stamped.
-    Images SHOULD not be displayed by non-friends by default, to prevent unwanted viewing of offensive material & to reduce attack surface.
-    All received posts must be verified to be stored and/or displayed to the user.
-
-    All data being transfered MUST be encrypted to the end node receiving the data, then the data MUST be encrypted the node(s) transporting/storing the data,
-
-     Posts have two settings:
-        • Friends only:
-            ◦ Posts MUST be encrypted to all trusted peers via AES256-HMAC-SHA256 and PGP signed (signed before encryption) and time stamped to prevent replaying. A temporary RSA key for use in every post (or message) is exchanged every X many configured post (or message), for use in addition with PGP and the HMAC.
-        • Public:
-            ◦ Posts MUST be PGP signed, and MUST NOT use any encryption.
-## Private Messages
-
-    Private messages are messages that can have attached images. They MUST be encrypted via AES256-HMAC-SHA256 and PGP signed (signed before encryption) and time stamped to prevent replaying. A temporary EdDSA key for use in every message is exchanged every X many configured messages (or posts), for use in addition with PGP and the HMAC.
-    When both peers are online messages SHOULD be dispatched directly between peers.
-    All messages must be verified prior to being displayed.
-
-    Clients SHOULD allow configurable message padding.
 ## Spam mitigation
 
 To send or receive data, a node can optionally request that the other node generate a hash that when in hexadecimal representation contains a random string at a random location in the string. Clients will configure what difficulty to request, and what difficulty is acceptable for themselves to perform. Difficulty should correlate with recent network & disk usage and data size. Friends can be configured to have less strict (to non existent) limits, separately from strangers. (proof of work).

onionr/api.py

@@ -20,44 +20,50 @@
 import flask
 from flask import request, Response, abort
 from multiprocessing import Process
-import configparser, sys, random, threading, hmac, hashlib, base64, time, math, gnupg, os, logger
+import sys, random, threading, hmac, hashlib, base64, time, math, os, logger, config
 
 from core import Core
-import onionrutils
+import onionrutils, onionrcrypto
 class API:
-    ''' Main http api (flask)'''
+    '''
+        Main HTTP API (Flask)
+    '''
     def validateToken(self, token):
         '''
-        Validate if the client token (hmac) matches the given token
+            Validate that the client token (hmac) matches the given token
         '''
         if self.clientToken != token:
             return False
         else:
             return True
 
-    def __init__(self, config, debug):
+    def __init__(self, debug):
-        ''' Initialize the api server, preping variables for later use
+        '''
-        This initilization defines all of the API entry points and handlers for the endpoints and errors
+            Initialize the api server, preping variables for later use
 
+            This initilization defines all of the API entry points and handlers for the endpoints and errors
             This also saves the used host (random localhost IP address) to the data folder in host.txt
         '''
-        if os.path.exists('dev-enabled'):
+
+        config.reload()
+
+        if config.get('devmode', True):
             self._developmentMode = True
             logger.set_level(logger.LEVEL_DEBUG)
-            logger.warn('DEVELOPMENT MODE ENABLED (THIS IS LESS SECURE!)')
         else:
             self._developmentMode = False
             logger.set_level(logger.LEVEL_INFO)
 
-        self.config = config
         self.debug = debug
         self._privateDelayTime = 3
         self._core = Core()
+        self._crypto = onionrcrypto.OnionrCrypto(self._core)
         self._utils = onionrutils.OnionrUtils(self._core)
         app = flask.Flask(__name__)
-        bindPort = int(self.config['CLIENT']['PORT'])
+        bindPort = int(config.get('client')['port'])
         self.bindPort = bindPort
-        self.clientToken = self.config['CLIENT']['CLIENT HMAC']
+        self.clientToken = config.get('client')['client_hmac']
+        if not os.environ.get("WERKZEUG_RUN_MAIN") == "true":
             logger.debug('Your HMAC token: ' + logger.colors.underline + self.clientToken)
 
         if not debug and not self._developmentMode:
@@ -75,6 +81,7 @@ class API:
                 Simply define the request as not having yet failed, before every request.
             '''
             self.requestFailed = False
+
             return
 
         @app.after_request
@@ -87,6 +94,7 @@ class API:
             resp.headers["Content-Security-Policy"] = "default-src 'none'"
             resp.headers['X-Frame-Options'] = 'deny'
             resp.headers['X-Content-Type-Options'] = "nosniff"
+
             return resp
 
         @app.route('/client/')
@@ -112,6 +120,7 @@ class API:
             elapsed = endTime - startTime
             if elapsed < self._privateDelayTime:
                 time.sleep(self._privateDelayTime - elapsed)
+
             return resp
 
         @app.route('/public/')
@@ -125,14 +134,14 @@ class API:
                 pass
             elif action == 'ping':
                 resp = Response("pong!")
-            elif action == 'setHMAC':
+            elif action == 'getHMAC':
-                pass
+                resp = Response(self._crypto.generateSymmetric())
+            elif action == 'getSymmetric':
+                resp = Response(self._crypto.generateSymmetric())
             elif action == 'getDBHash':
                 resp = Response(self._utils.getBlockDBHash())
             elif action == 'getBlockHashes':
                 resp = Response(self._core.getBlockList())
-            elif action == 'getPGP':
-                resp = Response(self._utils.exportMyPubkey())
             # setData should be something the communicator initiates, not this api
             elif action == 'getData':
                 resp = self._core.getData(data)
@@ -140,6 +149,16 @@ class API:
                     abort(404)
                     resp = ""
                 resp = Response(resp)
+            elif action == 'pex':
+                response = ','.join(self._core.listAdders())
+                if len(response) == 0:
+                    response = 'none'
+                resp = Response(response)
+            elif action == 'kex':
+                response = ','.join(self._core.listPeers())
+                if len(response) == 0:
+                    response = 'none'
+                resp = Response(response)
             else:
                 resp = Response("")
 
@@ -149,26 +168,36 @@ class API:
         def notfound(err):
             self.requestFailed = True
             resp = Response("")
-            #resp.headers = getHeaders(resp)
+
             return resp
+
         @app.errorhandler(403)
         def authFail(err):
             self.requestFailed = True
             resp = Response("403")
+
             return resp
+
         @app.errorhandler(401)
         def clientError(err):
             self.requestFailed = True
             resp = Response("Invalid request")
+
             return resp
-
+        if not os.environ.get("WERKZEUG_RUN_MAIN") == "true":
             logger.info('Starting client on ' + self.host + ':' + str(bindPort) + '...')
-        logger.debug('Client token: ' + logger.colors.underline + self.clientToken)
 
+        try:
             app.run(host=self.host, port=bindPort, debug=True, threaded=True)
+        except Exception as e:
+            logger.error(str(e))
+            logger.fatal('Failed to start client on ' + self.host + ':' + str(bindPort) + ', exiting...')
+            exit(1)
 
     def validateHost(self, hostType):
-        ''' Validate various features of the request including:
+        '''
+            Validate various features of the request including:
+
             If private (/client/), is the host header local?
             If public (/public/), is the host header onion or i2p?
 

onionr/bitpeer

@@ -0,0 +1 @@
+Subproject commit a74e826e9c69e643ead7950f9f76a05ab8664ddc

onionr/btc.py

@@ -20,8 +20,9 @@
 from bitpeer.node import *
 from bitpeer.storage.shelve import ShelveStorage
 import logging, time
+import socks, sys
 class OnionrBTC:
-	def __init__(self, lastBlock='00000000000000000021ee6242d08e3797764c9258e54e686bc2afff51baf599', lastHeight=510613):
+	def __init__(self, lastBlock='00000000000000000021ee6242d08e3797764c9258e54e686bc2afff51baf599', lastHeight=510613, torP=9050):
 		stream = logging.StreamHandler()
 		logger = logging.getLogger('halfnode')
 		logger.addHandler(stream)
@@ -29,9 +30,15 @@ class OnionrBTC:
 
 		LASTBLOCK = lastBlock
 		LASTBLOCKINDEX = lastHeight
-		self.node = Node ('BTC', ShelveStorage ('./btc-blocks.db'), lastblockhash=LASTBLOCK, lastblockheight=LASTBLOCKINDEX)
+		self.node = Node ('BTC', ShelveStorage ('data/btc-blocks.db'), lastblockhash=LASTBLOCK, lastblockheight=LASTBLOCKINDEX, torPort=torP)
 
 		self.node.bootstrap ()
 		self.node.connect ()
 		self.node.loop ()
 
+if __name__ == "__main__":
+    torPort = int(sys.argv[1])
+    bitcoin = OnionrBTC(torPort)
+    while True:
+        print(bitcoin.node.getBlockHash(bitcoin.node.getLastBlockHeight())) # Using print on purpose, do not change to logger
+        time.sleep(5)

onionr/colors.py

@@ -1,23 +0,0 @@
-'''
-Simply define terminal control codes (mainly colors)
-'''
-class Colors:
-    def __init__(self):
-        '''
-        PURPLE='\033[95m'
-        BLUE='\033[94m'
-        GREEN='\033[92m'
-        YELLOW='\033[93m'
-        RED='\033[91m'
-        BOLD='\033[1m'
-        UNDERLINE='\033[4m'
-        RESET="\x1B[m"
-        '''
-        self.PURPLE='\033[95m'
-        self.BLUE='\033[94m'
-        self.GREEN='\033[92m'
-        self.YELLOW='\033[93m'
-        self.RED='\033[91m'
-        self.BOLD='\033[1m'
-        self.UNDERLINE='\033[4m'
-        self.RESET="\x1B[m"

onionr/communicator.py

@@ -19,93 +19,144 @@ and code to operate as a daemon, getting commands from the command queue databas
     You should have received a copy of the GNU General Public License
     along with this program.  If not, see <https://www.gnu.org/licenses/>.
 '''
-import sqlite3, requests, hmac, hashlib, time, sys, os, logger
+import sqlite3, requests, hmac, hashlib, time, sys, os, math, logger, urllib.parse, random
-import core, onionrutils
+import core, onionrutils, onionrcrypto, onionrproofs, btc, config, onionrplugins as plugins
+
 class OnionrCommunicate:
     def __init__(self, debug, developmentMode):
-        ''' OnionrCommunicate
+        '''
+            OnionrCommunicate
 
             This class handles communication with nodes in the Onionr network.
         '''
+
         self._core = core.Core()
         self._utils = onionrutils.OnionrUtils(self._core)
+        self._crypto = onionrcrypto.OnionrCrypto(self._core)
+        '''
+        logger.info('Starting Bitcoin Node... with Tor socks port:' + str(sys.argv[2]))
+        try:
+            self.bitcoin = btc.OnionrBTC(torP=int(sys.argv[2]))
+        except _gdbm.error:
+            pass
+        logger.info('Bitcoin Node started, on block: ' + self.bitcoin.node.getBlockHash(self.bitcoin.node.getLastBlockHeight()))
+        '''
+        #except:
+        #logger.fatal('Failed to start Bitcoin Node, exiting...')
+        #exit(1)
+
         blockProcessTimer = 0
         blockProcessAmount = 5
         heartBeatTimer = 0
-        heartBeatRate = 10
+        heartBeatRate = 5
+        pexTimer = 5 # How often we should check for new peers
+        pexCount = 0
         logger.debug('Communicator debugging enabled.')
         torID = open('data/hs/hostname').read()
 
-        # get our own PGP fingerprint
+        self.peerData = {} # Session data for peers (recent reachability, speed, etc)
-        fingerprintFile = 'data/own-fingerprint.txt'
+
-        if not os.path.exists(fingerprintFile):
-            self._core.generateMainPGP(torID)
-        with open(fingerprintFile,'r') as f:
-            self.pgpOwnFingerprint = f.read()
-        logger.info('My PGP fingerprint is ' + logger.colors.underline + self.pgpOwnFingerprint + logger.colors.reset + logger.colors.fg.green + '.')
         if os.path.exists(self._core.queueDB):
             self._core.clearDaemonQueue()
+
+        # Loads in and starts the enabled plugins
+        plugins.reload()
+
         while True:
             command = self._core.daemonQueue()
             # Process blocks based on a timer
             blockProcessTimer += 1
             heartBeatTimer += 1
+            pexCount += 1
+            if pexTimer == pexCount:
+                self.getNewPeers()
+                pexCount = 0
             if heartBeatRate == heartBeatTimer:
                 logger.debug('Communicator heartbeat')
                 heartBeatTimer = 0
             if blockProcessTimer == blockProcessAmount:
                 self.lookupBlocks()
-                self._core.processBlocks()
+                self.processBlocks()
                 blockProcessTimer = 0
-            #logger.debug('Communicator daemon heartbeat')
             if command != False:
                 if command[0] == 'shutdown':
-                    logger.warn('Daemon recieved exit command.')
+                    logger.info('Daemon recieved exit command.')
                     break
             time.sleep(1)
-        return
-    def getRemotePeerKey(self, peerID):
-        '''This function contacts a peer and gets their main PGP key.
 
-        This is safe because Tor or I2P is used, but it does not ensure that the person is who they say they are
-        '''
-        url = 'http://' + peerID + '/public/?action=getPGP'
-        r = requests.get(url, headers=headers)
-        response = r.text
-        return response
-    def shareHMAC(self, peerID, key):
-        '''This function shares an HMAC key to a peer
-        '''
         return
-    def getPeerProof(self, peerID):
+
-        '''This function gets the current peer proof requirement'''
+    def getNewPeers(self):
-        return
+        '''
-    def sendPeerProof(self, peerID, data):
+            Get new peers
-        '''This function sends the proof result to a peer previously fetched with getPeerProof'''
+        '''
+        peersCheck = 5 # Amount of peers to ask for new peers + keys
+        peersChecked = 0
+        peerList = list(self._core.listAdders()) # random ordered list of peers
+        newKeys = []
+        newAdders = []
+        if len(peerList) > 0:
+            maxN = len(peerList) - 1
+        else:
+            peersCheck = 0
+            maxN = 0
+
+        if len(peerList) > peersCheck:
+            peersCheck = len(peerList)
+
+        while peersCheck > peersChecked:
+            i = random.randint(0, maxN)
+            logger.info('Using ' + peerList[i] + ' to find new peers')
+            try:
+                newAdders = self.performGet('pex', peerList[i], skipHighFailureAddress=True)
+                self._utils.mergeAdders(newAdders)
+            except requests.exceptions.ConnectionError:
+                logger.info(peerList[i] + ' connection failed')
+                continue
+            else:
+                try:
+                    logger.info('Using ' + peerList[i] + ' to find new keys')
+                    newKeys = self.performGet('kex', peerList[i], skipHighFailureAddress=True)
+                    # TODO: Require keys to come with POW token (very large amount of POW)
+                    self._utils.mergeKeys(newKeys)
+                except requests.exceptions.ConnectionError:
+                    logger.info(peerList[i] + ' connection failed')
+                    continue
+                else:
+                    peersChecked += 1
         return
 
     def lookupBlocks(self):
-        '''Lookup blocks and merge new ones'''
+        '''
-        peerList = self._core.listPeers()
+            Lookup blocks and merge new ones
+        '''
+        peerList = self._core.listAdders()
         blocks = ''
         for i in peerList:
-            lastDB = self._core.getPeerInfo(i, 'blockDBHash')
+            lastDB = self._core.getAddressInfo(i, 'DBHash')
             if lastDB == None:
                 logger.debug('Fetching hash from ' + i + ' No previous known.')
             else:
-                logger.debug('Fetching hash from ' + i + ', ' + lastDB + ' last known')
+                logger.debug('Fetching hash from ' + str(i) + ', ' + lastDB + ' last known')
             currentDB = self.performGet('getDBHash', i)
+            if currentDB != False:
+                logger.debug(i + " hash db (from request): " + currentDB)
+            else:
+                logger.warn("Error getting hash db status for " + i)
             if currentDB != False:
                 if lastDB != currentDB:
                     logger.debug('Fetching hash from ' + i + ' - ' + currentDB + ' current hash.')
                     blocks += self.performGet('getBlockHashes', i)
-            if currentDB != lastDB:
                 if self._utils.validateHash(currentDB):
-                    self._core.setPeerInfo(i, "blockDBHash", currentDB)
+                    self._core.setAddressInfo(i, "DBHash", currentDB)
-                else:
+        if len(blocks.strip()) != 0:
-                    logger.warn("Peer " + i + " returned malformed hash")
+            logger.debug('BLOCKS:' + blocks)
         blockList = blocks.split('\n')
         for i in blockList:
+            if len(i.strip()) == 0:
+                continue
+            if self._utils.hasBlock(i):
+                continue
             logger.debug('Exchanged block (blockList): ' + i)
             if not self._utils.validateHash(i):
                 # skip hash if it isn't valid
@@ -114,31 +165,100 @@ class OnionrCommunicate:
             else:
                 logger.debug('Adding ' +  i + ' to hash database...')
                 self._core.addToBlockDB(i)
+
         return
 
-    def performGet(self, action, peer, data=None, type='tor'):
+    def processBlocks(self):
-        '''Performs a request to a peer through Tor or i2p (currently only tor)'''
+        '''
+            Work with the block database and download any missing blocks
+
+            This is meant to be called from the communicator daemon on its timer.
+        '''
+
+        for i in self._core.getBlockList(True).split("\n"):
+            if i != "":
+                logger.warn('UNSAVED BLOCK: ' + i)
+                data = self.downloadBlock(i)
+
+        return
+
+    def downloadBlock(self, hash):
+        '''
+            Download a block from random order of peers
+        '''
+
+        peerList = self._core.listAdders()
+        blocks = ''
+        for i in peerList:
+            hasher = hashlib.sha3_256()
+            data = self.performGet('getData', i, hash)
+            if data == False or len(data) > 10000000:
+                continue
+            hasher.update(data.encode())
+            digest = hasher.hexdigest()
+            if type(digest) is bytes:
+                digest = digest.decode()
+            if digest == hash.strip():
+                self._core.setData(data)
+                if data.startswith('-txt-'):
+                    self._core.setBlockType(hash, 'txt')
+                logger.info('Successfully obtained data for ' + hash)
+                if len(data) < 120:
+                    logger.debug('Block text:\n' + data)
+            else:
+                logger.warn("Failed to validate " + hash)
+
+        return
+
+    def urlencode(self, data):
+        '''
+            URL encodes the data
+        '''
+
+        return urllib.parse.quote_plus(data)
+
+    def performGet(self, action, peer, data=None, skipHighFailureAddress=False, peerType='tor'):
+        '''
+            Performs a request to a peer through Tor or i2p (currently only Tor)
+        '''
+
         if not peer.endswith('.onion') and not peer.endswith('.onion/'):
             raise PeerError('Currently only Tor .onion peers are supported. You must manually specify .onion')
+
+        # Store peer in peerData dictionary (non permanent)
+        if not peer in self.peerData:
+            self.peerData[peer] = {'connectCount': 0, 'failCount': 0, 'lastConnectTime': math.floor(time.time())}
         socksPort = sys.argv[2]
         '''We use socks5h to use tor as DNS'''
-        proxies = {'http': 'socks5h://127.0.0.1:' + str(socksPort), 'https': 'socks5h://127.0.0.1:' + str(socksPort)}
+        proxies = {'http': 'socks5://127.0.0.1:' + str(socksPort), 'https': 'socks5://127.0.0.1:' + str(socksPort)}
         headers = {'user-agent': 'PyOnionr'}
-        url = 'http://' + peer + '/public/?action=' + action
+        url = 'http://' + peer + '/public/?action=' + self.urlencode(action)
         if data != None:
-            url = url + '&data=' + data
+            url = url + '&data=' + self.urlencode(data)
         try:
-            r = requests.get(url, headers=headers, proxies=proxies)
+            if skipHighFailureAddress and self.peerData[peer]['failCount'] > 10:
+                retData = False
+                logger.debug('Skipping ' + peer + ' because of high failure rate')
+            else:
+                logger.debug('Contacting ' + peer + ' on port ' + socksPort)
+                r = requests.get(url, headers=headers, proxies=proxies, timeout=(15, 30))
+                retData = r.text
         except requests.exceptions.RequestException as e:
             logger.warn(action + " failed with peer " + peer + ": " + str(e))
-            return False
+            retData = False
-        return r.text
+
+        if not retData:
+            self.peerData[peer]['failCount'] += 1
+        else:
+            self.peerData[peer]['connectCount'] += 1
+            self.peerData[peer]['lastConnectTime'] = math.floor(time.time())
+        return retData
 
 
 shouldRun = False
 debug = True
 developmentMode = False
-if os.path.exists('dev-enabled'):
+if config.get('devmode', True):
     developmentMode = True
 try:
     if sys.argv[1] == 'run':
@@ -149,4 +269,5 @@ if shouldRun:
     try:
         OnionrCommunicate(debug, developmentMode)
     except KeyboardInterrupt:
+        sys.exit(1)
         pass

onionr/config.py

@@ -0,0 +1,114 @@
+'''
+    Onionr - P2P Microblogging Platform & Social network
+
+    This file deals with configuration management.
+'''
+'''
+    This program is free software: you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License
+    along with this program.  If not, see <https://www.gnu.org/licenses/>.
+'''
+
+import os, json, logger
+
+_configfile = os.path.abspath('data/config.json')
+_config = {}
+
+def get(key, default = None):
+    '''
+        Gets the key from configuration, or returns `default`
+    '''
+
+    if is_set(key):
+        return get_config()[key]
+    return default
+
+def set(key, value = None, savefile = False):
+    '''
+        Sets the key in configuration to `value`
+    '''
+
+    global _config
+    if value is None:
+        del _config[key]
+    else:
+        _config[key] = value
+
+    if savefile:
+        save()
+
+def is_set(key):
+    return key in get_config() and not get_config()[key] is None
+
+def check():
+    '''
+        Checks if the configuration file exists, creates it if not
+    '''
+
+    try:
+        if not os.path.exists(os.path.dirname(get_config_file())):
+            os.path.mkdirs(os.path.dirname(get_config_file()))
+        if not os.path.isfile(get_config_file()):
+            open(get_config_file(), 'a', encoding="utf8").close()
+            save()
+    except:
+        logger.warn('Failed to check configuration file.')
+
+def save():
+    '''
+        Saves the configuration data to the configuration file
+    '''
+
+    check()
+    try:
+        with open(get_config_file(), 'w', encoding="utf8") as configfile:
+            json.dump(get_config(), configfile, indent=2, sort_keys=True)
+    except:
+        logger.warn('Failed to write to configuration file.')
+
+def reload():
+    '''
+        Reloads the configuration data in memory from the file
+    '''
+
+    check()
+    try:
+        with open(get_config_file(), 'r', encoding="utf8") as configfile:
+            set_config(json.loads(configfile.read()))
+    except:
+        logger.warn('Failed to parse configuration file.')
+
+def get_config():
+    '''
+        Gets the entire configuration as an array
+    '''
+    return _config
+
+def set_config(config):
+    '''
+        Sets the configuration to the array in arguments
+    '''
+    global _config
+    _config = config
+
+def get_config_file():
+    '''
+        Returns the absolute path to the configuration file
+    '''
+    return _configfile
+
+def set_config_file(configfile):
+    '''
+        Sets the path to the configuration file
+    '''
+    global _configfile
+    _configfile = os.abs.abspath(configfile)

onionr/core.py

@@ -1,7 +1,7 @@
 '''
     Onionr - P2P Microblogging Platform & Social network
 
-    Core Onionr library, useful for external programs. Handles peer processing and cryptography.
+    Core Onionr library, useful for external programs. Handles peer & data processing
 '''
 '''
     This program is free software: you can redistribute it and/or modify
@@ -17,12 +17,12 @@
     You should have received a copy of the GNU General Public License
     along with this program.  If not, see <https://www.gnu.org/licenses/>.
 '''
-import sqlite3, os, sys, time, math, gnupg, base64, tarfile, getpass, simplecrypt, hashlib, nacl, logger
+import sqlite3, os, sys, time, math, base64, tarfile, getpass, simplecrypt, hashlib, nacl, logger
-from Crypto.Cipher import AES
+#from Crypto.Cipher import AES
-from Crypto import Random
+#from Crypto import Random
 import netcontroller
 
-import onionrutils
+import onionrutils, onionrcrypto, btc
 
 if sys.version_info < (3, 6):
     try:
@@ -38,54 +38,90 @@ class Core:
         '''
         self.queueDB = 'data/queue.db'
         self.peerDB = 'data/peers.db'
-        self.ownPGPID = ''
         self.blockDB = 'data/blocks.db'
         self.blockDataLocation = 'data/blocks/'
-        self._utils = onionrutils.OnionrUtils(self)
+        self.addressDB = 'data/address.db'
 
         if not os.path.exists('data/'):
             os.mkdir('data/')
         if not os.path.exists('data/blocks/'):
             os.mkdir('data/blocks/')
-
         if not os.path.exists(self.blockDB):
             self.createBlockDB()
             
-        return
+        self._utils = onionrutils.OnionrUtils(self)
+        # Initialize the crypto object
+        self._crypto = onionrcrypto.OnionrCrypto(self)
 
-    def generateMainPGP(self, myID):
-        ''' Generate the main PGP key for our client. Should not be done often.
-        Uses own PGP home folder in the data/ directory. '''
-        # Generate main pgp key
-        gpg = gnupg.GPG(homedir='./data/pgp/')
-        input_data = gpg.gen_key_input(key_type="RSA", key_length=1024, name_real=myID, name_email='anon@onionr', testing=True)
-        #input_data = gpg.gen_key_input(key_type="RSA", key_length=1024)
-        key = gpg.gen_key(input_data)
-        logger.info("Generating PGP key, this will take some time..")
-        while key.status != "key created":
-            time.sleep(0.5)
-            print(key.status)
-        logger.info("Finished generating PGP key")
-        # Write the key
-        myFingerpintFile = open('data/own-fingerprint.txt', 'w')
-        myFingerpintFile.write(key.fingerprint)
-        myFingerpintFile.close()
         return
 
     def addPeer(self, peerID, name=''):
-        ''' Add a peer by their ID, with an optional name, to the peer database.'''
+        '''
-        ''' DOES NO SAFETY CHECKS if the ID is valid, but prepares the insertion. '''
+            Adds a public key to the key database (misleading function name)
+
+            DOES NO SAFETY CHECKS if the ID is valid, but prepares the insertion
+        '''
         # This function simply adds a peer to the DB
-        if not self._utils.validateID(peerID):
+        if not self._utils.validatePubKey(peerID):
             return False
         conn = sqlite3.connect(self.peerDB)
         c = conn.cursor()
         t = (peerID, name, 'unknown')
-        c.execute('insert into peers (id, name, dateSeen) values(?, ?, ?);', t)
+        c.execute('INSERT INTO peers (id, name, dateSeen) VALUES(?, ?, ?);', t)
         conn.commit()
         conn.close()
         return True
 
+    def addAddress(self, address):
+        '''Add an address to the address database (only tor currently)'''
+        if self._utils.validateID(address):
+            conn = sqlite3.connect(self.addressDB)
+            c = conn.cursor()
+            t = (address, 1)
+            c.execute('INSERT INTO adders (address, type) VALUES(?, ?);', t)
+            conn.commit()
+            conn.close()
+            return True
+        else:
+            return False
+
+    def removeAddress(self, address):
+        '''Remove an address from the address database'''
+        if self._utils.validateID(address):
+            conn = sqlite3.connect(self.addressDB)
+            c = conn.cursor()
+            t = (address,)
+            c.execute('Delete from adders where address=?;', t)
+            conn.commit()
+            conn.close()
+            return True
+        else:
+            return False 
+
+    def createAddressDB(self):
+        '''
+            Generate the address database
+
+            types:
+                1: I2P b32 address
+                2: Tor v2 (like facebookcorewwwi.onion)
+                3: Tor v3
+        '''
+        conn = sqlite3.connect(self.addressDB)
+        c = conn.cursor()
+        c.execute('''CREATE TABLE adders(
+            address text,
+            type int,
+            knownPeer text,
+            speed int,
+            success int,
+            DBHash text,
+            failure int
+            );
+        ''')
+        conn.commit()
+        conn.close()
+
     def createPeerDB(self):
         '''
             Generate the peer sqlite3 database and populate it with the peers table.
@@ -93,12 +129,10 @@ class Core:
         # generate the peer database
         conn = sqlite3.connect(self.peerDB)
         c = conn.cursor()
-        c.execute('''
+        c.execute('''CREATE TABLE peers(
-        create table peers(
             ID text not null,
             name text,
-        pgpKey text,
+            adders text,
-        hmacKey text,
             blockDBHash text,
             forwardKey text,
             dateSeen not null,
@@ -107,6 +141,8 @@ class Core:
         ''')
         conn.commit()
         conn.close()
+        return
+
     def createBlockDB(self):
         '''
             Create a database for blocks
@@ -114,26 +150,37 @@ class Core:
             hash - the hash of a block
             dateReceived - the date the block was recieved, not necessarily when it was created
             decrypted - if we can successfully decrypt the block (does not describe its current state)
-        dataObtained - if the data has been obtained for the block
+            dataType - data type of the block
+            dataFound - if the data has been found for the block
+            dataSaved - if the data has been saved for the block
         '''
         if os.path.exists(self.blockDB):
             raise Exception("Block database already exists")
         conn = sqlite3.connect(self.blockDB)
         c = conn.cursor()
-        c.execute('''create table hashes(
+        c.execute('''CREATE TABLE hashes(
             hash text not null,
             dateReceived int,
             decrypted int,
+            dataType text,
             dataFound int,
-            dataSaved int
+            dataSaved int);
-        );
         ''')
         conn.commit()
         conn.close()
+
+        return
+
     def addToBlockDB(self, newHash, selfInsert=False):
-        '''add a hash value to the block db (should be in hex format)'''
+        '''
+            Add a hash value to the block db
+
+            Should be in hex format!
+        '''
         if not os.path.exists(self.blockDB):
             raise Exception('Block db does not exist')
+        if self._utils.hasBlock(newHash):
+            return
         conn = sqlite3.connect(self.blockDB)
         c = conn.cursor()
         currentTime = math.floor(time.time())
@@ -141,41 +188,57 @@ class Core:
             selfInsert = 1
         else:
             selfInsert = 0
-        data = (newHash, currentTime, 0, 0, selfInsert)
+        data = (newHash, currentTime, 0, '', 0, selfInsert)
-        c.execute('INSERT into hashes values(?, ?, ?, ?, ?);', data)
+        c.execute('INSERT INTO hashes VALUES(?, ?, ?, ?, ?, ?);', data)
         conn.commit()
         conn.close()
 
+        return
+
     def getData(self,hash):
-        '''simply return the data associated to a hash'''
+        '''
+            Simply return the data associated to a hash
+        '''
         try:
             dataFile = open(self.blockDataLocation + hash + '.dat')
             data = dataFile.read()
             dataFile.close()
         except FileNotFoundError:
             data = False
+
         return data
 
     def setData(self, data):
-        '''set the data assciated with a hash'''
+        '''
+            Set the data assciated with a hash
+        '''
         data = data.encode()
         hasher = hashlib.sha3_256()
         hasher.update(data)
         dataHash = hasher.hexdigest()
+        if type(dataHash) is bytes:
+            dataHash = dataHash.decode()
         blockFileName = self.blockDataLocation + dataHash + '.dat'
         if os.path.exists(blockFileName):
-            raise Exception("Data is already set for " + dataHash)
+            pass # TODO: properly check if block is already saved elsewhere
+            #raise Exception("Data is already set for " + dataHash)
         else:
             blockFile = open(blockFileName, 'w')
             blockFile.write(data.decode())
             blockFile.close()
+
+        conn = sqlite3.connect(self.blockDB)
+        c = conn.cursor()
+        c.execute("UPDATE hashes SET dataSaved=1 WHERE hash = '" + dataHash + "';")
+        conn.commit()
+        conn.close()
+
         return dataHash
 
     def dataDirEncrypt(self, password):
         '''
             Encrypt the data directory on Onionr shutdown
         '''
-        # Encrypt data directory (don't delete it in this function)
         if os.path.exists('data.tar'):
             os.remove('data.tar')
         tar = tarfile.open("data.tar", "w")
@@ -186,12 +249,13 @@ class Core:
         encrypted = simplecrypt.encrypt(password, tarData)
         open('data-encrypted.dat', 'wb').write(encrypted)
         os.remove('data.tar')
+
         return
+
     def dataDirDecrypt(self, password):
         '''
             Decrypt the data directory on startup
         '''
-        # Decrypt data directory
         if not os.path.exists('data-encrypted.dat'):
             return (False, 'encrypted archive does not exist')
         data = open('data-encrypted.dat', 'rb').read()
@@ -204,13 +268,15 @@ class Core:
             tar = tarfile.open('data.tar')
             tar.extractall()
             tar.close()
+
         return (True, '')
+
     def daemonQueue(self):
         '''
             Gives commands to the communication proccess/daemon by reading an sqlite3 database
+
+            This function intended to be used by the client. Queue to exchange data between "client" and server.
         '''
-        # This function intended to be used by the client
-        # Queue to exchange data between "client" and server.
         retData = False
         if not os.path.exists(self.queueDB):
             conn = sqlite3.connect(self.queueDB)
@@ -226,7 +292,7 @@ class Core:
                 retData = row
                 break
             if retData != False:
-                c.execute('delete from commands where id = ?', (retData[3],))
+                c.execute('DELETE FROM commands WHERE id=?;', (retData[3],))
         conn.commit()
         conn.close()
 
@@ -241,12 +307,16 @@ class Core:
         conn = sqlite3.connect(self.queueDB)
         c = conn.cursor()
         t = (command, data, date)
-        c.execute('INSERT into commands (command, data, date) values (?, ?, ?)', t)
+        c.execute('INSERT INTO commands (command, data, date) VALUES(?, ?, ?)', t)
         conn.commit()
         conn.close()
+
         return
+
     def clearDaemonQueue(self):
-        '''clear the daemon queue (somewhat dangerousous)'''
+        '''
+            Clear the daemon queue (somewhat dangerous)
+        '''
         conn = sqlite3.connect(self.queueDB)
         c = conn.cursor()
         try:
@@ -256,58 +326,59 @@ class Core:
             pass
         conn.close()
 
-    def generateHMAC(self):
+        return
+    
+    def listAdders(self, randomOrder=True, i2p=True):
         '''
-        generate and return an HMAC key
+            Return a list of addresses
         '''
-        key = base64.b64encode(os.urandom(32))
+        conn = sqlite3.connect(self.addressDB)
-        return key
+        c = conn.cursor()
+        if randomOrder:
+            addresses = c.execute('SELECT * FROM adders ORDER BY RANDOM();')
+        else:
+            addresses = c.execute('SELECT * FROM adders;')
+        addressList = []
+        for i in addresses:
+            addressList.append(i[0])
+        conn.close()
+        return addressList
 
     def listPeers(self, randomOrder=True):
-        '''Return a list of peers
+        '''
+            Return a list of public keys (misleading function name)
 
             randomOrder determines if the list should be in a random order
         '''
         conn = sqlite3.connect(self.peerDB)
         c = conn.cursor()
         if randomOrder:
-            peers = c.execute('SELECT * FROM peers order by RANDOM();')
+            peers = c.execute('SELECT * FROM peers ORDER BY RANDOM();')
         else:
             peers = c.execute('SELECT * FROM peers;')
         peerList = []
         for i in peers:
-            peerList.append(i[0])
+            peerList.append(i[2])
         conn.close()
+
         return peerList
 
-    def processBlocks(self):
-        '''
-        Work with the block database and download any missing blocks
-        This is meant to be called from the communicator daemon on its timer.
-        '''
-        for i in self.getBlockList(True).split("\n"):
-            if i != "":
-                print('UNSAVED BLOCK:', i)
-        return
     def getPeerInfo(self, peer, info):
         '''
-        get info about a peer
+            Get info about a peer from their database entry
 
             id text             0
             name text,          1
-        pgpKey text,        2
+            adders text,        2
-        hmacKey text,       3
+            forwardKey text,    3
-        blockDBHash text,   4
+            dateSeen not null,  4
-        forwardKey text,    5
+            bytesStored int,    5
-        dateSeen not null,  7
+            trust int           6
-        bytesStored int,    8
-        trust int           9
         '''
-        # Lookup something about a peer from their database entry
         conn = sqlite3.connect(self.peerDB)
         c = conn.cursor()
         command = (peer,)
-        infoNumbers = {'id': 0, 'name': 1, 'pgpKey': 2, 'hmacKey': 3, 'blockDBHash': 4, 'forwardKey': 5, 'dateSeen': 6, 'bytesStored': 7, 'trust': 8}
+        infoNumbers = {'id': 0, 'name': 1, 'adders': 2, 'forwardKey': 3, 'dateSeen': 4, 'bytesStored': 5, 'trust': 6}
         info = infoNumbers[info]
         iterCount = 0
         retVal = ''
@@ -319,28 +390,109 @@ class Core:
                 else:
                     iterCount += 1
         conn.close()
+
         return retVal
+
     def setPeerInfo(self, peer, key, data):
-        '''update a peer for a key'''
+        '''
+            Update a peer for a key
+        '''
         conn = sqlite3.connect(self.peerDB)
         c = conn.cursor()
         command = (data, peer)
         # TODO: validate key on whitelist
-
+        if key not in ('id', 'name', 'pubkey', 'blockDBHash', 'forwardKey', 'dateSeen', 'bytesStored', 'trust'):
-        c.execute('UPDATE peers SET ' + key + ' = ? where id=?', command)
+            raise Exception("Got invalid database key when setting peer info")
+        c.execute('UPDATE peers SET ' + key + ' = ? WHERE id=?', command)
         conn.commit()
         conn.close()
+        return
+
+    def getAddressInfo(self, address, info):
+        '''
+            Get info about an address from its database entry
+
+            address text, 0
+            type int, 1
+            knownPeer text, 2
+            speed int, 3
+            success int, 4
+            DBHash text, 5
+            failure int 6
+        '''
+        conn = sqlite3.connect(self.addressDB)
+        c = conn.cursor()
+        command = (address,)
+        infoNumbers = {'address': 0, 'type': 1, 'knownPeer': 2, 'speed': 3, 'success': 4, 'DBHash': 5, 'failure': 6}
+        info = infoNumbers[info]
+        iterCount = 0
+        retVal = ''
+        for row in c.execute('SELECT * from adders where address=?;', command):
+            for i in row:
+                if iterCount == info:
+                    retVal = i
+                    break
+                else:
+                    iterCount += 1
+        conn.close()
+        return retVal
+
+    def setAddressInfo(self, address, key, data):
+        '''
+            Update an address for a key
+        '''
+        conn = sqlite3.connect(self.addressDB)
+        c = conn.cursor()
+        command = (data, address)
+        # TODO: validate key on whitelist
+        if key not in ('address', 'type', 'knownPeer', 'speed', 'success', 'DBHash', 'failure'):
+            raise Exception("Got invalid database key when setting address info")
+        c.execute('UPDATE adders SET ' + key + ' = ? WHERE address=?', command)
+        conn.commit()
+        conn.close()
+        return
 
     def getBlockList(self, unsaved=False):
-        '''get list of our blocks'''
+        '''
+            Get list of our blocks
+        '''
         conn = sqlite3.connect(self.blockDB)
         c = conn.cursor()
         retData = ''
         if unsaved:
-            execute = 'SELECT hash FROM hashes where dataSaved != 1;'
+            execute = 'SELECT hash FROM hashes WHERE dataSaved != 1;'
         else:
             execute = 'SELECT hash FROM hashes;'
         for row in c.execute(execute):
             for i in row:
                 retData += i + "\n"
+
         return retData
+
+    def getBlocksByType(self, blockType):
+        '''
+            Returns a list of blocks by the type
+        '''
+        conn = sqlite3.connect(self.blockDB)
+        c = conn.cursor()
+        retData = ''
+        execute = 'SELECT hash FROM hashes WHERE dataType=?;'
+        args = (blockType,)
+        for row in c.execute(execute, args):
+            for i in row:
+                retData += i + "\n"
+
+        return retData.split('\n')
+
+    def setBlockType(self, hash, blockType):
+        '''
+            Sets the type of block
+        '''
+
+        conn = sqlite3.connect(self.blockDB)
+        c = conn.cursor()
+        c.execute("UPDATE hashes SET dataType='" + blockType + "' WHERE hash = '" + hash + "';")
+        conn.commit()
+        conn.close()
+
+        return

onionr/gui.py

@@ -14,3 +14,57 @@
     You should have received a copy of the GNU General Public License
     along with this program.  If not, see <https://www.gnu.org/licenses/>.
 '''
+from tkinter import *
+import os, sqlite3, core
+class OnionrGUI:
+    def __init__(self, myCore):
+        self.root = Tk()
+        self.myCore = myCore # onionr core
+        self.root.title("PyOnionr")
+
+        w = Label(self.root, text="Onionr", width=10)
+        w.config(font=("Sans-Serif", 22))
+        w.pack()
+        scrollbar = Scrollbar(self.root)
+        scrollbar.pack(side=RIGHT, fill=Y)
+
+        self.listedBlocks = []
+
+        idText = open('./data/hs/hostname', 'r').read()
+        idLabel = Label(self.root, text="ID: " + idText)
+        idLabel.pack(pady=5)
+
+        self.sendEntry = Entry(self.root)
+        sendBtn = Button(self.root, text='Send Message', command=self.sendMessage)
+        self.sendEntry.pack()
+        sendBtn.pack()
+
+        self.listbox = Listbox(self.root, yscrollcommand=scrollbar.set, height=15)
+
+        #listbox.insert(END, str(i))
+        self.listbox.pack(fill=BOTH)
+
+        scrollbar.config(command=self.listbox.yview)
+        self.root.after(2000, self.update)
+        self.root.mainloop() 
+
+    def sendMessage(self):
+        messageToAdd = '-txt-' + self.sendEntry.get()
+        addedHash = self.myCore.setData(messageToAdd)
+        self.myCore.addToBlockDB(addedHash, selfInsert=True)
+        self.myCore.setBlockType(addedHash, 'txt')
+        self.sendEntry.delete(0, END)
+
+    def update(self):
+        for i in self.myCore.getBlocksByType('txt'):
+            if i.strip() == '' or i in self.listedBlocks:
+                continue
+            blockFile = open('./data/blocks/' + i + '.dat')
+            self.listbox.insert(END, str(blockFile.read().replace('-txt-', '')))
+            blockFile.close()
+            self.listedBlocks.append(i)
+            self.listbox.see(END)
+        blocksList = os.listdir('./data/blocks/') # dir is your directory path
+        number_blocks = len(blocksList)
+
+        self.root.after(10000, self.update)

onionr/logger.py

@@ -78,60 +78,82 @@ _type = OUTPUT_TO_CONSOLE | USE_ANSI # the default settings for logging
 _level = LEVEL_DEBUG # the lowest level to log
 _outputfile = './output.log' # the file to log to
 
-'''
-    Set the settings for the logger using bitwise operators
-'''
 def set_settings(type):
+    '''
+        Set the settings for the logger using bitwise operators
+    '''
+
     global _type
     _type = type
 
-'''
-    Get settings from the logger
-'''
 def get_settings():
+    '''
+        Get settings from the logger
+    '''
+
     return _type
 
-'''
-    Set the lowest log level to output
-'''
 def set_level(level):
+    '''
+        Set the lowest log level to output
+    '''
+
     global _level
     _level = level
 
-'''
-    Get the lowest log level currently being outputted
-'''
 def get_level():
+    '''
+        Get the lowest log level currently being outputted
+    '''
+
     return _level
 
-'''
+def set_file(outputfile):
-    Outputs raw data to console without formatting
+    '''
-'''
+        Set the file to output to, if enabled
+    '''
+
+    global _outputfile
+    _outputfile = outputfile
+
+def get_file():
+    '''
+        Get the file to output to
+    '''
+
+    return _outputfile
+
 def raw(data):
+    '''
+        Outputs raw data to console without formatting
+    '''
+
     if get_settings() & OUTPUT_TO_CONSOLE:
         print(data)
     if get_settings() & OUTPUT_TO_FILE:
         with open(_outputfile, "a+") as f:
             f.write(colors.filter(data) + '\n')
 
-'''
+def log(prefix, data, color = ''):
+    '''
         Logs the data
         prefix : The prefix to the output
         data   : The actual data to output
         color  : The color to output before the data
-'''
+    '''
-def log(prefix, data, color = ''):
+
     output = colors.reset + str(color) + '[' + colors.bold + str(prefix) + colors.reset + str(color) + '] ' + str(data) + colors.reset
     if not get_settings() & USE_ANSI:
         output = colors.filter(output)
 
     raw(output)
 
-'''
+def readline(message = ''):
+    '''
         Takes in input from the console, not stored in logs
         message: The message to display before taking input
-'''
+    '''
-def input(message = 'Enter input: '):
+
     color = colors.fg.green + colors.bold
     output = colors.reset + str(color) + '... ' + colors.reset + str(message) + colors.reset
 
@@ -139,14 +161,16 @@ def input(message = 'Enter input: '):
         output = colors.filter(output)
 
     sys.stdout.write(output)
-    return raw_input()
 
-'''
+    return input()
+
+def confirm(default = 'y', message = 'Are you sure %s? '):
+    '''
         Displays an "Are you sure" message, returns True for Y and False for N
         message: The confirmation message, use %s for (y/n)
         default: which to prefer-- y or n
-'''
+    '''
-def confirm(default = 'y', message = 'Are you sure %s? '):
+
     color = colors.fg.green + colors.bold
 
     default = default.lower()
@@ -163,7 +187,8 @@ def confirm(default = 'y', message = 'Are you sure %s? '):
         output = colors.filter(output)
 
     sys.stdout.write(output.replace('%s', confirm))
-    inp = raw_input().lower()
+
+    inp = input().lower()
 
     if 'y' in inp:
         return True

onionr/netcontroller.py

@@ -17,11 +17,14 @@
     You should have received a copy of the GNU General Public License
     along with this program.  If not, see <https://www.gnu.org/licenses/>.
 '''
+
 import subprocess, os, random, sys, logger, time, signal
+
 class NetController:
-    '''NetController
+    '''
         This class handles hidden service setup on Tor and I2P
     '''
+
     def __init__(self, hsPort):
         self.torConfigLocation = 'data/torrc'
         self.readyState = False
@@ -36,9 +39,14 @@ class NetController:
                     os.remove(self.torConfigLocation)
                 torrc.close()
         '''
+
         return
+
     def generateTorrc(self):
-        '''generate a torrc file for our tor instance'''
+        '''
+            Generate a torrc file for our tor instance
+        '''
+
         if os.path.exists(self.torConfigLocation):
             os.remove(self.torConfigLocation)
         torrcData = '''SocksPort ''' + str(self.socksPort) + '''
@@ -48,50 +56,83 @@ HiddenServicePort 80 127.0.0.1:''' + str(self.hsPort) + '''
         torrc = open(self.torConfigLocation, 'w')
         torrc.write(torrcData)
         torrc.close()
+
         return
 
     def startTor(self):
-        '''Start Tor with onion service on port 80 & socks proxy on random port
         '''
+            Start Tor with onion service on port 80 & socks proxy on random port
+        '''
+
         self.generateTorrc()
+
         if os.path.exists('./tor'):
             torBinary = './tor'
+        elif os.path.exists('/usr/bin/tor'):
+            torBinary = '/usr/bin/tor'
         else:
             torBinary = 'tor'
+
         try:
             tor = subprocess.Popen([torBinary, '-f', self.torConfigLocation], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
         except FileNotFoundError:
             logger.fatal("Tor was not found in your path or the Onionr directory. Please install Tor and try again.")
             sys.exit(1)
+        else:
+            # Test Tor Version
+            torVersion = subprocess.Popen([torBinary, '--version'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+            for line in iter(torVersion.stdout.readline, b''):
+                if 'Tor 0.2.' in line.decode():
+                    logger.warn("Running 0.2.x Tor series, no support for v3 onion peers")
+                    break
+            torVersion.kill()
+
         # wait for tor to get to 100% bootstrap
         for line in iter(tor.stdout.readline, b''):
             if 'Bootstrapped 100%: Done' in line.decode():
                 break
             elif 'Opening Socks listener' in line.decode():
-                logger.debug(line.decode())
+                logger.debug(line.decode().replace('\n', ''))
         else:
             logger.fatal('Failed to start Tor. Try killing any other Tor processes owned by this user.')
             return False
+
         logger.info('Finished starting Tor')
         self.readyState = True
+
         myID = open('data/hs/hostname', 'r')
-        self.myID = myID.read()
+        self.myID = myID.read().replace('\n', '')
         myID.close()
+
         torPidFile = open('data/torPid.txt', 'w')
         torPidFile.write(str(tor.pid))
         torPidFile.close()
+
         return True
+
     def killTor(self):
-        '''properly kill tor based on pid saved to file'''
+        '''
+            Properly kill tor based on pid saved to file
+        '''
+
         try:
             pid = open('data/torPid.txt', 'r')
             pidN = pid.read()
             pid.close()
         except FileNotFoundError:
             return
+
         try:
             int(pidN)
         except:
             return
+
+        try:
             os.kill(int(pidN), signal.SIGTERM)
             os.remove('data/torPid.txt')
+        except ProcessLookupError:
+            pass
+        except FileNotFoundError:
+            pass
+
+        return

onionr/onionr.py

@@ -20,8 +20,8 @@
     You should have received a copy of the GNU General Public License
     along with this program.  If not, see <https://www.gnu.org/licenses/>.
 '''
-import sys, os, configparser, base64, random, getpass, shutil, subprocess, requests, time, logger
+import sys, os, base64, random, getpass, shutil, subprocess, requests, time, platform
-import gui, api, core
+import api, core, gui, config, logger, onionrplugins as plugins
 from onionrutils import OnionrUtils
 from netcontroller import NetController
 
@@ -30,20 +30,44 @@ try:
 except ImportError:
     raise Exception("You need the PySocks module (for use with socks5 proxy to use Tor)")
 
-class Onionr:
+ONIONR_TAGLINE = 'Anonymous P2P Platform - GPLv3 - onionr.voidnet.tech'
-    def __init__(self):
+ONIONR_VERSION = '0.0.0' # for debugging and stuff
-        '''Main Onionr class. This is for the CLI program, and does not handle much of the logic.
+API_VERSION = '1' # increments of 1; only change when something fundemental about how the API works changes. This way other nodes knows how to communicate without learning too much information about you.
-        In general, external programs and plugins should not use this class.
 
+class Onionr:
+    cmds = {}
+    cmdhelp = {}
+
+    def __init__(self):
         '''
+            Main Onionr class. This is for the CLI program, and does not handle much of the logic.
+            In general, external programs and plugins should not use this class.
+        '''
+
         try:
             os.chdir(sys.path[0])
         except FileNotFoundError:
             pass
-        if os.path.exists('dev-enabled'):
+
+        # Load global configuration data
+
+        exists = os.path.exists(config.get_config_file())
+        config.set_config({'devmode': True, 'log': {'file': {'output': True, 'path': 'data/output.log'}, 'console': {'output': True, 'color': True}}}) # this is the default config, it will be overwritten if a config file already exists. Else, it saves it
+        config.reload() # this will read the configuration file into memory
+
+        settings = 0b000
+        if config.get('log', {'console': {'color': True}})['console']['color']:
+            settings = settings | logger.USE_ANSI
+        if config.get('log', {'console': {'output': True}})['console']['output']:
+            settings = settings | logger.OUTPUT_TO_CONSOLE
+        if config.get('log', {'file': {'output': True}})['file']['output']:
+            settings = settings | logger.OUTPUT_TO_FILE
+            logger.set_file(config.get('log', {'file': {'path': 'data/output.log'}})['file']['path'])
+        logger.set_settings(settings)
+
+        if config.get('devmode', True):
             self._developmentMode = True
             logger.set_level(logger.LEVEL_DEBUG)
-            logger.warn('DEVELOPMENT MODE ENABLED (THIS IS LESS SECURE!)')
         else:
             self._developmentMode = False
             logger.set_level(logger.LEVEL_INFO)
@@ -51,7 +75,7 @@ class Onionr:
         self.onionrCore = core.Core()
         self.onionrUtils = OnionrUtils(self.onionrCore)
 
-        # Get configuration and Handle commands
+        # Handle commands
 
         self.debug = False # Whole application debugging
 
@@ -69,15 +93,15 @@ class Onionr:
                 os.mkdir('data/')
                 os.mkdir('data/blocks/')
 
-        if not os.path.exists('data/peers.db'):
+        if not os.path.exists(self.onionrCore.peerDB):
             self.onionrCore.createPeerDB()
             pass
+        if not os.path.exists(self.onionrCore.addressDB):
+            self.onionrCore.createAddressDB()
 
         # Get configuration
-        self.config = configparser.ConfigParser()
+
-        if os.path.exists('data/config.ini'):
+        if not exists:
-            self.config.read('data/config.ini')
-        else:
             # Generate default config
             # Hostname should only be set if different from 127.x.x.x. Important for DNS rebinding attack prevention.
             if self.debug:
@@ -87,16 +111,290 @@ class Onionr:
                     randomPort = random.randint(1024, 65535)
                     if self.onionrUtils.checkPort(randomPort):
                         break
-            self.config['CLIENT'] = {'CLIENT HMAC': base64.b64encode(os.urandom(32)).decode('utf-8'), 'PORT': randomPort, 'API VERSION': '0.0.0'}
+            config.set('client', {'participate': 'true', 'client_hmac': base64.b64encode(os.urandom(32)).decode('utf-8'), 'port': randomPort, 'api_version': API_VERSION}, True)
-            with open('data/config.ini', 'w') as configfile:
+
-                self.config.write(configfile)
+        self.cmds = {
+            '': self.showHelpSuggestion,
+            'help': self.showHelp,
+            'version': self.version,
+            'config': self.configure,
+            'start': self.start,
+            'stop': self.killDaemon,
+            'stats': self.showStats,
+
+            'enable-plugin': self.enablePlugin,
+            'enplugin': self.enablePlugin,
+            'enableplugin': self.enablePlugin,
+            'enmod': self.enablePlugin,
+            'disable-plugin': self.disablePlugin,
+            'displugin': self.disablePlugin,
+            'disableplugin': self.disablePlugin,
+            'dismod': self.disablePlugin,
+            'reload-plugin': self.reloadPlugin,
+            'reloadplugin': self.reloadPlugin,
+            'reload-plugins': self.reloadPlugin,
+            'reloadplugins': self.reloadPlugin,
+
+            'listpeers': self.listPeers,
+            'list-peers': self.listPeers,
+
+            'addmsg': self.addMessage,
+            'addmessage': self.addMessage,
+            'add-msg': self.addMessage,
+            'add-message': self.addMessage,
+            'pm': self.sendEncrypt,
+
+            'gui': self.openGUI,
+
+            'addpeer': self.addPeer,
+            'add-peer': self.addPeer,
+            'add-address': self.addAddress,
+            'addaddress': self.addAddress,
+
+            'connect': self.addAddress
+        }
+
+        self.cmdhelp = {
+            'help': 'Displays this Onionr help menu',
+            'version': 'Displays the Onionr version',
+            'config': 'Configures something and adds it to the file',
+            'start': 'Starts the Onionr daemon',
+            'stop': 'Stops the Onionr daemon',
+            'stats': 'Displays node statistics',
+            'enable-plugin': 'Enables and starts a plugin',
+            'disable-plugin': 'Disables and stops a plugin',
+            'reload-plugin': 'Reloads a plugin',
+            'list-peers': 'Displays a list of peers',
+            'add-peer': 'Adds a peer (?)',
+            'add-msg': 'Broadcasts a message to the Onionr network',
+            'pm': 'Adds a private message to block',
+            'gui': 'Opens a graphical interface for Onionr'
+        }
+
         command = ''
         try:
             command = sys.argv[1].lower()
         except IndexError:
             command = ''
         finally:
-            if command == 'start':
+            self.execute(command)
+
+        if not self._developmentMode:
+            encryptionPassword = self.onionrUtils.getPassword('Enter password to encrypt directory: ')
+            self.onionrCore.dataDirEncrypt(encryptionPassword)
+            shutil.rmtree('data/')
+
+        return
+
+    '''
+        THIS SECTION HANDLES THE COMMANDS
+    '''
+
+    def getCommands(self):
+        return self.cmds
+
+    def getHelp(self):
+        return self.cmdhelp
+
+    def addCommand(self, command, function):
+        cmds[str(command).lower()] = function
+
+    def addHelp(self, command, description):
+        cmdhelp[str(command).lower()] = str(description)
+
+    def configure(self):
+        '''
+            Displays something from the configuration file, or sets it
+        '''
+
+        if len(sys.argv) >= 4:
+            config.reload()
+            config.set(sys.argv[2], sys.argv[3], True)
+            logger.debug('Configuration file updated.')
+        elif len(sys.argv) >= 3:
+            config.reload()
+            logger.info(logger.colors.bold + sys.argv[2] + ': ' + logger.colors.reset + str(config.get(sys.argv[2], logger.colors.fg.red + 'Not set.')))
+        else:
+            logger.info(logger.colors.bold + 'Get a value: ' + logger.colors.reset + sys.argv[0] + ' ' + sys.argv[1] + ' <key>')
+            logger.info(logger.colors.bold + 'Set a value: ' + logger.colors.reset + sys.argv[0] + ' ' + sys.argv[1] + ' <key> <value>')
+
+
+    def execute(self, argument):
+        '''
+            Executes a command
+        '''
+        argument = argument[argument.startswith('--') and len('--'):] # remove -- if it starts with it
+
+        # define commands
+        commands = self.getCommands()
+
+        command = commands.get(argument, self.notFound)
+        command()
+
+    '''
+        THIS SECTION DEFINES THE COMMANDS
+    '''
+
+    def version(self, verbosity=5):
+        '''
+            Displays the Onionr version
+        '''
+        logger.info('Onionr ' + ONIONR_VERSION + ' (' + platform.machine() + ') - API v' + API_VERSION)
+        if verbosity >= 1:
+            logger.info(ONIONR_TAGLINE)
+        if verbosity >= 2:
+            logger.info('Running on ' + platform.platform() + ' ' + platform.release())
+
+    def sendEncrypt(self):
+        '''
+            Create a private message and send it
+        '''
+
+        while True:
+            try:
+                peer = logger.readline('Peer to send to: ')
+            except KeyboardInterrupt:
+                break
+            else:
+                if self.onionrUtils.validateID(peer):
+                    break
+                else:
+                    logger.error('Invalid peer ID')
+        else:
+            try:
+                message = logger.readline("Enter a message: ")
+            except KeyboardInterrupt:
+                pass
+            else:
+                logger.info("Sending message to " + peer)
+                self.onionrUtils.sendPM(peer, message)
+
+
+    def openGUI(self):
+        '''
+            Opens a graphical interface for Onionr
+        '''
+
+        gui.OnionrGUI(self.onionrCore)
+
+    def listPeers(self):
+        '''
+            Displays a list of peers (?)
+        '''
+
+        logger.info('Peer list:\n')
+        for i in self.onionrCore.listPeers():
+            logger.info(i)
+
+    def addPeer(self):
+        '''
+            Adds a peer (?)
+        '''
+
+        try:
+            newPeer = sys.argv[2]
+        except:
+            pass
+        else:
+            logger.info("Adding peer: " + logger.colors.underline + newPeer)
+            self.onionrCore.addPeer(newPeer)
+
+        return
+
+    def addAddress(self):
+        '''Adds a Onionr node address'''
+        try:
+            newAddress = sys.argv[2]
+        except:
+            pass
+        else:
+            logger.info("Adding address: " + logger.colors.underline + newAddress)
+            if self.onionrCore.addAddress(newAddress):
+                logger.info("Successfully added address")
+            else:
+                logger.warn("Unable to add address")
+
+        return
+
+    def addMessage(self):
+        '''
+            Broadcasts a message to the Onionr network
+        '''
+
+        while True:
+            messageToAdd = '-txt-' + logger.readline('Broadcast message to network: ')
+            if len(messageToAdd) >= 1:
+                break
+
+        addedHash = self.onionrCore.setData(messageToAdd)
+        self.onionrCore.addToBlockDB(addedHash, selfInsert=True)
+        self.onionrCore.setBlockType(addedHash, 'txt')
+
+        return
+
+    def enablePlugin(self):
+        '''
+            Enables and starts the given plugin
+        '''
+
+        if len(sys.argv) >= 3:
+            plugin_name = sys.argv[2]
+            logger.info('Enabling plugin \"' + plugin_name + '\"...')
+            plugins.enable(plugin_name)
+        else:
+            logger.info(sys.argv[0] + ' ' + sys.argv[1] + ' <plugin>')
+
+        return
+
+    def disablePlugin(self):
+        '''
+            Disables and stops the given plugin
+        '''
+
+        if len(sys.argv) >= 3:
+            plugin_name = sys.argv[2]
+            logger.info('Disabling plugin \"' + plugin_name + '\"...')
+            plugins.disable(plugin_name)
+        else:
+            logger.info(sys.argv[0] + ' ' + sys.argv[1] + ' <plugin>')
+
+        return
+
+    def reloadPlugin(self):
+        '''
+            Reloads (stops and starts) all plugins, or the given plugin
+        '''
+
+        if len(sys.argv) >= 3:
+            plugin_name = sys.argv[2]
+            logger.info('Reloading plugin \"' + plugin_name + '\"...')
+            plugins.stop(plugin_name)
+            plugins.start(plugin_name)
+        else:
+            logger.info('Reloading all plugins...')
+            plugins.reload()
+
+        return
+
+    def notFound(self):
+        '''
+            Displays a "command not found" message
+        '''
+
+        logger.error('Command not found.')
+
+    def showHelpSuggestion(self):
+        '''
+            Displays a message suggesting help
+        '''
+
+        logger.info('Do ' + logger.colors.bold + sys.argv[0] + ' --help' + logger.colors.reset + logger.colors.fg.green + ' for Onionr help.')
+
+    def start(self):
+        '''
+            Starts the Onionr daemon
+        '''
+
         if os.path.exists('.onionr-lock'):
             logger.fatal('Cannot start. Daemon is already running, or it did not exit cleanly.\n(if you are sure that there is not a daemon running, delete .onionr-lock & try again).')
         else:
@@ -107,58 +405,70 @@ class Onionr:
             self.daemon()
             if not self.debug and not self._developmentMode:
                 os.remove('.onionr-lock')
-            elif command == 'stop':
-                self.killDaemon()
-            elif command in ('addmsg', 'addmessage'):
-                while True:
-                    messageToAdd = input('Broadcast message to network: ')
-                    if len(messageToAdd) >= 1:
-                        break
-                addedHash = self.onionrCore.setData(messageToAdd)
-                self.onionrCore.addToBlockDB(addedHash, selfInsert=True)
-            elif command == 'stats':
-                self.showStats()
-            elif command == 'help' or command == '--help':
-                self.showHelp()
-            elif command == '':
-                logger.info('Do ' + logger.colors.bold + sys.argv[0] + ' --help' + logger.colors.reset + logger.colors.fg.green + ' for Onionr help.')
-            else:
-                logger.error('Invalid command.')
 
-        if not self._developmentMode:
-            encryptionPassword = self.onionrUtils.getPassword('Enter password to encrypt directory: ')
-            self.onionrCore.dataDirEncrypt(encryptionPassword)
-            shutil.rmtree('data/')
-        return
     def daemon(self):
-        ''' Start the Onionr communication daemon
         '''
+            Starts the Onionr communication daemon
+        '''
+
         if not os.environ.get("WERKZEUG_RUN_MAIN") == "true":
-            net = NetController(self.config['CLIENT']['PORT'])
+            if self._developmentMode:
+                logger.warn('DEVELOPMENT MODE ENABLED (THIS IS LESS SECURE!)')
+            net = NetController(config.get('client')['port'])
             logger.info('Tor is starting...')
             if not net.startTor():
                 sys.exit(1)
             logger.info('Started Tor .onion service: ' + logger.colors.underline + net.myID)
+            logger.info('Our Public key: ' + self.onionrCore._crypto.pubKey)
             time.sleep(1)
             subprocess.Popen(["./communicator.py", "run", str(net.socksPort)])
             logger.debug('Started communicator')
-        api.API(self.config, self.debug)
+        api.API(self.debug)
+
         return
+
     def killDaemon(self):
-        '''Shutdown the Onionr Daemon'''
+        '''
+            Shutdown the Onionr daemon
+        '''
+
         logger.warn('Killing the running daemon')
-        net = NetController(self.config['CLIENT']['PORT'])
+        net = NetController(config.get('client')['port'])
         try:
             self.onionrUtils.localCommand('shutdown')
         except requests.exceptions.ConnectionError:
             pass
         self.onionrCore.daemonQueueAdd('shutdown')
         net.killTor()
+
         return
+
     def showStats(self):
-        '''Display statistics and exit'''
+        '''
+            Displays statistics and exits
+        '''
+
         return
-    def showHelp(self):
+
-        '''Show help for Onionr'''
+    def showHelp(self, command = None):
+        '''
+            Show help for Onionr
+        '''
+
+        helpmenu = self.getHelp()
+
+        if command is None and len(sys.argv) >= 3:
+            for cmd in sys.argv[2:]:
+                self.showHelp(cmd)
+        elif not command is None:
+            if command.lower() in helpmenu:
+                logger.info(logger.colors.bold + command  + logger.colors.reset + logger.colors.fg.blue + ' : ' + logger.colors.reset +  helpmenu[command.lower()])
+            else:
+                logger.warn(logger.colors.bold + command  + logger.colors.reset + logger.colors.fg.blue + ' : ' + logger.colors.reset + 'No help menu entry was found')
+        else:
+            self.version(0)
+            for command, helpmessage in helpmenu.items():
+                self.showHelp(command)
         return
+
 Onionr()

onionr/onionrcrypto.py

@@ -0,0 +1,97 @@
+'''
+    Onionr - P2P Microblogging Platform & Social network
+
+    This file handles Onionr's cryptography.
+'''
+'''
+    This program is free software: you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License
+    along with this program.  If not, see <https://www.gnu.org/licenses/>.
+'''
+import nacl.signing, nacl.encoding, nacl.public, os
+
+class OnionrCrypto:
+    def __init__(self, coreInstance):
+        self._core = coreInstance
+        self._keyFile = 'data/keys.txt'
+        self.pubKey = None
+        self.privKey = None
+
+        # Load our own pub/priv Ed25519 keys, gen & save them if they don't exist
+        if os.path.exists(self._keyFile):
+            with open('data/keys.txt', 'r') as keys:
+                keys = keys.read().split(',')
+                self.pubKey = keys[0]
+                self.privKey = keys[1]
+        else:
+            keys = self.generatePubKey()
+            self.pubKey = keys[0]
+            self.privKey = keys[1]
+            with open(self._keyFile, 'w') as keyfile:
+                keyfile.write(self.pubKey + ',' + self.privKey)
+        return
+
+    def edVerify(self, data, key):
+        '''Verify signed data (combined in nacl) to an ed25519 key'''
+        key = nacl.signing.VerifyKey(key=key, encoder=nacl.encoding.Base32Encoder)
+        retData = ''
+        if encodeResult:
+            retData = key.verify(data.encode(), encoder=nacl.encoding.Base64Encoder) # .encode() is not the same as nacl.encoding
+        else:
+            retData = key.verify(data.encode())
+        return retData
+    
+    def edSign(self, data, key, encodeResult=False):
+        '''Ed25519 sign data'''
+        key = nacl.signing.SigningKey(seed=key, encoder=nacl.encoding.Base32Encoder)
+        retData = ''
+        if encodeResult:
+            retData = key.sign(data.encode(), encoder=nacl.encoding.Base64Encoder) # .encode() is not the same as nacl.encoding
+        else:
+            retData = key.sign(data.encode())
+        return retData
+
+    def pubKeyEncrypt(self, data, pubkey, anonymous=False):
+        '''Encrypt to a public key (Curve25519, taken from base32 Ed25519 pubkey)'''
+        retVal = ''
+        if self.privKey != None and not anonymous:
+            ownKey = nacl.signing.SigningKey(seed=self.privKey, encoder=nacl.encoding.Base32Encoder())
+            key = nacl.signing.VerifyKey(key=pubkey, encoder=nacl.encoding.Base32Encoder).to_curve25519_public_key()
+            ourBox = nacl.public.Box(ownKey, key)
+            retVal = ourBox.encrypt(data.encode(), encoder=nacl.encoding.RawEncoder)
+        elif anonymous:
+            key = nacl.signing.VerifyKey(key=pubkey, encoder=nacl.encoding.Base32Encoder).to_curve25519_public_key()
+            anonBox = nacl.public.SealedBox(key)
+            retVal = anonBox.encrypt(data.encode(), encoder=nacl.encoding.RawEncoder)
+        return retVal
+
+    def pubKeyDecrypt(self, data, peer):
+        '''pubkey decrypt (Curve25519, taken from Ed25519 pubkey)'''
+        return
+
+    def symmetricPeerEncrypt(self, data):
+        '''Salsa20 encrypt data to peer (with mac)'''
+        return
+
+    def symmetricPeerDecrypt(self, data, peer):
+        '''Salsa20 decrypt data from peer (with mac)'''
+        return
+    
+    def generateSymmetric(self, data, peer):
+        '''Generate symmetric key'''
+        return
+
+    def generatePubKey(self):
+        '''Generate a Ed25519 public key pair, return tuple of base64encoded pubkey, privkey'''
+        private_key = nacl.signing.SigningKey.generate()
+        public_key = private_key.verify_key.encode(encoder=nacl.encoding.Base32Encoder())
+        return (public_key.decode(), private_key.encode(encoder=nacl.encoding.Base32Encoder()).decode())

onionr/onionrevents.py

@@ -0,0 +1,53 @@
+'''
+    Onionr - P2P Microblogging Platform & Social network
+
+    This file deals with configuration management.
+'''
+'''
+    This program is free software: you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License
+    along with this program.  If not, see <https://www.gnu.org/licenses/>.
+'''
+
+import config, logger, onionrplugins as plugins
+
+def event(event_name, data = None, onionr = None):
+    '''
+        Calls an event on all plugins (if defined)
+    '''
+
+    for plugin in plugins.get_enabled_plugins():
+        try:
+            call(plugins.get_plugin(plugin), event_name, data, onionr)
+        except:
+            logger.warn('Event \"' + event_name + '\" failed for plugin \"' + plugin + '\".')
+
+def call(plugin, event_name, data = None, onionr = None):
+    '''
+        Calls an event on a plugin if one is defined
+    '''
+
+    if not plugin is None:
+        try:
+            attribute = 'on_' + str(event_name).lower()
+
+            # TODO: Use multithreading perhaps?
+            if hasattr(plugin, attribute):
+                logger.debug('Calling event ' + str(event_name))
+                getattr(plugin, attribute)(onionr, data)
+
+            return True
+        except:
+            logger.warn('Failed to call event ' + str(event_name) + ' on module.')
+            return False
+    else:
+        return True

onionr/onionrplugins.py

@@ -0,0 +1,234 @@
+'''
+    Onionr - P2P Microblogging Platform & Social network
+
+    This file deals with management of modules/plugins.
+'''
+'''
+    This program is free software: you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License
+    along with this program.  If not, see <https://www.gnu.org/licenses/>.
+'''
+
+import os, re, importlib, config, logger
+import onionrevents as events
+
+_pluginsfolder = 'data/plugins/'
+_instances = dict()
+
+def reload(stop_event = True):
+    '''
+        Reloads all the plugins
+    '''
+
+    check()
+
+    try:
+        enabled_plugins = get_enabled_plugins()
+
+        if stop_event is True:
+            logger.debug('Reloading all plugins...')
+        else:
+            logger.debug('Loading all plugins...')
+
+        if stop_event is True:
+            for plugin in enabled_plugins:
+                stop(plugin)
+
+        for plugin in enabled_plugins:
+            start(plugin)
+
+        return True
+    except:
+        logger.error('Failed to reload plugins.')
+
+    return False
+
+
+def enable(name, start_event = True):
+    '''
+        Enables a plugin
+    '''
+
+    check()
+
+    if exists(name):
+        enabled_plugins = get_enabled_plugins()
+        enabled_plugins.append(name)
+        config_plugins = config.get('plugins')
+        config_plugins['enabled'] = enabled_plugins
+        config.set('plugins', config_plugins, True)
+
+        events.call(get_plugin(name), 'enable')
+
+        if start_event is True:
+            start(name)
+
+        return True
+    else:
+        logger.error('Failed to enable plugin \"' + name + '\", disabling plugin.')
+        disable(name)
+
+        return False
+
+
+def disable(name, stop_event = True):
+    '''
+        Disables a plugin
+    '''
+
+    check()
+
+    if is_enabled(name):
+        enabled_plugins = get_enabled_plugins()
+        enabled_plugins.remove(name)
+        config_plugins = config.get('plugins')
+        config_plugins['enabled'] = enabled_plugins
+        config.set('plugins', config_plugins, True)
+
+    if exists(name):
+        events.call(get_plugin(name), 'disable')
+
+        if stop_event is True:
+            stop(name)
+
+def start(name):
+    '''
+        Starts the plugin
+    '''
+
+    check()
+
+    if exists(name):
+        try:
+            plugin = get_plugin(name)
+
+            if plugin is None:
+                raise Exception('Failed to import module.')
+            else:
+                events.call(plugin, 'start')
+
+            return plugin
+        except:
+            logger.error('Failed to start module \"' + name + '\".')
+    else:
+        logger.error('Failed to start nonexistant module \"' + name + '\".')
+
+    return None
+
+def stop(name):
+    '''
+        Stops the plugin
+    '''
+
+    check()
+
+    if exists(name):
+        try:
+            plugin = get_plugin(name)
+
+            if plugin is None:
+                raise Exception('Failed to import module.')
+            else:
+                events.call(plugin, 'stop')
+
+            return plugin
+        except:
+            logger.error('Failed to stop module \"' + name + '\".')
+    else:
+        logger.error('Failed to stop nonexistant module \"' + name + '\".')
+
+    return None
+
+def get_plugin(name):
+    '''
+        Returns the instance of a module
+    '''
+
+    check()
+
+    if str(name).lower() in _instances:
+        return _instances[str(name).lower()]
+    else:
+        _instances[str(name).lower()] = importlib.import_module(get_plugins_folder(name, False).replace('/', '.') + 'main')
+        return get_plugin(name)
+
+def get_plugins():
+    '''
+        Returns a list of plugins (deprecated)
+    '''
+
+    return _instances
+
+def exists(name):
+    '''
+        Return value indicates whether or not the plugin exists
+    '''
+
+    return os.path.isdir(get_plugins_folder(str(name).lower()))
+
+def get_enabled_plugins():
+    '''
+        Returns a list of the enabled plugins
+    '''
+
+    check()
+
+    config.reload()
+
+    return config.get('plugins')['enabled']
+
+def is_enabled(name):
+    '''
+        Return value indicates whether or not the plugin is enabled
+    '''
+
+    return name in get_enabled_plugins()
+
+def get_plugins_folder(name = None, absolute = True):
+    '''
+        Returns the path to the plugins folder
+    '''
+
+    path = ''
+
+    if name is None:
+        path = _pluginsfolder
+    else:
+        # only allow alphanumeric characters
+        path = _pluginsfolder + re.sub('[^0-9a-zA-Z]+', '', str(name).lower()) + '/'
+
+    if absolute is True:
+        path = os.path.abspath(path)
+
+    return path
+
+def check():
+    '''
+        Checks to make sure files exist
+    '''
+
+    config.reload()
+
+    if not config.is_set('plugins'):
+        logger.debug('Generating plugin config data...')
+        config.set('plugins', {'enabled': []}, True)
+
+    if not os.path.exists(os.path.dirname(get_plugins_folder())):
+        logger.debug('Generating plugin data folder...')
+        os.makedirs(os.path.dirname(get_plugins_folder()))
+
+    if not exists('test'):
+        os.makedirs(get_plugins_folder('test'))
+        with open(get_plugins_folder('test') + '/main.py', 'a') as main:
+            main.write("print('Running')\n\ndef on_test(onionr = None, data = None):\n    print('received test event!')\n    return True\n\ndef on_start(onionr = None, data = None):\n    print('start event called')\n\ndef on_stop(onionr = None, data = None):\n    print('stop event called')\n\ndef on_enable(onionr = None, data = None):\n    print('enable event called')\n\ndef on_disable(onionr = None, data = None):\n    print('disable event called')\n")
+        enable('test')
+    return

onionr/onionrproofs.py

@@ -0,0 +1,86 @@
+'''
+    Onionr - P2P Microblogging Platform & Social network
+
+    Proof of work module
+'''
+'''
+    This program is free software: you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License
+    along with this program.  If not, see <https://www.gnu.org/licenses/>.
+'''
+import nacl.encoding, nacl.hash, nacl.utils, time, math, threading, binascii, logger
+import btc
+class POW:
+    def pow(self, reporting=False):
+        startTime = math.floor(time.time())
+        self.hashing = True
+        self.reporting = reporting
+        iFound = False # if current thread is the one that found the answer
+        answer = ''
+        heartbeat = 200000
+        hbCount = 0
+        blockCheck = 300000 # How often the hasher should check if the bitcoin block is updated (slows hashing but prevents less wasted work)
+        blockCheckCount = 0
+        block = ''#self.bitcoinNode.getBlockHash(self.bitcoinNode.getLastBlockHeight())
+        while self.hashing:
+            if blockCheckCount == blockCheck:
+                if self.reporting:
+                    logger.debug('Refreshing Bitcoin block')
+                block = ''#self.bitcoinNode.getBlockHash(self.bitcoinNode.getLastBlockHeight())
+                blockCheckCount = 0
+            blockCheckCount += 1
+            hbCount += 1
+            token = nacl.hash.blake2b(nacl.utils.random() + block.encode()).decode()
+            if self.mainHash[0:self.difficulty] == token[0:self.difficulty]:
+                self.hashing = False
+                iFound = True
+                break
+        if iFound:
+            endTime = math.floor(time.time())
+            if self.reporting:
+                logger.info('Found token ' + token)
+                logger.info('took ' + str(endTime - startTime))
+            self.result = token
+    
+    def __init__(self, difficulty, bitcoinNode):
+        self.foundHash = False
+        self.difficulty = difficulty
+
+        logger.debug('Computing difficulty of ' + str(self.difficulty))
+
+        self.mainHash = nacl.hash.blake2b(nacl.utils.random()).decode()
+        self.puzzle = self.mainHash[0:self.difficulty]
+        self.bitcoinNode = bitcoinNode
+        logger.debug('trying to find ' + str(self.mainHash))
+        tOne = threading.Thread(name='one', target=self.pow, args=(True,))
+        tTwo = threading.Thread(name='two', target=self.pow)
+        tThree = threading.Thread(name='three', target=self.pow)
+        tOne.start()
+        tTwo.start()
+        tThree.start()
+        return
+
+    def shutdown(self):
+        self.hashing = False
+        self.puzzle = ''
+    
+    def changeDifficulty(self, newDiff):
+        self.difficulty = newDiff
+
+    def getResult(self):
+        '''Returns the result then sets to false, useful to automatically clear the result'''
+        try:
+            retVal = self.result
+        except AttributeError:
+            retVal = False
+        self.result = False
+        return retVal

onionr/onionrutils.py

@@ -18,29 +18,78 @@
     along with this program.  If not, see <https://www.gnu.org/licenses/>.
 '''
 # Misc functions that do not fit in the main api, but are useful
-import getpass, sys, requests, configparser, os, socket, gnupg, hashlib, logger
+import getpass, sys, requests, os, socket, hashlib, logger, sqlite3, config
+import nacl.signing, nacl.encoding
+
 if sys.version_info < (3, 6):
     try:
         import sha3
     except ModuleNotFoundError:
         logger.fatal('On Python 3 versions prior to 3.6.x, you need the sha3 module')
         sys.exit(1)
+
 class OnionrUtils:
-    '''Various useful functions'''
+    '''
+        Various useful function
+    '''
     def __init__(self, coreInstance):
         self.fingerprintFile = 'data/own-fingerprint.txt'
         self._core = coreInstance
         return
-    def localCommand(self, command):
+
-        '''Send a command to the local http API server, securely. Intended for local clients, DO NOT USE for remote peers.'''
+    def sendPM(self, user, message):
-        config = configparser.ConfigParser()
+        '''High level function to encrypt a message to a peer and insert it as a block'''
-        if os.path.exists('data/config.ini'):
-            config.read('data/config.ini')
-        else:
         return
-        requests.get('http://' + open('data/host.txt', 'r').read() + ':' + str(config['CLIENT']['PORT']) + '/client/?action=' + command + '&token=' + config['CLIENT']['CLIENT HMAC'])
+    
+    def incrementAddressSuccess(self, address):
+        '''Increase the recorded sucesses for an address'''
+        increment = self._core.getAddressInfo(address, 'success') + 1
+        self._core.setAddressInfo(address, 'success', increment)
+        return
+        
+    def decrementAddressSuccess(self, address):
+        '''Decrease the recorded sucesses for an address'''
+        increment = self._core.getAddressInfo(address, 'success') - 1
+        self._core.setAddressInfo(address, 'success', increment)
+        return
+
+    def mergeKeys(self, newKeyList):
+        '''Merge ed25519 key list to our database'''
+        retVal = False
+        if newKeyList != False:
+            for key in newKeyList:
+                if not key in self._core.listPeers(randomOrder=False):
+                    if self._core.addPeer(key):
+                        retVal = True
+        return retVal
+
+    
+    def mergeAdders(self, newAdderList):
+        '''Merge peer adders list to our database'''
+        retVal = False
+        if newAdderList != False:
+            for adder in newAdderList:
+                if not adder in self._core.listAdders(randomOrder=False):
+                    if self._core.addAddress(adder):
+                        retVal = True
+        return retVal
+
+    def localCommand(self, command):
+        '''
+            Send a command to the local http API server, securely. Intended for local clients, DO NOT USE for remote peers.
+        '''
+
+        config.reload()
+
+        # TODO: URL encode parameters, just as an extra measure. May not be needed, but should be added regardless.
+        requests.get('http://' + open('data/host.txt', 'r').read() + ':' + str(config.get('client')['port']) + '/client/?action=' + command + '&token=' + str(config.get('client')['client_hmac']))
+
+        return
+
     def getPassword(self, message='Enter password: ', confirm = True):
-        '''Get a password without showing the users typing and confirm the input'''
+        '''
+            Get a password without showing the users typing and confirm the input
+        '''
         # Get a password safely with confirmation and return it
         while True:
             print(message)
@@ -50,14 +99,18 @@ class OnionrUtils:
                 pass2 = getpass.getpass()
                 if pass1 != pass2:
                     logger.error("Passwords do not match.")
-                    input()
+                    logger.readline()
                 else:
                     break
             else:
                 break
+
         return pass1
-    def checkPort(self, port, host = ''):
+
-        '''Checks if a port is available, returns bool'''
+    def checkPort(self, port, host=''):
+        '''
+            Checks if a port is available, returns bool
+        '''
         # inspired by https://www.reddit.com/r/learnpython/comments/2i4qrj/how_to_write_a_python_script_that_checks_to_see/ckzarux/
         sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
         retVal = False
@@ -68,37 +121,58 @@ class OnionrUtils:
                 retVal = True
         finally:
             sock.close()
+
         return retVal
+
     def checkIsIP(self, ip):
-        '''Check if a string is a valid ipv4 address'''
+        '''
+            Check if a string is a valid IPv4 address
+        '''
         try:
             socket.inet_aton(ip)
         except:
             return False
         else:
             return True
-    def exportMyPubkey(self):
-        '''Export our PGP key if it exists'''
-        if not os.path.exists(self.fingerprintFile):
-            raise Exception("No fingerprint found, cannot export our PGP key.")
-        gpg = gnupg.GPG(homedir='./data/pgp/')
-        with open(self.fingerprintFile,'r') as f:
-            fingerprint = f.read()
-        ascii_armored_public_keys = gpg.export_keys(fingerprint)
-        return ascii_armored_public_keys
 
     def getBlockDBHash(self):
-        '''Return a sha3_256 hash of the blocks DB'''
+        '''
+            Return a sha3_256 hash of the blocks DB
+        '''
         with open(self._core.blockDB, 'rb') as data:
             data = data.read()
         hasher = hashlib.sha3_256()
         hasher.update(data)
         dataHash = hasher.hexdigest()
+
         return dataHash
 
+    def hasBlock(self, hash):
+        '''
+            Check for new block in the list
+        '''
+        conn = sqlite3.connect(self._core.blockDB)
+        c = conn.cursor()
+        if not self.validateHash(hash):
+            raise Exception("Invalid hash")
+        for result in c.execute("SELECT COUNT() FROM hashes where hash='" + hash + "'"):
+            if result[0] >= 1:
+                conn.commit()
+                conn.close()
+                return True
+            else:
+                conn.commit()
+                conn.close()
+                return False
+
     def validateHash(self, data, length=64):
-        '''Validate if a string is a valid hex formatted hash'''
+        '''
+            Validate if a string is a valid hex formatted hash
+        '''
         retVal = True
+        if data == False or data == True:
+            return False
+        data = data.strip()
         if len(data) != length:
             retVal = False
         else:
@@ -106,9 +180,25 @@ class OnionrUtils:
                 int(data, 16)
             except ValueError:
                 retVal = False
+
         return retVal
+
+    def validatePubKey(self, key):
+        '''Validate if a string is a valid base32 encoded Ed25519 key'''
+        retVal = False
+        try:
+            nacl.signing.SigningKey(seed=key, encoder=nacl.encoding.Base32Encoder)
+        except nacl.exceptions.ValueError:
+            pass
+        else:
+            retVal = True
+        return retVal
+
+
     def validateID(self, id):
-        '''validate if a user ID is a valid tor or i2p hidden service'''
+        '''
+            Validate if an address is a valid tor or i2p hidden service
+        '''
         idLength = len(id)
         retVal = True
         idNoDomain = ''
@@ -146,5 +236,5 @@ class OnionrUtils:
                     retVal = False
             if not idNoDomain.isalnum():
                 retVal = False
-        return retVal
 
+        return retVal

onionr/tests.py

@@ -14,7 +14,7 @@
     You should have received a copy of the GNU General Public License
     along with this program.  If not, see <https://www.gnu.org/licenses/>.
 '''
-import unittest, sys, os, base64, tarfile, shutil, simplecrypt, logger
+import unittest, sys, os, base64, tarfile, shutil, simplecrypt, logger, btc
 
 class OnionrTests(unittest.TestCase):
     def testPython3(self):
@@ -23,18 +23,21 @@ class OnionrTests(unittest.TestCase):
             self.assertTrue(False)
         else:
             self.assertTrue(True)
+
     def testNone(self):
-        logger.debug('--------------------------')
+        logger.debug('-'*26 + '\n')
         logger.info('Running simple program run test...')
-        # Test just running ./onionr with no arguments
+
-        blank = os.system('./onionr.py')
+        blank = os.system('./onionr.py --version')
         if blank != 0:
             self.assertTrue(False)
         else:
             self.assertTrue(True)
+
     def testPeer_a_DBCreation(self):
-        logger.debug('--------------------------')
+        logger.debug('-'*26 + '\n')
         logger.info('Running peer db creation test...')
+
         if os.path.exists('data/peers.db'):
             os.remove('data/peers.db')
         import core
@@ -44,22 +47,27 @@ class OnionrTests(unittest.TestCase):
             self.assertTrue(True)
         else:
             self.assertTrue(False)
+
     def testPeer_b_addPeerToDB(self):
-        logger.debug('--------------------------')
+        logger.debug('-'*26 + '\n')
         logger.info('Running peer db insertion test...')
+
         import core
         myCore = core.Core()
         if not os.path.exists('data/peers.db'):
             myCore.createPeerDB()
-        if myCore.addPeer('facebookcorewwwi.onion') and not myCore.addPeer('invalidpeer.onion'):
+        if myCore.addPeer('6M5MXL237OK57ITHVYN5WGHANPGOMKS5C3PJLHBBNKFFJQOIDOJA====') and not myCore.addPeer('NFXHMYLMNFSAU==='):
             self.assertTrue(True)
         else:
             self.assertTrue(False)
+
     def testData_b_Encrypt(self):
         self.assertTrue(True)
         return
-        logger.debug('--------------------------')
+
+        logger.debug('-'*26 + '\n')
         logger.info('Running data dir encrypt test...')
+
         import core
         myCore = core.Core()
         myCore.dataDirEncrypt('password')
@@ -67,11 +75,14 @@ class OnionrTests(unittest.TestCase):
             self.assertTrue(True)
         else:
             self.assertTrue(False)
+
     def testData_a_Decrypt(self):
         self.assertTrue(True)
         return
-        logger.debug('--------------------------')
+
+        logger.debug('-'*26 + '\n')
         logger.info('Running data dir decrypt test...')
+
         import core
         myCore = core.Core()
         myCore.dataDirDecrypt('password')
@@ -79,34 +90,82 @@ class OnionrTests(unittest.TestCase):
             self.assertTrue(True)
         else:
             self.assertTrue(False)
-    def testPGPGen(self):
+
-        logger.debug('--------------------------')
+    def testConfig(self):
-        logger.info('Running PGP key generation test...')
+        logger.debug('-'*26 + '\n')
-        if os.path.exists('data/pgp/'):
+        logger.info('Running simple configuration test...')
-            self.assertTrue(True)
+
-        else:
+        import config
-            import core, netcontroller
+
-            myCore = core.Core()
+        config.check()
-            net = netcontroller.NetController(1337)
+        config.reload()
-            net.startTor()
+        configdata = str(config.get_config())
-            torID = open('data/hs/hostname').read()
+
-            myCore.generateMainPGP(torID)
+        config.set('testval', 1337)
-            if os.path.exists('data/pgp/'):
+        if not config.get('testval', None) is 1337:
-                self.assertTrue(True)
-    def testHMACGen(self):
-        logger.debug('--------------------------')
-        logger.info('Running HMAC generation test...')
-        # Test if hmac key generation is working
-        import core
-        myCore = core.Core()
-        key = myCore.generateHMAC()
-        if len(key) > 10:
-            self.assertTrue(True)
-        else:
             self.assertTrue(False)
+
+        config.set('testval')
+        if not config.get('testval', None) is None:
+            self.assertTrue(False)
+
+        config.save()
+        config.reload()
+
+        if not str(config.get_config()) == configdata:
+            self.assertTrue(False)
+
+        self.assertTrue(True)
+
+    def testBitcoinNode(self):
+        # temporarily disabled- this takes a lot of time the CI doesn't have
+        self.assertTrue(True)
+        #logger.debug('-'*26 + '\n')
+        #logger.info('Running bitcoin node test...')
+
+        #sbitcoin = btc.OnionrBTC()
+
+    def testPluginReload(self):
+        logger.debug('-'*26 + '\n')
+        logger.info('Running simple plugin reload test...')
+
+        import onionrplugins
+        try:
+            onionrplugins.reload('test')
+            self.assertTrue(True)
+        except:
+            self.assertTrue(False)
+
+    def testPluginStopStart(self):
+        logger.debug('-'*26 + '\n')
+        logger.info('Running simple plugin restart test...')
+
+        import onionrplugins
+        try:
+            onionrplugins.start('test')
+            onionrplugins.stop('test')
+            self.assertTrue(True)
+        except:
+            self.assertTrue(False)
+
+    def testPluginEvent(self):
+        logger.debug('-'*26 + '\n')
+        logger.info('Running plugin event test...')
+
+        import onionrplugins as plugins, onionrevents as events
+
+        plugins.start('test')
+        if not events.call(plugins.get_plugin('test'), 'test'):
+            self.assertTrue(False)
+
+        events.event('test', data = {'tests': self})
+
+        self.assertTrue(True)
+
     def testQueue(self):
-        logger.debug('--------------------------')
+        logger.debug('-'*26 + '\n')
         logger.info('Running daemon queue test...')
+
         # test if the daemon queue can read/write data
         import core
         myCore = core.Core()
@@ -124,4 +183,32 @@ class OnionrTests(unittest.TestCase):
         if command[0] == 'testCommand':
             if myCore.daemonQueue() == False:
                 logger.info('Succesfully added and read command')
+
+    def testHashValidation(self):
+        logger.debug('-'*26 + '\n')
+        logger.info('Running hash validation test...')
+
+        import core
+        myCore = core.Core()
+        if not myCore._utils.validateHash("$324dfgfdg") and myCore._utils.validateHash("f2ca1bb6c7e907d06dafe4687e579fce76b37e4e93b7605022da52e6ccc26fd2") and not myCore._utils.validateHash("f2ca1bb6c7e907d06dafe4687e579fce76b37e4e93b7605022da52e6ccc26fd$"):
+            self.assertTrue(True)
+        else:
+            self.assertTrue(False)
+
+    def testAddAdder(self):
+        logger.debug('-'*26 + '\n')
+        logger.info('Running address add+remove test')
+
+        import core
+        myCore = core.Core()
+        if not os.path.exists('data/address.db'):
+            myCore.createAddressDB()
+        if myCore.addAddress('facebookcorewwwi.onion') and not myCore.removeAddress('invalid'):
+            if myCore.removeAddress('facebookcorewwwi.onion'):
+                self.assertTrue(True)
+            else:
+                self.assertTrue(False)
+        else:
+            self.assertTrue(False)
+
 unittest.main()

onionr/timedHmac.py

@@ -30,6 +30,7 @@ class TimedHMAC:
         generatedHMAC = hmac.HMAC(base64.b64decode(base64Key).decode(), digestmod=self.hashAlgo)
         generatedHMAC.update(data + expire)
         self.HMACResult = generatedHMAC.hexdigest()
+
         return
 
     def check(self, data):

readme.md

@@ -1,18 +1,38 @@
-# Onionr
+![Onionr logo](./docs/onionr-logo.png)
 
 [![Build Status](https://travis-ci.org/beardog108/onionr.svg?branch=master)](https://travis-ci.org/beardog108/onionr)
+[![Open Source Love](https://badges.frapsoft.com/os/v3/open-source.png?v=103)](https://github.com/ellerbrock/open-source-badges/)
 
-P2P microblogging platform and social network, using Tor & I2P.
+
+Anonymous P2P platform, using Tor & I2P.
 
 Major work in progress.
 
 ***THIS SOFTWARE IS NOT USABLE OR SAFE YET.***
 
 
+**Roadmap/features:**
+
+* [X] Fully p2p/decentralized, no trackers or other single points of failure
+* [X] High level of anonymity
+* [ ] End to end encryption where applicable
+* [X] Optional non-encrypted blocks, useful for blog posts or public file sharing
+* [ ] Easy API system for integration to websites
+
 # Development
 
 This software is in heavy development. If for some reason you want to get involved, get in touch first.
 
+**Onionr API and functionality is subject to non-backwards compatible change during development**
+
+# Donate
+
+Bitcoin/Bitcoin Cash: 1onion55FXzm6h8KQw3zFw2igpHcV7LPq
+
 ## Disclaimer
 
 The Tor Project, I2P developers, and anyone else do not own, create, or endorse this project, and are not otherwise involved.
+
+The badges (besides travis-ci build) are by Maik Ellerbrock is licensed under a Creative Commons Attribution 4.0 International License.
+
+The onion in the Onionr logo is adapted from [this](https://commons.wikimedia.org/wiki/File:Red_Onion_on_White.JPG) image by Colin on Wikimedia under a Creative Commons Attribution-Share Alike 3.0 Unported license

requirements.txt

@@ -1,11 +1,8 @@
 PyNaCl==1.2.1
-gnupg==2.3.1
+requests==2.12.4
 Flask==0.12.2
-requests==2.18.4
-urllib3==1.22
 simple_crypt==4.1.7
+urllib3==1.19.1
 sha3==0.2.1
-pycrypto==2.6.1
-pynacl==1.2.1
 PySocks==1.6.8
 bitpeer.py==0.4.7.5

test.sh

@@ -1,3 +0,0 @@
-#!/bin/sh
-cd onionr
-./tests.py