bug fixes and performence improvements

docs/whitepaper.md

@@ -39,21 +39,21 @@ When designing Onionr we had these main goals in mind:
 
 At its core, Onionr is merely a description for storing data in self-verifying packages ("blocks"). These blocks can be encrypted to a user (or for one's self), encrypted symmetrically, or not at all. Blocks can be signed by their creator, but regardless, they are self-verifying due to being identified by a sha3-256 hash value; once a block is created, it cannot be modified.
 
-Onionr exchanges a list of blocks between all nodes. By default, all nodes download and share all other blocks, however this is configurable. Blocks do not rely on any particular order of reciept or transport mechanism.
+Onionr exchanges a list of blocks between all nodes. By default, all nodes download and share all other blocks, however this is configurable. Blocks do not rely on any particular order of receipt or transport mechanism.
 
 ## User IDs
 
 User IDs are simply Ed25519 public keys. They are represented in Base32 format, or encoded using the [PGP Word List](https://en.wikipedia.org/wiki/PGP_word_list).
 
-Public keys can be generated deterministicly with a password using a key derivation function (Argon2id). This password can be shared between many users in order to share data anonymously among a group, using only 1 password. This is useful in some cases, but is risky, as if one user causes the key to be compromised and does not notify the group or revoke the key, there is no way to know.
+Public keys can be generated deterministically with a password using a key derivation function (Argon2id). This password can be shared between many users in order to share data anonymously among a group, using only 1 password. This is useful in some cases, but is risky, as if one user causes the key to be compromised and does not notify the group or revoke the key, there is no way to know.
 
 ## Nodes
 
-Although Onionr is transport agnostic, the only supported transports in the reference implemetation are Tor .onion services and I2P hidden services. Nodes announce their address on creation by connecting to peers specified in a bootstrap file. Peers in the bootstrap file have no special permissions aside from being default peers.
+Although Onionr is transport agnostic, the only supported transports in the reference implementation are Tor .onion services and I2P hidden services. Nodes announce their address on creation by connecting to peers specified in a bootstrap file. Peers in the bootstrap file have no special permissions aside from being default peers.
 
 ### Node Profiling
 
-To mitigate maliciously slow or unreliable nodes, Onionr builds a profile on nodes it connects to. Nodes are assigned a score, which raises based on the amount of successful block transfers, speed, and reliabilty of a node, and reduces the score based on how unreliable a node is. If a node is unreachable for over 24 hours after contact, it is forgotten. Onionr can also prioritize connection to 'friend' nodes.
+To mitigate maliciously slow or unreliable nodes, Onionr builds a profile on nodes it connects to. Nodes are assigned a score, which raises based on the amount of successful block transfers, speed, and reliability of a node, and reduces the score based on how unreliable a node is. If a node is unreachable for over 24 hours after contact, it is forgotten. Onionr can also prioritize connection to 'friend' nodes.
 
 ## Block Format
 
@@ -90,3 +90,5 @@ This can be done either by the creator of the block prior to generation, or by a
 In addition, randomness beacons such as the one operated by [NIST](https://beacon.nist.gov/home) or the hash of the latest blocks in a cryptocurrency network could be used to affirm that a block was at least not *created* before a given time.
 
 # Direct Connections
+
+We propose a method of using Onionr's block sync system to enable direct connections between peers by having one peer request to connect to another using the peer's public key. Since the request is within a standard block, proof of work must be used to request connection, and in addition neither party knows the other's .onion or .i2p hostname, let alone IP address. If the requested peer is available and wishes to accept the connection,Onionr will generate a temporary .onion address for the other peer to connect to. Alternatively, a reverse connection may be formed, which is faster to establish but requires message brokering instead of a standard socket.

onionr/config.py

@@ -105,7 +105,7 @@ def check():
             open(get_config_file(), 'a', encoding="utf8").close()
             save()
     except:
-        logger.warn('Failed to check configuration file.')
+        logger.debug('Failed to check configuration file.')
 
 def save():
     '''

onionr/core.py

@@ -130,7 +130,7 @@ class Core:
 
         events.event('pubkey_add', data = {'key': peerID}, onionr = None)
 
-        conn = sqlite3.connect(self.peerDB, timeout=10)
+        conn = sqlite3.connect(self.peerDB, timeout=30)
         hashID = self._crypto.pubKeyHashID(peerID)
         c = conn.cursor()
         t = (peerID, name, 'unknown', hashID, 0)
@@ -160,7 +160,7 @@ class Core:
         if type(address) is None or len(address) == 0:
             return False
         if self._utils.validateID(address):
-            conn = sqlite3.connect(self.addressDB, timeout=10)
+            conn = sqlite3.connect(self.addressDB, timeout=30)
             c = conn.cursor()
             # check if address is in database
             # this is safe to do because the address is validated above, but we strip some chars here too just in case
@@ -193,7 +193,7 @@ class Core:
         '''
 
         if self._utils.validateID(address):
-            conn = sqlite3.connect(self.addressDB, timeout=10)
+            conn = sqlite3.connect(self.addressDB, timeout=30)
             c = conn.cursor()
             t = (address,)
             c.execute('Delete from adders where address=?;', t)
@@ -213,7 +213,7 @@ class Core:
         '''
 
         if self._utils.validateHash(block):
-            conn = sqlite3.connect(self.blockDB, timeout=10)
+            conn = sqlite3.connect(self.blockDB, timeout=30)
             c = conn.cursor()
             t = (block,)
             c.execute('Delete from hashes where hash=?;', t)
@@ -261,7 +261,7 @@ class Core:
             raise Exception('Block db does not exist')
         if self._utils.hasBlock(newHash):
             return
-        conn = sqlite3.connect(self.blockDB, timeout=10)
+        conn = sqlite3.connect(self.blockDB, timeout=30)
         c = conn.cursor()
         currentTime = self._utils.getEpoch() + self._crypto.secrets.randbelow(301)
         if selfInsert or dataSaved:
@@ -318,7 +318,7 @@ class Core:
                 #blockFile.write(data)
                 #blockFile.close()
                 onionrstorage.store(self, data, blockHash=dataHash)
-                conn = sqlite3.connect(self.blockDB, timeout=10)
+                conn = sqlite3.connect(self.blockDB, timeout=30)
                 c = conn.cursor()
                 c.execute("UPDATE hashes SET dataSaved=1 WHERE hash = ?;", (dataHash,))
                 conn.commit()
@@ -341,7 +341,7 @@ class Core:
         if not os.path.exists(self.queueDB):
             self.dbCreate.createDaemonDB()
         else:
-            conn = sqlite3.connect(self.queueDB, timeout=10)
+            conn = sqlite3.connect(self.queueDB, timeout=30)
             c = conn.cursor()
             try:
                 for row in c.execute('SELECT command, data, date, min(ID), responseID FROM commands group by id'):
@@ -367,7 +367,7 @@ class Core:
         retData = True
 
         date = self._utils.getEpoch()
-        conn = sqlite3.connect(self.queueDB, timeout=10)
+        conn = sqlite3.connect(self.queueDB, timeout=30)
         c = conn.cursor()
         t = (command, data, date, responseID)
 
@@ -410,7 +410,7 @@ class Core:
         '''
             Clear the daemon queue (somewhat dangerous)
         '''
-        conn = sqlite3.connect(self.queueDB, timeout=10)
+        conn = sqlite3.connect(self.queueDB, timeout=30)
         c = conn.cursor()
 
         try:
@@ -428,7 +428,7 @@ class Core:
         '''
             Return a list of addresses
         '''
-        conn = sqlite3.connect(self.addressDB, timeout=10)
+        conn = sqlite3.connect(self.addressDB, timeout=30)
         c = conn.cursor()
         if randomOrder:
             addresses = c.execute('SELECT * FROM adders ORDER BY RANDOM();')
@@ -456,7 +456,7 @@ class Core:
             randomOrder determines if the list should be in a random order
             trust sets the minimum trust to list
         '''
-        conn = sqlite3.connect(self.peerDB, timeout=10)
+        conn = sqlite3.connect(self.peerDB, timeout=30)
         c = conn.cursor()
 
         payload = ''
@@ -505,7 +505,7 @@ class Core:
             trust int           4
             hashID text         5
         '''
-        conn = sqlite3.connect(self.peerDB, timeout=10)
+        conn = sqlite3.connect(self.peerDB, timeout=30)
         c = conn.cursor()
 
         command = (peer,)
@@ -531,7 +531,7 @@ class Core:
             Update a peer for a key
         '''
 
-        conn = sqlite3.connect(self.peerDB, timeout=10)
+        conn = sqlite3.connect(self.peerDB, timeout=30)
         c = conn.cursor()
 
         command = (data, peer)
@@ -563,7 +563,7 @@ class Core:
             introduced  10
         '''
 
-        conn = sqlite3.connect(self.addressDB, timeout=10)
+        conn = sqlite3.connect(self.addressDB, timeout=30)
         c = conn.cursor()
 
         command = (address,)
@@ -588,7 +588,7 @@ class Core:
             Update an address for a key
         '''
 
-        conn = sqlite3.connect(self.addressDB, timeout=10)
+        conn = sqlite3.connect(self.addressDB, timeout=30)
         c = conn.cursor()
 
         command = (data, address)
@@ -609,7 +609,7 @@ class Core:
         if dateRec == None:
             dateRec = 0
 
-        conn = sqlite3.connect(self.blockDB, timeout=10)
+        conn = sqlite3.connect(self.blockDB, timeout=30)
         c = conn.cursor()
 
         # if unsaved:
@@ -630,7 +630,7 @@ class Core:
             Returns the date a block was received
         '''
 
-        conn = sqlite3.connect(self.blockDB, timeout=10)
+        conn = sqlite3.connect(self.blockDB, timeout=30)
         c = conn.cursor()
 
         execute = 'SELECT dateReceived FROM hashes WHERE hash=?;'
@@ -646,7 +646,7 @@ class Core:
             Returns a list of blocks by the type
         '''
 
-        conn = sqlite3.connect(self.blockDB, timeout=10)
+        conn = sqlite3.connect(self.blockDB, timeout=30)
         c = conn.cursor()
 
         if orderDate:
@@ -665,7 +665,7 @@ class Core:
 
     def getExpiredBlocks(self):
         '''Returns a list of expired blocks'''
-        conn = sqlite3.connect(self.blockDB, timeout=10)
+        conn = sqlite3.connect(self.blockDB, timeout=30)
         c = conn.cursor()
         date = int(self._utils.getEpoch())
 
@@ -683,7 +683,7 @@ class Core:
             Sets the type of block
         '''
 
-        conn = sqlite3.connect(self.blockDB, timeout=10)
+        conn = sqlite3.connect(self.blockDB, timeout=30)
         c = conn.cursor()
         c.execute("UPDATE hashes SET dataType = ? WHERE hash = ?;", (blockType, hash))
         conn.commit()
@@ -710,7 +710,7 @@ class Core:
         if key not in ('dateReceived', 'decrypted', 'dataType', 'dataFound', 'dataSaved', 'sig', 'author', 'dateClaimed', 'expire'):
             return False
 
-        conn = sqlite3.connect(self.blockDB, timeout=10)
+        conn = sqlite3.connect(self.blockDB, timeout=30)
         c = conn.cursor()
         args = (data, hash)
         c.execute("UPDATE hashes SET " + key + " = ? where hash = ?;", args)

onionr/static-data/bootstrap-nodes.txt

@@ -0,0 +1 @@
+svlegnabtuh3dq6ncmzqmpnxzik5mita5x22up4tai2ekngzcgqbnbqd.onion

onionr/static-data/default_config.json

@@ -79,7 +79,7 @@
     "peers" : {
         "minimum_score" : -100,
         "max_stored_peers" : 5000,
-        "max_connect" : 10
+        "max_connect" : 50
     },
 
     "timers" : {