Category: golang | nickler's

Data-Driven De-Anonymization in Bitcoin

2016-06-25T23:41:00+00:00

Abstract

We analyse the performance of several clustering algorithms in the digital peer- to-peer currency Bitcoin. Clustering in Bitcoin refers to the task of finding addresses that belongs to the same wallet as a given address. In order to assess the effectiveness of clustering strategies we exploit a vulner- ability in the implementation of Connection Bloom Filtering to capture ground truth data about 37,585 Bitcoin wallets and the addresses they own. In addition to well-known clustering techniques, we introduce two new strategies, apply them on addresses of the collected wallets and evaluate precision and recall using the ground truth. Due to the nature of the Connection Bloom Filtering vulnerability the data we collect is not without errors. We present a method to correct the performance metrics in the presence of such inaccuracies. Our results demonstrate that even modern wallet software can not protect its users properly. Even with the most basic clustering technique known as multi- input heuristic, an adversary can guess on average 68.59% addresses of a victim. We show that this metric can be further improved by combining several more sophisticated heuristics.

Read full thesis

Guessing Bitcoin's P2P Connections

2015-03-06T14:22:00+00:00

The paper Deanonymisation of clients in Bitcoin P2P network (2014) by Biryukov, Khovratovich and Pustogarov (BKP), who describe an attack on Bitcoin Core clients, has started some discussion lately. The main idea of the paper is to first get a set of nodes $E_v$ to which your victim $v$ is directly connected to (“entry nodes”). Second, for each transaction $t$ record the $10$ nodes $P_t$ which first propagate $t$. The authors experimentally show that if $|E_v \cap P_t|$ is bigger than $3$ then there is a pretty high chance that $t$ actually originated from $v$. However, both attack stages basically require a Sybil attack – the attacker has to be connected to a lot of nodes a lot of times. ‘A lot’ means that in their experiments they had 50 connection to each full node (~250) in the test network. As a result, such an attack seems to be powerful, but certainly won’t be undetected.

In this post I show that the first stage of the attack, namely learning the nodes a victim is directly connected to can be done with a single connection to the victim. In addition to BKP’s attack, knowing all outbound peers of a client could significantly increase the success probability of a double spend. Note that all experiments are based on Bitcoin Core 0.9.4, but 0.10.0 shows the same behavior.

TLDR The attacker can reliably guess all of the outbound connections of a victim by making a selection from the known addresses of a victim based on the timestamp of the addresses.

Update A fix has been merged to bitcoind. The timestamp is not updated anymore when receiving a message from a connected peer. Instead, it is only updated when the peer disconnects. The fix is released in bitcoin core 0.10.1.

Learning connections using addr propagation

When a node $n$ connects to another peer $p$ in the network it advertises its address using the “addr” message. The peer will select a number of its own peers at random which are “responsible” for $n$’s address. Then the address is forwarded to responsible peers to spread the knowledge about $n$ in the network. The number of responsible peers is either $1$ or $2$ depending on whether the address is reachable by $p$.

BKP’s attack works by recording the set of peers that first propagated a victim’s address. In order to have good chance to be in the set of responsible peers for the address, the attacker has to hold a significant number of connections to each full node in the network. Note that it is possible to have multiple connections from a single public address to a peer.

The getaddr message

It turns out that an attacker can simply infer the peers of a victim by sending getaddr messages to him.

In bitcoin, the address structures that are send via the addr message do not only contain the IP adress and port but also a timestamp. The timestamp’s role is ensuring that terminated nodes vanish from the networks knowledge and it is regular refreshed by the nodes which have an interaction (more about that later) with the peer at that address. Bitcoin nodes usually record the addresses they hear about and send them in a reply to a getaddr using the addr message.

The following experiments show that an attacker can guess some or all of the direct peers of a victim by sorting the known addresses of the victim based on the timestamp.

A minor obstacle is that a node replies to a single getaddr message only with maximal 2500 addrs selected uniformly at random. In order to get a certain percentage $\tau$ of the known addresses of a node the attacker has to send multiple getaddr messages and record the percentage that is new to her.

S = {}
while(true):
    send_getaddr()
    T = response()
    tau' = 1 - ((|T - S|) / |T|) 
    S <- S $\cup$ T
    if \tau' > \tau
        break
return S

Experiments show that if we wait 10 seconds after each getaddr request it takes around $3.5$ minutes to collect $\tau$ percent addresses ($13,500$ in this case).

Experimental results

I set up a victim node $v$, which is just a regular bitcoin node. The attacker $a$ is a node that connects to $v$ via the P2P network and queries the known nodes of $v$. Second, $a$ connects to $v$ via the RPC interface and gets the true peers.

The attacker code (btcP2PStruct) is available on github. Thanks to the btcwire package it is very simple to write this kind of code.

You can find all the data to produce the graphs in the project repository.

First we consider the case where $v$ does not accept incoming connections (“client” in BKP’s terms). $v$ was running for 2 days and I recorded data for every hour but I will only discuss the last measurement because the data is very similar.

Note that $v$ returned $12,868$ known addresses. Also, a client usually has maximally 8 peers due to the default maximum number of outbound connections. This implies that an attacker can not start start this attack on a client that is not connected to her. Here we see that if the attacker obtains all peers of $v$ (without any false positives in this case).

Next, the case for the full node, which I left running for 8 days.

Again it is evident that an attacker can reliably determine all outbound connections of the victim using a threshold of 20 minutes. However, inbound peers can only be detected very poorly.

The reason for finding all outbound peers is is this logic in bitcoin-core which refreshes the timestamp on every message of outbound nodes.

Reducing false positives

BKP mention a neat trick how to determine if two nodes $v_1$ and $v_2$ are connected. First, the attacker connect to $v_1$ and $v_2$ and send addr messages containing bogus addresses to $v_1$. Then, she counts the number of times one of these addresses is received from $v_2$. However, the authors leave open how many messages you need send to be certain about the hypothesis.

As we already know, the address is forwarded only to two responsible nodes so we have to compute the probabilities of our node being responsible. Using the binomial distribution we can compute the likelihood of receiving a certain number of addresses back given that we sent a certain number of addresses.

I’ve done the math using this code and some assumptions regarding the structure (edges are uniformly iid). Also, the attacker has to know or approximate the number of peers of a node, which can be done with a similar method than the one described. Connect two times to the victim, send and note the ratio of returned addr messages. If you can not connect to the node, it will most likely have 8 peers.

This theoretical model shows that that if $v_1$ is a full node and $v_2$ is a client then we need about 2000 messages to determine if they are connected with 95% probability. Similarly, if $v_1$ and $v_2$ are full nodes, the attacker needs to send 20000 messages.

However, in order to remain polite in the network this attack needs start from a candidate set of nodes. Therefore, it could be a useful method to remove the false positives which were obtained with the “getaddr”-fingerprint.

Conclusion

It should be pointed out that even if you know a victim’s entry nodes you can not simply connect to those few and listen for transactions. This is because “trickling” prevents estimating the origin of a transaction without further assumptions or doing BKP’s Sybil attack. However, knowing all outbound peers of a client could significantly increase the success probability of a double spend.

Update The fix removes the update every 20 minutes and updates on disconnect

Privacy in BitcoinJ

2015-02-12T15:53:00+00:00

As part of my epic quest to apply supervised machine learning to the blockchain in order to discover transaction patterns, I reviewed various wallet implementations in the hope of finding privacy leaks.

tl;dr If you are using a wallet that is built upon BitcoinJ, such as Android Wallet, Multibit and Hive Wallet, you have almost zero wire privacy. An attacker who manages to connect to your wallet is easily able to figure out all addresses you control. This is not very likely to get fixed in the near future.

Update: Mike Hearn’s reply addresses additional problems and improvements. There was also accompanying discussion on reddit.

Bloom Filters for SPV Nodes

A Bloom filter is a probabilistic data structure that is used to test whether an element is a member of a set. Bitcoin SPV nodes that use BIP 37 (we call them thin clients from now on) put all public keys they are interested in into the Bloom filter and send the filter to their peers. Upon receiving a new transaction, peers query the Bloom filter and only relay the transaction to the BIP 37 node if the query returned true. Thus, thin clients normally only receive the transactions they are really interested in, i.e. mostly transactions that include one of the wallet’s keys.

The advantage of using a Bloom filter instead of just broadcasting all your pubkeys is that a Bloom filter is faster and more space-efficient at the cost of some false positives. That means the thin client will receive transactions that include pubkeys which were not put into the filter. Usually, the parameters of a Bloom filter are computed such that a certain target false positive rate (fp) is achieved. We want the fp rate to be relatively small (say 0.05%) to reduce bandwidth usage.

Bloom Filters and Privacy

BIP 37 states:

Privacy: Because Bloom filters are probabilistic, with the false positive rate chosen by the client, nodes can trade off precision vs bandwidth usage. A node with access to lots of bandwidth may choose to have a high fp rate, meaning the remote peer cannot accurately know which transactions belong to the client and which don’t.

This has created a misunderstanding between what is ideally possible with Bloom filters and how the reality looks like. I’ll focus on BitcoinJ because it is the most widely used implementation of BIP 37, but similar vulnerabilities might exist in other implementations as well. Unfortunately, in the current BitcoinJ implementation Bloom filters are just as bad for your privacy as broadcasting your pubkeys directly to your peers.

A Simple Attack

The main idea behind this vulnerability is that BitcoinJ puts both pubkey and pubkeyhash into the Bloom filter which substantially reduces the false positive rate.

If you create a completely fresh wallet, BitcoinJ holds 271 pubkeys and computes the parameters of the Bloom filter such that the fp rate for (271*2)+100 elements is equal to 0.05%. Because bitcoinj initially puts only 271*2 elements into the filter (pubkey and corresponding pubkeyhash) the effective false positive rate is fp=0.000146.

The vulnerability is that if a pubkey is truly in the filter then querying both pubkey and pubkeyhash must return true. Because the pubkeyhash is just another almost uniformly random string, the probability of a false positive for the attacker is fp' = fp^2 = 0.0000000213. I obtained around 56 million pubkeys from the blockchain (mid-January), which theoretically results in 56 million * fp' = 1.19 expected false positives when scanning the blockchain.

Experimental results

I ran 20 crawlers since the beginning of December and collected 70,000 distinct filters until now. These crawlers just listen for a filterload message and try to be really polite by disconnecting after 2 minutes and not sending anything. The probability that a randomly selected DNS seed returns at least one of the crawlers is 4.3%.

In fact, most of the Bloom filters from recent BitcoinJ versions show a experimental false positive rate around 0.000146. The experimental fp rate is computed by querying the filter with millions of elements which are certainly not pubkeys. Android Wallet 4.16, 4.17, 4.18 for example use the most recent BitcoinJ version (12.2) and make up 52% of the data. However, there is also MultiBit 0.5.18 whose effective fp rate is smaller than 0.00000001.

We are currently starting to analyze all filters using the described “attack” and we expect that this will take several weeks. What we’ve already seen is that the theoretical fp' really holds, i.e. if you create a fresh wallet and scan the whole blockchain you most likely get one false positive pubkey.

(Slightly) More Difficult Attacks

You might think that the problem is easily fixed by trading off bandwidth for more privacy and increase the fp rate to fp = sqrt(0.0005) = 0.0224. On the one hand this might seriously impact the bandwidth of mobile clients, and on the other, there is another another general class of vulnerabilities concerning Bloom filters: If an attacker manages to obtain multiple, different filters from the same Wallet, he can compute the intersection of pubkeys that match the filters and therefore removes the false positive noise similar to the “simple attack”. Different filters mean that they have different total size of a different Nonce. Sending different filters can happen in BitcoinJ due to multiple reasons, for example

Restart. BitcoinJ stores the filter’s nonce in volatile memory.
Creation of new keys. When the wallet creates many new keys the filter gets ‘full’ and thus has to be recomputed.
Measured false positive rate is too high. BitcoinJ measures the false positive rate of transactions it receives. When it becomes too high the filter is recomputed.

Conclusion

I do think this is a critical privacy leak as it doesn’t require a sophisticated attack and wallets have practically been broadcasting all their pubkeys for years. Not only the addresses you see in your wallet, but also a lot of your future addresses have been exposed. From now on you should assume that the kind of bulk data collection I did is happening. It is difficult to say how accurate and stealthy targeted attacks would be.

According to Mike Hearn, the creator of BitcoinJ, the problems have been known from the start but fixing these issues is far from trivial because “lying consistently is hard”. I fully agree with this. Someone needs to make it their project for a few months.

There are some simple ideas to slightly improve the current status such as deploying nodes that broadcast fake bloom filters. Arthur Gervais et al., 2014 were the first to publish an academic paper on the topic and propose some more or less vague suggestions. One idea I find interesting is that thin clients should be able to install multiple filters at their peers such that no pubkey is shared between the filters. Thus, instead of recomputing the filter when the wallet creates new addresses, it would create an entirely fresh filter for the next keys. One disadvantage is that at the moment multiple filters per peer is not supported by the bitcoin wire protocol. Another issue with Bloom filters is that an attacker could safely assume that the probability is higher for two pubkeys to belong to the same person if they are closer in the transaction graph. As a countermeasure the wallet could deliberately put existing foreign pubkeys that are close into the filter.

I feel sorry for the people whose privacy has been potentially compromised unknowingly by malicious parties and we certainly won’t give away the data set but nonetheless it is really exciting what can be found in the data. If you have suggestions what to look out for and what would be interesting (not necessarily concerning machine learning) feel free to contact me.