Category: bitcoin | nickler's

MuSig2: Simple Two-Round Schnorr Multisignatures

2020-11-29T23:25:00+00:00

Abstract: Multi-signatures enable a group of signers to produce a single signature on a given message. Recently, Drijvers et al. (S&P’19) showed that all thus far proposed two-round multi-signature schemes in the DL setting (without pairings) are insecure under concurrent sessions, i.e., if a single signer participates in multiple signing sessions concurrently. While Drijvers et al. improve the situation by constructing a secure two-round scheme, saving a round comes with the price of having less compact signatures. In particular, the signatures produced by their scheme are more than twice as large as Schnorr signatures, which arguably are the most natural and compact among all practical DL signatures and are therefore becoming popular in cryptographic applications (e.g., support for Schnorr signature verification has been proposed to be included in Bitcoin). If one needs a multi-signature scheme that can be used as a drop-in replacement for Schnorr signatures, then one is either forced to resort to a three-round scheme such as MuSig (Maxwell et al., DCC 2019) or MSDL-pop (Boneh, Drijvers, and Neven, ASIACRYPT 2018), or to accept that signing sessions are only secure when run sequentially, which may be hard to enforce in practice, e.g., when the same signing key is used by multiple devices.

In this work, we propose MuSig2, a novel and simple two-round multi-signature scheme variant of the MuSig scheme. Our scheme is the first multi-signature scheme that simultaneously i) is secure under concurrent signing sessions, ii) supports key aggregation, iii) outputs ordinary Schnorr signatures, iv) needs only two communication rounds, and v) has similar signer complexity as regular Schnorr signatures. Furthermore, our scheme is the first multi-signature scheme in the DL setting that supports preprocessing of all but one rounds, effectively enabling a non-interactive signing process, without forgoing security under concurrent sessions. The combination of all these features makes MuSig2 highly practical. We prove the security of MuSig2 under the one-more discrete logarithm (OMDL) assumption in the random oracle model, and the security of a more efficient variant in the combination of the random oracle and algebraic group models.

BIP-{Schnorr,Taproot,Tapscript}

2020-11-29T22:59:00+00:00

BIP-Schnorr Abstract: This document proposes a standard for 64-byte Schnorr signatures over the elliptic curve secp256k1.
BIP-Taproot Abstract: This document proposes a new SegWit version 1 output type, with spending rules based on Taproot, Schnorr signatures, and Merkle branches.
BIP-Tapscript Abstract: This document specifies the semantics of the initial scripting system under BIP341 [BIP-Taproot].

Some highlights:

2018-01-23: Taproot idea is proposed
2018-04-28: Work on BIP-Schnorr and what was known as “BIP-metas” (MErkle branches, TAproot, and Schnorr signatures) begins
2018-07-06: BIP-Schnorr draft is published
2018-09-25: libsecp256k1 schnorrsig PR opened
2019-05-06: BIP-Taproot and BIP-Tapscript proposals are published
2020-01-21: Bitcoin Core taproot PR opened
2020-09-11: libsecp256k1 schnorrsig PR merged
2020-10-15: Bitcoin Core taproot PR merged
2021-05-01: Bitcoin Core release with “Speedy Trial” activation parameters
2021-06-13 (block 687456): Taproot LOCKED-IN
Block 709632: Taproot activation

Protip: If you have troubles memorizing BIP numbers (like me), achow101 observed that BIP-Taproot’s number, 341, are the reversed digits of BIP-143. The segwit version 0 transaction digest is defined in BIP-143 and version 1 digest is defined in BIP-341.

MuSig-DN: Schnorr Multisignatures with Verifiably Deterministic Nonces

2020-11-29T22:53:00+00:00

Abstract: MuSig is a multi-signature scheme for Schnorr signatures, which supports key aggregation and is secure in the plain public key model. Standard derandomization techniques for discrete logarithm-based signatures such as RFC 6979, which make the signing procedure immune to catastrophic failures in the randomness generation, are not applicable to multi-signatures as an attacker could trick an honest user into producing two different partial signatures with the same randomness, which would reveal the user’s secret key.

In this paper, we propose a variant of MuSig in which signers generate their nonce deterministically as a pseudorandom function of the message and all signers’ public keys and prove that they did so by providing a non-interactive zero-knowledge proof to their cosigners. The resulting scheme, which we call MuSig-DN, is the first Schnorr multi-signature scheme with deterministic signing. Therefore its signing protocol is robust against failures in the randomness generation as well as attacks trying to exploit the statefulness of the signing procedure, e.g., virtual machine rewinding attacks. As an additional benefit, a signing session in MuSig-DN requires only two rounds instead of three as required by all previous Schnorr multi-signatures including MuSig. To instantiate our construction, we identify a suitable algebraic pseudorandom function and provide an efficient implementation of this function as an arithmetic circuit. This makes it possible to realize MuSig-DN efficiently using zero-knowledge proof frameworks for arithmetic circuits which support inputs given in Pedersen commitments, e.g., Bulletproofs. We demonstrate the practicality of our technique by implementing it for the secp256k1 elliptic curve used in Bitcoin.

Read more on the Blockstream Blog or watch the pre-recorded talk we presented at the CCS 2020 conference.

X-only Pubkeys and Insecure MuSig Shortcuts

2019-11-19T13:35:00+00:00

There are two posts I recently contributed to Blockstream’s engineering blog expanding on the talk I gave at The Lightning Conference 2019. Cross-posting them here because they fit the theme of this blog:

Reducing Bitcoin Transaction Sizes with x-only Pubkeys

This article is about the recent introduction of so-called x-only pubkeys to the Bitcoin Improvement Proposal BIP-schnorr […] significantly reducing the weight of every transaction output without any loss in security. By removing the Y-coordinate byte from compressed public keys currently used in Bitcoin, public keys end up with a 32-byte representation. We are going to look at how it works, why that’s useful, and sketch a security proof.
Insecure Shortcuts in MuSig

Using BIP-Schnorr-based multisignatures, no matter how many signers are involved, the result is a single public key and a single signature indistinguishable from a regular, single-signer BIP-Schnorr signature. This article is about optimizing implementations of multisignature protocols and why seemingly harmless changes can totally break the security.

Secure Protocols on BIP-taproot

2019-06-25T22:42:00+00:00

At Breaking Bitcoin 2019 in Amsterdam I gave a talk about how to build secure protocols on BIP-taproot or more specifically how to avoid the dangers we learned about so far. There was not enough time to cover everything. The talk also gives an introduction to how to use our MuSig implementation in libsecp256k1-zkp. The video recording is on youtube (slides). Thanks to kanzure there’s also a transcript of the talk.

Erratum: MuSig nonces can not be pre-shared. Only nonce commitments. See https://github.com/ElementsProject/secp256k1-zkp/pull/73 for details.

nix-bitcoin

2019-06-25T22:39:00+00:00

nix-bitcoin (github.com/fort-nix/nix-bitcoin) is a project I contribute to in my spare time that provides nix packages and nixos modules for easily installing Bitcoin nodes and higher layer protocols. The initial idea was to build myself a lightning node in a reproducible way. I talked more about the motivation and how to use it at the LightningHackdayMUC (video, slides).

Schnorr and Taproot in Lightning

2018-09-04T13:24:00+00:00

Last weekend a bunch of hackers assembled for the 3rd Lightning Netword Hackday in Berlin. The event was packed with interesting sessions, neat hacks and exciting discussions which were concluded with the traditional dinner & drinks at ROOM77. I gave a talk about “Schnorr and Taproot in Lightning” (slides, video) focusing on privacy and security implications.

Blind Signatures in Scriptless Scripts

2018-07-31T14:53:00+00:00

At the recent Building on Bitcoin conference in Lisbon I gave a talk about a few new ideas in the scriptless scripts framework. The first part was mainly about blind coinswaps, which is a way to swap bitcoins with a tumbler without revealing which coin are swapped. The second part about how to exchange ecash tokens peer-to-peer using scriptless scripts and Brands credentials. You can find the talk on youtube and the slides here. Thanks to kanzure there’s also a transcript of the talk.

EDIT: I’ve added a note about the security of Blind Schnorr signatures against forgery to the slides. In short, a naive implementation of the scheme is vulnerable to Wagner’s attack. An attacker can forge a signature using 65536 parallel signing sessions and O(2^32) work.

The 2016 Backdoored Cryptocurrency Contest Winner

2016-08-31T22:43:00+00:00

Thanks to a quite a bit of luck I won this years version of the Underhanded Crypto Contest focused on crypto currencies. The submission consists of a writeup and – of course – backdoored code.

Data-Driven De-Anonymization in Bitcoin

2016-06-25T23:41:00+00:00

Slides

Abstract

We analyse the performance of several clustering algorithms in the digital peer- to-peer currency Bitcoin. Clustering in Bitcoin refers to the task of finding addresses that belongs to the same wallet as a given address. In order to assess the effectiveness of clustering strategies we exploit a vulner- ability in the implementation of Connection Bloom Filtering to capture ground truth data about 37,585 Bitcoin wallets and the addresses they own. In addition to well-known clustering techniques, we introduce two new strategies, apply them on addresses of the collected wallets and evaluate precision and recall using the ground truth. Due to the nature of the Connection Bloom Filtering vulnerability the data we collect is not without errors. We present a method to correct the performance metrics in the presence of such inaccuracies. Our results demonstrate that even modern wallet software can not protect its users properly. Even with the most basic clustering technique known as multi- input heuristic, an adversary can guess on average 68.59% addresses of a victim. We show that this metric can be further improved by combining several more sophisticated heuristics.

Read full thesis

A Validation-cost Metric for Bitcoin

2015-12-13T20:36:00+00:00

This is the transcript of a talk I gave at the Scaling Bitcoin Conference 2015 in Hong Kong. See this mailing list post by Greg Maxwell for a general summary of the scaling measures that Bitcoin Core developers are adopting. The slides accompanying the transcript can be found here.

Motivation

As we’ve seen over the last two days scalability is a multidimensional problem. One of the main topics of research is increasing the blocksize to increase transaction throughput. The assumption is that as technological progress is continuing and transaction throughput is increased accordingly, the cost for runnning a fully validating node stays constant.

However, blocksize proposals usually only directly change one aspect of full node costs – the blocksize. The actual costs for a node are composed of multiple factors, such as the resources required to validate a block or to store the utxos. And these factors are not necessarily in a linear relationship with each other. This has been discussed more detailed in Mark’s talk at the previous Scaling Bitcoin conference.

The most prominent example for showing non-linear relationships consists of putting as many OP_CHECKSIG operations into a single transaction as possible. For each checksig operation, the whole transaction is hashed and a signature is verified. Assuming 1MB blocks, it is possible to create a block that takes more than 10 minutes to validate on my 2014 laptop. It is clear that each proposal that increases blocksize also needs a strategy to deal with these non-linearities.

One of those strategies is to put a hard limit the number of signature verifications and the number of bytes that are hashed for a block to be valid. We see some problems with this approach: First, as it stands there is no intuitive way to choose these limits nor how they grow with the blocksize. Second, there are other factors that influence validation cost, which might not relevant now, but could get significant in bigger blocks if not properly limited. For example, it is possible to create a 1MB block that takes 5 seconds to validate on my laptop, which just consists of as many HASH opcodes as possible. And third, placing hard limits on certain factors completely ignores the relationship between those factors.

These relationships exist, because thiose factors influence validation cost in some way. This brings us to the concept of cost metrics.

The goal of the cost metric approach is to tie consensus rules to actual resource requirements. The idea is that cost of a block is a function of certain block properties. As an example, the block cost could be represented by a weighted sum of block size, validation cost and utxo growth.

When we have agreed on such a cost metric, we can get rid of the hard limits and instead introduce a new consensus rule that blocks need to cost less than a threshold to be valid.

Validation cost

One aspect of a full cost function are validation-cost. We can view validation cost as the time it take to validate a block on a reference machine. Then we can introduce a threshold saying that a block is not allowed to exceed 30 seconds validation time on a reference machine. In other words, we want to find a function from block features like the number of bytes that are hashed for signature validation to validation time on the reference machine. To do that, we assume a simple model function that states that the validation duration is a linear combination of block features, collect data about the actual validation duration on that machine and then fit the model to the data.

The one dimensional situation is depicted in the right, there is one data point for each block consisting of the number of bytes that were hashed and the time it took to validate. With this data it is possible to determine the effect or coefficient of hashing on validation time which is represented as a line in the plot. This coefficient can then be used in a consensus rule.

MAYBE: If we assume that the resources involved grow at the same speed, this kind of metric can be naturally scaled by multiplying the whole equation with the inverse of the growth factor.

Experiments

Validation cost is affected first and foremost by OP_CHECKSIG, that is signature verification and hashing the transaction. Bitcoin Core already limits the number of OP_CHECKSIGs but this is insufficient for our case because what counts are the number of OP_CHECKSIGs that are executed. We built on Gavin Andresen’s code to count those factors while validating transactions. We also record hashing via the OP_HASH opcodes, and how many bytes are written and removed from the stack. And the number of inputs which loosely corresponds to the number of lookups in the utxo set. And of course we also measured our dependent variable, the ConnectBlock duration on the reference machine.

As a reference machine we used my laptop, which has two 3gHz i7 cores. To collect block feature data and the corresponding ConnectBlock duration, we reindexed mainchain, testchain and custom regtest chains which for example consisted hard-to-validate blocks. I found out that I could comfortably use the computer while using only 5GB of 8GB RAM, so I set the dbcache option to 3GB. dbcache determines how much data is cached in memory We ran Bitcoin Core version 0.11.2 with libsecp validation and disabled checkpoints.

Result

After estimating the coefficients using linear regression, we get useful information like for each kilobyte of hashing validation takes 0.005 millisecond longer for each signature verification it takes 0.1 millisecond longer. Other features do not play a comparably significant role at the moment, even though it is possible to create a block that takes around 5 seconds to validate and only consists of hash opcodes.

The validation cost function fit is very accurate: for a random test selection of test and mainnet we get an average absolute error of less than 4 ms. Most importantly, the estimated function is able to predict hard-to-validate blocks very accurately: The one tested example was a block that took 130.4ms to validate, 131.7 was predicted.

Cost Metric

So, now we derived a validation cost metric that corresponds to validation time on a reference machine and we can define a new consensus rule that would require a block to have a smaller validation cost than some threshold. After picking a threshold, there would be a situation like in this plot, where x-axis is the block size, y-axis validation time and the green area represents the space of valid blocks.

However, picking another threshold is difficult. because there is no one size fits all solution: (1) you don’t want to constrain potential use cases but (2) and you also don’t want want to sum validation time and bandwidth worst cases.

On the other hand, we can try to relate bandwidth requirements and validation cost using a simple weighted sum for example and then pick a single threshold.

And this is exactly the idea behind the cost-metric, find all factors affecting node cost and how exactly they influence node costs and then pick a reasonable cost threshold. And what this idea really entails is moving away from blocksize proposals to arguing about total node costs.

Now the question is how exactly do you convert bandwidth requirements, validation time to cost? Does it make sense to trade off one second of network latency with one second of validation duration? How do we bring additional cost factors in, like utxo set size? How future-proof is that solution?

There is certainly no single correct answer to these questions. We can, however, show the advantages of a cost function while building on existing block size proposals. Most block size proposals consider average use at a specific maximum block size. So in terms of cost threshold it would make a lot of sense to allow maximum sized blocks only in combination with average validation time. In this way we can prevent blocks that have both a worst-case size and worst-case validation time. We get the average validation duration for a specific block size using the data we collected earlier with the reference machine.

Also we set a hard limit validation cost of 10 seconds, which seems reasonable because the maximum validation time on the reference machine was 6 seconds. to the average validation time at the maximum blocksize. Then we allow to linearly interpolate between the maximum validation time at half of the maximum blocksize

This shows an advantages of a cost metric: we constrain the worst case by bringing it closer to the average case, and still allow possible future use-cases which require a lot of validation resources.

So far, the cost of maintaining the utxos has not played a role in Bitcoin. In fact with a 1MB block, the worst case utxo set size increase is almost 1MB, whereas the average over the past year is an increase of around 11kilobyte. Finding a reasonable place in the cost function is even more complicated than validation and bandwidth resources, in part because they are long-term costs. The current situation with Bitcoin is that there is no incentive to avoid increasing the utxo set size if possible. This can be as simple as moving bytes from the scriptSig to the scriptPubKey. What we can do with the cost function is placing a slight incentive to include transactions that reduce the utxo set size and thereby cheapen them. The proposed way to do this is allowing a larger validation costs when the block reduces the utxo set size. This aligns well with the fact that blocks that sweep a lot of utxos have rather extreme validation costs due to the high ratio of inputs to outputs and we want these blocks to be valid because they are extremely beneficial.

In order to determine a specific function one can compute the maximum possible decrease of utxo set size for a block of maximum size. Then linearly interpolate such that for each byte the utxo set is reduced the maximum allowed validation costs are increased until we reach let’s say half of the remaining validation cost. This rule does not give the utxo set size the prominent place in the cost function it would deserve but at least moves incentives in the right direction.

This cost function can trivially grow with the blocksize, by multiplying the validation cost limit and average validation cost with the same scaling factor. So if the blocksize is doubled, then double max validation cost point and double max validation cost and double average transaction

This situation is shown in the plot for 1MB, 2MB and 4MB maximum block sizes.

It ensures that the worst case validation time scales as fast as the block size, which is an implicit assumption underlying many blocksize proposals. Also it guarantees that average blocks are always allowed to have the maximum block size.

Conclusion

In conclusion, Bitcoin places various resource requirements on full nodes. And it is essential that blocksize proposals account at least for the most important ones, or extreme worst cases are . A cost metric helps with that because it sets the requirements in relation to each other.

We’ve seen that estimating a function for validation cost only, is straightforward, when assuming a reference machine, collecting data and fitting a linear function.

A more complete cost function that includes bandwidth, validation and utxo requirements is difficult to derive from the bottom up. But as we showed we can build on existing blocksize proposals to get some of the advantages of a cost metric,

such as confining the worst-case while
allowing to trade-off various block aspects
and setting the right incentives.

Can Miners Exploit a Block Size Increase?

2015-06-23T10:49:00+00:00

We don’t know yet. But modelling is in my opinion a useful tool to investigate potential effects. Therefore, I used Gavin Andresen’s mining simulator to create some more or less plausible scenarios. You can find the resulting plots on github.

Fuzzing Bitcoin Consensus

2015-05-09T23:39:00+00:00

TLDR I ran afl-fuzz against libbitcoinconsensus to discover interesting Bitcoin scripts and used them to search for Bitcoin reimplementations vulnerable to forking. This discovered two bugs in btcd by Conformal. See the bitcoinconsensus_testcases repository for the discovered Bitcoin scripts.

Forks

One of the things that must not happen during regular Bitcoin operation are forks. A fork occurs when there is a new block $B_{i+1}$ which is a valid successor to block $B_i$ for some set of Bitcoin nodes $N_v$ and invalid for the remaining nodes $N_{\neg v}$. Therefore, miners in $N_v$ will mine new blocks on top of $B_{i+1}$ and miners in $N_{\neg v}$ will still mine on $B_i$. As long as the majority of hashpower is in $N_{\neg v}$, the chain divergence will be resolved after some time, because $N_{\neg v}$’s chain will eventually get longer than $N_v$’s chain and then the nodes in $N_v$ will switch to $N_{\neg v}$’s chain. This is due to the nature of the blockchain: nodes always trust the longest valid chain (more exact: the chain with the most proof of work).

Consider for example the case of an update to the Bitcoin reference implementation that restricts valid signature encodings. $N_v$ are the nodes running the old Bitcoin version and $N_{\neg v}$ run the new version. As soon as the hash power of $N_{\neg v}$ exceeds some threshold the new consensus rule can be safely activated. In the context of Bitcoin updates this is called a softfork: a valid block becomes invalid in the new version. On the other hand, a hardfork occurs when an invalid block is valid in a new version, for example by raising the maximum block size limit. Then nodes that run the old version are represented by $N_{\neg v}$. Even if the majority of hashpower is in $N_v$, the nodes in $N_{\neg v}$ can never switch to $N_v$’s chain because some blocks are invalid for them. Therefore, in the case of a hardfork all nodes are required to update.

Fuzzing

Forks in practice do not only happen deliberately because of updating mechanisms but can also be triggered by bugs. Bitcoin reimplementations such as libbitcoin, btcd, bitcore and toshi are particularly vulnerable to these bugs because they have to match exactly the behavior of the Bitcoin reference implementation. In order to abstract part of the consensus critical code and allow other projects to use it, Bitcoin Core developers created the bitcoinconsensus library. I am not aware of any reimplementation that already adopted libbitcoinconsensus. Right now, it only has a single function bitcoinconsensus_script_verify, which takes an output script and a transaction and returns if the transaction is allowed to spend the output.

Among other conditions, a transaction is valid if the top stack item is different from 0 after script execution. Bitcoin script is much more powerful than just verifying signatures and therefore I was curious to find interesting scripts, i.e. scripts that trigger unusual edge cases. I’ve recently heard about successes with afl-fuzz whose heuristic using code coverage seemed to be particularly well suited for the task. Also, it has the capability to minimize a set of inputs such that the code coverage stays the same. After fuzzing libbitcoinconsensus for two weeks I supplied the inputs to btcd’s txscript, a reimplementation in golang, and checked if the outputs differ.

Btcd bugs

The first bug I found was in btcd’s implementation of the OP_IFDUP opcode. This opcode pushes the top stack element on the stack if it differs from 0. Because of a type conversion in btcd, a stack element that exceeds 4 bytes would have never been copied, which differs from bitcoinconsensus’ implementation of the opcode. The second bug concerned the representation of the result of OP_EQUAL. This opcode compares the two top stack elements and pushes the result on the stack. In Bitcoin Core, if the comparison fails an empty byte array is pushed on the stack. Btcd however pushed a byte array containing 0. This means that the following script would be valid in bitcoinconsensus and invalid in btcd (Note that OP_0 pushes an empty byte array to the stack):

OP_0 OP_0 OP_TRUE OP_EQUAL OP_EQUAL

Both bugs would have triggered hardforks. An attacker could simply broadcast a transaction with the affected scripts and it would be mined subsequently. Btcd would have not been able to include the block into its chain and would become stuck on the last block. Therefore, an attacker could create a block on top of btcd’s chain paying a merchant running btcd without affecting his ‘real’ coins on the main chain. Note that the attacker would not race against the hashpower of Bitcoin miners.

Dave Collins from the btcd team fixed these issues very fast and additionally improved the test coverage in Bitcoin Core for the affected and more opcodes. Additionally, he was so kind to award me with 0.5 bitcoin for the find.

Conclusion

You can find the result of the fuzzing, the code to produce them and test reimplementation in the bitcoinconsensus_testcases repository. If you are interested you can start fuzzing yourself and submit a pull request with new scripts you found. Also, I’ve executed the testcases only with btcd and bitcore so far.

Bitcoin Seeder DoS Vulnerability

2015-03-17T22:23:00+00:00

The DNS parser of the Bitcoin Seeder was vulnerable to a denial of service attack. A specially crafted DNS request could trigger infinite recursive function calls that lead to a stack overflow. See the exploit. The vulnerability was fixed in commit 11e935b.

Guessing Bitcoin's P2P Connections

2015-03-06T14:22:00+00:00

The paper Deanonymisation of clients in Bitcoin P2P network (2014) by Biryukov, Khovratovich and Pustogarov (BKP), who describe an attack on Bitcoin Core clients, has started some discussion lately. The main idea of the paper is to first get a set of nodes $E_v$ to which your victim $v$ is directly connected to (“entry nodes”). Second, for each transaction $t$ record the $10$ nodes $P_t$ which first propagate $t$. The authors experimentally show that if $|E_v \cap P_t|$ is bigger than $3$ then there is a pretty high chance that $t$ actually originated from $v$. However, both attack stages basically require a Sybil attack – the attacker has to be connected to a lot of nodes a lot of times. ‘A lot’ means that in their experiments they had 50 connection to each full node (~250) in the test network. As a result, such an attack seems to be powerful, but certainly won’t be undetected.

In this post I show that the first stage of the attack, namely learning the nodes a victim is directly connected to can be done with a single connection to the victim. In addition to BKP’s attack, knowing all outbound peers of a client could significantly increase the success probability of a double spend. Note that all experiments are based on Bitcoin Core 0.9.4, but 0.10.0 shows the same behavior.

TLDR The attacker can reliably guess all of the outbound connections of a victim by making a selection from the known addresses of a victim based on the timestamp of the addresses.

Update A fix has been merged to bitcoind. The timestamp is not updated anymore when receiving a message from a connected peer. Instead, it is only updated when the peer disconnects. The fix is released in bitcoin core 0.10.1.

Learning connections using addr propagation

When a node $n$ connects to another peer $p$ in the network it advertises its address using the “addr” message. The peer will select a number of its own peers at random which are “responsible” for $n$’s address. Then the address is forwarded to responsible peers to spread the knowledge about $n$ in the network. The number of responsible peers is either $1$ or $2$ depending on whether the address is reachable by $p$.

BKP’s attack works by recording the set of peers that first propagated a victim’s address. In order to have good chance to be in the set of responsible peers for the address, the attacker has to hold a significant number of connections to each full node in the network. Note that it is possible to have multiple connections from a single public address to a peer.

The getaddr message

It turns out that an attacker can simply infer the peers of a victim by sending getaddr messages to him.

In bitcoin, the address structures that are send via the addr message do not only contain the IP adress and port but also a timestamp. The timestamp’s role is ensuring that terminated nodes vanish from the networks knowledge and it is regular refreshed by the nodes which have an interaction (more about that later) with the peer at that address. Bitcoin nodes usually record the addresses they hear about and send them in a reply to a getaddr using the addr message.

The following experiments show that an attacker can guess some or all of the direct peers of a victim by sorting the known addresses of the victim based on the timestamp.

A minor obstacle is that a node replies to a single getaddr message only with maximal 2500 addrs selected uniformly at random. In order to get a certain percentage $\tau$ of the known addresses of a node the attacker has to send multiple getaddr messages and record the percentage that is new to her.

S = {}
while(true):
    send_getaddr()
    T = response()
    tau' = 1 - ((|T - S|) / |T|) 
    S <- S $\cup$ T
    if \tau' > \tau
        break
return S

Experiments show that if we wait 10 seconds after each getaddr request it takes around $3.5$ minutes to collect $\tau$ percent addresses ($13,500$ in this case).

Experimental results

I set up a victim node $v$, which is just a regular bitcoin node. The attacker $a$ is a node that connects to $v$ via the P2P network and queries the known nodes of $v$. Second, $a$ connects to $v$ via the RPC interface and gets the true peers.

The attacker code (btcP2PStruct) is available on github. Thanks to the btcwire package it is very simple to write this kind of code.

You can find all the data to produce the graphs in the project repository.

First we consider the case where $v$ does not accept incoming connections (“client” in BKP’s terms). $v$ was running for 2 days and I recorded data for every hour but I will only discuss the last measurement because the data is very similar.

Note that $v$ returned $12,868$ known addresses. Also, a client usually has maximally 8 peers due to the default maximum number of outbound connections. This implies that an attacker can not start start this attack on a client that is not connected to her. Here we see that if the attacker obtains all peers of $v$ (without any false positives in this case).

Next, the case for the full node, which I left running for 8 days.

Again it is evident that an attacker can reliably determine all outbound connections of the victim using a threshold of 20 minutes. However, inbound peers can only be detected very poorly.

The reason for finding all outbound peers is is this logic in bitcoin-core which refreshes the timestamp on every message of outbound nodes.

Reducing false positives

BKP mention a neat trick how to determine if two nodes $v_1$ and $v_2$ are connected. First, the attacker connect to $v_1$ and $v_2$ and send addr messages containing bogus addresses to $v_1$. Then, she counts the number of times one of these addresses is received from $v_2$. However, the authors leave open how many messages you need send to be certain about the hypothesis.

As we already know, the address is forwarded only to two responsible nodes so we have to compute the probabilities of our node being responsible. Using the binomial distribution we can compute the likelihood of receiving a certain number of addresses back given that we sent a certain number of addresses.

I’ve done the math using this code and some assumptions regarding the structure (edges are uniformly iid). Also, the attacker has to know or approximate the number of peers of a node, which can be done with a similar method than the one described. Connect two times to the victim, send and note the ratio of returned addr messages. If you can not connect to the node, it will most likely have 8 peers.

This theoretical model shows that that if $v_1$ is a full node and $v_2$ is a client then we need about 2000 messages to determine if they are connected with 95% probability. Similarly, if $v_1$ and $v_2$ are full nodes, the attacker needs to send 20000 messages.

However, in order to remain polite in the network this attack needs start from a candidate set of nodes. Therefore, it could be a useful method to remove the false positives which were obtained with the “getaddr”-fingerprint.

Conclusion

It should be pointed out that even if you know a victim’s entry nodes you can not simply connect to those few and listen for transactions. This is because “trickling” prevents estimating the origin of a transaction without further assumptions or doing BKP’s Sybil attack. However, knowing all outbound peers of a client could significantly increase the success probability of a double spend.

Update The fix removes the update every 20 minutes and updates on disconnect

Privacy in BitcoinJ

2015-02-12T15:53:00+00:00

As part of my epic quest to apply supervised machine learning to the blockchain in order to discover transaction patterns, I reviewed various wallet implementations in the hope of finding privacy leaks.

tl;dr If you are using a wallet that is built upon BitcoinJ, such as Android Wallet, Multibit and Hive Wallet, you have almost zero wire privacy. An attacker who manages to connect to your wallet is easily able to figure out all addresses you control. This is not very likely to get fixed in the near future.

Update: Mike Hearn’s reply addresses additional problems and improvements. There was also accompanying discussion on reddit.

Bloom Filters for SPV Nodes

A Bloom filter is a probabilistic data structure that is used to test whether an element is a member of a set. Bitcoin SPV nodes that use BIP 37 (we call them thin clients from now on) put all public keys they are interested in into the Bloom filter and send the filter to their peers. Upon receiving a new transaction, peers query the Bloom filter and only relay the transaction to the BIP 37 node if the query returned true. Thus, thin clients normally only receive the transactions they are really interested in, i.e. mostly transactions that include one of the wallet’s keys.

The advantage of using a Bloom filter instead of just broadcasting all your pubkeys is that a Bloom filter is faster and more space-efficient at the cost of some false positives. That means the thin client will receive transactions that include pubkeys which were not put into the filter. Usually, the parameters of a Bloom filter are computed such that a certain target false positive rate (fp) is achieved. We want the fp rate to be relatively small (say 0.05%) to reduce bandwidth usage.

Bloom Filters and Privacy

BIP 37 states:

Privacy: Because Bloom filters are probabilistic, with the false positive rate chosen by the client, nodes can trade off precision vs bandwidth usage. A node with access to lots of bandwidth may choose to have a high fp rate, meaning the remote peer cannot accurately know which transactions belong to the client and which don’t.

This has created a misunderstanding between what is ideally possible with Bloom filters and how the reality looks like. I’ll focus on BitcoinJ because it is the most widely used implementation of BIP 37, but similar vulnerabilities might exist in other implementations as well. Unfortunately, in the current BitcoinJ implementation Bloom filters are just as bad for your privacy as broadcasting your pubkeys directly to your peers.

A Simple Attack

The main idea behind this vulnerability is that BitcoinJ puts both pubkey and pubkeyhash into the Bloom filter which substantially reduces the false positive rate.

If you create a completely fresh wallet, BitcoinJ holds 271 pubkeys and computes the parameters of the Bloom filter such that the fp rate for (271*2)+100 elements is equal to 0.05%. Because bitcoinj initially puts only 271*2 elements into the filter (pubkey and corresponding pubkeyhash) the effective false positive rate is fp=0.000146.

The vulnerability is that if a pubkey is truly in the filter then querying both pubkey and pubkeyhash must return true. Because the pubkeyhash is just another almost uniformly random string, the probability of a false positive for the attacker is fp' = fp^2 = 0.0000000213. I obtained around 56 million pubkeys from the blockchain (mid-January), which theoretically results in 56 million * fp' = 1.19 expected false positives when scanning the blockchain.

Experimental results

I ran 20 crawlers since the beginning of December and collected 70,000 distinct filters until now. These crawlers just listen for a filterload message and try to be really polite by disconnecting after 2 minutes and not sending anything. The probability that a randomly selected DNS seed returns at least one of the crawlers is 4.3%.

In fact, most of the Bloom filters from recent BitcoinJ versions show a experimental false positive rate around 0.000146. The experimental fp rate is computed by querying the filter with millions of elements which are certainly not pubkeys. Android Wallet 4.16, 4.17, 4.18 for example use the most recent BitcoinJ version (12.2) and make up 52% of the data. However, there is also MultiBit 0.5.18 whose effective fp rate is smaller than 0.00000001.

We are currently starting to analyze all filters using the described “attack” and we expect that this will take several weeks. What we’ve already seen is that the theoretical fp' really holds, i.e. if you create a fresh wallet and scan the whole blockchain you most likely get one false positive pubkey.

(Slightly) More Difficult Attacks

You might think that the problem is easily fixed by trading off bandwidth for more privacy and increase the fp rate to fp = sqrt(0.0005) = 0.0224. On the one hand this might seriously impact the bandwidth of mobile clients, and on the other, there is another another general class of vulnerabilities concerning Bloom filters: If an attacker manages to obtain multiple, different filters from the same Wallet, he can compute the intersection of pubkeys that match the filters and therefore removes the false positive noise similar to the “simple attack”. Different filters mean that they have different total size of a different Nonce. Sending different filters can happen in BitcoinJ due to multiple reasons, for example

Restart. BitcoinJ stores the filter’s nonce in volatile memory.
Creation of new keys. When the wallet creates many new keys the filter gets ‘full’ and thus has to be recomputed.
Measured false positive rate is too high. BitcoinJ measures the false positive rate of transactions it receives. When it becomes too high the filter is recomputed.

Conclusion

I do think this is a critical privacy leak as it doesn’t require a sophisticated attack and wallets have practically been broadcasting all their pubkeys for years. Not only the addresses you see in your wallet, but also a lot of your future addresses have been exposed. From now on you should assume that the kind of bulk data collection I did is happening. It is difficult to say how accurate and stealthy targeted attacks would be.

According to Mike Hearn, the creator of BitcoinJ, the problems have been known from the start but fixing these issues is far from trivial because “lying consistently is hard”. I fully agree with this. Someone needs to make it their project for a few months.

There are some simple ideas to slightly improve the current status such as deploying nodes that broadcast fake bloom filters. Arthur Gervais et al., 2014 were the first to publish an academic paper on the topic and propose some more or less vague suggestions. One idea I find interesting is that thin clients should be able to install multiple filters at their peers such that no pubkey is shared between the filters. Thus, instead of recomputing the filter when the wallet creates new addresses, it would create an entirely fresh filter for the next keys. One disadvantage is that at the moment multiple filters per peer is not supported by the bitcoin wire protocol. Another issue with Bloom filters is that an attacker could safely assume that the probability is higher for two pubkeys to belong to the same person if they are closer in the transaction graph. As a countermeasure the wallet could deliberately put existing foreign pubkeys that are close into the filter.

I feel sorry for the people whose privacy has been potentially compromised unknowingly by malicious parties and we certainly won’t give away the data set but nonetheless it is really exciting what can be found in the data. If you have suggestions what to look out for and what would be interesting (not necessarily concerning machine learning) feel free to contact me.

Evolving Bitcoin Trade Bot

2012-11-28T21:02:00+00:00

Bagalute is a trade bot, designed for buying a currency (bitcoin) at the the right time. As a decision criterium it uses the relative strength index (RSI). The bot looks at the price development of the last twelve hours and runs an evolutionary algorithm to determine the optimal parameters for the RSI in that time frame. Then, it uses these parameters for the next time frame. Currently the bot uses for debugging purposes a dummy interface to trade. The actual success of the bot was sparsely tested but had negative results, maybe due to a general downtrend of prices. I wrote it during my 2nd semester break