Joel Monegro, Aug 2016

Fat protocols

Here’s one way to think about the differences between the Internet and the Blockchain. The previous generation of shared protocols (TCP/IP, HTTP, SMTP, etc.) produced immeasurable amounts of value, but most of it got captured and re-aggregated on top at the applications layer, largely in the form of data (think Google, Facebook and so on). The Internet stack, in terms of how value is distributed, is composed of “thin” protocols and “fat” applications. As the market developed, we learned that investing in applications produced high returns whereas investing directly in protocol technologies generally produced low returns.

This relationship between protocols and applications is reversed in the blockchain application stack. Value concentrates at the shared protocol layer and only a fraction of that value is distributed along at the applications layer. It’s a stack with “fat” protocols and “thin” applications.

We see this very clearly in the two dominant blockchain networks, Bitcoin and Ethereum. The Bitcoin network has a $10B market cap yet the largest companies built on top are worth a few hundred million at best, and most are probably overvalued by “business fundamentals” standards. Similarly, Ethereum has a $1B market cap even before the emergence of a real breakout application on top and only a year after its public release.

There are two things about most blockchain-based protocols that cause this to happen: the first is the shared data layer, and the second is the introduction cryptographic “access” token with some speculative value.

But an open network and a shared data layer alone are not not enough of an incentive to promote adoption. The second component, the protocol token[1] which is used to access the service provided by the network (transactions in the case of Bitcoin, computing power in the case of Ethereum, file storage in the case of Sia and Storj, and so on) fills that gap.

Albert’s post will help you understand how tokens incentivize protocol development. Here, I’m going focus on how tokens incentivize protocol adoption and how they affect value distribution via what I will call the token feedback loop.

When a token appreciates in value, it draws the attention of early speculators, developers and entrepreneurs. They become stakeholders in the protocol itself and are financially invested in its success. Then some of these early adopters, perhaps financed in part by the profits of getting in at the start, build products and services around the protocol, recognizing that its success would further increase the value of their tokens. Then some of these become successful and bring in new users to the network and perhaps VCs and other kinds of investors. This further increases the value of the tokens, which draws more attention from more entrepreneurs, which leads to more applications, and so on.

There are two things I want to point out about this feedback loop. First is how much of the initial growth is driven by speculation. Because most tokens are programmed to be scarce, as interest in the protocol grows so does the price per token and thus the market cap of the network. Sometimes interest grows a lot faster than the supply of tokens and it leads to bubble-style appreciation.

With the exception of deliberately fraudulent schemes, this is a good thing. Speculation is often the engine of technological adoption [2]. Both aspects of irrational speculation — the boom and the bust — can be very beneficial to technological innovation. The boom attracts financial capital through early profits, some of which are reinvested in innovation (how many of Ethereum’s investors were re-investing their Bitcoin profits, or DAO investors their Ethereum profits?), and the bust can actually support the adoption long-term adoption of the new technology as prices depress and out-of-the-money stakeholders look to be made whole by promoting and creating value around it (just look at how many of today’s Bitcoin companies were started by early adopters after the crash of 2013).

The second aspect worth pointing out is what happens towards the end of the loop. When applications begin to emerge and show early signs of success (whether measured by increased usage or by the attention (or capital) paid by financial investors), two things happen in the market for a protocol’s token: new users are drawn to the protocol, increasing demand for tokens (since you need them to access the service — see Albert’s analogy of tickets in a fair), and existing investors hold onto their tokens anticipating future price increases, further constraining supply. The combination forces up the price (assuming sufficient scarcity in new token creation), the newly-increased market cap of the protocol attracts new entrepreneurs and new investors, and the loop repeats itself.

What’s significant about this dynamic is the effect it has on how value is distributed along the stack: the market cap of the protocol always grows faster than the combined value of the applications built on top, since the success of the application layer drives further speculation at the protocol layer. And again, increasing value at the protocol layer attracts and incentivises competition at the application layer. Together with a shared data layer, which dramatically lowers the barriers to entry, the end result is a vibrant and competitive ecosystem of applications and the bulk value distributed to a widespread pool of shareholders. This is how tokenized protocols become “fat” and its applications “thin”.

This is a big shift. The combination of shared open data with an incentive system that prevents “winner-take-all” markets changes the game at the application layer and creates an entire new category of companies with fundamentally different business models at the protocol layer. Many of the established rules about building businesses and investing in innovation don’t apply to this new model and today we probably have more questions than answers. But we’re quickly learning the ins and outs of this market through our blockchain portfolio and in typical USV fashion we’re going to share that knowledge as we go along.

Thin applications

Joel Monegro, Jan 2020

I predicted this architecture would dominate new online services within a decade as crypto took over the web. So it’s been fun to review how things have developed five years into that idea. The most obvious flaw was thinking we’d build everything on top of Bitcoin (Ethereum hadn’t launched). Now we have a multitude of blockchains to choose from, which is much better. We also refer to “overlay networks” as layer 2. And today, the “Web3 Application Stack” would be a better name. But the overall framework appears to stand.

A Cryptoservices Architecture

A Cryptoservices Architecture

A cryptoservices architecture is great for startups. Entrepreneurs can launch new applications quickly and cheaply by outsourcing a lot of the functionality to various networks. And every app is on equal footing when it comes to protocol costs and resources (unlike web infrastructure like AWS where the smaller you are, the more expensive it is). The companies above stand out because they brought fully-featured products to the market before their first real rounds of funding. They’re a first look at the level of capital efficiency that’s possible for “thin” applications using this new model, compared to the increasing amount of capital web companies need to raise to compete with incumbents.

Bring Your Own Data

As a crypto user, you bring your own data. Nobody has monopoly control. When you log into a crypto app by connecting your wallet, you’re sharing the “keys” it needs to find your information across the relevant networks. You can share them with any app, so your data comes with you as you move from interface to interface. And you keep control of the “private key” (basically a password) needed to operate on it, like signing messages or authorizing transactions. So you have effective custody over your data and nobody can manipulate it without consent (unless you delegated your keys to a custodian).

Value Capture vs. Investment Returns

For example, buying $10 million worth of ETH at ~$15 billion network value will get you just about 0.06% of the network (based on current supply), and to provide a 5x return (to $50 million) Ethereum needs to add $60 billion dollars to its market cap. Meanwhile, a $1 million seed investment in a successful application business for 10% of the company will yield the same $50 million with “just” another $490 million in value – and probably for less than $10 million when you account for follow-ons.

But networks and companies process value in very different ways. The forces that take a public network from $20 to $90 billion in value are very different from those which take a business from $10 to $500 million. Token prices are chaotically determined every time there is a trade in the public markets, where networks gain and lose more value per investment dollar faster than private companies can. The complexity adds a lot of leverage to every investment dollar flowing in and out. Meanwhile, business value may be a well-known function, but private, early-stage investments can be riskier in unpredictable ways.

P2B2C

Finally, the idea of user-staking is about leveraging tokens to distribute value and upside to users. In general, it works by having users stake an amount of the application’s own token (not a protocol’s) to unlock benefits like discounts or rewards – but there’s a lot of variations. At first glance, they look similar to the loyalty/reward systems used to retain customers in other hyper-commoditized markets like airlines and credit cards. Except those programs provide no upside at all. The innovation is in designing token models that allow users to profit from the application’s growth. It goes beyond marginal benefits like discounts by including users in the upside of the business.

I’m most fascinated by the user-staking models because they represent a genuine business model innovation. The examples above are built more like traditional web applications. They are more centralized and custodial than thinner apps like Zerion. But what I love about their staking models is how they change the user-service relationship. Web users are locked-in by force through the centralization of data. Crypto applications, even if they’re built more traditionally, don’t have that same ability to lock you in. But user-staking creates a kind of “opt-in” economic lock-in that benefits the user by turning them into stakeholders in the success of the service. It creates defensibility through user-ownership instead of user lock-in. This presents a universe of fascinating consequences, to be explored in future work.

Preethi Kasireddy, Feb 2019

Many of us are guilty of describing blockchains as “trustless” systems. However, I’ve come to realize that the term “trustless” is ambiguous, confusing, and most importantly, inaccurate.

Blockchains don’t actually eliminate trust. What they do is minimize the amount of trust required from any single actor in the system. They do this by distributing trust among different actors in the system via an economic game that incentivizes actors to cooperate with the rules defined by the protocol.

Let me explain in more detail.

A truly trustless transactional system would look something like this:

Two people who are interested in transacting with one another change hands directly. They are physically present, and therefore can easily verify

1. Authenticity: the actual sender is handing over the money, and

2. No double spending: the money is not fake, it’s a real $10 bill

While theoretically flawless, this transactional system is limited. Consider: two individuals may trade with one another only when they are in close physical proximity. For economies to function at scale, a transactional system should enable transfers with anyone in the world, regardless of distance.

So, what we really want is this:

As you can see from the diagram above, the way we achieve this aim is by having an intermediary who can facilitate the transfer of value to make sure that the actual sender is sending the money and the money is real.

This begs the question: who serves as the wholly trustworthy intermediary?

In modern day transactional systems, the intermediary can be a bank (e.g. Chase Bank); a payment provider (e.g. Paypal); a remittance company (e.g. Western Union); a credit card (e.g. Visa), and so on.

In this centralized model, the bank authenticates you, and guarantees the recipient that they are getting real money.

In other words, unless there is a direct physical transfer of value from one individual to the other, there must be some intermediary that exists that we “trust”.

Blockchains are no different.

Blockchains define a protocol that allows two individuals to transact with one another in a “peer-to-peer” manner over the Internet. When you digitally transfer value from one account to another on the blockchain, you’re trusting the underlying blockchain system to both enable that transfer and ensure sender authenticity and currency validity.

In a “centralized” system, we trust a single third party (e.g. Chase Bank) to act as the intermediary who guarantees those two properties; in a “decentralized” system, our trust is placed elsewhere, namely in public-key cryptography and a “consensus mechanism” that allows us to determine the truth.

Public-Key Cryptography

Public key cryptography (or asymmetrical cryptography) uses:

1. a set of public keys visible to anyone, and

2. a set of private keys visible only to the owner

The private key generates a “digital signature” for each blockchain transaction that a user sends out. The signature ensures authenticity by:

1. confirming that the transaction is coming from the user, and

2. preventing the transaction from being altered by anyone once it has been issued

Changing the transaction message in any way will cause verification to fail.

Okay, so we’ve figured out that public-key cryptography helps us authenticate users in a peer-to-peer system. But to ensure no double spending, we need to keep track of who has what so that we can know whether someone is sending real digital money or fake digital money.

This is where the “consensus system” — which allows us to preserve a digitally shared truth — must come into play.

Machine Consensus (The Cryptoeconomic Protocol)

Blockchains have a shared ledger that gives us the absolute truth of the state of the system. It use mathematics, economics, and game theory to incentivize all parties in the system to reach a “consensus”, or coming to an agreement on a single state of this ledger.

Let’s take Bitcoin, for example. The Bitcoin protocol has a consensus algorithm called “Proof of Work” that holds the system together. For a transaction to be settled between two consumers, the algorithm requires that a set of nodes (called “miners”) compete to validate transactions by solving a complex algorithmic problem. In other words, Bitcoin “economically incentivizes” miners to purchase and use compute power to solve complex problems. These economic incentives include:

1. miners earning a transaction fee that users pay for carrying out a transaction, and

2. miners earning new Bitcoins for successfully solving the puzzle

Because of these economic incentives, miners are constantly watching the network so that they can gather a new set of transactions to fit into a new “block.” Then they use their computing resources to solve the complex algorithm in order to “prove” that they did some work.

The first miner to solve the algorithm adds the proof and the new block (and all the transactions in it) to the blockchain and broadcasts it to the network. At that point, everyone else in the network syncs the latest blockchain because it’s a “truth” everyone believes in.

Since miners are competing to run computations, there are times when multiple blocks get solved at the same time. This then creates a “fork” of multiple chains:

When there are forks like this, the network’s “canonical” chain is the one which is the “longest” — the one which the most amount of miners trusted and continued to work on.

Every new block that’s added to the blockchain in this manner adds more security to the system because an attacker who wants to create new blocks that overwrite a party of history would need to consistently solve for the puzzle faster than anyone else in the network. This is practically impossible to do, making it’s impossible to reverse engineer or alter the data inside these blocks. This is why users trust continue to trust the system.

So when we transact with one another on the blockchain, we are anchoring our trust in the miners who are giving up their resources to do some work to ensure no double spending.

Social Consensus (Governance)

Of course, even if the machine consensus works perfectly, we can never guarantee a 100% probability of reaching consensus on other important aspects required to maintain trust in the network. For example, when the underlying network needs to be upgraded, improved, or repaired, we need some way to trust that the network and all its constituents can appropriately handle the changes. In such cases, it’s very much a coordination effort amongst constituents, or what I would call a “social consensus” (e.g. governance).

For example, if the blockchain requires an improvement (e.g. better transaction logs), we need a governance mechanism that coordinates the interests of all parties involved (users, developers, investors, etc.) in coming up with the best solution. Or if there’s a controversy on the best path forward (e.g. a contentious fork), then a community needs to form a consensus on what to do next. If an agreement can’t be reached, the network forks, and people are forced to choose one side over another instead of everyone believing in a shared truth. Users would lose trust in the system because they would be unable to reasonably determine which chain was the “valid” chain.

Conclusion

When we say blockchains are “trustless,” what we mean is that there are mechanisms in place by which all parties in the system can reach a consensus on what the canonical truth is. Power and trust is distributed (or shared) among the network’s stakeholders (e.g. developers, miners, and consumers), rather than concentrated in a single individual or entity (e.g. banks, governments, and financial institutions).

Perhaps a more accurate way to describe blockchains is not as “trustless,” but as built on the basis of distributed trust: We are trusting everyone in aggregate.

Of course, this assumes that we trust that a majority of the power held in the system belongs to stakeholders who share similar values. Unfortunately, I don’t think we can claim — at least, not yet — to have figured out exactly what those shared values consist of. Hence the proliferation of blockchains and contentious forks in the past year … but that’s a long-winded topic for another day! 😊

Balaji Srinivasan, Jul 2017

We must be able to measure blockchain decentralization before we can improve it.

• Measure the extent of a given system’s decentralization

• Determine how much a given system modification improves or reduces decentralization

• Design optimization algorithms and architectures to maximize decentralization

The basic idea is to (a) enumerate the essential subsystems of a decentralized system, (b) determine how many entities one would need to be compromised to control each subsystem, and (c) then use the minimum of these as a measure of the effective decentralization of the system. The higher the value of this minimum Nakamoto coefficient, the more decentralized the system is.

To motivate this definition, we begin by giving some background on the related concepts of the Gini coefficient and Lorenz curve, and then display some graphs and calculations to look at the current state of centralization in the cryptocurrency ecosystem as a whole according to these measures. We then discuss the concept of measuring decentralization as an aggregate measure over the essential subsystems of Bitcoin and Ethereum. We conclude by defining the minimum Nakamoto coefficient as a proposed measure of system-wide decentralization, and discuss ways to improve this coefficient.

The Lorenz Curve and the Gini Coefficient

Even though they are typically concerns of different political factions, there are striking similarities between the concepts of “too much inequality” and “too much centralization”. Specifically, we can think of a non-uniform distribution of wealth as highly unequal and a non-uniform distribution of power as highly centralized.

The equation for the Gini coefficient can be calculated from the areas under the Lorenz curve and the so-called “Line of Equality” as shown below:

Intuitively, the more uniform the distribution of resources, the closer the Gini coefficient is to zero. Conversely, the more skewed to one party the distribution of resources, the closer the Gini coefficient is to one.

This captures our intuitive notions of centralization: in a highly centralized system with G=1, there is one decision maker and/or one entity to capture in order to compromise the system. Conversely, in a highly decentralized system with G=0, there are a multiplicity of decision makers who need to be captured in order to compromise the system. Hence, a low Gini coefficient means a high degree of decentralization.

Cryptocurrency: Gini Coefficients and Lorenz Curves

To build intuition, let’s look at the Lorenz curve and Gini coefficient for a simple example: the distribution of wealth across cryptocurrency market capitalizations. To do this, we took a snapshot of market capitalizations on July 15 2017 for the top 100 digital currencies, calculated the percentage of market share for each, and graphed it as a Lorenz curve with associated Gini coefficient:

If we measure the centralization of market capitalization across the top 100 cryptocurrencies, the Gini coefficient is 0.91. This fits our intuition as about 70% of the market capitalization as of July 2017 is held by the top two cryptocurrencies, namely Bitcoin and Ethereum.

Decentralized Systems are Composed of Subsystems

To apply this concept to the space of public blockchains, we need to make a distinction between a decentralized system and a decentralized subsystem. Specifically, a decentralized system (like Bitcoin) is composed of a set of decentralized subsystems (like mining, exchanges, nodes, developers, clients, and so on). Here are six of the subsystems that compose Bitcoin:

We will use these six subsystems to illustrate how to measure the decentralization of Bitcoin or Ethereum. Please note: you may decide to use different subsystems based on which ones you consider essential to decentralization of the system as a whole.

Now, one can argue that some of these decentralized subsystems may be more essential than others; for example, mining is absolutely required for Bitcoin to function, whereas exchanges (as important as they are) are not actually part of the Bitcoin protocol.

Let’s assume however that a given individual can draw a line that identifies the essential decentralized subsystems of a decentralized system. We can then stipulate that one can compromise a decentralized system if one can compromise any of its essential decentralized subsystems.

Quantifying Bitcoin and Ethereum Decentralization

Given these definitions, let’s now calculate Lorenz curves and Gini coefficients for the Bitcoin and Ethereum mining, client, developer, exchange, node, and owner subsystems. We can see how centralized each of them are according to the Gini coefficient and Lorenz curve measures.

Here are the curves for Bitcoin:

And here they are for Ethereum:

Let’s discuss each of these subsystems in turn by reference to the six panels in each of the figures above.

Mining Decentralization

Client Decentralization

Dev Decentralization

Exchange Decentralization

Node Decentralization

Ownership Decentralization

With that said, two points. First, while one would not want a Gini coefficient of exactly 1.0 for BTC or ETH (as then only one person would have all of the digital currency, and no one would have an incentive to help boost the network), in practice it appears that a very high level of wealth centralization is still compatible with the operation of a decentralized protocol. Second, as we show below, we think the Nakamoto coefficient is a better metric than the Gini coefficient for measuring holder concentration in particular as it obviates the issue of arbitrarily choosing a threshold.

Maximum Gini Coefficient: A Crude Measure of Blockchain Decentralization

Can we combine these sample measures of subsystem decentralization into a measure of system decentralization? A simple first approach would simply be to take the maximum Gini coefficient over all essential subsystems, as shown below:

So, by this measure, both Bitcoin and Ethereum have a maximum Gini Coefficient of ~0.92, because both of them have nodes with clients that are very highly concentrated in one codebase (Bitcoin Core for Bitcoin, and geth for Ethereum).

Crucially, a different choice of essential subsystems will change these values. For example, one may believe that the presence of a single codebase is not an impediment toward practical decentralization. If so, then Bitcoin’s maximum Gini coefficient would improve to 0.84, and the new decentralization bottleneck would be the node distribution across countries.

We certainly don’t argue that the particular choice of six subsystems here is the perfect one for measuring decentralization; we just wanted to gather some data to show what this kind of calculation would look like. We do argue that the maximum Gini coefficient metric starts to point in the right direction of identifying possible decentralization bottlenecks.

Minimum Nakamoto Coefficient: An Improved Measure of Blockchain Decentralization

However, the maximum Gini coefficient has one obvious issue: while a high value tracks with our intuitive notion of a “more centralized” system, the fact that each Gini coefficient is restricted to a 0–1 scale means that it does not directly measure the number of individuals or entities required to compromise a system.

Specifically, for a given blockchain suppose you have a subsystem of exchanges with 1000 actors with a Gini coefficient of 0.8, and another subsystem of 10 miners with a Gini coefficient of 0.7. It may turn out that compromising only 3 miners rather than 57 exchanges may be sufficient to compromise this system, which would mean the maximum Gini coefficient would have pointed to exchanges rather than miners as the decentralization bottleneck.

There are various ways to surmount this difficulty. For example, we might be able to come up with principled weights for the Gini coefficients of different subsystems prior to combining them.

An alternative approach is to define a spiritually similar metric based on the Lorenz curve from which the Gini coefficient is calculated, which we dub the “Nakamoto coefficient”. A visual is below. In this example, the Nakamoto coefficient of the given subsystem is 8, as it would require 8 entities to get to 51% control.

That is, we define the Nakamoto coefficient as the minimum number of entities in a given subsystem required to get to 51% of the total capacity. Aggregating this measure by taking the minimum of the minimum across subsystems then gives us the “minimum Nakamoto coefficient”, which is the number of entities we need to compromise in order to compromise the system as a whole.

The Nakamoto coefficient represents the minimum number of entities to compromise a given subsystem. The minimum Nakamoto coefficient is the minimum of the Nakamoto coefficients over all subsystems.

We can also define a “modified Nakamoto coefficient” if 51% is not the operative threshold across each subsystem. For example, perhaps one might require 75% of exchanges to be compromised in order to seriously degrade the system, but only 51% of miners.

That illustrates the concept. Here are graphs for all of the subsystems for Bitcoin and Ethereum again, this time with the Nakamoto coefficients calculated:

And here’s a table where we’ve assembled the Nakamoto coefficients of each subsystem:

As we can see, given these essential subsystems, we can say that the Nakamoto coefficient is 1 for both Bitcoin and Ethereum. Specifically, the compromise of the Bitcoin Core or geth codebases would compromise more than 51% of clients, which would result in compromise of their respective networks.

Increasing this for Ethereum would mean achieving a much higher market share for non-geth clients like Parity, after which point developer or mining centralization would become the next bottleneck. Increasing this for Bitcoin would similarly require widespread adoption of clients like btcd, bcoin, and the like.

The Minimum Nakamoto Coefficient depends on Subsystem Definitions

For example, if one considers “founder and spokesman” an essential subsystem, then the minimum Nakamoto coefficient for Ethereum would trivially be 1, as the compromise of Vitalik Buterin would compromise Ethereum.

Conversely, if one considers “number of distinct countries with substantial mining capacity” an essential subsystem, then the minimum Nakamoto coefficient for Bitcoin would again be 1, as the compromise of China (in the sense of a Chinese government crackdown on mining) would result in >51% of mining being compromised.

The selection of which essential subsystems best represent a particular decentralized system will be a topic of some debate that we consider outside the scope of this post. However, it is worth observing that the “founder and spokesman” and “China miner” compromises are two different kinds of attacks for two different chains. As such, if one thinks about comparing the minimum Nakamoto coefficient across coins, some degree of ecosystem diversity may quantitatively improve decentralization.

Conclusion

Many have said that decentralization is the most important property of systems like Bitcoin and Ethereum. If this is true, then it is critical to be able to quantify decentralization. The minimum Nakamoto coefficient is one such measure; as it increases, the minimum number of entities required to compromise the system increases. We believe this corresponds to the intuitive notion of decentralization.

The reason an explicit measure for quantifying decentralization is important is three-fold.

1. Measurement. First, quantitative measures like this can be computed unambiguously, recorded over time, and displayed in dashboards. This gives us the ability to track historical trends toward decentralization within subsystems and at the system level.

2. Improvement. Second, just like we measure performance, a measure like the Nakamoto coefficient allows us to begin measuring improvements and/or reductions of decentralization. This then allows us to begin attributing changes in decentralization to individual deployments of code or other kinds of network activities. For example, given scarce resources, we can measure whether deploying 1000 nodes or hiring two new client developers would provide a greater improvement in decentralization.

We recognize that there is plenty of room for debate over which subsystems of a decentralized system are essential. However, given a proposed essential subsystem we can now generate a Lorenz curve and a Nakamoto coefficient, and see whether this is plausibly a decentralization bottleneck for the system as a whole.

As such, we think the minimum Nakamoto coefficient is a useful first step towards quantifying decentralization.

Georgios Konstantopoulos, Dec 2017

Blockchains are inherently decentralized systems which consist of different actors who act depending on their incentives and on the information that is available to them.

Whenever a new transaction gets broadcasted to the network, nodes have the option to include that transaction to their copy of their ledger or to ignore it. When the majority of the actors which comprise the network decide on a single state, consensus is achieved.

These processes are described as consensus.

• What happens when an actor decides to not follow the rules and to tamper with the state of his ledger?

• What happens when these actors are a large part of the network, but not the majority?

In order to create a secure consensus protocol, it must be fault tolerant.

Firstly, we will talk briefly about the unsolvable Two Generals Problem. Then we will extend that to the Byzantine Generals’ Problem and discuss Byzantine Fault Tolerance in distributed and decentralized systems. Finally, we will discuss how all this relates to the blockchain space.

The Two generals Problem

In order for them to communicate and decide on a time, General 1 has to send a messenger across the enemy’s camp that will deliver the time of the attack to General 2. However, there is a possibility that the messenger will get captured by the enemies and thus the message won’t be delivered. That will result in General 1 attacking while General 2 and his army hold their grounds.

There is no way to guarantee the second requirement that each general be sure the other has agreed to the attack plan. Both generals will always be left wondering whether their last messenger got through.

Since the possibility of the message not getting through is always > 0, the generals can never reach an aggrement with 100% confidence.

The Byzantine Generals Problem

The leader-follower paradigm described in the Two Generals Problem is transformed to a commander-lieutenant setup. In order to achieve consensus here, the commander and every lieutenant must agree on the same decision (for simplicity attack or retreat).

Adding to IC2., it gets interesting that if the commander is a traitor, consensus must still be achieved. As a result, all lieutenants take the majority vote.

The algorithm to reach consensus in this case is based on the value of majority of the decisions a lieutenant observes.

Theorem: For any m, Algorithm OM(m) reaches consensus if there are more than 3m generals and at most m traitors.

This implies that the algorithm can reach consensus as long as 2/3 of the actors are honest. If the traitors are more than 1/3, consensus is not reached, the armies do not coordinate their attack and the enemy wins.

m = 0 → no traitors, each lieutenant obeys | m > 0 → each lieutenant’s final choice comes from the majority of all lieutenant’s choices

This should be more clear with a visual example from Lieutentant 2’s point of view— Let C be Commander and L{i} be Lieutenant i:

OM(1): Lieutenant 3 is a traitor — L2 point of view

Steps:

1. Commander sends v to all Lieutenants

2. L1 sends v to L2 | L3 sends x to L2

3. L2 ← majority(v,v,x) == v

The final decision is the majority vote from L1, L2, L3 and as a result consensus has been achieved

The important thing to remember is that the goal is for the majority of the lieutenants to choose the same decision, not a specific one.

Let’s examine the case of the commander being a traitor:

OM(1): Commander is a traitor

Steps:

1. Commander sends x, y, z to L1, L2, L3 respectively

2. L1 sends x to L2, L3 | L2 sends y to L1, L3 | L3 sends z to L1, L2

3. L1 ← majority(x,y,z) | L2 ← majority(x,y,z) | L3 ← majority(x,y,z)

They all have the same value and thus consensus is reached. Take a moment here to reflect that even if x, y, z are all different the value of majority(x, y, z) is the same for all 3 Lieutenants. In the case x,y,z are totally different commands, we can assume that they act on the default option retreat.

Byzantine Fault Tolerance

The algorithm mentioned in the previous section is Byzantine Fault Tolerant as long as the number of traitors do not exceed one third of the generals. Other variations exist which make solving the problem easier, including the use of digital signatures or by imposing communication restrictions between the peers in the network.

How does this all relate to blockchain?

Blockchains are decentralized ledgers which, by definition, are not controlled by a central authority. Due to the value stored in these ledgers, bad actors have huge economic incentives to try and cause faults. That said, Byzantine Fault Tolerance, and thus a solution to the Byzantine Generals’ Problem for blockchains is much needed.

In the absence of BFT, a peer is able to transmit and post false transactions effectively nullifying the blockchain’s reliability. To make things worse, there is no central authority to take over and repair the damage.

Conclusion

In this article, we discussed some basic concepts of consensus in distributed systems.

In the next article, we will discuss and compare some the algorithms that are used in blockchains in order to achieve Byzantine Fault Tolerance.

• DLT consensus

• PoS vs. PoW

Vitalik Buterin, Feb 2017

“Decentralization” is one of the words that is used in the cryptoeconomics space the most frequently, and is often even viewed as a blockchain’s entire raison d’être, but it is also one of the words that is perhaps defined the most poorly. Thousands of hours of research, and billions of dollars of hashpower, have been spent for the sole purpose of attempting to achieve decentralization, and to protect and improve it, and when discussions get rivalrous it is extremely common for proponents of one protocol (or protocol extension) to claim that the opposing proposals are “centralized” as the ultimate knockdown argument.

But there is often a lot of confusion as to what this word actually means. Consider, for example, the following completely unhelpful, but unfortunately all too common, diagram:

Three types of Decentralization

When people talk about software decentralization, there are actually three separate axes of centralization/decentralization that they may be talking about. While in some cases it is difficult to see how you can have one without the other, in general they are quite independent of each other. The axes are as follows:

• Architectural (de)centralization — how many physical computers is a system made up of? How many of those computers can it tolerate breaking down at any single time?

• Political (de)centralization — how many individuals or organizations ultimately control the computers that the system is made up of?

• Logical (de)centralization— does the interface and data structures that the system presents and maintains look more like a single monolithic object, or an amorphous swarm? One simple heuristic is: if you cut the system in half, including both providers and users, will both halves continue to fully operate as independent units?

We can try to put these three dimensions into a chart:

Note that a lot of these placements are very rough and highly debatable. But let’s try going through any of them:

• Traditional corporations are politically centralized (one CEO), architecturally centralized (one head office) and logically centralized (can’t really split them in half)

• Civil law relies on a centralized law-making body, whereas common law is built up of precedent made by many individual judges. Civil law still has some architectural decentralization as there are many courts that nevertheless have large discretion, but common law have more of it. Both are logically centralized (“the law is the law”).

• BitTorrent is logically decentralized similarly to how English is. Content delivery networks are similar, but are controlled by one single company.

• Blockchains are politically decentralized (no one controls them) and architecturally decentralized (no infrastructural central point of failure) but they are logically centralized (there is one commonly agreed state and the system behaves like a single computer)

Architectural centralization often leads to political centralization, though not necessarily — in a formal democracy, politicians meet and hold votes in some physical governance chamber, but the maintainers of this chamber do not end up deriving any substantial amount of power over decision-making as a result. In computerized systems, architectural but not political decentralization might happen if there is an online community which uses a centralized forum for convenience, but where there is a widely agreed social contract that if the owners of the forum act maliciously then everyone will move to a different forum (communities that are formed around rebellion against what they see as censorship in another forum likely have this property in practice).

Three reasons for Decentralization

The next question is, why is decentralization useful in the first place? There are generally several arguments raised:

• Fault tolerance— decentralized systems are less likely to fail accidentally because they rely on many separate components that are not likely.

• Collusion resistance — it is much harder for participants in decentralized systems to collude to act in ways that benefit them at the expense of other participants, whereas the leaderships of corporations and governments collude in ways that benefit themselves but harm less well-coordinated citizens, customers, employees and the general public all the time.

All three arguments are important and valid, but all three arguments lead to some interesting and different conclusions once you start thinking about protocol decisions with the three individual perspectives in mind. Let us try to expand out each of these arguments one by one.

Do blockchains as they are today manage to protect against common mode failure? Not necessarily. Consider the following scenarios:

• All nodes in a blockchain run the same client software, and this client software turns out to have a bug.

• All nodes in a blockchain run the same client software, and the development team of this software turns out to be socially corrupted.

• The research team that is proposing protocol upgrades turns out to be socially corrupted.

• In a proof of work blockchain, 70% of miners are in the same country, and the government of this country decides to seize all mining farms for national security purposes.

• The majority of mining hardware is built by the same company, and this company gets bribed or coerced into implementing a backdoor that allows this hardware to be shut down at will.

• In a proof of stake blockchain, 70% of the coins at stake are held at one exchange.

A holistic view of fault tolerance decentralization would look at all of these aspects, and see how they can be minimized. Some natural conclusions that arise are fairly obvious:

Note that the fault tolerance requirement in its naive form focuses on architectural decentralization, but once you start thinking about fault tolerance of the community that governs the protocol’s ongoing development, then political decentralization is important too.

However, once you adopt a richer economic model, and particularly one that admits the possibility of coercion (or much milder things like targeted DoS attacks against nodes), decentralization becomes more important. If you threaten one person with death, suddenly $50 million will not matter to them as much anymore. But if the $50 million is spread between ten people, then you have to threaten ten times as many people, and do it all at the same time. In general, the modern world is in many cases characterized by an attack/defense asymmetry in favor of the attacker — a building that costs $10 million to build may cost less than $100,000 to destroy, but the attacker’s leverage is often sublinear: if a building that costs $10 million to build costs $100,000 to destroy, a building that costs $1 million to build may realistically cost perhaps $30,000 to destroy. Smaller gives better ratios.

Finally, we can get to perhaps the most intricate argument of the three, collusion resistance. Collusion is difficult to define; perhaps the only truly valid way to put it is to simply say that collusion is “coordination that we don’t like”. There are many situations in real life where even though having perfect coordination between everyone would be ideal, one sub-group being able to coordinate while the others cannot is dangerous.

Blockchain advocates also make the point that blockchains are more secure to build on because they can’t just change their rules arbitrarily on a whim whenever they want to, but this case would be difficult to defend if the developers of the software and protocol were all working for one company, were part of one family and sat in one room. The whole point is that these systems should not act like self-interested unitary monopolies. Hence, you can certainly make a case that blockchains would be more secure if they were more discoordinated.

There are three ways to answer this:

• Don’t bother mitigating undesired coordination; instead, try to build protocols that can resist it.

• Try to find a happy medium that allows enough coordination for a protocol to evolve and move forward, but not enough to enable attacks.

• Try to make a distinction between beneficial coordination and harmful coordination, and make the former easier and the latter harder.

The third is a social challenge more than anything else; solutions in this regard may include:

• Social interventions that try to increase participants’ loyalty to the community around the blockchain as a whole and substitute or discourage the possibility of the players on one side of a market becoming directly loyal to each other.

• Promoting communication between different “sides of the market” in the same context, so as to reduce the possibility that either validators or developers or miners begin to see themselves as a “class” that must coordinate to defend their interests against other classes.

• Designing the protocol in such a way as to reduce the incentive for validators/miners to engage in one-to-one “special relationships”, centralized relay networks and other similar super-protocol mechanisms.

• Clear norms about what the fundamental properties that the protocol is supposed to have, and what kinds of things should not be done, or at least should be done only under very extreme circumstances.

This third kind of decentralization, decentralization as undesired-coordination-avoidance, is thus perhaps the most difficult to achieve, and tradeoffs are unavoidable. Perhaps the best solution may be to rely heavily on the one group that is guaranteed to be fairly decentralized: the protocol’s users.

Welcome to the Haruko digital assets knowledge portal.

We are excited to share a curated database of the best crypto thought pieces and resources, which we hope will be helpful in your digital asset journey.

Chris Dixon, Feb 2018

The first two eras of the internet

During the first era of the internet — from the 1980s through the early 2000s — internet services were built on open protocols that were controlled by the internet community. This meant that people or organizations could grow their internet presence knowing the rules of the game wouldn’t change later on. Huge web properties were started during this era including Yahoo, Google, Amazon, Facebook, LinkedIn, and YouTube. In the process, the importance of centralized platforms like AOL greatly diminished.

During the second era of the internet, from the mid 2000s to the present, for-profit tech companies — most notably Google, Apple, Facebook, and Amazon (GAFA) — built software and services that rapidly outpaced the capabilities of open protocols. The explosive growth of smartphones accelerated this trend as mobile apps became the majority of internet use. Eventually users migrated from open services to these more sophisticated, centralized services. Even when users still accessed open protocols like the web, they would typically do so mediated by GAFA software and services.

The good news is that billions of people got access to amazing technologies, many of which were free to use. The bad news is that it became much harder for startups, creators, and other groups to grow their internet presence without worrying about centralized platforms changing the rules on them, taking away their audiences and profits. This in turn stifled innovation, making the internet less interesting and dynamic. Centralization has also created broader societal tensions, which we see in the debates over subjects like fake news, state sponsored bots, “no platforming” of users, EU privacy laws, and algorithmic biases. These debates will only intensify in the coming years.

“Web 3”: the third era of the internet

One response to this centralization is to impose government regulation on large internet companies. This response assumes that the internet is similar to past communication networks like the phone, radio, and TV networks. But the hardware-based networks of the past are fundamentally different than the internet, a software-based network. Once hardware-based networks are built, they are nearly impossible to rearchitect. Software-based networks can be rearchitected through entrepreneurial innovation and market forces.

Why decentralization?

Decentralization is a commonly misunderstood concept. For example, it is sometimes said that the reason cryptonetwork advocates favor decentralization is to resist government censorship, or because of libertarian political views. These are not the main reasons decentralization is important.

Let’s look at the problems with centralized platforms. Centralized platforms follow a predictable life cycle. When they start out, they do everything they can to recruit users and 3rd-party complements like developers, businesses, and media organizations. They do this to make their services more valuable, as platforms (by definition) are systems with multi-sided network effects. As platforms move up the adoption S-curve, their power over users and 3rd parties steadily grows.

When they hit the top of the S-curve, their relationships with network participants change from positive-sum to zero-sum. The easiest way to continue growing lies in extracting data from users and competing with complements over audiences and profits. Historical examples of this are Microsoft vs. Netscape, Google vs. Yelp, Facebook vs. Zynga, and Twitter vs. its 3rd-party clients. Operating systems like iOS and Android have behaved better, although still take a healthy 30% tax, reject apps for seemingly arbitrary reasons, and subsume the functionality of 3rd-party apps at will.

For 3rd parties, this transition from cooperation to competition feels like a bait-and-switch. Over time, the best entrepreneurs, developers, and investors have become wary of building on top of centralized platforms. We now have decades of evidence that doing so will end in disappointment. In addition, users give up privacy, control of their data, and become vulnerable to security breaches. These problems with centralized platforms will likely become even more pronounced in the future.

Enter cryptonetworks

In short, cryptonetworks align network participants to work together toward a common goal — the growth of the network and the appreciation of the token. This alignment is one of the main reasons Bitcoin continues to defy skeptics and flourish, even while new cryptonetworks like Ethereum have grown alongside it.

Today’s cryptonetworks suffer from limitations that keep them from seriously challenging centralized incumbents. The most severe limitations are around performance and scalability. The next few years will be about fixing these limitations and building networks that form the infrastructure layer of the crypto stack. After that, most of the energy will turn to building applications on top of that infrastructure.

How decentralization wins

It’s one thing to say decentralized networks should win, and another thing to say they will win. Let’s look at specific reasons to be optimistic about this.

Software and web services are built by developers. There are millions of highly skilled developers in the world. Only a small fraction work at large technology companies, and only a small fraction of those work on new product development. Many of the most important software projects in history were created by startups or by communities of independent developers.

Decentralized networks can win the third era of the internet for the same reason they won the first era: by winning the hearts and minds of entrepreneurs and developers.

The lesson is that when you compare centralized and decentralized systems you need to consider them dynamically, as processes, instead of statically, as rigid products. Centralized systems often start out fully baked, but only get better at the rate at which employees at the sponsoring company improve them. Decentralized systems start out half-baked but, under the right conditions, grow exponentially as they attract new contributors.

In the case of cryptonetworks, there are multiple, compounding feedback loops involving developers of the core protocol, developers of complementary cryptonetworks, developers of 3rd party applications, and service providers who operate the network. These feedback loops are further amplified by the incentives of the associated token, which — as we’ve seen with Bitcoin and Ethereum — can supercharge the rate at which crypto communities develop (and sometimes lead to negative outcomes, as with the excessive electricity consumed by Bitcoin mining).

The question of whether decentralized or centralized systems will win the next era of the internet reduces to who will build the most compelling products, which in turn reduces to who will get more high quality developers and entrepreneurs on their side. GAFA has many advantages, including cash reserves, large user bases, and operational infrastructure. Cryptonetworks have a significantly more attractive value proposition to developers and entrepreneurs. If they can win their hearts and minds, they can mobilize far more resources than GAFA, and rapidly outpace their product development.

Centralized platforms often come bundled at launch with compelling apps: Facebook had its core socializing features and the iPhone had a number of key apps. Decentralized platforms, by contrast, often launch half-baked and without clear use cases. As a result, they need to go through two phases of product-market fit: 1) product-market fit between the platform and the developers/entrepreneurs who will finish the platform and build out the ecosystem, and 2) product-market fit between the platform/ecosystem and end users. This two-stage process is what causes many people — including sophisticated technologists — to consistently underestimate the potential of decentralized platforms.

The next era of the internet

Decentralized networks aren’t a silver bullet that will fix all the problems on the internet. But they offer a much better approach than centralized systems.

Or consider the problem of network governance. Today, unaccountable groups of employees at large platforms decide how information gets ranked and filtered, which users get promoted and which get banned, and other important governance decisions. In cryptonetworks, these decisions are made by the community, using open and transparent mechanisms. As we know from the offline world, democratic systems aren’t perfect, but they are a lot better than the alternatives.

Centralized platforms have been dominant for so long that many people have forgotten there is a better way to build internet services. Cryptonetworks are a powerful way to develop community-owned networks and provide a level playing field for 3rd-party developers, creators, and businesses. We saw the value of decentralized systems in the first era of the internet. Hopefully we’ll get to see it again in the next.

VIEWS EXPRESSED IN “CONTENT” (INCLUDING POSTS, PODCASTS, VIDEOS) LINKED ON THIS WEBSITE OR POSTED IN SOCIAL MEDIA AND OTHER PLATFORMS (COLLECTIVELY, “CONTENT DISTRIBUTION OUTLETS”) ARE MY OWN AND ARE NOT THE VIEWS OF AH CAPITAL MANAGEMENT, L.L.C. (“A16Z”) OR ITS RESPECTIVE AFFILIATES. AH CAPITAL MANAGEMENT IS AN INVESTMENT ADVISER REGISTERED WITH THE SECURITIES AND EXCHANGE COMMISSION. REGISTRATION AS AN INVESTMENT ADVISER DOES NOT IMPLY ANY SPECIAL SKILL OR TRAINING. THE POSTS ARE NOT DIRECTED TO ANY INVESTORS OR POTENTIAL INVESTORS, AND DO NOT CONSTITUTE AN OFFER TO SELL -- OR A SOLICITATION OF AN OFFER TO BUY -- ANY SECURITIES, AND MAY NOT BE USED OR RELIED UPON IN EVALUATING THE MERITS OF ANY INVESTMENT.

THE CONTENT SHOULD NOT BE CONSTRUED AS OR RELIED UPON IN ANY MANNER AS INVESTMENT, LEGAL, TAX, OR OTHER ADVICE. YOU SHOULD CONSULT YOUR OWN ADVISERS AS TO LEGAL, BUSINESS, TAX, AND OTHER RELATED MATTERS CONCERNING ANY INVESTMENT. ANY PROJECTIONS, ESTIMATES, FORECASTS, TARGETS, PROSPECTS AND/OR OPINIONS EXPRESSED IN THESE MATERIALS ARE SUBJECT TO CHANGE WITHOUT NOTICE AND MAY DIFFER OR BE CONTRARY TO OPINIONS EXPRESSED BY OTHERS. ANY CHARTS PROVIDED HERE ARE FOR INFORMATIONAL PURPOSES ONLY, AND SHOULD NOT BE RELIED UPON WHEN MAKING ANY INVESTMENT DECISION. CERTAIN INFORMATION CONTAINED IN HERE HAS BEEN OBTAINED FROM THIRD-PARTY SOURCES. WHILE TAKEN FROM SOURCES BELIEVED TO BE RELIABLE, I HAVE NOT INDEPENDENTLY VERIFIED SUCH INFORMATION AND MAKES NO REPRESENTATIONS ABOUT THE ENDURING ACCURACY OF THE INFORMATION OR ITS APPROPRIATENESS FOR A GIVEN SITUATION. THE CONTENT SPEAKS ONLY AS OF THE DATE INDICATED.

Cryptopedia, Aug 2021

Expecting no one to eat the cake you stored in the fridge isn’t an exercise in trustlessness. “Trust” and “trustless” are related — yet different — concepts.

While trustless is defined by Merriam Webster as “not deserving of trust,” in the blockchain space it means something entirely different. Trustlessness in the blockchain industry simply means you do not need to place your sole trust in any one stranger, institution, or other third party in order for a network or payment system to function. Trustless systems work and achieve consensus mainly through the code, asymmetric cryptography, and protocols of the blockchain network itself. The trustless environments that blockchains have created enable the peer-to-peer (P2P) sending and receiving of transactions, smart contract agreements, and more.

Whom Do You Trust?

Trust vs. Trustless in Crypto

The implications for global commerce are significant. Furthermore, the trustless component of blockchains goes far beyond payments and includes implications for crypto self-custody, smart contracts, and asset trading solutions.

Trustless Crypto Wallets

While such an exchange transaction is not trustless in the crypto sense, you may feel more comfortable using a crypto custodian in the same way that you feel more comfortable using a bank to store large sums of money. Holding your own wallet and keys comes with its own set of challenges and requirements to ensure security and access. However, after purchasing on a custodied exchange, you could also choose to withdraw your funds to a trustless non-custodial wallet where you have sole control of your cryptocurrency.

Do Decentralized Exchanges Make Trading Trustless?

“In Code We Trust” — Are There Truly “Trustless” Systems?

In crypto, you don’t necessarily have to trust any other person (or institution), but there is someone you must trust: yourself. While the self-custody of crypto is referred to as trustless, you must decide if you can trust yourself with this responsibility. This entails safely storing any passwords, having a recovery phrase, and following other best practices. If your passwords are lost or stolen, you might not be able to recover your funds.

Mohit Mamoria, Jun 2017

Unless you’re hiding under the rock, I am sure you’d have heard of Bitcoins and Blockchain. After all, they are the trending and media’s favorite topics these days — the buzzwords of the year. Even the people who’ve never mined a cryptocurrency or understand how it works, are talking about it. I have more non-technical friends than technical ones. They have been bugging me for weeks to explain this new buzzword to them. I guess there are thousands out there who feel the same. And when that happens, there comes a time to write something to which everyone can point the other lost souls to — that’s the purpose of this post — written in plain english that any regular internet user understands.

Blockchain: why do we even need something this complex?

“For every complex problem there is an answer that is clear, simple, and wrong.” — H. L. Mencken

Unlike every other post on the internet, instead of first defining the Blockchain, we’ll understand the problem it solves.

Imagine, Joe is your best friend. He is traveling overseas, and on the fifth day of his vacation, he calls you and says, “Dude, I need some money. I have run out of it.”

You reply, “Sending some right away,” and hung up.

You then call your account manager at your bank and tell him, “Please transfer $1000 from my account to Joe’s account.”

Your account manager replies, “Yes, sir.”

He opens up the register, checks your account balance to see if you have enough balance to transfer $1000 to Joe. Because you’re a rich man, you have plenty; thus, he makes an entry in the register like the following:

The Transaction Register

Note: We’re not talking about computers only to keep things simple.

You call Joe and tell him, “I’ve transferred the money. Next time, you’d go to your bank, you can withdraw the $1000 that I have just transferred.”

What just happened? You and Joe both trusted the bank to manage your money. There was no real movement of physical bills to transfer the money. All that was needed was an entry in the register. Or more precisely, an entry in the register that neither you nor Joe controls or owns.

And that is the problem of the current systems.

To establish trust between ourselves, we depend on individual third-parties.

For years, we’ve depended on these middlemen to trust each other. You might ask, “what is the problem depending on them?”

The problem is that they are singular in number. If a chaos has to be injected in the society, all it requires is one person/organization to go corrupt, intentionally or unintentionally.

• What if that register in which the transaction was logged gets burnt in a fire?

• What if, by mistake, your account manager had written $1500 instead of $1000?

• What if he did that on purpose?

For years, we have been putting all our eggs in one basket and that too in someone else’s.

Could there be a system where we can still transfer money without needing the bank?

To answer this question, we’ll need to drill down further and ask ourselves a better question (after all, only better questions lead to better answers).

Think about it for a second, what does transferring money means? Just an entry in the register. The better question would then be —

Is there a way to maintain the register among ourselves instead of someone else doing it for us?

Now, that is a question worth exploring. And the answer is what you might have already guessed. The blockchain is the answer to the profound question.

It is a method to maintain that register among ourselves instead of depending on someone else to do it for us.

Are you still with me? Good. Because now, when several questions have started popping in your mind, we will learn how this distributed register works.

Yes, but tell me, how does it work?

The requirement of this method is that there must be enough people who would like not to depend on a third-party. Only then this group can maintain the register on their own.

“It might make sense just to get some Bitcoin in case it catches on. If enough people think the same way, that becomes a self-fulfilling prophecy.” — Satoshi Nakamoto in 2009

How many are enough? At least three. For our example, we will assume ten individuals want to give up on banks or any third-party. Upon mutual agreement, they have details of each other’s accounts all the time — without knowing the other’s identity.

1. An Empty Folder

Everyone contains an empty folder with themselves to start with. As we’ll progress, all these ten individuals will keep adding pages to their currently empty folders. And this collection of pages will form the register that tracks the transactions.

2. When A Transaction Happens

Next, everyone in the network sits with a blank page and a pen in their hands. Everyone is ready to write any transaction that occurs within the system.

Now, if #2 wants to send $10 to #9.

To make the transaction, #2 shouts and tells everyone, “I want to transfer $10 to #9. So, everyone, please make a note of it on your pages.”

Everyone checks whether #2 has enough balance to transfer $10 to #9. If she has enough balance, everyone then makes a note of the transaction on their blank pages.

First transaction on the page

The transaction is then considered to be complete.

3. Transactions Continue Happening

As the time passes, more people in the network feel the need to transfer money to others. Whenever they want to make a transaction, they announce it to everyone else. As soon as a person listens to the announcement, (s)he writes it on his/her page.

This exercise continues until everyone runs out of space on the current page. Assuming a page has space to record ten transactions, as soon as the tenth transaction is made, everybody runs out of the space.

When page gets filled

It’s time to put the page away in the folder and bring out a new page and repeat the process from the step 2 above.

4. Putting Away The Page

Before we put away the page in our folders, we need to seal it with a unique key that everyone in the network agrees upon. By sealing it, we will make sure that no one can make any changes to it once its copies have been put away in everyone’s folder — not today, not tomorrow and not even after a year. Once in the folder, it will always stay in the folder — sealed. Moreover, if everyone trusts the seal, everyone trusts the contents of the page. And this sealing of the page is the crux of this method.

[Jargon Box] It is called ‘mining’ on the page to secure it, but for the simplicity of it, we’ll keep calling it ‘sealing.’

Earlier the third-party/middleman gave us the trust that whatever they have written in the register will never be altered. In a distributed and decentralized system like ours, this seal will provide the trust instead.

Interesting! How do we seal the page then?

Before we learn how we can seal the page, we’ll know how the seal works, in general. And as a pre-requisite to it is learning about something that I like to call…

The Magic Machine

Imagine a machine surrounded by thick walls. If you send a box with something inside it from the left, it will spit out a box containing something else.

[Jargon Box] This machine is called ‘Hash Function,’ but we aren’t in a mood to be too technical. So, for today, these are ‘The Magic Machines.’

The Magic Machine (aka Hashing Function)

Suppose, you send the number 4 inside it from the left, we’d find that it spat out the following word on its right: ‘dcbea.’

How did it convert the number 4 to this word? No one knows. Moreover, it is an irreversible process. Given the word, ‘dcbea,’ it is impossible to tell what the machine was fed on the left. But every time you’d feed the number 4 to the machine, it will always spit out the same word, ‘dcbea.’

hash(4) == dcbea

Given the word, ‘dcbea,’ it is impossible to tell what the machine was fed on the left. But every time you’d feed the number 4 to the machine, it will always spit out the same word, ‘dcbea.’

Let’s try sending in a different number. How about 26?

hash(26) == 94c8e

We got ‘94c8e’ this time. Interesting! So, the words can contain the numbers too.

What if I ask you the following question now:

“Can you tell me what should I send from the left side of the machine such that I get a word that starts with three leading zeroes from the right side of it? For example, 000ab or 00098 or 000fa or anything among the others.”

Predicting the input

Think about the question for a moment.

I’ve told you the machine has a property that we cannot calculate what we must send from the left after we’re given the expected output on the right. With such a machine given to us, how can we answer the question I asked?

I can think of one method. Why not try every number in the universe one by one until we get a word that starts with three leading zeroes?

Try everything to calculate the input

Being optimistic, after several thousand attempts, we’ll end up with a number that will yield the required output on the right.

It was extremely difficult to calculate the input given the output. But at the same time, it will always be incredibly easy to verify if the predicted input yields the required output. Remember that the machine spits out the same word for a number every time.

How difficult do you think the answer is if I give you a number, say 72533, and ask you the question, “Does this number, when fed into the machine, yields a word that starts with three leading zeroes?”

All you need to do is, throw the number in the machine and see what did you get on the right side of it. That’s it.

The most important property of such machines is that — “Given an output, it is extremely difficult to calculate the input, but given the input and the output, it is pretty easy to verify if the input leads to the output.”

We’ll remember this one property of the Magic Machines (or Hash Functions) through the rest of the post:

Given an output, it is extremely difficult to calculate the input, but given an input and output, it is pretty easy to verify if the input leads to the output.

How to use these machines to seal a page?

We’ll use this magic machine to generate a seal for our page. Like always, we’ll start with an imaginary situation.

Imagine I give you two boxes. The first box contains the number 20893. I, then, ask you, “Can you figure out a number that when added to the number in the first box and fed to the machine will give us a word that starts with three leading zeroes?”

This is a similar situation as we saw previously and we have learned that the only way to calculate such a number is by trying every number available in the entire universe.

After several thousand attempts, we’ll stumble upon a number, say 21191, which when added to 20893 (i.e. 21191 + 20893 = 42084) and fed to the machine, will yield a word that satisfies our requirements.

In such a case, this number, 21191 becomes the seal for the number 20893. Assume there is a page that bears the number 20893 written on it. To seal that page (i.e. no one can change the contents of it), we will put a badge labeled ‘21191’ on top of it. As soon as the sealing number (i.e. 21191) is stuck on the page, the page is sealed.

The sealed number

[Jargon Box] The sealing number is called ‘Proof Of Work,’ meaning that this number is the proof that efforts had been made to calculate it. We are good with calling it ‘sealing number’ for our purposes.

If anyone wants to verify whether the page was altered, all he would have to do is — add the contents of the page with the sealing number and feed to the magic machine. If the machine gives out a word with three leading zeroes, the contents were untouched. If the word that comes out doesn’t meet our requirements, we can throw away the page because its contents were compromised, and are of no use.

We’ll use a similar sealing mechanism to seal all our pages and eventually arrange them in our respective folders.

Finally, sealing our page…

To seal our page that contains the transactions of the network, we’ll need to figure out a number that when appended to the list of transactions and fed to the machine, we get a word that starts with three leading zeroes on the right.

Note: I have been using the phrase ‘word starting with three leading zeroes’ only as an example. It illustrates how Hashing Functions work. The real challenges are much more complicated than this.

Once that number is calculated after spending time and electricity on the machine, the page is sealed with that number. If ever, someone tries to change the contents of the page, the sealing number will allow anyone to verify the integrity of the page.

Now that we know about sealing the page, we will go back to the time when we had finished writing the tenth transaction on the page, and we ran out of space to write more.

As soon as everyone runs out of the page to write further transactions, they indulge in calculating the sealing number for the page so that it can be tucked away in the folder. Everyone in the network does the calculation. The first one in the network to figure out the sealing number announces it to everyone else.

Immediately on hearing the sealing number, everyone verifies if it yields the required output or not. If it does, everyone labels their pages with this number and put it away in their folders.

But what if for someone, say #7, the sealing number that was announced doesn’t yield the required output? Such cases are not unusual. The possible reasons for this could be:

• He might have misheard the transactions that were announced in the network

• He might have miswritten the transactions that were announced in the network

• He might have tried to cheat or be dishonest when writing transactions, either to favor himself or someone else in the network

No matter what the reason is, #7 has only one choice — to discard his page and copy it from someone else so that he too can put it in the folder. Unless he doesn’t put his page in the folder, he cannot continue writing further transactions, thus, forbidding him to be part of the network.

Whatever sealing number the majority agrees upon, becomes the honest sealing number.

Then why does everyone spend resources doing the calculation when they know that someone else will calculate and announce it to them? Why not sit idle and wait for the announcement?

Great question. This is where the incentives come in the picture. Everyone who is the part of the Blockchain is eligible for rewards. The first one to calculate the sealing number gets rewarded with free money for his efforts (i.e. expended CPU power and electricity).

Simply imagine, if #5 calculates the sealing number of a page, he gets rewarded with some free money, say $1, that gets minted out of thin air. In other words, the account balance of #5 gets incremented with $1 without decreasing anyone else’s account balance.

That’s how Bitcoin got into existence. It was the first currency to be transacted on a Blockchain (i.e. distributed registers). And in return, to keep the efforts going on in the network, people were awarded Bitcoins.

When enough people possess Bitcoins, they grow in value, making other people wanting Bitcoins; making Bitcoins grow in value even further; making even more people wanting Bitcoins; making them grow in value even further; and so on.

The rewards make everyone keep working in the network.

And once everyone tucks away the page in their folders, they bring out a new blank page and repeat the whole process all over again — doing it forever.

[Jargon Box] Think of a single page as a Block of transactions and the folder as the Chain of pages (Blocks), therefore, turning it into a Blockchain.

And that, my friends, is how Blockchain works.

Except that there’s one tiny thing I didn’t tell you. Yet.

Imagine there are five pages in the folder already — all sealed with a sealing number. What if I go back to the second page and modify a transaction to favor myself? The sealing number will let anyone detect the inconsistency in the transactions, right? What if I go ahead and calculate a new sealing number too for the modified transactions and label the page with that instead?

To prevent this problem of someone going back and modifying a page (Block) as well as the sealing number, there’s a little twist to how a sealing number is calculated.

Protecting modifications to the sealing numbers

Remember how I told you that I had given you two boxes — one containing the number 20893 and another empty for you to calculate? In reality, to calculate the sealing number in a Blockchain, instead of two boxes, there are three — two pre-filled and one to be calculated.

And when the contents of all those three boxes are added and fed to the machine, the answer that comes out from the right side must satisfy the required conditions.

We already know that one box contains the list of transactions and one box will contain the sealing number. The third box contains the output of the magic machine for the previous page.

With this neat little trick, we have made sure that every page depends on its previous page. Therefore, if someone has to modify a historical page, he would also have to change the contents and the sealing number of all the pages after that, to keep the chain consistent.

If one individual, out of the ten we imagined in the beginning, tries to cheat and modify the contents of the Blockchain (the folder containing the pages with the list of transactions), he would have to adjust several pages and also calculate the new sealing numbers for all those pages. We know how difficult it is to calculate the sealing numbers. Therefore, one dishonest guy in the network cannot beat the nine honest guys.

What will happen is, from the page the dishonest guy tries to cheat, he would be creating another chain in the network, but that chain would never be able to catch up with the honest chain — simply because one guy’s efforts and speed cannot beat cumulative efforts and speed of nine. Hence, guaranteeing that the longest chain in a network is the honest chain.

Longest chain is the honest chain.

Longest chain is the honest chain.

When I told you that one dishonest guy cannot beat nine honest guys, did it ring any bell in your head?

What if, instead of one, six guys turn dishonest?

In that case, the protocol will fall flat on its face. And it is known as “51% Attack”. If the majority of the individuals in the network decides to turn dishonest and cheat the rest of the network, the protocol will fail its purpose.

And that’s the only vulnerable reason why Blockchains might collapse if they ever will. Know that, it is unlikely to happen but we must all know the vulnerable points of the system. It is built on the assumption that the majority of a crowd is always honest.

And that, my friends, is all there is about Blockchains. If you ever find someone feeling left behind and wondering, “WTF is the Blockchain?” you know where you can point them to. Bookmark the link.

Can think of someone right now who should read this? The ‘Share’ button is all yours.

—

About the author

David Friedman, 2005

An old science fiction novel features a device that surrounds its bearer with an impenetrable bubble of force. The inventor rapidly discovers that every government and political faction on the planet wants what he has and is prepared to use any means, from persuasion to brute force, to get it. Our hero spends most of the book alternately listening to arguments, trying to decide who are the good guys and using his invention to help him escape attempts to capture him.

After about a hundred and fifty pages he realizes that he has been asking the wrong question. The answer to "what faction can be trusted with a monopoly over the shield" is "no." The right question is how the shield will affect the world--how it will alter the balance between freedom and oppression, individual and state, small and big. The answer to that is easy. A world where the random individual is armored against anything short of an atomic explosion will be, on net, a better and freer world than the one he is currently living in. He writes out an explanation of how the shield works and spends two days distributing the information to people all over the world. By the time Military Security--the most formidable of his pursuers--catches up with him, it is too late. The cat is out of the bag.

Poul Anderson's shield is fiction. The nearest real world equivalent is privacy--my control over other people's access to information about me. Neither my government nor my neighbor can punish my thoughts, because neither can read my mind. That is why thoughts are free. However much other people are offended by what I write, they cannot retaliate unless they know who wrote it, what he looks like, where he lives. That is why Salmon Rushdie is still alive despite the death sentence passed on the author of The Satanic Verses fifteen years ago by Iranian authorities.

Defensive weapons can be used for bad purposes; an impenetrable shield would be very useful for a bank robber. But it would be even more useful for the bank teller. Robbing banks would be harder in a world where everyone had the shield than in a world where nobody did.

The ability to control other people's access to information about you can be used for bad purposes too. That is the usual argument against privacy--"If you haven't done anything wrong, what do you have to hide?" The ability to conceal past crimes from the police and potential victims is useful to a robber. But the ability to conceal what I have that is worth stealing, where it is, how it is protected, is equally useful to the potential victim. Broadly stated, privacy gives each of us more control over his own life--which on average, if not in every case, is likely to lead to a freer world.

If I am a bad guy, the police are not the only people I might want to keep secrets from. When courting a wealthy widow, it helps if she does not know that my last three wives drowned in their bath tubs after taking out large life insurance policies. When borrowing money, it helps if the lender does not know that I have declared bankruptcy twice already.

But in a world of voluntary transactions--such as loans and marriages--my privacy does not require you to take me on faith. You have the option of not taking me. I have the power to keep my past defaults secret from a potential lender but he has the power to refuse to lend to me if I do. Privacy is my ability to control other people's access to information about me. That does not mean that they cannot get the information--only that they cannot get it without my permission. Someone who offers to take care of my children but refuses to allow me access to the records that would show whether or not he has ever been convicted of child abuse has already told me all I need to know.

In some contexts I am willing to let other people know things about me. In others I am eager to. If only lenders knew a little more about my finances I would not be interrupted at dinner by phone calls from people offering to refinance my nonexistent mortgage. If sellers were better informed about what sorts of things I was interested in buying, advertisements would be less of a nuisance and more of a service. Even in a world where I could keep information secret, I often would choose not to. Privacy provides me protection when I want it and only when I want it.

Privacy and Government

"Government is not reason. It is not eloquence. It is a force, like fire:

a dangerous servant and a terrible master."

George Washington

Privacy includes the ability to keep things secret from the government. The better I can do that, the less able government is to help me--I might be keeping secret my weakness for alcohol, or heroin, or gambling or pornography and so preventing the government from stepping in to protect me from myself. And the better other people can keep secrets from the government, the harder it is for the government to protect me from them. If you view government as a benevolent super being watching over you--a wise and kindly uncle with a long white beard--you will and should reject much of what I am saying.

But government is not Uncle Sam or a philosopher king. Government is a set of institutions through which human beings act for human purposes. Its special feature--what differentiates political action from the other ways in which we try to get what we want--is that government is permitted to use force to make people do things. A firm can try to fool me into giving it my money. A tax collector uses more direct methods. A preacher can try to persuade me to renounce my sins. The Drug Enforcement Administration, with the help of the local police, can arrange to have me locked up until I do.

Part of the genius of American political culture is the recognition that making it hard for governments to control people is not always a bad thing. Political mechanisms, even in a democracy, give us only very limited control over what government can do to us. Reducing government's ability to do bad things to us, at the cost of limiting its ability to protect us from bad things done to us by ourselves or by other people, may not be such a bad deal. And since government, unlike a private criminal, has overwhelming superiority of physical force, control over what information it can get about me is one of the few ways in which I can limit its ability to control me.

I have defined privacy and sketched the reasons why I think it is, on the whole, a good thing. The obvious next questions are where privacy comes from--what determines how much of it we have--and what we can and should do to get more of it.

Where Does Privacy Come From?

One of the things that determines how much control I have over other people's access to information about me is technology. If someone invents a mind reading machine or a reliable truth drug, my thoughts will no longer be as private as they now are. Or as free.

Another is custom--systems of social norms. The more willing my friends and neighbors are to gossip about something, the easier it is for information about that something to get from those who have it to those who want it. That is one reason why Israelis are better informed about how much money their friends and relations make than Americans are and modern Americans better informed about other people's sex lives than nineteenth century Britons were.

A final factor is law. In the U.S., the Fourth Amendment to the Constitution prohibits "unreasonable searches and seizures" and requires that search warrants shall only be issued with probable cause. The more narrowly courts interpret that restriction, the easier it is to keep secrets from the police. One important example is the series of cases that applied the restriction to wiretaps as well as physical searches. Later cases have ruled on to what extent the use of high tech devices to figure out what people are doing inside their houses--infrared photographs to spot illegal greenhouses growing marijuana, for example--is a search and so requires a warrant.

Law and technology interact in complicated ways. For your neighbor's nosy fifteen year old to use a scanner to listen to the phone calls you make on your wireless phone and tell his friends about them is illegal. It is also easy, making that particular legal protection of privacy in practice unenforceable. The substitute is technology--encryption of the signal from the handset to the base station. Similarly with cell phones.

As these examples suggest, technological developments can both decrease and increase privacy. So can law. Legal rules that ban or limit technologies for learning things about other people, such as laws against wiretaps, increase privacy. Legal rules that ban or limit technologies for preventing other people from learning things about us, such as restrictions on the use of encryption, decrease it.

Privacy and Technology: The Dark Side of the Force

It used to be that one reason to move from a village to the big city was to get more privacy. Walls were no higher in the city, windows no less transparent. But there were so many more people. In the village, interested neighbors could keep track of what who was doing with whom. In the city, nobody could keep track of everyone.

That form of privacy--privacy through obscurity--is doomed. I cannot keep track of the million people who share the city I live in. But the computer on my desk has enough space on its hard drive to hold a hundred pages of information on every man, woman and child in San Jose. With a few hundred dollars worth of additional storage, I could do it for everyone in California, for a few thousand, everyone in the country. And I can do more than store the information. If I had it I could search it--produce, in a matter of seconds, a list of those of my fellow citizens who are left handed gun owners with more than six children. Privacy through obscurity cannot survive modern data processing.

As it happens, I do not have a hundred pages worth of information on each of my fellow citizens. But with a little time and effort--too much for a single individual, but not too much for a government, a police department, or a large firm--I could. It is hard to pass through the world without leaving tracks. Somewhere there is a record of every car I have registered, every tax form I have filed, two marriages, one divorce, the birth of three children, thousands of posts to online forums on a wide variety of subjects, four published books, medical records and a great deal more.

Much such information, although not all of it, was publicly available in the past. But actually digging it up was a lot of work. The result was that most of us went through life reasonably sure that most of the people we met did not know much about us beyond what we chose to tell them. That will not be true in the future.

Data processing is one technology with the potential to sharply reduce privacy. Another is surveillance. One form--already common in England--is a video camera on a pole.

A video camera in a park connected to a screen with a police officer watching it is, at first glance, no more a violation of privacy than the same police officer standing in the park watching what is going on. It merely lets the officer do his watching somewhere warm and out of the wet. Add a video recorder and it is arguably an improvement, since the evidence it produces is less subject to mistake or misrepresentation than the memory of the policeman. And, judging by British experience, such surveillance cameras are an effective way of reducing crime. What's the problem?

To see the answer, add one more technology--face recognition software. Combine that with a database, put up enough cameras, and we have a record of where everyone was any time of the day and--with suitable cameras--night. The arresting officer, or the prosecuting attorney, no longer has to ask the defendant where he was at eight P.M. of July ninth. All he has to do is enter the defendant's social security number and the date and the computer will tell him. And, if the defendant was in a public place at the time, show him.

For a slightly lower tech version of the same issue, consider the humble phone tap. In the past, the main limit on how many phones got tapped by police was not the difficulty of getting a court order but the cost of implementing it. Phone taps are labor intensive--someone has to listen to a lot of phone calls in order to find the ones that matter.

That problem has now been solved. Voice recognition software originated by companies such as Dragon Systems and IBM lets computers convert speech into text--a boon for computer users who are slow typists. The same technology means that the police officer listening to someone else's phone calls can now be replaced by a computer. Only when it gets a hit, spots the words or phrases it has been programmed to listen for, does it need to call in a human being. Computers work cheap.

In an old comedy thriller (The President's Analyst, starring James Coburn) the hero, having temporarily escaped his pursuers and made it to a phone booth, calls a friendly CIA agent to come rescue him. When he tries to leave the booth, the door won't open. Down the road comes a phone company truck loaded with booths. The truck's crane picks up the one containing the analyst, deposits it in the back, replaces it with an empty booth and drives off.

A minute later a helicopter descends containing the CIA agent and a KGB agent who is his temporary ally. They look in astonishment at the empty phone booth. The American speaks first:

"It can't be. Every phone in America tapped?"

The response (you will have to imagine the Russian accent)

"Where do you think you are≠Russia?"

A great scene in a very funny movie--but it may not be a joke much longer. The digital wiretap bill, pushed through Congress by the FBI a few years ago, already requires phone companies to provide law enforcement with the ability to simultaneously tap one percent of all phones in a selected area. There is no obvious reason why that cannot be expanded in the future. My current estimate is that the dedicated hardware to do the listening part of the job--for every phone call in the U.S.--would cost less than a billion dollars. And it is getting cheaper.

So far I have been discussing technologies that already exist. Fast forward a little further and surveillance need no longer be limited to public places. Video cameras are getting smaller. It should not be all that long before we can build one with the size--and the aerodynamic characteristics--of a mosquito.

Here again, if we regard government law enforcement agents as unambiguously good guys, there is no problem. The better our record of where everyone was when, the easier it will be to catch and convict criminals.

The same technology would make keeping track of dissidents, or political opponents, or members of an unpopular religion, or people with the wrong sexual tastes, or people who read the wrong books, or anyone else, a great deal easier than it now is. It is true that the random government is rather less likely to have bad intentions than the random criminal. But if it does have bad intentions it can do a great deal more damage.

The technologies I have been discussing so far--database and face recognition software, surveillance hardware--have the potential to make this a much less private world. So do other technologies that I have not covered: improvements in lie detectors and interrogation drugs to learn what we think, biometric identification by fingerprints, retinal patterns, DNA to learn who we are, with or without our permission. The future implications of such developments are sufficiently strong to have convinced at least one thoughtful observer that the best we can hope for in the future is a transparent society, a world without privacy where the police can watch us but we can also watch them (Brin 1999). I would find the symmetry of that future more appealing if it did not conceal an important asymmetry: They can arrest us and we cannot arrest them.

But there are other technologies.

Encryption: A World of Strong Privacy

We start with an old problem: How to communicate with someone without letting other people know what you are saying. There are a number of familiar solutions. If worried about eavesdroppers, check under the eaves. To be safer still, hold your private conversation in the middle of a large, open field, or a boat in the middle of a lake. The fish are not interested and nobody else can hear.

That no longer works. The middle of a lake is still within range of a shotgun mike. Eaves do not have to contain eavesdroppers≠just a microphone and a transmitter. Phone lines can be tapped, cordless or cell phone messages intercepted. An email bounces through multiple computers on its way to its destinationãanyone controlling one of those computers can save a copy for himself.

The solution is encryption. Scramble the message. Provide the intended recipient with the formula for unscrambling it. Now it does not matter if someone intercepts your mail. He can't read it.

There is still a problem. In order to read my scrambled message you need the key≠the formula describing how to unscramble it. If I do not have a safe way of sending you messages, I may not have a safe way of sending you the key either. If I sent it by a trusted messenger but made a small mistake as to who he was really working for, someone else now has a copy and can use it decrypt my future messages to you.

About twenty-five years ago, this problem was solved. The solution is public key encryption. It works by using two keys, each of which decrypts what the other encrypts. One of the two--my public key--I make available to anyone who might want to send me a message. The other never leaves my hands. Someone who wants to communicate with me encrypts his messages with my public key. I use my private key to decrypt them.

Public key encryption provides a free bonus--digital signatures. In order to prove that a message was sent by me I can encrypt it using my private key. The recipient decrypts it using my public key. The fact that what comes out is text rather than gibberish proves it was encrypted with the matching private key--which only I have. Hence, unless I have been very careless, the message is from me.

Imagine a world where public key encryption is in general use. Add in related technologies such as anonymous digital money, to permit payments that leave no paper trail, and anonymous remailers, to keep who I am talking to, as well as what I am saying, private--for details see (Friedman 1996). In that world I can email someone--anyone--with reasonable certainty that nobody else can read the message. I can have telephone conversations without worrying about who may be listening. In that world I can if I wish establish an online persona--an identity defined by my digital signature--while keeping control over the link between that and my realspace persona. However much my online persona offends someone--even the rulers of Iran--there is very little anyone can do about it. It is hard to murder someone when you don't know his name, what he looks like, or what continent he is on.

I have been describing things we already know how to do. Most can already be done--using free software that runs on the computers most of us have. I now take a small step forward to add one more element to the mix: Virtual reality. Using goggles and earphones--if we are willing to step further into science fiction, direct links between mind and computer--we create the illusion of seeing, hearing, perhaps tasting and touching. The world of strong privacy expands from text messages and phone conversations to something very much like the real world we currently live in. Just let your fingers do the walking.

Combine and Stir

I have described two clusters of technologies. One--database, voice and text recognition, surveillance--has the potential to reduce privacy to the point where those who control the technology know very nearly everything that everyone does. The other--encryption, online communication, virtual reality--has the potential to increase privacy to the point where individuals have nearly total control over other people's access to information about them. What if we get both?

It will be an interesting world. Everything you do in realspace will be known to the authorities, perhaps to everyone--David Brin's Transparent Society. But most of the important stuff--all transactions involving information, ideas, arguments, beliefs--will have been moved to cyberspace, protected by the strong privacy of encryption. Freedom of speech will no longer depend on how the Supreme Court interprets the First Amendment. It will be protected, instead, by the laws of mathematics--which so far, at least, heavily favor defense over offense, encryption over cracking.

There will be--already have been--attempts to use law to block both futures. Supporters of privacy will try to get laws restricting the ability of law enforcement--and other people--to use technology to learn our secrets. Opponents of privacy will try to get laws restricting the ability of private individuals to use encryption to protect their secrets.

Technology and Law

There are at least two legal approaches to preserving privacy in the face of technologies such as computer databases and surveillance. One is to use law to prevent other people from getting information--a data base is of no use if there is nothing in it. The other is to permit other people to get information but use law to limit what they can do with it.

An example of the first approach is regulation of wire tapping and other forms of surveillance--both laws against private surveillance and laws restricting surveillance by law enforcement agents. Such restrictions can keep some information about me from getting to other people. But they do nothing to protect the vast amount of information that I generate by going about my daily life in the public view--buying and selling, marrying and getting divorced, writing and talking.

An example of the second approach is the web of restrictions, legal, contractual, and customary, on the use of confidential information. I cannot keep my doctor from having access to the medical information he creates when he examines me and uses when he prescribes for me. But I can, to some limited degree, prevent him from sharing that information with other people. Credit bureaus are free to collect information on people in order to advise other people as to whether to lend them money but, under current Federal law, they are only permitted to release that information in response to requests from people who have a legitimate need for it.

As the example of credit bureaus suggests, there are practical difficulties with protecting privacy by letting other people have information and then controlling what they do with it. Credit agencies could not serve their intended purpose at any reasonable cost if they engaged in an extensive investigation of everyone who asked for information. And even if the agency limits itself to giving the information to people who can prove they are entitled to it, there is no way it can control who they then give it to. It is probably prudent to assume that what the credit agency knows about you any else can know if he really wants to. The forms you sign when you shift to a new doctor include an extensive list of people to whom and circumstances under which your medical information will be made available, so it might be equally prudent not to rely too much on your medical privacy.

As long as we limit our options to current technologies for protecting privacy, the outlook does not look good. We might succeed in restricting the use of surveillance, wiretapping, and similar technologies, although attempts to restrict their use by law enforcement face serious opposition by those concerned with the threat of crime and terrorism. But most information about us is public, and once information is out it is hard to control how other people use it or who they give it to.

The technologies of strong privacy offer at least a partial solution. If I make a purchase with a credit card, I create a paper trail--someone, somewhere, knows what I bought. Even if I use cash, a purchase in real space requires me to walk into a store where someone sees me--the information about what I bought is now his as well as mine. In a world where the relevant software is a little better than it now is--say ten years in the future--that someone is a store video camera linked to facial recognition software linked to a database. Stores, after all, like to know who their customers are.

If, however, I buy something over the phone or over the internet, using the digital equivalent of cash--anonymous digital currency--only I know that I bought it. If the something is not a physical object that must be delivered to me but information--music, data, software--I can collect my purchase online without ever revealing my identity or location.

Thus the technologies of encryption and computer networking can permit us, to a considerable extent, to move through the world without leaving footprints behind. If I want to receive advertising based on my past purchases--as it happens I often do--I can choose to make those purchases under my real name and provide my real address. If I want to receive the advertising without making my acts publicly observable--perhaps I am purchasing pornography--I can do it via an online identity. The link that ties my realspace body to my cyberspace persona is under my control. I have privacy--control over other people's access to information about me.

If we go a little further into science fiction I could even have privacy from my doctor. He knows the information that an examination--via remote controlled devices--revealed about me. He does not need to know what my name is, my face looks like, or where I live. It is not likely that I would want to carry my privacy that far--but I could.

So far I have been considering ways in which we might preserve privacy against the threat posed by technology. But there is another side to the story. For those who think that we already have too much privacy, what I view as the solution may look more like the problem. There have already been attempts to restrict the use of encryption to protect privacy. There will be more.

Suppose I concede, at least for the purposes of argument, that it is possible to have too much privacy as well as too little. Further, and less plausibly, suppose I believed that the strong privacy provided by encryption is a serious problem. How might one use law to solve it?

One difficulty is that encryption regulation poses the problem summed up in the slogan--"when guns are outlawed, only outlaws have guns." The mathematics of public key encryption have been public for decades. The software to do it already exists in a variety of forms, some of them freely available. Given the nature of software, once you have a program you can make an unlimited number of copies. Keeping encryption software out of the hands of spies, terrorists, and competent criminals is not a practical option. They probably have it already, and if they don't they can easily get it. The only people affected by a law against encryption software are the law abiding.

What about banning or restricting the use of encryption--at least encryption that cannot be broken by law enforcement agents? To enforce such a ban law enforcement agencies could randomly monitor all communication systems, looking for illegally encrypted messages. One practical problem is the enormous volume of information flowing over computer networks. A second and even more intractable problem is that while it is easy enough to tell whether a message consists of text written in English, it is very much harder--in practice impossible--to identify other sorts of content well enough to be sure that they do not contain encrypted messages.

Consider a three million pixel digital photo. To conceal a million character long encrypted message--an average sized novel--I replace the least significant bit of each of the numbers describing the color of a pixel with one bit of the message. The photo is now a marginally worse picture than it was--but there is no way an FBI agent, or a computer working for an FBI agent, can know precisely what the photo ought to look like.

Short of banning communication over computer networks or at least restricting it to text messages, there is no way that law enforcement can keep sophisticated criminals, spies, or terrorists from using encryption. What can be done is to put limits on the encryption software used by the rest of us≠to insist, for example, that if AOL or Microsoft builds encryption into their programs it must contain a back door permitting properly authorized persons to read the message without the key.

This still leaves the problem of how to give law enforcement what it wants without imposing unacceptably high costs on the rest of us. Consider the description of adequate regulation given by Louis Freeh, at the time the head of the FBI--the ability to crack any encrypted message in half an hour. The equivalent in realspace would be legal rules that let properly authorized law enforcement agents open any lock in the country in half an hour. That includes not only the lock on your front door but the locks protecting bank vaults, trade secrets, lawyers' records, lists of contributors to unpopular causes, and much else.

Encryption provides the locks for cyberspace. If all legal encryption comes with a mandatory back door accessible in half an hour to any police officer with a court order, everything in cyberspace is vulnerable to a private criminal with the right contacts. Those locks have billions of dollars worth of stuff behind them≠money in banks, trade secrets in computers and in messages. If being a police officer gives you access to locks with billions of dollars behind them, in cash, diamonds, or information, some cops will become criminals and some criminals will become cops.

In one important way, the consequence for cyberspace is even worse than the equivalent in realspace. If a police officer opens a safe and pockets a stack of cash or a bag of diamonds, the owner can see that something is missing and demand it back. But when information is copied the original is still there. If the officer who has decrypted your communications or stored data assures you that he found nothing relevant to his investigation and so took nothing away, there is no way to prove he is lying.

For encryption regulation to be useful it must either prevent the routine use of encryption or make it easy for law enforcement agents to access encrypted data and messages. Not only would that seriously handicap routine transactions, it would make computer crime easier by restricting the technology best suited to defend against it. And what we get in exchange is protection not against the use of encryption by sophisticated criminals and terrorists≠there is no way of providing that≠but only against its use by ordinary people and unsophisticated criminals. It does not look like a very attractive deal.

Privacy, Freedom and Government

Some years ago Professor Etzioni, who has contributed a chapter to this volume, published a book arguing for some restrictions on privacy as ways of promoting the common good. In reading it, I was struck by two differences between our views that explain much of the difference in our conclusions.

The first was that I did, and he did not, define privacy in the context of freedom of association. Consider the question of airlines requiring their pilots to be tested for drugs and alcohol. Professor Etzioni regards that as a (desirable) restriction on the pilots' privacy. I agree that it is desirable but not that it restricts privacy.

In a society where privacy is protected you have a right not to be tested. You do not have a right to be hired to fly airplanes--and, if you choose to exercise your right not to be tested, you should not be surprised if the airline exercises its right not to hire you. The background legal principle is not that I have a right to be hired as a pilot or that United Airlines has a right to have me fly their planes. The background principle is that they can hire me to fly their planes if and only if both they and I agree. Given that principle of free association many--although not all--of the problems that Professor Etzioni sees with privacy vanish.

The second difference has to do with our different views of government. While Professor Etzioni makes occasional references to the risk of some future oppressive government misusing information, he does not take seriously similar concerns with regard to our current government. His implicit assumption is that government is to be viewed as a benevolent agent standing above the human struggle, not as a mechanism through which individuals seek to achieve their goals, often at the expense of other individuals. That is not a view that strikes me as realistic.

Conclusion

Privacy, like almost anything else, can be used for good or bad purposes. My thesis in this chapter is that, on net, more privacy makes the world a better place. It does so because it is an essentially defensive weapon, a way of reducing the ability of other people to control us.

Reducing the ability of other people to control us is not always a good thing--someone may, after all, want to control me for my own good or control you to keep you from hurting me. But we live in a world where too much control is more of a problem than too little. In the entire world over the past century, something on the order of ten million people have been killed by private murderers. Between a hundred and two hundred million have been killed by the governments that ruled them (Rummel (1999) estimates about 170 million from 1900 to 1987). Quite a lot of individual pain, suffering, injustice has been due to the acts of private individuals; some could have been prevented by better law enforcement. But mass pain, suffering and injustice has been very nearly a monopoly of governments. If governments were better able to control us, there would have been more of it. And at the individual level, while privacy can be used to protect criminals against police, it can also be used to protect victims against criminals.

It is tempting to try for the best of both worlds--to restrict the privacy of bad people while protecting that of good, permit governments to collect detailed information about us but only allow it to be used for good purposes. But somebody must decide who are the good people and the bad, what purposes are worthy or unworthy. Whoever that somebody is will have his own agenda, his own purposes. Angels are in short supply.

To put the matter differently, "cannot" is better protection than "may not." If we permit law enforcement agents to know everything about everybody but forbid them fro using that information against individuals with unpopular views or political opponents of the party in power, we are protected only by a "may not." The same is true if private parties are able to collect information but restricted in what they may do with it. If the law keeps the information from being collected in the first place, we are protected by a cannot--however corrupt or dishonest they are, or however convinced that they are working for a greater good, people cannot use information they do not have.

"Cannot" at one level may depend on "may not" at another. You cannot use information that you do not have. You do not have it because you may not collect it. But even if the law forbids wiretaps or unauthorized surveillance, a sufficiently determined agency--or a sufficiently competent private criminal--can violate the law. That is where technologies that support privacy come into the picture. In a world where encryption is routine it does you no good to tap my phone because you cannot understand what I am saying. It does no good to intercept my email because you cannot read it. "Cannot" is better than "may not."

We can and should fight a delaying action against the use of technology to restrict privacy. But in the long run technology--useful technology--is hard to stop. In the long run, the real battle will be the one fought in defense of technologies that protect privacy. That one we might win.

Bibliography

References

Brin, David (1999), The Transparent Society: Will Technology Force Us to Choose Between Privacy and Freedom? Perseus (The first chapter is webbed at http://www.kithrup.com/brin/tschp1.html)

Etzioni, Amitai (1999), The Limits of Privacy, Basic Books.

Friedman, David (1996) ≥A World of Strong Privacy: Promises and Perils of Encryption,≤ Social Philosophy and Policy, pp. 212-228. Webbed at http://www.daviddfriedman.com/Academic/Strong_Privacy/Strong_Privacy.html

Rummel, Rudolph J. (1999), Statistics of democide: Genocide and mass murder since 1900, Lit Verlag.

Further Reading

Anderson, Poul, Shield

Ellen Frankel Paul, Fred D. Miller, Jr., & Jeffrey Paul (Eds). (2000) The Right to Privacy, Cambridge University Press.

http://www.mega.nu:8080/ampp/rummel/20th.htm (detailed statistics on 20th century democide)

http://www.daviddfriedman.com/future_imperfect_draft/future_imperfect.html (Much more detailed account of encryption, surveillance, and much else.)

Eric Hughes, Mar 1993

Privacy is necessary for an open society in the electronic age. Privacy is not secrecy. A private matter is something one doesn't want the whole world to know, but a secret matter is something one doesn't want anybody to know. Privacy is the power to selectively reveal oneself to the world.

If two parties have some sort of dealings, then each has a memory of their interaction. Each party can speak about their own memory of this; how could anyone prevent it? One could pass laws against it, but the freedom of speech, even more than privacy, is fundamental to an open society; we seek not to restrict any speech at all. If many parties speak together in the same forum, each can speak to all the others and aggregate together knowledge about individuals and other parties. The power of electronic communications has enabled such group speech, and it will not go away merely because we might want it to.

Since we desire privacy, we must ensure that each party to a transaction have knowledge only of that which is directly necessary for that transaction. Since any information can be spoken of, we must ensure that we reveal as little as possible. In most cases personal identity is not salient. When I purchase a magazine at a store and hand cash to the clerk, there is no need to know who I am. When I ask my electronic mail provider to send and receive messages, my provider need not know to whom I am speaking or what I am saying or what others are saying to me; my provider only need know how to get the message there and how much I owe them in fees. When my identity is revealed by the underlying mechanism of the transaction, I have no privacy. I cannot here selectively reveal myself; I must always reveal myself.

Therefore, privacy in an open society requires anonymous transaction systems. Until now, cash has been the primary such system. An anonymous transaction system is not a secret transaction system. An anonymous system empowers individuals to reveal their identity when desired and only when desired; this is the essence of privacy.

Privacy in an open society also requires cryptography. If I say something, I want it heard only by those for whom I intend it. If the content of my speech is available to the world, I have no privacy. To encrypt is to indicate the desire for privacy, and to encrypt with weak cryptography is to indicate not too much desire for privacy. Furthermore, to reveal one's identity with assurance when the default is anonymity requires the cryptographic signature.

We cannot expect governments, corporations, or other large, faceless organizations to grant us privacy out of their beneficence. It is to their advantage to speak of us, and we should expect that they will speak. To try to prevent their speech is to fight against the realities of information. Information does not just want to be free, it longs to be free. Information expands to fill the available storage space. Information is Rumor's younger, stronger cousin; Information is fleeter of foot, has more eyes, knows more, and understands less than Rumor.

We must defend our own privacy if we expect to have any. We must come together and create systems which allow anonymous transactions to take place. People have been defending their own privacy for centuries with whispers, darkness, envelopes, closed doors, secret handshakes, and couriers. The technologies of the past did not allow for strong privacy, but electronic technologies do.

We the Cypherpunks are dedicated to building anonymous systems. We are defending our privacy with cryptography, with anonymous mail forwarding systems, with digital signatures, and with electronic money.

Cypherpunks write code. We know that someone has to write software to defend privacy, and since we can't get privacy unless we all do, we're going to write it. We publish our code so that our fellow Cypherpunks may practice and play with it. Our code is free for all to use, worldwide. We don't much care if you don't approve of the software we write. We know that software can't be destroyed and that a widely dispersed system can't be shut down.

Cypherpunks deplore regulations on cryptography, for encryption is fundamentally a private act. The act of encryption, in fact, removes information from the public realm. Even laws against cryptography reach only so far as a nation's border and the arm of its violence. Cryptography will ineluctably spread over the whole globe, and with it the anonymous transactions systems that it makes possible.

For privacy to be widespread it must be part of a social contract. People must come and together deploy these systems for the common good. Privacy only extends so far as the cooperation of one's fellows in society. We the Cypherpunks seek your questions and your concerns and hope we may engage you so that we do not deceive ourselves. We will not, however, be moved out of our course because some may disagree with our goals.

The Cypherpunks are actively engaged in making the networks safer for privacy. Let us proceed together apace.

Onward.

9 March 1993

Juan Ibanez, Mar 1993

It is often claimed that Bitcoin is revolutionary because it is the first triple-entry accounting system in history. But is this really so?

To answer this, we must first ask what triple-entry accounting is. A quick Google search will provide an answer like the following: triple-entry accounting is using blockchain to build a common ledger between two parties that would typically maintain two separate, redundant, double-entry ledgers.

This is not exactly right. Triple-entry accounting is a design for a common ledger, yes, but not all common ledgers are triple-entry. Triple-entry accounting is a particular model to build a common ledger through signed messages.

Blockchain technology, introduced by Bitcoin, presents a way to replace Ivan with a decentralized community of nodes, making the entire system more “trustless” (one does not need to personally trust a third party but only the architecture of the system itself). But is Bitcoin itself triple-entry?

February’s model shows the receipt as signed by Alice only. While February argues that this is called triple-entry accounting because there is a signed receipt held by three parties, this could be seen as deviating from triple-entry accounting as we described before. Signed receipts are held by three parties, yes, but shouldn’t they also be signed thrice?

A quest for hidden signatures

The second signature

Bookkeeping is keeping records of transactions, which are a bilateral event. In a sale, for example, somebody pays and somebody else delivers something in return. Bitcoin, however, is just a payment system. This means it only records (typically) one-half of the transaction, making the second signature almost unnecessary.

Todd Boyle’s triple-entry accounting looked at wide patterns of transactions as bilateral events in trade cycles (offer, acceptance, delivery, invoice, payment, etcetera). This made Bob’s acceptance necessary, e.g. for draft invoices, as there were two parties which were both giving something up. A signature was thus needed to show that both parties were consenting to give this up.

However, for a payment system in which payment is not linked to delivery, it makes little sense to attest Bob’s consent. It could be seen as a waste of time. This was the case of Ian Grigg’s original triple-entry accounting model: the Ricardo Payment System.

One can therefore argue that Bitcoin is triple-entry accounting even in the absence of a second signature, because this what triple-entry accounting looks like for a payment-only system.

Nevertheless, a second signature is found in payment systems: in the event of the receiver deciding to spend the value by signing another payment over it. In the act of signing a spend, the receiver confirms the receipt of previous funds, and in effect creates an asynchronous signature. This is broadly true of payment systems, whether they be Bitcoin, Ricardo or high street banks.

In summary, in the act of later spending the amount received, the receiver of a payment sends a signed message confirming acceptance of that payment. This also applies to Bitcoin’s UTXO (Unspent Transaction Output) model.

In UTXO, the transaction input (amount sent by the payer) or inputs must equal the transaction output (amount the payee receives). If it exceeds it, it is spent anyway and returned to the payer as a new output: change. As explained in the aforementioned paper:

“The UTXO model necessarily requires the new payer (and former payee) to sign off on a message that confirms receipt of a prior payment output in its totality (or of a mining reward which resulted in the creation of new bitcoins through a coinbase transaction).”

The third signature

So, second signatures can be found even in the absence of schemes to make Bitcoin transactions bilateral. What about the third ones?

Admittedly, we are using a broad concept of signature: a signature is any token that attests agreement at some point. Nonetheless, this goes in line with Ian Grigg’s design, not against it.

Conclusion

A triple-entry accounting system requires a signed receipt to be held by three parties in three places. If the transaction pattern requires this, all of the parties must have signed the receipt. Bitcoin does this and, in this sense, it is triple-entry.

Literature

Preeth Kasireddy, Nov 2018

Distributed systems can be difficult to understand, mainly because the knowledge surrounding them is distributed. But don’t worry, I’m well aware of the irony. While teaching myself distributed computing, I fell flat on my face many times. Now, after many trials and tribulations, I’m finally ready to explain the basics of distributed systems to you.

Blockchains have forced engineers and scientists to re-examine and question firmly entrenched paradigms in distributed computing.

I also want to discuss the profound effect that blockchain technology has had on the field. Blockchains have forced engineers and scientists to re-examine and question firmly entrenched paradigms in distributed computing. Perhaps no other technology has catalyzed progress faster in this area of study than blockchain.

Distributed systems are by no means new. Scientists and engineers have spent decades researching the subject. But what does blockchain have to do with them? Well, all the contributions that blockchain has made wouldn’t have been possible if distributed systems hadn’t existed first.

Unfortunately, much of the literature on distributed computing is either difficult to comprehend or dispersed across way too many academic papers. To make matters more complex, there are hundreds of architectures, all of which serve different needs. Boiling this down into a simple-to-understand framework is quite difficult.

Because the field is vast, I had to carefully choose what I could cover. I also had to make generalizations to mask some of the complexity. Please note, my goal is not to make you an expert in the field. Instead, I want to give you enough knowledge to jump-start your journey into distributed systems and consensus.

After reading this post, you’ll walk away with a stronger grasp of:

• What a distributed system is,

• The properties of a distributed system,

• What it means to have consensus in a distributed system,

• An understanding of foundational consensus algorithms (e.g. DLS and PBFT), and

• Why Nakamoto Consensus is a big deal.

I hope you’re ready to learn, because class is now in session.

What Is a Distributed System?

A distributed system involves a set of distinct processes (e.g., computers) passing messages to one another and coordinating to accomplish a common objective (i.e., solving a computational problem).

A distributed system is a group of computers working together to achieve a unified goal.

Simply put, a distributed system is a group of computers working together to achieve a unified goal. And although the processes are separate, the system appears as a single computer to end-user(s).

As I mentioned, there are hundreds of architectures for a distributed system. For example, a single computer can also be viewed as a distributed system: the central control unit, memory units, and input-output channels are separate processes collaborating to complete an objective.

In the case of an airplane, these discrete units work together to get you from Point A to Point B:

In this post, we’ll focus on distributed systems in which processes are spatially-separated computers.

Note: I may use the terms “node,” “peer,” “computer,” or “component” interchangeably with “process.” They all mean the same thing for the purposes of this post. Similarly, I may use the term “network” interchangeably with “system.”

Properties of a Distributed System

Every distributed system has a specific set of characteristics. These include:

A) Concurrency

The processes in the system operate concurrently, meaning multiple events occur simultaneously. In other words, each computer in the network executes events independently at the same time as other computers in the network.

This requires coordination.

Lamport, L (1978). Time, Clocks and Ordering of Events in a Distributed System

B) Lack of a global clock

For a distributed system to work, we need a way to determine the order of events. However, in a set of computers operating concurrently, it is sometimes impossible to say that one of two events occurred first, as computers are spatially separated. In other words, there is no single global clock that determines the sequence of events happening across all computers in the network.

1. Messages are sent before they are received.

2. Each computer has a sequence of events.

However, if we base the order entirely upon events heard by each individual computer, we can run into situations where this order differs from what a user external to the system perceives. Thus, the paper shows that the algorithm can still allow for anomalous behavior.

Finally, Lamport discusses how such anomalies can be prevented by using properly synchronized physical clocks.

Essentially, Lamport’s paper demonstrates that time and order of events are fundamental obstacles in a system of distributed computers that are spatially separated.

C) Independent failure of components

A critical aspect of understanding distributed systems is acknowledging that components in a distributed system are faulty. This is why it’s called “fault-tolerant distributed computing.”

It’s impossible to have a system free of faults. Real systems are subject to a number of possible flaws or defects, whether that’s a process crashing; messages being lost, distorted, or duplicated; a network partition delaying or dropping messages; or even a process going completely haywire and sending messages according to some malevolent plan.

It’s impossible to have a system free of faults.

These failures can be broadly classified into three categories:

• Crash-fail: The component stops working without warning (e.g., the computer crashes).

• Omission: The component sends a message but it is not received by the other nodes (e.g., the message was dropped).

• Byzantine: The component behaves arbitrarily. This type of fault is irrelevant in controlled environments (e.g., Google or Amazon data centers) where there is presumably no malicious behavior. Instead, these faults occur in what’s known as an “adversarial context.” Basically, when a decentralized set of independent actors serve as nodes in the network, these actors may choose to act in a “Byzantine” manner. This means they maliciously choose to alter, block, or not send messages at all.

With this in mind, the aim is to design protocols that allow a system with faulty components to still achieve the common goal and provide a useful service.

Given that every system has faults, a core consideration we must make when building a distributed system is whether it can survive even when its parts deviate from normal behavior, whether that’s due to non-malicious behaviors (i.e., crash-fail or omission faults) or malicious behavior (i.e., Byzantine faults).

Broadly speaking, there are two types of models to consider when making a distributed system:

1) Simple fault-tolerance

In a simple fault-tolerant system, we assume that all parts of the system do one of two things: they either follow the protocol exactly or they fail. This type of system should definitely be able to handle nodes going offline or failing. But it doesn’t have to worry about nodes exhibiting arbitrary or malicious behavior.

2A) Byzantine fault-tolerance

A simple fault-tolerant system is not very useful in an uncontrolled environment. In a decentralized system that has nodes controlled by independent actors communicating on the open, permissionless internet, we also need to design for nodes that choose to be malicious or “Byzantine.” Therefore, in a Byzantine fault-tolerant system, we assume nodes can fail or be malicious.

2B) BAR fault-tolerance

More formally, this is defined as the BAR model — one that specifies for both Byzantine and rational failures. The BAR model assumes three types of actors:

• Byzantine: Byzantine nodes are malicious and trying to screw you.

• Altruistic: Honest nodes always follow the protocol.

• Rational: Rational nodes only follow the protocol if it suits them.

D) Message passing

As I noted earlier, computers in a distributed system communicate and coordinate by “message passing” between one or more other computers. Messages can be passed using any messaging protocol, whether that’s HTTP, RPC, or a custom protocol built for the specific implementation. There are two types of message-passing environments:

1) Synchronous

In a synchronous system, it is assumed that messages will be delivered within some fixed, known amount of time.

Synchronous message passing is conceptually less complex because users have a guarantee: when they send a message, the receiving component will get it within a certain time frame. This allows users to model their protocol with a fixed upper bound of how long the message will take to reach its destination.

However, this type of environment is not very practical in a real-world distributed system where computers can crash or go offline and messages can be dropped, duplicated, delayed, or received out of order.

2) Asynchronous

In an asynchronous message-passing system, it is assumed that a network may delay messages infinitely, duplicate them, or deliver them out of order. In other words, there is no fixed upper bound on how long a message will take to be received.

What It Means to Have Consensus in a Distributed System

So far, we’ve learned about the following properties of a distributed system:

• Concurrency of processes

• Lack of a global clock

• Faulty processes

• Message passing

Next, we’ll focus on understanding what it means to achieve “consensus” in a distributed system. But first, it’s important to reiterate what we alluded to earlier: there are hundreds of hardware and software architectures used for distributed computing.

The most common form is called a replicated state machine.

Replicated State Machine

A replicated state machine is a deterministic state machine that is replicated across many computers but functions as a single state machine. Any of these computers may be faulty, but the state machine will still function.

In a replicated state machine, if a transaction is valid, a set of inputs will cause the state of the system to transition to the next state. A transaction is an atomic operation on a database. This means the operations either complete in full or never complete at all. The set of transactions maintained in a replicated state machine is known as a “transaction log.”

The logic for transitioning from one valid state to the next is called the “state transition logic.”

In other words, a replicated state machine is a set of distributed computers that all start with the same initial value. For each state transition, each of the processes decides on the next value. Reaching “consensus” means that all the computers must collectively agree on the output of this value.

In turn, this maintains a consistent transaction log across every computer in the system (i.e., they “achieve a common goal”). The replicated state machine must continually accept new transactions into this log (i.e., “provide a useful service”). It must do so despite the fact that:

1. Some of the computers are faulty.

2. The network is not reliable and messages may fail to deliver, be delayed, or be out of order.

3. There is no global clock to help determine the order of events.

And this, my friends, is the fundamental goal of any consensus algorithm.

The Consensus Problem, Defined

An algorithm achieves consensus if it satisfies the following conditions:

• Agreement: All non-faulty nodes decide on the same output value.

• Termination: All non-faulty nodes eventually decide on some output value.

Note: Different algorithms have different variations of the conditions above. For example, some divide the Agreement property into Consistency and Totality. Some have a concept of Validity or Integrity or Efficiency. However, such nuances are beyond the scope of this post.

Broadly speaking, consensus algorithms typically assume three types of actors in a system:

1. Proposers, often called leaders or coordinators.

2. Acceptors, processes that listen to requests from proposers and respond with values.

3. Learners, other processes in the system which learn the final values that are decided upon.

Generally, we can define a consensus algorithm by three steps:

Step 1: Elect

• Processes elect a single process (i.e., a leader) to make decisions.

• The leader proposes the next valid output value.

Step 2: Vote

• The non-faulty processes listen to the value being proposed by the leader, validate it, and propose it as the next valid value.

Step 3: Decide

• The non-faulty processes must come to a consensus on a single correct output value. If it receives a threshold number of identical votes which satisfy some criteria, then the processes will decide on that value.

• Otherwise, the steps start over.

It’s important to note that every consensus algorithm has different:

• Terminology (e.g., rounds, phases),

• Procedures for how votes are handled, and

• Criteria for how a final value is decided (e.g., validity conditions).

Nonetheless, if we can use this generic process to build an algorithm that guarantees the general conditions defined above, then we have a distributed system which is able to achieve consensus.

Simple enough, right?

FLP impossibility

… Not really. But you probably saw that coming!

Recall how we described the difference between a synchronous system and asynchronous system:

• In synchronous environments, messages are delivered within a fixed time frame

• In asynchronous environments, there’s no guarantee of a message being delivered.

This distinction is important.

Reaching consensus in a synchronous environment is possible because we can make assumptions about the maximum time it takes for messages to get delivered. Thus, in this type of system, we can allow the different nodes in the system to take turns proposing new transactions, poll for a majority vote, and skip any node if it doesn’t offer a proposal within the maximum time limit.

But, as noted earlier, assuming we are operating in synchronous environments is not practical outside of controlled environments where message latency is predictable, such as data centers which have synchronized atomic clocks.

In reality, most environments don’t allow us to make the synchronous assumption. So we must design for asynchronous environments.

If we cannot assume a maximum message delivery time in an asynchronous environment, then achieving termination is much harder, if not impossible. Remember, one of the conditions that must be met to achieve consensus is “termination,” which means every non-faulty node must decide on some output value.

This is formally known as the “FLP impossibility result.” How did it get this name? Well, I’m glad you asked!

Even a single faulty process makes it impossible to reach consensus among deterministic asynchronous processes.

This result was a huge bummer for the distributed computing space. Nonetheless, scientists continued to push forward to find ways to circumvent FLP impossibility.

At a high level, there are two ways to circumvent FLP impossibility:

1. Use synchrony assumptions.

2. Use non-determinism.

Let’s take a deep dive into each one right now.

Approach 1: Use Synchrony Assumptions

I know what you’re thinking: What the heck does this even mean?

Let’s revisit our impossibility result. Here’s another way to think about it: the FLP impossibility result essentially shows that, if we cannot make progress in a system, then we cannot reach consensus. In other words, if messages are asynchronously delivered, termination cannot be guaranteed. Recall that termination is a required condition that means every non-faulty node must eventually decide on some output value.

But how can we guarantee every non-faulty process will decide on a value if we don’t know when a message will be delivered due to asynchronous networks?

To be clear, the finding does not state that consensus is unreachable. Rather, due to asynchrony, consensus cannot be reached in a fixed time. Saying that consensus is “impossible” simply means that consensus is “not always possible.” It’s a subtle but crucial detail.

One way to circumvent this is to use timeouts. If no progress is being made on deciding the next value, we wait until a timeout, then start the steps all over again. As we’re about to see, this is what consensus algorithms like Paxos and Raft essentially did.

Paxos

Paxos works like this:

Phase 1: Prepare request

1. The proposer chooses a new proposal version number (n) and sends a “prepare request” to the acceptors.

2. If acceptors receive a prepare request (“prepare,” n) with n greater than that of any prepare request they had already responded to, the acceptors send out (“ack,” n, n’, v’) or (“ack,” n, ^ , ^).

3. Acceptors respond with a promise not to accept any more proposals numbered less than n.

4. Acceptors suggest the value (v) of the highest-number proposal that they have accepted, if any. Or else, they respond with ^.

Phase 2: Accept request

1. If the proposer receives responses from a majority of the acceptors, then it can issue an accept request (“accept,” n, v) with number n and value v.

2. n is the number that appeared in the prepare request.

3. v is the value of the highest-numbered proposal among the responses.

4. If the acceptor receives an accept request (“accept,” n, v), it accepts the proposal unless it has already responded to a prepare request with a number greater than n.

Phase 3: Learning phase

1. Whenever an acceptor accepts a proposal, it responds to all learners (“accept,” n, v).

2. Learners receive (“accept,” n, v) from a majority of acceptors, decide v, and send (“decide,” v) to all other learners.

3. Learners receive (“decide,” v) and the decided v.

Phew! Confused yet? I know that was a quite a lot of information to digest.

But wait… There’s more!

As we now know, every distributed system has faults. In this algorithm, if a proposer failed (e.g., because there was an omission fault), then decisions could be delayed. Paxos dealt with this by starting with a new version number in Phase 1, even if previous attempts never ended.

I won’t go into details, but the process to get back to normal operations in such cases was quite complex since processes were expected to step in and drive the resolution process forward.

The main reason Paxos is so hard to understand is that many of its implementation details are left open to the reader’s interpretation: How do we know when a proposer is failing? Do we use synchronous clocks to set a timeout period for deciding when a proposer is failing and we need to move on to the next rank? 🤷‍

This design choice ended up becoming one of the biggest downsides of Paxos. It’s not only incredibly difficult to understand but difficult to implement as well. In turn, this made the field of distributed systems incredibly hard to navigate.

By now, you’re probably wondering where the synchrony assumption comes in.

In Paxos, although timeouts are not explicit in the algorithm, when it comes to the actual implementation, electing a new proposer after some timeout period is necessary to achieve termination. Otherwise, we couldn’t guarantee that acceptors would output the next value, and the system could come to a halt.

Raft

One important new thing we learned from Raft is the concept of using a shared timeout to deal with termination. In Raft, if you crash and restart, you wait at least one timeout period before trying to get yourself declared a leader, and you are guaranteed to make progress.

But Wait… What about ‘Byzantine’ Environments?

While traditional consensus algorithms (such as Paxos and Raft) are able to thrive in asynchronous environments using some level of synchrony assumptions (i.e. timeouts), they are not Byzantine fault-tolerant. They are only crash fault-tolerant.

Crash-faults are easier to handle because we can model the process as either working or crashed — 0 or 1. The processes can’t act maliciously and lie. Therefore, in a crash fault-tolerant system, a distributed system can be built where a simple majority is enough to reach a consensus.

In an open and decentralized system (such as public blockchains), users have no control over the nodes in the network. Instead, each node makes decisions toward its individual goals, which may conflict with those of other nodes.

In a Byzantine system where nodes have different incentives and can lie, coordinate, or act arbitrarily, you cannot assume a simple majority is enough to reach consensus. Half or more of the supposedly honest nodes can coordinate with each other to lie.

For example, if an elected leader is Byzantine and maintains strong network connections to other nodes, it can compromise the system. Recall how we said we must model our system to either tolerate simple faults or Byzantine faults. Raft and Paxos are simple fault-tolerant but not Byzantine fault-tolerant. They are not designed to tolerate malicious behavior.

The ‘Byzantine General’s Problem’

Here’s why:

If x nodes are faulty, then the system needs to operate correctly after coordinating with n minus x nodes (since x nodes might be faulty/Byzantine and not responding). However, we must prepare for the possibility that the x that doesn’t respond may not be faulty; it could be the x that does respond. If we want the number of non-faulty nodes to outnumber the number of faulty nodes, we need at least n minus x minus x > x. Hence, n > 3x + 1 is optimal.

However, the algorithms demonstrated in this paper are only designed to work in a synchronous environment. Bummer! It seems we can only get one or the other (Byzantine or Asynchronous) right. An environment that is both Byzantine and Asynchronous seems much harder to design for.

Why?

In short, building a consensus algorithm that can withstand both an asynchronous environment and a Byzantine one is… well, that would sort of be like making a miracle happen.

Algorithms like Paxos and Raft were well-known and widely used. But there was also a lot of academic work that focused more on solving the consensus problem in a Byzantine + asynchronous setting.

So buckle your seatbelts…

We’re going on a field trip…

To the land of…

Theoretical academic papers!

Okay, okay — I’m sorry for building that up. But you should be excited! Remember that whole “making a miracle” thing we discussed earlier? We’re going to take a look at two algorithms (DLS and PBFT) that brought us closer than ever before to breaking the Byzantine + asynchronous barrier.

The DLS Algorithm

As you may recall, in a synchronous system, there is a known fixed upper bound on the time required for a message to be sent from one processor to another. In an asynchronous system, no fixed upper bounds exist. Partial synchrony lies somewhere between those two extremes.

The paper explained two versions of the partial synchrony assumption:

• Assume that fixed bounds exist for how long messages take to get delivered. But they are not known a priori. The goal is to reach consensus regardless of the actual bounds.

• Assume the upper bounds for message delivery are known, but they’re only guaranteed to hold starting at some unknown time (also called “Global Standardization Time,” GST). The goal is to design a system that can reach consensus regardless of when this time occurs.

Here’s how the DLS algorithm works:

A series of rounds are divided into “trying” and “lock-release” phases.

1. Each round has a proposer and begins with each of the processes communicating the value they believe is correct.

2. The proposer “proposes” a value if at least N − x processes have communicated that value.

3. When a process receives the proposed value from the proposer, it must lock on the value and then broadcast that information.

4. If the proposer receives messages from x + 1 processes that they locked on some value, it commits that as the final value.

DLS was a major breakthrough because it created a new category of network assumptions to be made — namely, partial synchrony — and proved consensus was possible with this assumption. The other imperative takeaway from the DLS paper was separating the concerns for reaching consensus in a Byzantine and asynchronous setting into two buckets: safety and liveness.

Safety

This is another term for the “agreement” property we discussed earlier, where all non-faulty processes agree on the same output. If we can guarantee safety, we can guarantee that the system as a whole will stay in sync. We want all nodes to agree on the total order of the transaction log, despite failures and malicious actors. A violation of safety means that we end up with two or more valid transaction logs.

Liveness

This is another term for the “termination” property we discussed earlier, where every non-faulty node eventually decides on some output value. In a blockchain setting, “liveness” means the blockchain keeps growing by adding valid new blocks. Liveness is important because it’s the only way that the network can continue to be useful — otherwise, it will stall.

As we know from the FLP impossibility, consensus can’t be achieved in a completely asynchronous system. The DLS paper argued that making a partial synchrony assumption for achieving the liveness condition is enough to overcome FLP impossibility.

Thus, the paper proved that the algorithms don’t need to use any synchrony assumption to achieve the safety condition.

Pretty straightforward, right? Don’t worry if it’s not. Let’s dig a little deeper.

Remember that if nodes aren’t deciding on some output value, the system just halts. So, if we make some synchrony assumptions (i.e., timeouts) to guarantee termination and one of those fails, it makes sense that this would also bring the system to a stop.

But if we design an algorithm where we assume timeouts (to guarantee correctness), this carries the risk of leading to two valid transaction logs if the synchrony assumption fails.

Designing a distributed system is always about trade-offs.

This would be far more dangerous than the former option. There’s no point in having a useful service (i.e., liveness) if the service is corrupt (i.e., no safety). Basically, having two different blockchains is worse than having the entire blockchain come to a halt.

A distributed system is always about trade-offs. If you want to overcome a limitation (e.g., FLP impossibility), you must make a sacrifice somewhere else. In this case, separating the concerns into safety and liveness is brilliant. It lets us build a system that is safe in an asynchronous setting but still needs some form of timeouts to keep producing new values.

Despite everything that the DLS paper offered, DLS was never widely implemented or used in a real-world Byzantine setting. This is probably due to the fact that one of the core assumptions in the DLS algorithm was to use synchronous processor clocks in order to have a common notion of time. In reality, synchronous clocks are vulnerable to a number of attacks and wouldn’t fare well in a Byzantine fault-tolerant setting.

The PBFT Algorithm

“Practical” in this sense meant it worked in asynchronous environments like the internet and had some optimizations that made it faster than previous consensus algorithms. The paper argued that previous algorithms, while shown to be “theoretically possible,” were either too slow to be used or assumed synchrony for safety.

And as we’ve explained, that can be quite dangerous in an asynchronous setting.

In a nutshell, the PBFT algorithm showed that it could provide safety and liveness assuming (n-1)/3 nodes were faulty. As we previously discussed, that’s the minimum number of nodes we need to tolerate Byzantine faults. Therefore, the resiliency of the algorithm was optimal.

The algorithm provided safety regardless of how many nodes were faulty. In other words, it didn’t assume synchrony for safety. The algorithm did, however, rely on synchrony for liveness. At most, (n-1)/3 nodes could be faulty and the message delay did not grow faster than a certain time limit. Hence, PBFT circumvented FLP impossibility by using a synchrony assumption to guarantee liveness.

The algorithm moved through a succession of “views,” where each view had one “primary” node (i.e., a leader) and the rest were “backups.” Here’s a step-by-step walkthrough of how it worked:

1. A new transaction happened on a client and was broadcast to the primary.

2. The primary multicasted it to all the backups.

3. The backups executed the transaction and sent a reply to the client.

4. The client wanted x + 1 replies from backups with the same result. This was the final result, and the state transition happened.

If the leader was non-faulty, the protocol worked just fine. However, the process for detecting a bad primary and reelecting a new primary (known as a “view change”) was grossly inefficient. For instance, in order to reach consensus, PBFT required a quadratic number of message exchanges, meaning every computer had to communicate with every other computer in the network.

Note: Explaining the PBFT algorithm in full is a blog post all on its own! We’ll save that for another day ;).

While PBFT was an improvement over previous algorithms, it wasn’t practical enough to scale for real-world use cases (such as public blockchains) where there are large numbers of participants. But hey, at least it was much more specific when it came to things like failure detection and leader election (unlike Paxos).

It’s important to acknowledge PBFT for its contributions. It incorporated important revolutionary ideas that newer consensus protocols (especially in a post-blockchain world) would learn from and use.

For instance, Tendermint rotates a new leader every round. If the current round’s leader doesn’t respond within a set period of time, the leader is skipped and the algorithm simply moves to the next round with a new leader. This actually makes a lot more sense than using point-to-point connections every time there needs to be a view-change and a new leader elected.

Approach 2: Non-Determinism

As we’ve learned, most Byzantine fault-tolerant consensus protocols end up using some form of synchrony assumption to overcome FLP impossibility. However, there is another way to overcome FLP impossibility: non-determinism.

Enter: Nakamoto Consensus

As we just learned, in traditional consensus, f(x) is defined such that a proposer and a bunch of acceptors must all coordinate and communicate to decide on the next value.

Traditional consensus doesn’t scale well.

This is too complex because it requires knowing every node in the network and every node communicating with every other node (i.e., quadratic communication overhead). Simply put, it doesn’t scale well and doesn’t work in open, permissionless systems where anyone can join and leave the network at any time.

The brilliance of Nakamoto Consensus is making the above probabilistic. Instead of every node agreeing on a value, f(x) works such that all of the nodes agree on the probability of the value being correct.

Wait, what does that even mean?

Byzantine-fault tolerant

Rather than electing a leader and then coordinating with all nodes, consensus is decided based on which node can solve the computation puzzle the fastest. Each new block in the Bitcoin blockchain is added by the node that solves this puzzle the fastest. The network continues to build on this timestamped chain, and the canonical chain is the one with the most cumulative computation effort expended (i.e., cumulative difficulty).

The longest chain not only serves as proof of the sequence of blocks, but proof that it came from the largest pool of CPU power. Therefore, as long as a majority of CPU power is controlled by honest nodes, they’ll continue to generate the longest chain and outpace attackers.

Block rewards

Nakamoto Consensus works by assuming that nodes will expend computational efforts for the chance of deciding the next block. The brilliance of the algorithm is economically incentivizing nodes to repeatedly perform such computationally expensive puzzles for the chance of randomly winning a large reward (i.e., a block reward).

Sybil resistance

The proof of work required to solve this puzzle makes the protocol inherently Sybil-resistant. No need for PKI or any other fancy authentication schemes.

Peer-to-peer gossip

Not “technically” safe in asynchronous environments

In Nakamoto consensus, the safety guarantee is probabilistic — we’re growing the longest chain, and each new block lowers the probability of a malicious node trying to build another valid chain.

Therefore, Nakamoto Consensus does not technically guarantee safety in an asynchronous setting. Let’s take a second to understand why.

For a system to achieve the safety condition in an asynchronous setting, we should be able to maintain a consistent transaction log despite asynchronous network conditions. Another way to think about it is that a node can go offline at any time and then later come back online, and use the initial state of the blockchain to determine the latest correct state, regardless of network conditions. Any of the honest nodes can query for arbitrary states in the past, and a malicious node cannot provide fraudulent information that the honest nodes will think is truthful.

Nakamoto Consensus does not technically guarantee safety in an asynchronous setting.

In previous algorithms discussed in this post, this is possible because we are deterministically finalizing a value at every step. As long as we terminate on each value, we can know the past state. However, the reason Bitcoin is not “technically” asynchronously safe is that it is possible for there to be a network partition that lets an attacker with a sufficiently large hash power on one side of the partition to create an alternative chain faster than the honest chain on the other side of the partition — and on this alternative chain, he can try to change one of his own transactions to take back money he spent.

Admittedly, this would require the attacker to gain a lot of hashing power and spend a lot of money.

In essence, the Bitcoin blockchain’s immutability stems from the fact that a majority of miners don’t actually want to adopt an alternative chain. Why? Because it’s too difficult to coordinate enough hash power to get the network to adopt an alternative chain. Put another way, the probability of successfully creating an alternative chain is so low, it’s practically negligible.

Nakamoto vs. Traditional Consensus

For practical purposes, Nakamoto Consensus is Byzantine fault-tolerant. But it clearly doesn’t achieve consensus the way consensus researchers would traditionally assume. Therefore, it was initially seen as being completely out of the Byzantine fault-tolerant world.

By design, the Nakamoto Consensus makes it possible for any number of nodes to have open participation, and no one has to know the full set of participants. The importance of this breakthrough can’t be overstated. Thank you, Nakamoto.

Simpler than previous consensus algorithms, it eliminates the complexity of point-to-point connections, leader election, quadratic communication overhead, etc. You just launch the Bitcoin protocol software on any computer and start mining.

This makes it easily deployable in a real-world setting. It is truly the more “practical” cousin of PBFT.

Conclusion

And there you have it — the brief basics of distributed systems and consensus. It has been a long, winding journey of research, roadblocks, and ingenuity for distributed computing to get this far. I hope this post helps you understand the field at least a tiny it better.

Nakamoto Consensus is truly an innovation that has allowed a whole new wave of researchers, scientists, developers, and engineers to continue breaking new ground in consensus protocol research.

Chris McCann, Jun 2021

Decentralized Finance (DeFi) is redefining the future of finance. There is a major shift going on in the underlying infrastructure powering financial applications, and it’s changing the way we think about permission and control, transparency and risks.

The goal of this report is to provide an introduction of the new emerging area of DeFi infrastructure powering DeFi apps today. While it’s easy to get caught up in the hype and speculation within the space, I’ll focus on the key components of DeFi applications, their key differentiation compared to traditional finance, potential risks, and longer term implications these DeFi apps are causing.

Major Structural Commonalities Across DeFi Apps

The major categories of DeFi apps include decentralized exchanges, lending platforms, stablecoins, synthetic assets, insurance, among others. While diverse in scope, all of these DeFi apps share a major set of commonalities including:

1. Using underlying blockchains as the core ledger

2. Open source and transparent by default

3. Interoperable and programmable (composability)

4. Open and accessible to all (permissionless)

Using Underlying Blockchains as the Core Ledger

Compared to traditional financial applications which use core banking systems (Fiserv, Jack Henry, FIS, etc.) as the underlying ledgers of record, DeFi apps use blockchains as their underlying core ledger.

A few of the most prominent blockchains used to build DeFi apps include Ethereum, Solana, and Binance Chain, etc. These underlying blockchains store the ledger state of what is deposited into the DeFi apps, what is stored within the smart contracts, all of the transactions, and withdrawals.

All of the core accounting functions to ensure matching inputs and outputs are handled by the blockchain itself, the DeFi apps don’t need to create external systems to reconcile balances, because all of the transactions are queryable across the various block explorers.

In addition, compared to the traditional system there is no separate process of settling & clearing transactions. The transaction processing, clearing, and settling all happen at the same time when the transaction is broadcasted. Although it is advisable to wait around ~21 blocks or more to ensure finality on the blockchain itself.

Open Source and Transparent by Default

Compared to traditional financial applications which are all closed-source and built on top of proprietary systems, DeFi applications are typically entirely open sourced and built on top of open underlying blockchains.

Banking “APIs”

This causes three interesting properties:

1. Composability — The DeFi app itself can be forked, remixed, and reused in many other applications (more on this below).

2. Transparency — Since the DeFi app is open source, it is completely auditable to know exactly what the smart contract is doing in terms of functions, user permissions, and user data.

3. Auditability — Since the underlying blockchain itself is open sourced, the entire flow of funds is completely auditable including collateral in the system, trading volume, defaults, etc.

Unlike the traditional financial system (which is opaque), runs on a fractional reserve system, and is prone to market shocks — the DeFi system is completely transparent and over-collateralized — which allows DeFi companies to weather downturns much more efficiently.

Interoperable and Programmable

In order for developers to gain the trust of users, the majority of the DeFi apps are completely open source — including the front end and the smart contracts themselves. In addition, since DeFi apps all run on top of a common platform (the underlying blockchain) these DeFi apps are completely interoperable with each other and can be programmed to work with any other DeFi app in the ecosystem.

Contrast this to the traditional financial system where;

• Infrastructure Fragmentation — Traditional financial apps are not built on top of common infrastructure.

• Siloed Applications — Traditional financial apps are typically proprietary to one banking institution. For example, all of Wells Fargo’s “fintech apps” work together but not across different banking institutions.

• Developer Unfriendly — Traditional financial apps are not made for other developers to build services on top of.

The traditional financial system does have common standards; however, it’s extremely hard to reach consensus across market participants because financial institutions view their software as their competitive moat instead of using products as a differentiating factor.

One of the biggest reasons why we have seen so much innovation within the DeFi space is because the systems are interoperable, it allows the developer ecosystem to have more creative expression on the products and services they create. On top of this, developers don’t need to waste time reinventing the wheel, but rather can build upon common frameworks and focus on the things that make their products special.

Open and accessible to all

With traditional financial applications, new users typically need to go through a lengthy onboarding process, income verifications, credit checks, or even in person meetings — just to be able to use a given financial product.

With DeFi applications, all you need is a wallet address to interact with these systems. DeFi apps don’t ask for income verification, they don’t need credit checks, and in most cases they don’t even need to know who you are outside of the wallet address you are using.

This is one of the most under-appreciated aspects of DeFi products.

Traditional Fintech Architectures vs. DeFi Architecture

Here is a more architectural diagram on the main technical differences between a traditional fintech app and DeFi app (simplified for brevity’s sake):

Here is a more direct comparison chart on some of the key differences between centralized and decentralized financial applications:

DeFi Infrastructure — Market Map

Below is a market map of two different DeFi ecosystems, one built on the Solana ecosystem and the other built on the Ethereum ecosystem.

The reason why I am picking these two ecosystems to focus on is to show the breadth of DeFi apps being built across two different underlying protocols. I also believe Solana is the most interesting new layer one protocol because of its high transaction throughput (50K+ transactions per second), sub second latency & transaction confirmation times, and fast growing ecosystem of developers building DeFi apps on top of the Solana protocol.

While similar in structure, each underlying protocol has its own ecosystem built on top which is largely independent of the other. Below are some of the further explanations of each layer and the tradeoffs between them.

Base Layer (Layer One)

The base layer is the blockchain in which the core ledger itself sits. Ethereum is the most dominant layer one today, and Solana is the most promising new entrant with faster transaction speeds, more throughput, and cheaper transactions.

Node Infrastructure

A never ending amount of data needs to be queried about the underlying ledger (retrieving blocks, finding transactions, syncing data, writing transactions, etc). In the Ethereum ecosystem, a whole industry sprung up to solve this need (Infura, Alchemy, etc.).

Contrast this with Solana where the underlying ledger is fast enough and in sync enough that teams can just query Solana’s RPC nodes directly (this might not last forever though).

Layer Two

On Ethereum, there are various layer two solutions primarily used for scaling since Etheruem itself cannot handle all of the transactions on itself. Two of the promising scaling solutions include Matic, Optimism, among others.

On Solana, since there is only one layer to build upon (no layer 2 scaling solution needed) there are no specialized integrations needed and no mismatches with the underlying ledger which is processing settlement.

Order Book Aggregation

When new DeFi projects are built on top of Solana (DEX, AMM, Options, etc.), they can pull orders from Serum and push orders back into Serum, greatly reducing the cold start challenge most new financial applications face.

The best way to think about it is to think of it as “networked liquidity” and an “order management” system which is used by the majority of projects within the Solana ecosystem.

One of the more innovative examples of combining a CLOB (Serum) and an AMM is Raydium (very similar to Uniswap v3). The combining of these systems allows for passive LPs with active market making using Serum.

DeFi Toolset

There are a set of common tools needed to operate most of these DeFi apps, either from the perspective of developers or end users. These services don’t have direct traditional finance analogies but they include:

• Wallets — The main interface people use to store assets & interface with DeFi apps.

• Oracles — On-chain data feeds DeFi apps use to reference prices and execute transactions against (example: liquidations).

• Block Explorers & Analytics — Tools like Block Explorers were created to allow people to query the blockchain ledger itself directly. These are used most often when verifying transactions.

• Stablecoins — The two main assets used in DeFi ecosystems include the underlying native protocol token (ETH or SOL) and ideally on-chain stablecoins (USDC, Dai, or Pai).

• Front-Ends — A new emerging layer which creates easy to use front-end applications to interact with multiple DeFi projects at once, or to simplify transactions. This includes both Zapper.fi within the Ethereum Ecosystem or Step Finance within the Solana ecosystem.

DeFi Apps

The DeFi apps themselves are composed of all of the core financial applications which can be used directly, or embedded into other various apps within the crypto ecosystem.

Potential Missing Pieces of DeFi Infrastructure

When comparing and contrasting DeFi infrastructure with traditional financial infrastructure, there were a few pieces that don’t exist yet in the decentralized world that could be interesting to explore.

A few to highlight below:

• Consumer Applications — In the traditional financial world, consumers typically act with consumer apps (ex. Robinhood, Chime, Transferwise) not the underlying protocols themselves. The front-ends of the DeFi space could be greatly improved and intermediate much more of the total consumer experience. In general, the UI/UX of most DeFi apps are still very difficult to use from a consumer perspective.

• CRM — The DeFi space doesn’t really have a concept of customer relationship management nor typically collects any amount of consumer data. While great from a privacy perspective, there is great value in understanding the customer better.

• Notifications — Notifications or alerts don’t really exist at all in the DeFi space at all. On a more broader level there aren’t any great methods to communicate with users either.

• Product Analytics — There are tools to measure blockchain activity, but not to measure engagement within DeFi applications.

• Security — DeFi products do typically conduct security audits; however, none of the security audits guarantee the most common protections consumers are accustomed to in the traditional financial world. On top of this, the demand for security auditors outstrips the supply, so it’s a big bottleneck.

• Transaction Rollbacks — In traditional finance, if you make a mistake, a financial institution can initiate a rollback of the transaction. This does not yet exist in DeFi.

• Custody — Right now, most DeFi projects need to be interacted with from an individual wallet perspective. None of the custodians allow you to interact with DeFi apps.

• Developer Platforms — Most of the developers in the crypto space are building right on top of the layer one protocol itself. There are no concepts of developer platforms or middleware just yet.

• Identity — One of the biggest complaints from the traditional finance world about DeFi is the pseudonymity of users. Ideally there needs to be a way to keep out the bad actors while persevering consumer privacy.

Future of Financial Applications

After meeting hundreds of founders and seeing progress teams are making, one thing is very clear — the pace of innovation in DeFi is 10x faster vs. that of traditional fintech apps.

In traditional finance:

• The underlying ledgers are not open source nor developer friendly.

• There are a whole host of “banking as a service” applications just to wrap underlying partner banks in developer friendly platforms.

• Fintech apps are very challenging regulatory wise and typically take years of development before releasing a single product.

Contrast that to DeFi where:

• Everything is open source including the ledger itself.

• All of the transactions are public.

• Everything is built from the perspective of developers building applications on top of protocols.

• New DeFi apps are built and released in weeks, not years.

We at Race Capital believe that DeFi developers will forever change how the finance world works. We are incredibly bullish about the DeFi infrastructure stack and community.

Thanks!

Linda Xie, Jan 2020

In this beginner’s guide to decentralized finance (“DeFi”) we review the following:

• Stablecoins. A building block of decentralized finance. Unlike cryptocurrencies like Bitcoin or Ethereum that are known for their price volatility, a stablecoin is engineered to remain “stable” at exactly 1.00 units of fiat. Most stablecoins are pegged to the USD, but some are in other fiat currencies like the Chinese RMB.

• Decentralized lending. Programmatically take out a loan on the blockchain. No bank account required.

• Decentralized exchanges. Buy and sell cryptocurrencies through a blockchain, rather than a centralized exchange like Coinbase. In principle, a machine can trade on these!

• Collateralization. Provide digital assets to collateralize your decentralized loans, providing the lender some recourse in the event of default.

• Decentralized Identity. Identities are used in the context of smart contracts for things like assessing your creditworthiness for a decentralized loan.

• Composability. Snapping together DeFi functions that do different things, much like software libraries. For example, if one contract takes in crypto and generates interest, the second contract could automatically reinvest that interest.

• Risk management. High returns in DeFi are often accompanied by even higher risks. Fortunately, new tools are arising to help hedge these risks.

Let’s go through these concepts one by one.

Stablecoins

In general, the first two types of stablecoins have proven most popular. Whether it is fiat or crypto providing the collateral, people appear to want certainty around price stability. With that said, there are ongoing experiments around the third class of stablecoin, with people looking to combine both the crypto-collateralization and algorithmic elements.

Decentralized lending

Given functional stablecoins like USDC and DAI, we can start rebuilding pieces of the traditional financial system as automated smart contracts. One of the most fundamental is the concept of borrowing and lending.

Centralized order books for lending and borrowing

To understand what is being shown in this snapshot, start with the left hand side. The first row represents a borrower in the market who is willing to take out a 30-day loan for $4,421.58 at a 0.0265% daily rate. Right below them is another borrower willing to take out a 30-day loan for $34,292.38 at a slightly lower 0.0263% interest rate. On the right hand side are the lenders. The first row represents a lender willing to lend out up to $8,199.32 for 2 days at a 0.027418% daily rate. The next row is someone willing to lend $255.68 for two days at the slightly higher rate of 0.027436% per day. And so on and so forth. This is how a centralized order book for loans works. In the above example, the highest rate a borrower is willing to accept is 0.0265% daily interest, while the lowest rate a lender is willing to give is 0.027418% daily interest. One of these two parties will give in and either raise or lower their price of money, and then a deal will be made. Bitfinex provides the service to set up the order book and match users, and then takes a cut off the top of each loan for their trouble.

Decentralized order books for lending and borrowing

Some decentralized services for lending and borrowing take this to the next level. Rather than building an order book and facilitating matches, they allow users to directly lend or borrow from the smart contract itself, which dynamically raises or lowers interest rates to match. For example, if a large amount of crypto is borrowed from the smart contract, a higher interest rate is charged to borrowers. Moreover, in order to borrow funds, a user needs to provide collateral to the smart contract, providing an amount greater than what was borrowed so the loan is overcollateralized.

Decentralized exchange

Decentralized crypto exchanges attempt to put services like Coinbase Pro on the blockchain. That is, they aim to facilitate trades of different cryptocurrencies between two parties.

To understand the use case, start with centralized cryptocurrency exchanges. These exchanges, like Coinbase Pro, act as an intermediary and custodian where two parties deposit their assets and are able to trade with each other. While they have worked at scale to facilitate billions of dollars in trades, centralized exchanges do present a single point of failure that can be hacked, censor transactions, or prevent certain people from trading.

Decentralized exchanges aim to address this by using smart contracts to reduce or eliminate middleman.The dream is fully peer-to-peer exchange of all digital assets.

Decentralized identity

One issue with the decentralized lending and borrowing services mentioned thus far is that they require quite a lot of collateral. This overcollateralization requirement can be a highly inefficient use of capital -- and many people do not have the extra funds in the first place to provide as collateral. However, people are working on decentralized identity and reputation systems that will reduce the collateralization requirements. One of the first applications would be building blockchain analogs of fiat-based credit bureaus like Experian, TransUnion, and Equifax that institutions like banks rely on for credit scores. Now, to anticipate an objection, it’s certainly true that credit bureaus can put certain groups such as international and young people at a disadvantage. But newer services like Lending Club have addressed the problem of overreliance on FICO scores by offering additional data points like home ownership, income, and length of employment.

Composability

As DeFi matures, we should expect the libraries to start getting used outside the crypto community. Eventually you’ll be able to add one line of code to add a full decentralized marketplace to a video game, or another line of code to allow merchants in your e-commerce store to earn interest on their balance.

Risks

While DeFi is fascinating, it is important to acknowledge the risks that come with it. Let’s enumerate some classes of risk:

• Smart contract risk. Many of these systems are new and need more time to be battle tested. When protocols are interacting with each other, the smart contract risk compounds. If one protocol has a critical smart contract bug, then it can cause the entire system to be vulnerable. It would be wise to avoid putting too much capital in any of these systems during the early days.

• Collateralization and volatility risk. There are also risks associated with specific collateral types used to back loans. Overcollateralization reduces volatility risk, but if the price of the collateralized asset falls too quickly, a margin call isn’t guaranteed to cover the full amount that was borrowed. However this should be less of a risk with a reasonable collateralization ratio and vetted collateral types. Another potential issue is that the interest rate volatility on many DeFi platforms could make it impractical for someone to participate. There will likely be interest rate swaps or other methods to lock in a rate for a premium but this also adds its own complexities.

• Regulatory risk. DeFi platforms have varying degrees of decentralization, and we have not yet seen court cases that test all the claims being made. We’ll have to see what happens here.

Resources

The DeFi space is large and growing larger. Hundreds of millions of dollars worth of cryptocurrency has already been deployed and the space’s future potential is massive. This article only covered a handful of use cases, but we’ve provided some further reading below to understand DeFi further and keep up with the developments.

Understanding DeFi

Keeping up with DeFi

Thank you to Balaji Srinivasan, Will Warren, and Jordan Clifford for reviewing this post!

Disclaimer: Linda Xie is a Managing Director of Scalar Capital Management, LLC, an investment manager focused on cryptoassets. This post is not investment advice.

Defiant, Mar 2021

At The Defiant, we are convinced finance will become increasingly decentralized and open. It just makes sense; our current financial system is running on infrastructure that was built in the 1970s; messaging networks like SWIFT and ACH that simply aren’t made to transact value.

Blockchains have value at their core. They are money networks, and as such, they make for more efficient financial systems. They cut out intermediaries, improving speed and cost. They are permissionless and global, so that essential tools to grow savings and make payments and transfers, are at the hands of anyone with an internet connection, anywhere in the world.

A financial system that’s running on distributed networks and powered by cryptocurrencies allows users to maintain control over their assets and information. For the first time ever, any individual can own a piece of the networks and applications they use. They can participate in these protocols’ governance and benefit from their growth. It’s a new paradigm for the internet and for money.

We are witnessing the birth of the internet of money. It’s here, growing and maturing. You can use it right now.

The Defiant is the leading information company at this intersection of tech and money. It’s the only content platform producing unbiased, objective journalism, that’s focused on decentralized finance. We cover the news in this space day to day, across our newsletter, podcast, YouTube channel, and website.

For those who are new to this ecosystem (and even for those who have been around for longer), the flood of new terms, apps, and concepts can feel overwhelming.

This guide is meant to be a one-stop-shop to get started. These are all the DeFi basics you will need to start exploring the internet of money.

🤝 Share this guide with your friends, family, colleagues —anyone who you think would benefit from learning about the future of finance and how to start taking control of their assets.

Here are the first two articles you need to read as you get started on your DeFi journey.

Definition: Decentralized finance, or DeFi, is the ecosystem of financial applications being built with blockchain technology and without banks.

Characteristics:

1. Non-Custodial

2. Open

3. Transparent

4. Decentralized

History:

• Origins of the term

• Bitcoin and MakerDAO

In this guide we’ll be helping you take your first steps into the boundless world of Decentralized Finance, better known as DeFi.

We guide you through:

1. Setting up an account at a crypto exchange

2. Buying ETH

3. Setting up an Ethereum wallet

4. Withdrawing to MetaMask

5. How to use Uniswap and trade from ETH to DAI

6. How to lend stablecoins on DeFi protcols

7. How to connect your wallet to portfolio trackers

DeFi 101

Learn: Theory and Fundamentals

1. What is blockchain?

2. What is Interoperability and Why is it Important?

3. What is Layer 1?

4. What is Layer 2?

5. Layer 1 vs Layer 2

6. What is DeFi?

7. How Bitcoin, Ethereum, and Other Blockchains Fit Within DeFi

8. The Top DeFi Wallets

9. What are Stablecoins?

10. What is a DEX?

11. What is Liquidity Providing?

12. What is Yield Farming?

Risks & Hacks

1. Risks of DeFi

2. DeFi Insurance and how Nexus Mutual Works

3. Yearn Loses $11M in 2021’s First DeFi Hack

4. For All You Degens Farming in Public: Here's Your Privacy Toolbox

5. Blockchain Devs Are Neglecting Data-Related Risks, Chainlink's Sergey Nazarov Says

6. You stole $37M, now what? The $CREAM $ALPHA exploit unpacked

7. Flash Loans, and Attacks -- explained

8. DeFi Dre and the Invisible Hacked Pickles

Get Started: Tutorials & Guides

1. Getting Started With DeFi

2. Setting Up MetaMask

3. What is Gas and how to reduce it?

4. How to become a liquidity provider on UniSwap?

5. DeFi Safety Tips

6. DeFi Privacy Tool Box

Ideas and Big Picture

"I'm Not Super Bullish on DeFi. We're Using This Tech to Enrich a Small Group of People." James Prestwich.

"Basic Financial Services is a 21st-Century Fundamental Human Need." OMG Network's, Vansa Chatikavanij.

The Bangkok-based project formerly known as OmiseGo has spearheaded research and work on a scaling technology for Ethereum called Plasma. Scaling solutions will be key for Ethereum and therefore DeFi, to continue growing. The network is already at capacity, and gas prices are getting prohibitively high, especially for complex DeFi transactions.

"Decentralized Money Shouldn't be Traded on Centralized Exchanges." Loopring Founder, Daniel Wang.

"For the First Time, You Can Buy a Piece of the Art, and a Piece of the Gallery." MetaKovan.

This podcast episode is with MetaKovan, the pseudonymous investor who holds the largest known NFT collection in his Metapruse fund, which he leads with his (also pseudonymous) partner Twobadour.

"I'm Imagining a World Where Every Song Has an Investable Layer." DJ 3LAU.

"We've Been Creating Value for Instagram and TikTok With Very Little Actually Accruing to Us." Trevor McFedries.

"It Turns Out Music Does Have Value. We've Been Pricing It Incorrectly For 20 Years." RAC.

Anupam Majumdar and Dan Carr, Feb 2022

Crypto’s Evolution From a ‘Store Of Value’ to a ‘Medium Of Exchange’

Despite its recent dip, 2021 was a breakthrough year for crypto, with its market capitalization rising by 188% to reach ~$3 trillion in November 2021. Crypto’s profile and growth to date has derived from its utility as a store of value, but we are finally starting to see its potential as a ‘medium of exchange’. In this article, we examine how crypto payments are gaining relevancy in mainstream commerce, illustrate the current payments use cases, and evaluate their growth outlook. Bitcoin, the first cryptocurrency, was introduced in 2009 as an alternative medium of exchange to fiat-based currencies. This first generation of cryptocurrencies (bitcoin then others) never gained meaningful traction as payment methods due to their high volatility, low transaction processing speeds, and lack of acceptance. Bitcoin was highly successful as a validation of blockchain technology, which subsequently inspired the development of new generations of crypto across a wide range of applications (as shown in figure 1). Today, select cryptocurrencies (e.g., Solano) and next generation crypto innovations such as stablecoins (e.g., Tether, USDC, Dai) and central bank digital currencies (CBDCs) are driving a radical shift in consumer and merchant perceptions and accelerating usage of crypto payments.

FIGURE 1: Evolution of Crypto & Relevance as a Medium of Exchange

Source: Flagship market observations Drivers of Crypto as a 'Medium of Exchange’ As we outline in figure 2, crypto currencies are evolving beyond a ‘store of value’ to relevance as a ‘medium of exchange’. Emergence of new technological innovations such as stablecoins, NFTs (non-fungible tokens, which are essentially digitized tokens of ownership against digital assets) and DeFi (decentralized finance, a new breed of financial securities and investment vehicles built on the blockchain network) have accelerated consumer curiosity and adoption. Traditional PSPs have invested in crypto as a growth vertical and have further invested in strategic M&A. Visa and Mastercard have also adapted their rails to settle select crypto directly. Lastly, while regulations continue to be a double-edged sword, greater clarity in some markets is encouraging adoption by traditional actors.

FIGURE 2: Building Blocks of Crypto Payments Adoption

*Non-fungible Tokens (NFTs) and Decentralized Finance (DeFi) Source: Theblockcrypto.com, Chainanalysis.com, Flagship Advisory Partners analysis Current State of Crypto Payments

As illustrated in figure 3, crypto payments primary use case today is on-ramping and off-ramping fiat to crypto via exchanges (where volumes are huge). Several specialized fintechs that offer fiat on-ramps (buying crypto in exchange for fiat currency) have emerged on this basis (e.g., Ramp, MoonPay, Wyre), and many traditional PSPs have also thrived supporting these volumes (e.g., Checkout.com, Nuvei, Worldpay). New merchant segments, beyond exchanges, have also emerged (e.g., NFT marketplaces, GameFi, DeFi) accelerating the opportunity for C2B crypto acceptance. In traditional ecommerce, we have seen an acceleration of crypto acceptance as a payment method by merchants in select verticals such as online gaming, luxury travel, digital and adult services. In the last 12 months, Visa and Mastercard have partnered with over 80 crypto exchanges, enabling these exchanges to issue branded cards. These cards have accelerated the acceptance of crypto towards C2B online commerce. Visa recently announced the total card volumes exceeded $2.5 billion in its fiscal Q1 of 2022, already accounting for 70% from last year’s volumes. Select businesses are also facilitating B2C and B2B payouts using crypto, though this is specific to select verticals, mainly addressing payouts to contract and gig workers (independent, online platform contractors). Crypto payments are also evident in C2C remittances today, providing an economical way to remit payments especially for consumers in emerging markets.

FIGURE 3: Crypto Payments Proposition Models Overview

Source: Flagship market observations

As we illustrate in figure 4, we estimate crypto payment volumes at €0.5 trillion today, a tiny fraction compared to the global aggregated C2B, B2B and B2C payments turnover. C2B Fiat on/off ramps account for 95% of this share, so general purchasing and other use cases remain small.

FIGURE 4: Global Volumes: Traditional Payments vs. Crypto Payments (USD trn.; 2021)

Global includes payment volumes in B2B, B2C, C2B and C2C services Source: Visa, Flagship analysis

While crypto payments are small today, we expect volumes to accelerate and grow rapidly in the next 12-18 months. As illustrated in figure 5, C2B crypto payment adoption will continue to grow as newer generations of crypto mature with the ability to support rapid, simple and low cost means of payment in everyday life. We consider cross-border C2C to be potentially disruptive to traditional fiat remittance payments, as consumers are likely to benefit from economical price points and to be able to remit money in real time. B2C and B2B may take a little longer to mature, as traditional businesses will take time to embrace crypto.

FIGURE 5: Crypto Payments Maturity by Use Case

Source: Flagship market observations

Our expectation for acceleration of crypto use cases is based on the growth drivers that we outline in figure 6. In the near term, we anticipate mainstream PSPs to play a central role in driving consumerization of crypto. New blockchain applications in the form of NFTs will gain momentum and be accessible through fiat rails. We anticipate card schemes and regulators to play an important role in creating the network and the rules for direct crypto settlement. In the medium term, we do expect a tipping point when crypto becomes more mainstream, due to 1) regulatory clarity and sovereign, acceleration of CBDCs, and 2) acceleration of new commerce activity in platforms such as Metaverses and platforms built on Web 3.0.

FIGURE 6: Growth Drivers

Source: Flagship market observations

Conclusion

Payments has been an elusive aspiration of the cryptocurrency ecosystem since its inception many years ago. However, the key building blocks are now in place and a series of growth drivers are coming that we expect will accelerate the digital enablement of crypto payments in everyday commerce. We remain optimistic of the upsides in crypto and its appeal to both consumers and merchants as a long-term vehicle of commerce.

Aleksandar Svetski, Dec 2019

A high level primer on Bitcoin’s most well known second layer tech.

We’re now in the home stretch of the first edition of the Bitcoin Times.

In the last chapter, we closed out with comparisons between Bitcoin & the internet. In the next few, we’ll give some high level understanding of the concepts of second layer tech, with the focus being on Lightning.

Giving Lightning the explanation and time it deserves is out of the scope of this paper, but I’ll attempt to give you enough of an understanding to go further down the rabbit hole with….

Imagine a network, where each of the participants are not only route ‘users’, but also route operators. Where every participant becomes a node, that strengthens and broadens the network for not only themselves, but for everybody using it.

The best analogy I can think of is the internet (once again). The internet really exploded, when we became not only ‘consumers of content’, but also creators and routers of this data and content.

Testament to this explosion is the company which arguably capitalized on this the most. Facebook is barely 15yrs old and is one of the largest in the world. It gave everyone a forum to consume, create and share content; in other words — everyone was a node that made Facebook more valuable (and not just in relation to its market cap).

Conceptual way to understand the network effect layers have

What happens when you apply that same concept of read / write / route to money and payments?

In short — it changes the game.

Whilst not entirely accurate: Bitcoin is like the internet (one transformed information, the other money) and Lightning is a little like Facebook in that it makes money a content type that everyone can collectively participate in.

Money has never had that kind of fluidity, and it’s this fluidity that Lightning represents at a high level.

BUT…You might say: “Wait a minute. Not everyone can route money. They’re not a bank! How can we trust them”? That’s where Bitcoin comes in.

Lightning is technically able to be applied or “anchored” onto other networks, but its maximum utility comes from doing so on a network that gives the highest guarantee of immutability.

That’s the entire point, and how we unlock Lightning’s potential.

If you can refer back to something that has prioritized security, stability, resistance to censorship and shutdown, then you can begin to really abstract and build financial complexity on top of it; without worrying about the potential of error, compromise, fraud or failure. Bitcoin + Lightning is where the future is at.

Lightning enables:

1. Instant Payments: Because we’re not worrying about block confirmation times, payment speed is measured in milliseconds to seconds. It’s truly peer to peer, and as fast as data can move.

2. Scale: When all participants are also nodes, you don’t get the congestion we have in today’s archaic, centralised payment networks. You get true scale; capable of millions to billions of transactions per second across the network. This blows away any high-speed blockchain or any other legacy payment rails by many orders of magnitude.

3. Low Cost: Non custodial micro payments (e.g: pay per action/ click) are truly possible at fractions of cents. This is foundation for the set of use cases yet to emerge

4. Security: By anchoring to a source of truth (aka; Bitcoin) with simple, robust “smart contracts” one can ensure the integrity of the second layer without recording them on chain (via a complex version of netting).

How does Lightning work?

Lightning is a second layer technology. By using the native smart-contract scripting language of a network (such as Bitcoin) to anchor or connect to, it’s possible to create a secure second layer ‘network’ of participants who are able to process and route transactactions at high volume and high speed.

For example:

Dingus & Wingus decide they want to transact. Lots of times. Instead of bothering everyone on the core network and having every single validator on the core network have to record their transactions, they decide to open up what’s called a “payment channel”. Think of it like putting some money on your transaction account.

They can then transact between each other, back and forth, as much as they want — each time netting off against the prior transaction. After a certain period of aggregating these transactions, and updating the final net state; they could choose to close out the channel by broadcasting the final, net result to the underlying network (e.g: Bitcoin) and settle.

It’s important to note that this final, net entry can be closed out at any time by either party — without any trust or custodianship — by broadcasting the most recent version to the blockchain.

A very sophisticated, technical diagram.

Closing a channel is also how the network deals with cases of attempted fraud or “bad acting”. The last valid, signed set of transactions between both parties wins - and there are some incentive / disincentive rules that help to ensure it’s in everyone’s economic self-interest to do the right thing (i.e.; attempt to defraud the other user, but last signature shows otherwise, you lose the funds you committed to the channel).

This is all similar to how legal contracts function. One does not go to court every time a contract is made (that’s analogous to doing ‘everything on the blockchain’). Only in the event of non-cooperation is the court involved, and by making the transactions and scripts parsable and thus “anchoring” to the underlying network, these smart-contracts can be enforced and the result settled.

The potential

A payment channel between two participants is just the beginning. It’s a building block for a larger network. In fact, the network only forms when numerous payment channels join to form a web. In this way, two participants who are not directly connected can transact with each other.

Let’s say Dingus wants to pay Pingus. He can still do it even if he doesn’t have a direct connection (payment channel) with him, but as long as Wingus from earlier can connect them via a chain, i.e. route.

Another… very sophisticated, technical diagram.

The exciting part is that as the network grows, you won’t necessarily even need to set up a dedicated channel to send funds to a certain person. Instead, you will be able to send payment to someone using channels that you’re already connected with. The system will automatically find the shortest route.

By creating an entire network of these two-party ledger entries, it’s possible to find payments paths across the network similar to how packets are routed on the internet.

As all the pieces of the puzzle come together, one starts to see the magnitude of this innovation.

This is how everything else in nature works, along with the functional systems of cooperation we’ve built throughout the millennia.

You abstract the small, you settle when you need to — whether that be at closure, or on disagreement.

In fact, it’s a big part of how modern banking evolved (because it’s more efficient).

The difference (and beauty) with Bitcoin is that you can’t influence or manipulate it (remember: immutability as a service) so it will be the ultimate arbiter & settle as per the original rules.

It’s time to move to a new model where the arbiter / settlement function is digital and owned by the commons, not by the few.

The Future is on Lightning

What could the future hold?

Aside from the potential of doing billions of transactions per second (seeing as though speed is what everyone wants), and transforming payments and value transfer from things that happen at a time and place, to something that “streams” over time and space, perhaps something a little easier to imagine is the launch of a Bitcoin bank.

One where anyone, anywhere in the world could set up an account in seconds, and begin participating in global commerce.

Where all reserves are held and denominated in Bitcoin, on Bitcoin — and the organisation can be held 100% accountable because it’s all transparent and able to be queried.

Could we open up the ability to lend, borrow, spend, save, trade and interact globally — without worrying about exchange rates, inflation and manipulation?

I don’t know. Maybe. Or maybe I’m thinking way too small. I don’t know.

What I do know is that the real innovation is yet to come — and those innovations (like Facebook and cat videos on the internet) will not be skeuomorphic.

They will not be something we can predict or even imagine today. Myself and my team at Amber are making inroads in the new world, and we’ll continue to be at the cutting edge — but we are well and truly at the beginning — and all we can do is keep pushing the envelope.

Sometimes images speak louder than words

In the next & final section of The Bitcoin Times Edition 1, we’ll review Money as the fabric of society, how money functions, and we’ll close out the paper with some final thoughts.

Thankyou for sticking with us along this journey.

William Peasteron, Jun 2020

Actual farmers measure yield as the total amount of a crop that’s grown. Accordingly, DeFi proponents have now latched onto the farming metaphor and memed into existence “yield farmers,” i.e. folks who measure yield as the amount of interest that’s grown atop underlying crypto assets like Dai, USDC, and USDT when put to use in DeFi platforms like Compound.

The DeFi arena was already catching fire in 2020 before yield farming exploded onto the scene, but things have definitely kicked into overdrive in the sector thanks to the beginning of Compound’s COMP governance token distribution system in June. Simply put, the scheme rewards Compound users with COMP.

Before the distribution system started, Compound was the second-largest DeFi project per total value locked in its smart contracts. Yet just days after the system’s launch, Compound’s now decisively the largest DeFi project and its COMP token has the largest market cap of any DeFi token at press time. Why? Droves of traders have migrated to Compound to use the platform in order to “farm” COMP.

This buzz around the biggest happening in DeFi this year so far has had more than a few crypto users renewing their focus on various yield farming activities already available in the ecosystem, e.g. through projects like Balancer, Curve, and Synthetix. That said, yield farming is not necessarily new, but the surge of attention around such cryptonative opportunities absolutely is. Let’s dive deeper into DeFi’s hottest meme right now to better wrap our heads around what it means for us users.

COMP Farming

InstaDApp’s made yield farming easy for Compound users.

BAL Farming

To that end, BAL is the governance token of Balancer. Of the 100 million BAL ever to be minted , up to 65 million have been set aside to reward liquidity providers. 145,000 BAL are now being distributed to these providers on a weekly basis, which has led to traders parking their funds in Balancer pools to yield farm the BAL awards.

sUSD Liquidity Trial

Back in March, Synthetix launched an incentives program for liquidity providers of sUSD, the native stablecoin of Synthetix, through the Curve and iearn exchange protocols. The 4-week test campaign was designed so that 32,000 SNX tokens would be “distributed pro-rata to liquidity providers who stake their Curve LP tokens.”

The system worked as follows: users deposited sUSD and another supported stablecoin (think USDC, USDT, or Dai) into iearn, at which point they’d receive an allocation of Curve.fi sUSD/y.curve.fi tokens. Depositors could then take these tokens to Mintr, the Synthetix ecosystem’s decentralized minting hub, and stake them in order to qualify for the trial’s SNX rewards. The idea, then, was that liquidity providers would get regular pool APY and incentives in the form of SNX, for providing liquidity for the Synthetix ecosystem. Obviously, that’s a very attractive campaign for yield farmers, where you’re earning interest on your parked digital assets as well as additional liquid digital assets you can instantly sell for profits on any DEX.

The Curve-Ren-Synthetix Farming Meld

Because of the unique way the system has been set up, participants who provide WBTC, renBTC, and sBTC liquidity to the new BTC Curve liquidity pool will simultaneously earn BAL, SNX, REN, and CRV (Curve’s coming reward token). That’s a yield farmer’s dream come true, but how’s it possible? First off, the Synthetix and Ren teams have created a Balancer pool composed of SNX and REN tokens. This pool will generate BAL from Balancer’s liquidity mining campaign, as well as liquidity provider (LP) rewards in the form of BPT, which is simply a wrapped combination of SNX and REN.

For 10 weeks, then, this generated BAL and BPT will go to depositors who provide liquidity to the BTC Curve liquidity pool. Additionally, the LPs will also be allocated CRV tokens for servicing Curve.

Futureswap

Why only 3 days? Because the demand to use Futureswap was so explosive in that span that the project’s builders saw enough and decided to shutter the Alpha platform early just to be cautious. But those who used the exchange while it was live saw potential. In a follow-up analysis, the Futureswap team said high volume during the Alpha “translated into the outperforming of holding equal value amounts of ETH/DAI for liquidity providers of over 550% annually.” That margin will certainly get most yield farmers’ attention.

Conclusion

Be curious, but don’t be reckless. Yield farming is new and isn’t going anywhere, so there’s no need to rush in. And in zooming out a bit, the fact that yield farming isn’t going anywhere makes it yet another ace for Ethereum when it comes to fostering interesting things that users want.

Yuga.eth, Jan 22

One of the most distinctive characteristics of Decentralized Finance (DeFi) is the prevalence of the concept of “yield.” Every day, new protocols advertise absurdly high numbers in an attempt to attract customers: 97% APR on ThisCoin, 69,420% APY on ThatCoin, and so forth. Of course, this property of DeFi is also what makes newcomers suspicious. How could a new protocol on a blockchain you’ve never heard of possibly earn a billion percent when the rate for a typical savings account is 0.5%?

This article will demystify the notion of DeFi yield. We will lay out the most common mechanisms by which DeFi protocols provide yield to their customers, and in particular, we will make the following argument:

DeFi yield comes from the value of the underlying DeFi protocol. Thus, locking up your money with a yield-generating DeFi protocol is a bet that the protocol itself is intrinsically valuable.

Let’s dive in.

Definition of “Yield”

Denominating Value

Notice how I used the word “value” above. How does one denominate “value”? The fact is, it’s up to you: understanding denomination of value is the key to understanding DeFi yield.

As an example, let’s suppose that a DeFi protocol offers 1,000% yield for staking its native asset, ThisCoin. ThisCoin is currently trading at $2 per coin. You take $100, buy 50 ThisCoins, and stake the entire sum and expect to receive 50 * 1,000% = 500 ThisCoins at the end of a year.

A year passes, and you indeed do receive 500 ThisCoins back: you have achieved 1,000% yield! But now, each ThisCoin costs just 1 cent. That means that your 500 ThisCoins are worth $5, and in dollar terms, you suffered a 95% loss. On the other hand, if each ThisCoin costs $20 per coin, then you now have a $10,000 stack, for a 10,000% yield in dollar terms.

The point is this: whether you value the 1,000% yield in ThisCoin more than the 95% loss in US Dollar value is up to you, depending on whether you believe ThisCoin holds more intrinsic value than the US Dollar. (In this case, it probably doesn’t.)

The example serves to illustrate that to understand a protocol’s yield, you must understand the assets by which the yield is being disbursed. The potential fluctuations in the value of the yield-disbursed assets is a risk that needs to be taken into consideration.

Calculating Yield

Even with the proper denomination of value, there can be a great deal of variability in the way that yield is calculated. Here are some common aspects that you should be aware of when you see a sky-high number for the yield.

• Lookback Period: The APY figures you see on DeFi protocols can be based on data from a time period in the past: it could be the past day, the past week, the past year, or anything else. Given the variability of market fluctuations, protocol performance, etc. the projected APY can change wildly. Thus, projected APY may or may not resemble actual APY depending on past and future market conditions. Mitigate this risk by understanding the time horizon on which protocol APY figures are computed.

Yield Mechanisms

DEXs / AMMs

For example, in the last 24 hours, the TraderJoe USDC-AVAX pool received $156K in trading fees, disbursed to the LPs. Given the pool’s overall $166M value, this corresponds to an APR of $156K * 365 / $166M = 34.2% for liquidity providers.

But fundamentally, the source of yields for DEX and AMM LPs is clear: they come from trading fees. The traders are willing to pay these fees because the DEXs and AMMs are providing a valuable service: a liquidity pool and an automatic algorithm to swap assets. The more popular the trading pair, the greater the trading volume, the greater the fees, and thus, the greater the yields for LPs. In other words, the yield derives from the value of the underlying protocol

Lending Pools

Borrowing / lending pools are similar to DEXs / AMMs in that they are all about providing liquidity. The difference is that the liquidity is provided in the form of debt rather than sales.

For example, borrowers may want to take out a loan in USDC in order to pay for their groceries. To do this, they deposit ETH. Two scenarios would result in a liquidation of their ETH and a closure of their loan: 1) If the price of ETH dropped enough, or 2) If the loan accrued enough interest. So long as neither of these events happen, the loan remains open. Once the loan is paid back, the borrower gets back their ETH.

Again, likening this to the TradFi setting: in the centralized world, it is the banks and other institutions (e.g. Fannie Mae and Freddie Mac) that capture the interest rate fees, passing on what are generally very low rates to depositors. In DeFi, the lenders themselves get to capture this value, and the interest rates are set algorithmically by open-source code.

Staking

Many protocols use the term “staking” to refer to any locking-up-of-assets, but this is not correct. Staking, in its true sense, is to risk loss of value for potential gain of value.

Proof-of-Stake is about ensuring that the transactions on a blockchain have integrity - there is no double-spending, fake accounting, and so on. To ensure that a series of transactions have integrity, validator nodes do the work of checking that every transaction is valid according to the rules of the blockchain.

How can we trust that the validators will do honest work? This is where Proof-of-Stake comes in. At a high level, it works as follows:

1. Users stake some amount of their asset with a validator.

   1. Once staked, there is usually a minimum lock-up period, ranging from weeks to months. Eventually, users can unstake their asset.

2. For every honest piece of work that the validator does (e.g. computing transaction integrity, voting on correct blocks etc.), they are rewarded.

   1. The rewards are disbursed to the stakers, and this is the yield.

3. For every dishonest piece of work that the validator does (e.g. violates rules, cheats, goes offline, etc.), they are slashed (i.e. some portion of funds are taken away).

   1. The penalties are taken proportionately from the stakers.

4. Validators verify and vote on each other’s work to decide on honesty / dishonesty.

As a de facto successor to Proof-of-Work, Proof-of-Stake is a monumental achievement in the field of distributed consensus. By locking up your assets with a validator, you are earning yield by contributing to the security of the underlying network, which is intrinsically valuable, and you are risking the possibility of losses if the validator is dishonest or broken.

Yield Optimizers

If you don’t want to think too hard about how to generate yield, an optimizer might be for you. These are giga-brained mechanisms that take your asset, perform a bunch of DeFi operations behind the scene, and return a larger amount of the asset to you in the end. The best analogy for this type of operation might be a mutual fund or hedge fund: you aren’t really doing the investing, you are outsourcing that to an algorithm.

Gro Protocol’s Capital Allocation.

Yield optimizers are attractive because they amalgamate a diverse range of strategies so that if any single one fails, you might still end up making money with the other ones. They also save you the time of having to research how exactly any single protocol works. That said, you assume the risk of trusting the authors of the optimizer strategy - the yield you’ll receive is a function of how good the optimizing strategy is.

Derivatives

Derivatives are a double-edged sword. On one hand, the ingenuity of these protocols are filling a niche that will undoubtedly become more valuable as DeFi matures as an industry. On the other hand, these are fairly complicated pieces of financial technology that the average DeFi participant cannot be expected to be knowledgeable about. If you are going to get your yield from derivative-based protocols, it’s important that you understand how the structure of the derivative directly relates to your yield!

Governance

Governance tokens are one of the largest sources of advertised yield, and yet they are also arguably the hardest to understand.

When DeFi protocols are starting out, they often reward Liquidity Providers and other early participants with the protocol’s own governance tokens, a process known as “liquidity mining.” They will also bootstrap liquidity pools on DEXes for their governance tokens, so that the tokens become exchangeable for common assets like USDC and ETH. This induces a market price for the token.

The moral of the story here is that if a significant portion of the yield for a DeFi protocol comes in the form of its native governance token, you should apply extra scrutiny to it, because in this case, your yield literally derives from the value of the underlying protocol.

Rebasing Tokens

Rebasing tokens are fundamentally defined by the eponymous mechanism of rebasing, which just means that if you lock up the currency in a staking contract, the amount of the currency that you hold increases by a certain percentage at a regular time period, like 8 hours. The frequency of this compounding is what leads to astronomically high APYs. For example, currently, OlympusDAO offers rewards of 0.3265% per epoch (8 hours) on staked OHM, which results in a 3,450% APY.

Of course, if you’ve been reading this post closely, you’ll easily see that an increase in the underlying currency amount does not necessarily imply an increase in value. That is, if the number that represents the amount of currency does not meaningfully correspond to something intrinsically valuable - like software that facilitates financial exchange, or a real-world asset like the US dollar, or a share of future revenues of a protocol - then the currency amount going up does not necessarily imply that you own more value.

Recently, rebasing tokens had a reckoning as more participants realized that their high APYs didn’t actually mean they owned more value. Here are the charts of OHM, TIME (Wonderland) and BTRFLY ([redacted]) market cap:

OHM market cap has declined 85% over 3 months.

TIME market cap has been extremely volatile and declined 52% over the past 3 months.

BTRFLY market cap has declined 48% in 3 months.

“Reserve Currencies” showcase how DeFi can quickly become untethered from reality. As new participants see extremely high numbers, they become excited and buy in. This drives the price up, leading to more participants buying and staking, creating a flywheel effect. If at any point people had stopped to consider why the numbers were going up they might have realized that their high APYs were not based on value, but on an inflationary monetary instrument.

TIME in particular has seen quite a bit of drama recently, with accusations that the manager of the treasury is a convicted financial felon:

But if the currency doesn’t even attempt to make its “amount” meaningfully reflect an underlying source of value, then they shouldn’t be seen as a reliable deliverer of DeFi Yield.

Conclusion

There are many ways of earning yield through DeFi. Some are low-risk and some are high-risk, but in all cases, it behooves you to understand where the yield is coming from. The straighter the line between the underlying value of the protocol and the yield being disbursed to you, the surer you can be that there won’t be a rugpull down the line. Since DeFi Yield comes from the value of the underlying protocol, your key job is to determine the value of DeFi protocols. And I’ll be here to help you do that :)

Jump Crypto

TL;DR

• This piece uses a conceptual lens to study yield farming, i.e., earning compounding returns on crypto assets. It illuminates the fundamental exchange of value that yield farming entails.

• Yield farmers passively provide five forms of value: operating the network, lending to traders, providing liquidity, governing protocols, and raising visibility.

• Yield farmers are compensated for these activities by a mix of users, non-farming protocol owners, and each other.

Introduction

Warren Buffet warns that you should "never invest in a business you cannot understand." So an investor could be forgiven for looking at yield farming — a landscape full of free money ("airdrops") and scams ("rug pulls"), ruled by characters calling themselves "degens" and "apes" — and running away.

However, a closer examination reveals that yield farming is simply another type of business activity, one which provides value by bearing risk. Obscured by the frenzy of new protocols, misspelled slang, and coordinated capital flows, yield farming rewards entrepreneurial efforts to build new platforms.

In this article, we examine yield farming through a fundamental economic interpretation. In particular, we ask two questions:

1. What core value do yield farmers create, for which they are compensated?

2. Who pays that compensation, whether explicitly or implicitly?

For many traditional businesses, the answers are straightforward. For instance, our local sandwich shop provides a range of services — sourcing quality ingredients, assembling sandwiches, and wiping tables — and diners pay for those services directly. Yield farming is not much more complicated. Jargon aside, most yield farming strategies engage in five core types of economic activity, in passive and delegated ways:

1. Farmers support network operations, such as validating transactions.

2. Farmers provide lending for traders.

3. Farmers provide liquidity to token holders.

4. Farmers provide management and governance to protocols.

5. Farmers enhance protocols’ marketing efforts.

This piece examines yield farming through these simple, abstract lenses to help current and potential farmers understand their opportunities and evaluate their risk profiles. Since specific strategies are constantly rising and falling, we will avoid diving too deep into the mechanics of any particular trade. Instead, we will explore the underlying conceptual bedrock, which is broadly applicable to both current and future farming opportunities.

What is Yield Farming?

• Managing passive strategies: Yield farming is hard work, and farmers must constantly find ideas, manage risks, and rebalance positions. However, we consider these strategies passive because, once the upfront cost to find an opportunity is incurred, the deployed position earns a return with little to no further action. This framing contrasts with active ways of earning compensation, e.g., running validator nodes or managing algorithmic market-making strategies, which require ongoing technical labor.

• Well-defined interest: To make our definition more useful, we focus on strategies that have some well-defined schedule of interest payouts. There are many forms of well-defined interest schedules: fixed and floating, simple and complex. The schedule itself is what distinguishes yield farming from simple buy-and-hold strategies, e.g., buying coins or NFTs hoping for price appreciation, as these do not have any definite payouts.

Yield farming can further be defined, with respect to a traditional buy-and-hold strategy, as a way to earn extra returns while holding exposure to those same positions. To help better understand this definition, consider some examples from the world of traditional finance.

• An investor who holds cash in her standard bank account would not be yield farming, since this is the default action and does not involve managing a strategy. But an investor who constantly opens new high-yield savings accounts to earn bonuses and promotional rates would be yield farming cash positions.

• Similarly, a consumer who uses a single credit card would not be yield farming, but one who actively manages a portfolio of cards to maximize miles, bonuses, and other rewards would be. In this case, the underlying asset would be a line of credit.

• An investor who simply holds a stock would not be yield farming, since again this is the default buy-and-hold action and earns no well-defined interest. However, an investor who boosts returns by lending their shares to short sellers would be.

• Fixed-income investors who buy and hold fall into a grey area. If we consider cash as the underlying asset, an investor who lends that cash to a borrower would indeed be yield farming, since they would be earning interest passively on a cash position. But in many cases, we tend to think of a fixed-income security, e.g., a bond, as the underlying asset, in which case the investor is not earning any extra yield.

Extra returns do not come freely, of course. In exchange for this compensation, yield farmers take on risks and provide a range of passive benefits to the protocol. We will now outline the five ways they do so.

Activity #1: Network Operations

The most basic function in crypto is correctly and securely operating the network. This is done by node operators, usually called validators, who process transactions in exchange for payments in the network’s native token. While many networks rely on computationally-intensive problem-solving to incentivize good behavior ("proof-of-work" networks), others rely on validators posting collateral ("proof-of-stake" networks). That collateral can be partially or fully seized if a validator underperforms or misbehaves.

In turn, the first major form of yield farming is for farmers to delegate tokens to high-quality validators, i.e., validators with reliable and honest performance, in exchange for a share of proceeds. If yield farmers allocate to low-quality validators, those validators will face negative consequences, i.e., forfeited collateral, and farmers will bear that burden.

Consider the two original questions: what economic value do farmers create and who compensates them?

• Yield farmers allocate their holdings towards high-quality validators, allowing the network to run more efficiently and securely.

• Rewards are paid by network participants, who pay fees to validators in exchange for using the network. Validators then remit a portion of those fees back to farmers.

Activity #2: Lending

The explosion of decentralized protocols, known as "DeFi Summer" in mid-2020, dramatically changed the face of crypto lending activity. Prior to this point, lending relied mostly on large and centralized institutions, but since 2020 a new wave of decentralized protocols has allowed individual traders to participate in a wide range of lending activities.

Thus, the second major form of yield farming is a generalization of the first. Rather than only lending to validators, traders can lend cryptocurrency positions to anyone. In particular, yield farmers place tokens in funding pools, and borrowers automatically borrow from those pools using suitable collateral.

Many examples of these types of protocols exist, with Aave, Compound, and Anchor as some of the most popular. These protocols typically accept deposits in a base asset — one that already exists outside the protocol, e.g., UST in the case of Anchor — which can be lent out to borrowers. These protocols keep track of deposits by issuing the lender a new synthetic token (e.g., "aUST" for Anchor), which lenders can use to redeem the original position and the accrued interest later.

For now, almost all protocols have focused on over-collateralized lending. Lenders thus face the risk of the collateral depreciating more quickly than it can be liquidated. However, a few protocols such as TrueFi and Goldfinch are expanding into the uncollateralized lending space by vetting borrowers, using their knowledge of some off-chain, real-world information. Future yield farmers will likely choose between making fully collateralized loans and taking on the default risk of borrowers more directly.

As before, we ask ourselves the same two questions: What economic value do farmers generate and who pays them?

• Yield farmers allocate holdings towards capital-constrained traders, which allows for those traders to express views on asset prices more efficiently. In the future, yield farmers may also provide value by allocating holdings towards higher-quality borrowers and projects.

• Farmers are compensated by borrowers, who pay continuous interest back to farmers (with protocols taking a cut). While some protocols temporarily guarantee fixed interest rates, most use floating rates that allocate supply and demand.

Activity #3: Liquidity

Liquidity provision, like lending, has been democratized and massively expanded by DeFi protocols. Previously, only centralized exchanges and professional market makers could successfully muster the capital, technical expertise, and ongoing attention needed to provide liquidity. Today, retail traders can passively do so too.

This is the third major form of yield farming. Farmers deposit cryptocurrency positions into common liquidity pools (known as "automated market makers" or AMMs). Traders who need liquidity can swap tokens against these pools — often paying explicit fees in addition to spreads to do so. Liquidity providers earn these fees and/or spreads by facilitating two-way liquidity, but also bear the risk of capital losses if the fundamental exchange rate changes (and does not revert). This contrasts with active liquidity providers, who frequently adjust their positions as the exchange rate drifts.

Examples of major protocols running liquidity pools for yield farmers include Curve, Uniswap, Sushiswap, among others. These pools act as centralized liquidity hubs, facilitating trades between many different pairs of assets.

In returning to the same two questions, the answer to the first question — what value do farmers provide? — is straightforward.

• Yield farmers provide liquidity to those who need it, allowing them to enter and exit positions with minimal market impact.

But answering the second question — who pays for that value? — is more complicated. In particular, there are three ways that liquidity providers are compensated.

• First, AMMs distribute direct rewards — namely, trading fees and spreads — to liquidity providers. The rewards are directly paid by the users who take liquidity from the pool.

• Second, AMMs issue rewards in their own native tokens, e.g., Curve issues the CRV token. (Note that Curve was indeed the first protocol to successfully operationalize this model, which has since become wildly popular and widely imitated.) In addition to their direct economic value, these tokens usually come with special rewards schemes (and governance rights).

  • For instance, a farmer on Curve can boost their rewards on a pool up to 2.5x from the base amount by holding a substantial amount of CRV tokens relative to the liquidity they provide.

  • As a result, rewards are paid by two constituencies. First, farmers who have small CRV positions earn lower fees from liquidity provision, implicitly subsidizing farmers with large CRV positions. Second, non-farmers who hold CRV positions are diluted through CRV supply inflation, thus again subsidizing farmers.

• Third, individual protocols may pay for rewards to those who provide liquidity for their specific token. In practice, they operationalize this by buying governance tokens of the large liquidity protocols (directly or indirectly) to redirect extra rewards to their token’s pool. This method is discussed more in the next section.

Activity #4: Management and Governance

Although "code is law" is frequently cited as the blockchain ideal, much of crypto remains a hands-on activity. Capital needs to be directed, protocols upgraded, and systemic threats addressed. Most such tasks are accomplished via active and decentralized means — individual stakeholders write and review code, vote on proposals, etc. But in the last few years, pooled management systems have grown in popularity, particularly in directing capital.

The fourth major form of yield farming is to power those pooled systems and thus manage tokens in passive and delegated ways. For instance, the protocol Convex has been highly successful in directing liquidity on the Curve platform across liquidity pools. The protocol Yearn has found similar success at a higher level, allocating assets across multiple lending and liquidity protocols.

This reminds us that a single yield farming strategy may provide value in multiple ways. For instance, a yield farmer could provide liquidity on Curve directly, or she could provide liquidity on Curve through Convex. She earns rewards for providing liquidity in both cases, but earns extra rewards for doing so more efficiently in the latter case.

The answers to our two core questions — the value provided and the compensation received — are more subtle than in previous sections. In particular, there are two ways in which yield farmers earn their keep through managing protocols. The first subset is value-creating:

• Yield farmers create surplus through more efficient management. Using the same examples of Convex and Yearn, these protocols can reallocate liquidity towards particular markets more quickly and cheaply than a group of disaggregated traders. (Note that aligning incentives towards the most useful markets is a subject of lively and evolving debate.)

• Farmers are compensated both explicitly and implicitly:

  • First, farmers earn better returns by redirecting resources towards high-value opportunities. They are then compensated by their new end users, such as borrowers or liquidity utilizers.

  • Second, farmers save on transaction costs by utilizing such services. This can be highly significant — fixed fees on the Ethereum network sometimes exceed $100 per transaction, which makes larger transactions much more attractive. (Imagine an ATM that charged $100 to withdraw. You’d want to take out a lot each time!) Pooled protocols amortize these fees, so the portfolio can be rebalanced more frequently. Farmers can also save on individual effort through automated portfolio management, where the protocol automatically identifies and transfers funds towards the current best opportunities.

However, farmers can also profit from management through value extraction:

• Farmers can take advantage of protocol-based rewards (as with Curve) either through pooled structures ("aggregators") or individually, but aggregators make more efficient use of reward-boosting mechanisms. Thus, on the margin, the increased compensation is funded by small liquidity providers and token holders who do not provide liquidity. In short, value is being transferred rather than created.

Activity #5: Marketing

Thus, the final value provided by yield farmers is enhanced visibility and trust through asset allocation. To keep that value within the system, protocols often reward illiquidity as a means of "leasing" attention and usage, which gives them time to build and mature. Specifically, protocols ask farmers to purchase and lock tokens in exchange for token distributions — with larger rewards for longer lockups. Of course, part of this exchange is that locked holders, who cannot respond to market conditions, bear substantial macroeconomic price risk relative to liquid ones.

Looking at our core questions of fundamental economic value and compensation for the last time, we arrive at the following:

• Yield farmers provide increased TVL to a given protocol, driving higher awareness and more usage.

• Yield farmers are compensated by the protocol, which typically provides rewards in the native token. This means that, in the short run, non-farmers pay for these rewards by bearing the inflationary burdens. However, in the long run, the protocol hopes to be successful in creating value and attracting new users. In this case, later generations of holders pay for the marketing benefits provided by the early (yield farming) generations.

Conceptually, this channel is the most nebulous and prone to Ponzi-like dynamics. Indeed, protocols must grapple with key questions, such as whether they can keep users after the implicit marketing budget ends. But there is some precedent for success from the startup world, where "blitzscaling" — the strategy by which VC-funded startups massively subsidize users to gain market share before raising prices — has proved effective if finicky. Crypto is now copying that playbook, incurring deep expenditures upfront with the chance of realizing enormous value over time. Channeling the Lindy effect, protocols hope to become a fixture on the crypto scene in the short run to ensure success in the long run.

Rewards-driven marketing is a tightrope that must we must tread carefully. Protocols that offer no rewards will languish without attention. On the other hand, highly aggressive protocols (e.g., ones that offer 1000% staking yields) attract only yield farmers, and there are few non-farmers to power the rewards. This leads to short-term spikes from pools of "mercenary liquidity," but long-term collapses.

Financialization and Scaling Yield Farming

Despite their numerous variations, yield farming strategies at their core are fairly simple. Farmers passively provide value to protocols, in exchange for which they receive both direct and indirect compensation. However, there is one final ingredient in the mix: financialization. Crypto has developed a robust ecosystem of financial protocols, allowing yield farmers to transfer assets freely and to take generous leverage. This is how many yield farmers multiply high base yields to even more enticing levels (e.g., 20% to 100%).

While a further discussion of the use of leverage in yield farming is out of scope for this piece, we will mention one common example: liquid staking. This is where farmers deposit a base token into a protocol and receive a synthetic tradable token in exchange. As long as the synthetic token is accepted as collateral by other lending protocols, farmers can leverage their position. Namely, a farmer could deposit the base token (e.g., ETH), receive a synthetic token representing that claim (e.g., sETH), borrow ETH against that synthetic token as collateral, deposit that ETH, receive sETH against it, etc. This sequence is by no means the only one, and the ability to short or fractionalize adds even more complexity to the mix.

But it is important to stress that, while financialization can amplify yield farming opportunities, the underpinning theory remains the same. Farmers are compensated for bearing risk and passively providing value, while financialization connects different parts of the ecosystem and expands the menu of risk-return profiles.

Conclusion

At their core, yield farmers and traditional farmers are not so different. Both take risks (whether price risk or weather risk), provide value (whether creating buzz or creating food), and earn returns. We hope that, with the fog of jargon and insider lingo lifted, yield farming can be seen as just another economic activity that helps the crypto ecosystem run more efficiently.

But, like traditional farming, yield farming is not an easy business. Despite the conceptual simplicity, yield farming in practice involves an ability to find under-appreciated opportunities, move crypto positions rapidly, and understand nuanced risks in smart contracts. Aspiring yield farmers should brace themselves for mistakes and only deploy the holdings they are prepared to lose.

As the industry matures, we believe that these problems will ease and that a broader set of individuals will be able to participate safely. Yield farming will then deliver on crypto’s broader promise — democratizing finance and allowing anyone, regardless of sophistication or wealth, to provide these core sources of value.

Angle, Jul 2022

AMM stands for “Automated Market Maker”. It’s basically a smart contract that manages liquidity provided by Liquidity Providers to create an on-chain marketplace. We use them every time we do on-chain swaps.

The most known AMMs (Uniswap and its forks, Curve, Balancer) respect certain properties:

• They do not need any human input and are described by mathematical invariants, which dictates what swaps they authorize. The most known of is Uniswap V2’s invariant x*y = k. The smart contract holds x token X and y token Y, and allows any swap dx for dy that preserve the invariant. So any swap such that (x+dx)*(y-dy) = k.

• In addition to being driven by a mathematical formula, they are “path independent”, meaning that if we don’t account for accrued fees, their states only depend on the prices. So as a LP, if you add liquidity and remove it at the same price, you can’t have made a loss.

Let’s do a step-by-step example to better understand. Assume you have 1000€ to invest on the agEUR / ETH pair, and that the starting price of ETH is 1000€. If you do nothing and hold only one of the 2 assets, your returns in € after a year would be:

Holding agEUR versus holding ETH

It may seem complex, but this graph basically says that you’ll have 1000€ if you’ve held agEUR or whatever 1 ETH will be worth a year from now if you’ve held ETH.

Now assume you want to provide liquidity in Uniswap V2. You’ll have to bring 500 agEUR and 0.5 ETH at first. Would it be better than holding those 500 agEUR and 0.5 ETH ? Without taking fees accrued by the AMM into account, after a year you’ll have:

Uni V2 LP without fees versus holding 50% agEUR / 50% ETH

See the difference between the red curve and the purple curve? This is the infamous Impermanent Loss a.k.a. “IL”, that is to say what you’ve lost by enabling others to swap your assets.

There are numerous proposals out there of protocols or magical techniques to “eradicate” IL, but it is somehow inherent to AMMs: if you allow traders to swap your liquidity, then you’ll be left with a different asset balance than what you brought, which may be worst than having let them idle in the first place. So how do LPs earn ? Thanks to fees and the “path independent” property: with a reasonable price change, after some time the pool will be back to a state close to its original one and will have accrue some fees.

Let’s add add these fees to the previous curve. Let’s assume the volume is 100 agEUR / days for a year and the pool has a 0.25% fee. The graph would then be:

Uni V2 LP versus holding 50% agEUR / 50% ETH

Now we see all the potential benefits of LPing: in this scenario, without price change, there would be a ~9% APR !

Let’s now deal with Uniswap V3. Briefly, it’s is a complexification of Uniswap V2: the mathematical invariant is the same, but LPs can “concentrate” their liquidity on a price range by adding a “virtual” liquidity to the pool. Let’s not dig into the maths this time but focus on what the implications for LPs are.

When adding liquidity on Uniswap V3, you choose a price range, which defines the amount of “virtual” liquidity you’re also adding to the pool. The more your range is small, the more “virtual” liquidity you add to the pool, which’d mean more IL but bigger swaps so higher fees to move the pool price.

In the extreme case of a range 0-infinity, you’re back in the Uni-V2 case. Let’s take the example of a 500–2000 range and compare Uniswap V3, V2 and holding:

Uni V2 LP versus Uni V3 LP versus holding 50% agEUR / 50% ETH, without fees

What happens ? When the price reaches 500, everything has been swapped to ETH so the Uniswap V3 position only contains ETH, and when it reaches 2000, there is only agEUR left in the position. The IL is much stronger that Uniswap V2. But it also means that to move the price of the AMM, more liquidity needs to be swapped, so there will be more arbitrages and more fees earned by LPs. But how much ?

For the purpose of this simulation, we can therefore 2 types of volumes:

• a daily “intrinsic” volume: all the swaps that are not an arbitrage, and that are not made in order to move the price of the pool. It includes for example all the retail, spot orders. This volume will not be higher depending on the AMM type.

• a volume due to arbitrages between market places, that aims to move the price of the pool. This one can be estimated given a daily price change and the formula of the AMM: you can compute the swap size needed to move the pool price, hence the fees earned.

Let’s assume a mean price deviation of 10%, implying more volume and fees for the UniV3 pool:

Uni V2 LP versus Uni V3 LP versus holding 50% agEUR / 50% ETH

Uni V3 versus holding ETH until 1500, then holding agEUR

Antonio Juliano, May 2019

To date, the biggest sector by far for decentralized applications has been lending & borrowing crypto assets. Several high quality products have been built that allow users to borrow and lend directly on the Ethereum blockchain with no intermediaries. Decentralized lending products are available to anyone, anywhere, and require only an Ethereum wallet to use. These products are already seeing real usage today with total USD volumes in the hundreds of millions.

Use Cases

Lending

Crypto holders can lend on decentralized lending platforms to earn passive income on their holdings through interest fees paid by borrowers. This is an attractive option to lenders as they can earn relatively low risk interest on their existing holdings without entrusting their private keys to a 3rd party centralized service.

Borrowing

The dominant use case for borrowing crypto is margin trading. Borrowing allows traders to get leverage which multiplies gains and losses while trading, as well as short selling, a trading strategy which makes money when the price of an asset goes down.

Margin trading involves borrowing an asset, and then immediately selling it. For example, you could borrow DAI (a stablecoin) and then buy ETH with it, which would let you buy more ETH than you would otherwise be able to (giving you leverage on ETH). Of the 4 top lending protocols discussed below, all except for dYdX focus on just the borrowing & lending side of margin trading, meaning traders have to go to other exchanges to execute the sell of the borrowed funds.

Collateralized Borrowing

Currently, all relevant decentralized lending platforms use a form of borrowing called collateralized borrowing. Collateralized borrowing means that borrowers must lock up collateral of greater value than the value of their borrow. The collateral serves to ensure lenders will be repaid even if the borrower never repays the loan.

For example, say you want to borrow $100 worth of ZRX, you would have to lock up more than $100 worth of collateral in another asset. Say you choose to lock up $150 worth of ETH. Now if you default on your loan, the lender who lent you $100 of ZRX can just seize your ETH, which is worth even more.

However, the price of both ZRX and ETH can change over time. Say the price of ETH falls, and now your collateral is only worth $90. Now the lender could not get their money back by seizing your ETH anymore, because it wouldn’t be worth as much as your debt. This is where the concept of a liquidation comes in.

A liquidation is when your borrow is automatically repaid by selling off some of your collateral to buy back the asset you owe to the lender. Liquidations occur when your borrow falls below some required level of collateralization (usually between 115%–150%).

Decentralized Lending Platforms

MakerDAO is both the most complex and widely used decentralized lending platform available today. MakerDAO is the creator of DAI, which is a cryptocurrency with a target price of $1 (known as a stablecoin).

On MakerDAO, there are no lenders, and the only asset available to borrow is DAI. Borrowers can borrow a newly created supply of DAI by locking up ETH as collateral, and must maintain a 150% collateralization ratio. The interest rate on DAI is global, and is set through governance by MKR token holders. The interest rate has recently been fairly volatile, increasing from 2.5% to 19.5% in a bit over a month.

Opening a MakerDAO CDP via cdp.makerdao.com

Assets: DAI (Borrow Only), ETH (Collateral Only) [MakerDAO plans to add more collateral assets with their upcoming release of muli-collateral DAI]

Interest Rates: Variable. Set through governance by MKR token holders

Collateral Requirement: 150% minimum

Liquidation Penalty: 13%

Compound uses a money market based approach, with global pools of capital for each supported asset. Each asset has a global borrow and lend interest rate which all borrowers pay & lenders earn. These interest rates are variable and set algorithmically based on the percent of each pool that is being borrowed.

Borrowers must collateralize their accounts with 150% of the value being borrowed. For example if you want to borrow 1 ETH from Compound, you could deposit 225 DAI into the Compound DAI pool and then borrow 1 ETH from the ETH pool (assuming 1 ETH = 150 DAI — this would be 150% collateralized). Borrows on compound have unlimited duration.

Borrowing REP & Lending DAI on compound.finance

Assets: DAI, ETH, ZRX, REP, BAT

Interest Rates: Variable. Set algorithmically based on supply & demand

Collateral Requirement: 150% minimum

Liquidation Penalty: 5%

Dharma utilizes peer-to-peer lending to match individual lenders directly with borrowers. On Dharma, borrowers and lenders enter into fixed term / interest rate loans.

Lenders can use Dharma to offer fixed-term loans of up to 90 days, and start earning interest only after they are matched with a borrower. Lenders’ funds are locked for the duration of the loan.

Borrowers on Dharma lock up collateral equal to 150% the value of the assets being borrowed. Borrows have a maximum 90 day duration, and fixed interest rates for the duration of the loan.

Borrowing DAI on dharma.io

Assets: DAI, ETH

Interest Rates: Fixed. Rates set centrally by Dharma

Collateral Requirement: 150% initial, 125% minimum

Liquidation Penalty: Unclear (?)

The main difference between dYdX and the previously mentioned lending platforms is dYdX natively supports trading in addition to borrowing / lending, meaning that traders can margin trade without leaving the platform for another exchange. dYdX’s product is targeted at margin traders, and is more complex than either Compound or Dharma, while also supporting more functionality.

Under the hood dYdX uses a pooled based lending approach and algorithmic variable interest rates similar to Compound. There are no lockup periods or maximum durations while lending on dYdX.

On dYdX borrowers must lock up 125% collateral, meaning dYdX offers the highest leverage (up to 4x) of any decentralized lending platform. Borrows / positions on dYdX are limited to 28 days.

Opening a Leveraged Long position on trade.dydx.exchange

Assets: DAI, ETH, USDC

Interest Rates: Variable. Set algorithmically based on supply & demand

Collateral Requirement: 125% initial / 115% minimum

Liquidation Penalty: 5%

Conclusion

Within the past year we’ve seen the emergence of quality decentralized lending platforms. The ability to borrow and lend on a completely open platform is a fundamental advancement in financial markets, and is already seeing real volume today.

Emiri, Apr 2022

This deep-dive is a handbook that will cover all the different types of AMMs that currently exist in the DeFi ecosystem.

2021 was the year of the L1 trade. From SOLUNAVAX at the beginning of the year to FOAN towards the end of the year (shoutout white wolf), every L1 caught a bid. Something even a blind person could see is that whenever a new L1 caught a bid, there would be a massive rotation of capital into that ecosystem. Everytime that happened, the native DEX always performed the best.

DEXs are a cornerstone piece of any DeFi ecosystem. Majority of the liquidity within a DeFi ecosystem will flow through its native DEX. As you may know, DEXs use something called an Automated Market Maker (AMM) to facilitate smooth on-chain trading. The innovation in the AMM space over the last 3 years has been rampant. Everyone is trying to make the next AMM that is faster, more capital efficient, more gas-optimized, and better for Liquidity Providers (LPs). I was doing some research and wanted an organized list of all the different types of AMMs out there, but the information was very fragmented. So I said fuck it, and decided to make my own handbook that dives into all the different types of AMMs that are currently out there. I will only be covering the AMMs that are truly unique from one another, I will not be covering forks because I’ll probably have kids by the time I finish writing. Why AMMs?

If you have only interacted with CEXs or are from TradFi then the concept of an AMM might be new to you. All centralized exchanges use something called a Centralized Limit Order Book or CLOB for short. A CLOB simply matches bids and asks according to price and time priority. Users can make 2 types of orders here, a limit order & a market order. This works well for centralized exchanges, so why do we need AMMs for DEXs?

The first problem is gas fees. Most of us have probably lost mini-fortunes in gas fees. Using a CLOB on-chain would probably cost the equivalent of a small countries GDP in gas fees. Constantly adjusting orders and placing orders would simply not be possible because you will get charged gas fees on every single transaction.

Another problem is frontrunning. The transparent nature of the blockchain becomes a problem for CLOBs. It is not a problem for market orders since AMMs only allow market orders, the problem is for limit orders. Suppose a large trader has a limit order set for a specific market, their orders will be streamed to every market participant and will most certainly be frontrun. It becomes extremely difficult to execute trades the way you want. In centralized exchanges, these orders are mostly obfuscated for the end market participant which makes frontrunning difficult.

In comes AMMs.

What are AMMs?

If you already know how AMMs work, then skip this section

To facilitate gas-efficient trading on-chain, AMMs use something called liquidity pools. A liquidity pool is basically a smart contract that holds the tokens of a trading pair in a certain ratio. To get liquidity into these pools you have users called liquidity providers or LPs. Suppose the trading pair is ETH/USDC. LPs are incentivized to provide equal amounts of liquidity to this pool (usually 50/50) and in return they get an LP token. This LP token represents their position in the liquidity pool and it entitles them to receive a portion of the trading fees generated from the pool. When LPs want to take their liquidity out, they simply burn the LP token. Every time a swap is facilitated by a liquidity pool, there will be a price adjustment which is determined by a deterministic pricing algorithm. This is the Automated Market Maker. Now traders can directly buy and sell from these different liquidity pools through market orders making an on-chain trading experience feasible.

As time has gone on, different issues were encountered with AMMs which drove devs to make their own versions of this model. This deep-dive will look at all the different variations that exist.

This will be the order of this guide, so you can skip go to the AMMs you want to read about

1. Uniswap

2. Curve Finance

3. Balancer

4. Bancor

5. Crocswap

6. Muffin

7. Pendle

8. Primitive

9. Cowswap

10. Osmosis

11. Dopex

12. YieldSpace

13. Sudoswap

1) Uniswap

Uniswap has been live since November 2018. Since it’s been live, it has gone through 3 stages of evolution. From V1 to V3, the team has always tried to bring innovative solutions to make the user experience better. Uniswap V1:

The first version of Uniswap established the general structure of how it will work. They have basic liquidity pools where each pool is its own smart contract. Only two tokens can be swapped in these pools, and when LPs provide liquidity they have to do so in a 50/50 ratio. So if the pair is ETH/DAI, they have to provide both tokens in equal proportions. For doing so, the LPs on Uniswap receive rewards, there is a standard 0.30% trading fee collected by Uniswap across all pools which are proportionally redistributed to LPs. The AMM model used by Uniswap is called the “Constant Product Market Maker”. This follows the infamous x*y=K formula. This formula basically means that the product of the quantities of the two tokens in a pool must always remain constant. At the time of release, Uniswap was one of a kind. But they had one major issue with V1. It only supported ERC-20/ETH pairs. Uniswap V2:

While ERC-20/ERC-20 swaps was one of the main features, there were other notable features. The first one was on-chain price feeds through the use of price oracles. Price oracles constantly feed the smart contract information about the price of a certain asset. The oracles were made to be resistant to price manipulation and fulfilled the purpose of improving the user experience. The other feature was flash swaps, it allows a user to receive the output tokens from the contract before giving the input tokens. While it may seem counter-intuitive to give tokens before they’ve been paid for, Ethereum transaction are atomic so if the user does not pay the input token amount by the end of the transaction, the transaction will get rolled back. Flash swaps are good for capital free arbitrage or instant leverage.

While the Uniswap team could’ve been satisfied with their success so far, they wanted to take it a step further to improve the user experience.

Uniswap V3:

V3 came a year after V2 with the main focus being on capital-efficiency. This is done through concentrated liquidity & multiple fee tiers. Previously, when LPs would provide liquidity to a pool it would be in the 0 to infinity price range. Their liquidity would be distributed evenly across the price curve. This is not efficient because all the liquidity does not get used. Take the example of a USDC/DAI pair. This pair would mostly trade within the $0.99 to $1.1 price range. With liquidity being distributed evenly across the curve, majority of it is not being used. The same can applied for any other token pairs.

Hence, they introduced concentrated liquidity. With this LPs can pick a range within which they would like to provide liquidity and can keep adjusting their position based on market conditions. For our above example most LPs would keep their range as $0.99 to $1.1. But for an ETH/DAI pair for example, some LPs may choose a $2k to $3k range while others may choose a $2.8k to $3.2k. The tighter the range made by the LP and the longer price trades within their range, the more fees they accumulate as rewards. So LPs are incentivized to actively manage their positions to keep the Uniswap AMM very capital-efficient.

2) Curve Finance

Curve is a DeFi powerhouse, it’s such an integral part of DeFi that a literal war has been fought over it. Let’s look at why.

The Plain pools are like any other liquidity pool, they facilitate swaps between ERC-20 tokens. Some pools have more than 2 tokens in them, the most popular being the tripool (DAI/USDC/USDT). In the lending pools, the underlying tokens are lent out to different protocols such as Compound or Yearn and a wrapped version of the token can be traded in the lending pools. Suppose the token is DAI, if it’s cDAI then it’s on compound but if it’s yDAI then it’s on Yearn. This way you can earn lending interest as well as trading fees. The metapool is a way for LPs of the plain pools mentioned above to earn additional trading fees by depositing LP tokens into the metapool. In the metapool, stablecoins are paired against base pool LP tokens. Suppose you are a LP for the tripool. You receive the LP token 3CRV. You can now deposit this 3CRV token into the GUSD metapool. People can trade this, but LPs get the new LP token gusd3CRV which can be put in gauges to earn additional rewards. Factory pools were made in collaboration with Yearn, it is basically a permissionless way for anyone to create their own metapool. This can be an individual or a protocol but it is most often a protocol.

The Gauges responsible for giving LPs their CRV rewards. The gauges monitor the activity and usage by LPs and automatically assign a portion of the CRV inflation to be directed to LPs. Through this they can lock their CRV for the infamous veCRV to get more governance control.

Curve has now launched a V2 of their protocol. V2 involves the launch of crypto pools. A new way to trade non-pegged assets with low risk. The whitepaper is extremely complex but I’ll try and simplify it. They basically use concentrated liquidity just like Uni V3, but for this AMM rather than concentrating liquidity around price = 1 like they do for stableswaps, it is concentrated around current price. The additional feature is their internal oracle which is used to automatically adjust the range where liquidity is provided rather than having users do it manually like they do on Uni V3.

3) Balancer

The team at Balancer saw the constant product market making formula of having two assets in a pool at a fixed ratio as problem. There was no customizability and lack of options. Hence, Balancer created smart pools. The Balancer smart pools use the constant mean formula. This allows the creation of liquidity pools with upto 8 tokens with no fixed ratios and varying weights allocated to each tokens. This opens up options for both traders and liquidity providers because there is more customizability. This customizability also means that there is third player in the Balancer AMM, not only traders and LPs. They are the controllers. They are responsible for managing the pool. This typically involves just rebalancing the weights depending on market demand and volatility. An added benefit is that it very easy to create your own pool. Anybody can do it. Just go to “pool management” tab and plug in your wallet, provide the liquidity and you’re good to go. This is good for individuals if there is no pool up to their liking or if they see an opportunity to create pool that will be demanded by others. Projects can also use this feature to attract liquidity.

4) Bancor

Bancor is an OG DeFi protocol, they created the first ever AMM. Their AMM is called omnipool. Unlike other AMMs where you can create a pool with any token, Bancor’s omnipools require all tokens to be paired against their native token BNT. The problem with traditional AMMs is fragmented liquidity, some pairs will have deeper liquidity than other pools which makes the trading experience worse. This is because there is no common denominator. For example, a random token (Token ABC for example) will have deeper liquidity when paired against ETH, and weaker liquidity when it’s paired against USDT or USDC. To combat this liquidity issue, Bancor decided to have a common denominator in the form of BNT. Now users have a central liquidity pool to trade from which gives every token equally good liquidity thereby making the overall trading experience much smoother.

However, Bancor saw 2 key problems with its AMM design, involuntary token exposure and impermanent loss. Hence, they created Bancor v2.1

Bancor v2.1 allows single sided exposure and provides impermanent loss insurance. For single sided exposure, Bancor allows LPs to provide only one token rather than forcing them to provide both. The LPs will have 100% exposure only to the token which they have provided to the pool and will accrue fees and rewards based on that. Impermanent loss insurance works like this, if a user deposits $100k of a token into a pool, Bancor matches it with a $100k deposit of BNT. Now both the user and the protocol are accruing fees and rewards. Once the user withdraws liquidity both the user and Bancors LP token gets burned at the same time. The protocol checks to see if the user has suffered any impermanent loss and accordingly distributes the accumulated fees from their LP position to the user.

5) Crocswap

Crocswap is some big brain stuff, it took me a lot of time to wrap my head around this one but I finally got it.

It’s an Ethereum-based AMM where the entire DEX is run on one single smart contract. Typically, DEXs are run by having a separate smart contract for each liquidity pool, but by running the entire DEX on one contract you will see major gas & tax savings since tokens aren’t always being reshuffled between contracts. Within this smart contract, there are multiple lightweight data structures which represent individual liquidity pools. This opens up the doors to smoothly conduct many different multi-pool operations.

Crocswap also makes significant improvements for LPs. They have a dual-liquidity model. Classical liquidity (or ambient liquidity) of Uniswap v2 and concentrated liquidity of Uniswap v3 both co-exist in Crocswap. Combined with this, LPs provide liquidity through range orders. Basically, there is a pre-determined price grid where LPs can place orders. As price hits the different ticks, liquidity is added or removed depending on the nature of the order at that tick. This LP model has two key benefits, the first is that it makes the ambient liquidity LP tokens fungible, and the second is that the LPs position automatically compounds rewards rather than having to manually collect rewards. All in all, Crocswap combines the best features of multiple different AMM variations, and puts it into a sleek, cheap, & efficient one contract exchange design.

6) Muffin

Muffin (formerly Deliswap) focuses on two key features for their AMM. Concentrated liquidity & multiple fee tier pools.

We already know what concentrated liquidity is, it’s the same mechanism used by UniV3 and Curve V2. An issue that other AMMs have is fee-tiers. Some AMMs like Uniswap have a fixed fee tier at 0.30% while other AMMs offer 1 or 2 fee-tiers per pool. If you want more fee-tiers then you have to make a new pool of the same token which fragments liquidity. Muffin wants to offer LPs more granular control over where they provide liquidity to generate the most fees rather than having liquidity distributed evenly across all fee-tiers. Hence, they created the “multiple fee-tiers per pool” model. Within one pool, there can be up to 6 fee-tiers. Each fee-tier will act like its own inner-pool which will have its own liquidity and price. Depending on the state of the market, LPs can choose what price range and fee-tier they would like to provide liquidity in. This creation of more choice makes the system more capital-efficient and generates more rewards for LPs.

7) Pendle

The ultimate aim of the AMM is to minimize time-dependent impermanent loss that arises from using tokens with a time-decay. To do this, they use a formula similar to Uniswaps x*y=k but they just add an additional account for weight and time. So the pool starts off with a curve similar to uniswaps, but as time passes the curve shifts in such a way to ensure that the price of the YT keeps falling as time passes. The curve shifting is governed by a time-decay pricing model which is similar to the options pricing model in tradfi.

8) Primitive

Primitive is a spot & derivative token exchange with the key innovations coming from the derivative token side. For the spot exchange, they use the concentrated liquidity method. When it comes to derivatives, they use a variation on AMMs called Replicating Market Maker (RMM). RMM is a way to make the on-chain derivative trading experience more user-friendly and more efficient. The Primitive team saw two problems, current on-chain derivatives depend solely on oracles. Oracles present security risks and can be a central point of failure. The other issue is that LPs get a LP token to represent their share in the pool which is essentially a derivative, while there are trading markets for these LP tokens, none of these derivative are inherently financially useful. Using the RMM, you can create or offer any type of derivative payoff products. I’ll go over some examples outlined by the team to give you some clarity. In lending markets like Fuse these LP tokens can be shorted which expands the derivative payoff range for call & put options, for liquidity strategy protocol like Ribbon and Charm you can create a basket of LP tokens to create any type of novel DeFi product, lastly you can also create binary options by being able to sell the rights to one side of the LP token. 9) Cowswap

Cowswap is built on top off the CoW protocol. To understand Cowswap lets understand CoW protocol first.

There are 2 key features that make the CoW protocol unique, batch auctions & Coincidence of Wants (Hence the name CoW). Batch auctions are when orders are placed off-chain after which they are aggregated into batches to eventually be settled on-chain. The benefit of batch auctions is that it simplifies the Coincidence of Wants (CoWs). CoWs are when two parties hold an item that the other wants, they can exchange them directly without the need for any type of intermediary. Similarly, when two traders hold an asset that the other wants they can exchange them easily through batch auctions. This means that CoW protocol doesn’t need direct access to on-chain liquidity which gives them significant MEV protection.

Built on top of this is the DEX Cowswap. It acts as a meta DEX aggregator or a DEX aggregator of DEX aggregators because it gives users the best price and cheapest execution across all AMMs and aggregators on Ethereum. To do this it doesn’t use any existing AMM or constant function market maker model, it replaces these mechanisms with a new party called “solvers”. The solvers are incentivized to submit the most optimal settlement solution for a designated batch. The solvers are incentivized to compete against each other to find the most optimized solution. The winning solver is then appropriately rewarded with tokens. Anyone can become a solver after fulfilling some of the minor prerequisites.

10) Osmosis

Osmosis is an AMM native to the Cosmos ecosystem. For those of you who aren’t familiar with Cosmos, they follow a hub and zone architecture where each zone is an application specific blockchain. Osmosis is one of these Zones which lives in an interoperable Cosmos ecosystem. So It’s an L1 AMM with IBC built-in which means it is interoperable with every other zone in the Cosmos ecosystem. The main feature of Osmosis is customizability.

Other AMMs have most things hard-coded. They either have a “constant product” or “constant sum” or “constant mean” or any other type of formula, the bonding curves are pre-determined, the fee structures and tiers are also pre-determined. Osmosis essentially gives users the tools to create their own AMM pool with maximum customizability. People can set their own parameters for bonding curves, have two token pools or multi-weighted asset pools, they can use any of the already established AMM formulas or make their own one. For LPing the rewards are decided through governance, when coupled with customizability this creates different strategic incentive games.

With this customizability for liquidity pools, there needs to be some form of governance for decision making. So in Osmosis, the LP tokens received don’t only accrue rewards but they also represent the share of decision-making power a LP has for that specific pool. The longer a LP is locked into a pool the more rewards and governance power they get.

Essentially, with this customizability, Osmosis creates an “AMM as a serviced infrastructure” model. With the complexity of new types of assets in DeFi, you need options in terms of finding the optimal AMM for each type of asset while not requiring people to take on the massive task of building their own AMM.

11) Dopex

Dopex is a very ambitious protocol. Their aim is to create a crypto options protocol that runs entirely on-chain.

At the heart of Dopex is their Single Staking Options Vault (SSOVs). These SSOVs allow users to deposit their assets in the contract, the protocol then sells these assets as call options to buyers while the depositor earns rewards for locking up their assets. Other than SSOVs Dopex also has multiple other types of pools to allow users to trade options. They have pools including Options pools and volume pools. Underlying this architecture is the Dopex AMM.

Before we get into the AMM, you need to know what a volatility smile is. A volatility smile is a result of plotting the strike price and implied volatility of a group of options onto a graph. When implied volatility is plotted at each strike price, the graph forms a smiley face. With that established let’s get into the AMM.

The Dopex AMM uses a black-scholes pricing model. This pricing formula gives traders a theoretical price of an option by using option volatility as an input. Using option volatility helps understand how the price of an asset will move in the future. So Dopex uses chainlink adapters to get information on implied volatility and asset prices, this information is used with the black-scholes pricing formula to determine the volatility smile. This gives users accurate option prices on-chain while taking into account real market risk and behavior.

12) YieldSpace

The Yield Protocol created the concept of fyTokens. These fyTokens are synthetic tokens that have a certain maturity date. Once this maturity date is reached, the fyToken holder can redeem the original asset or target asset at its original price. Before maturity the price will be free-floating. Suppose the target asset is DAI, then the fyToken will be fyDAI which will be free-floating until maturity when it can be redeemed for 1 DAI. For those familiar with tradfi, it is similar to a zero-coupon bond. It is a debt security instrument where traders can buy the bond at a significant discount and then redeem it for its face-value or par value at maturity. The difference between purchase price and par value is the profit. While existing AMM formulas can be used to trade tokens with price maturity, they are by no means optimal. Using Uniswaps constant product formula would lead to higher price-impact and fees for traders when close to maturity making it capital-inefficient. Using the constant sum formula doesn’t allow for price-discovery at all which will discourage trading. Hence, the YieldSpace AMM uses something called the “constant power sum formula”.

X^1-t + Y^1-t = k

X represent the reserves of the target token, Y represents the reserves of the fyToken, and t represents the time to maturity. This formula is basically a combination of the constant product and constant sum formula making it optimal for trading tokens with maturity dates. When t=1 the formula acts like a constant product formula and when t=0 it acts like a constant sum formula.

This is done to ensure that the marginal interest rate of the fyToken that is offered by the pool is equal to the ratio of the fyToken and original token reserves in the pool. Suppose the reserves in the pool are 110 fyDAI and 100 DAI the marginal interest rate offered will be 10%. So the allocation of fyToken to the pool changes according to the current interest rate offered by the fyToken. If interest rate rises then fyToken allocation rises and if interest rate falls then fyToken allocation falls.

13) Sudoswap

I saved Sudoswap for last because this is the most unique AMM. It’s an AMM for NFTs. Rather than using off-chain orderbooks, Sudo uses on-chain liquidity pools to allow low slippage swaps for NFTs. There are 3 main types of pools that users will interact with. There is a buy-only pool, a sell-only pool, and a both pool. Each pool is a separate contract managed by a single address. A buy-only pool will always have ETH and will be ready to give a quote to buy the NFT, a sell-only pool will always contain NFT and will always be ready to give a quote to sell the NFT for ETH, and in both pool, you have both a whole number NFTs and ETH, users can the either buy from that pool which increase the amount of ETH in the pool, or sell which decreases the amount of ETH in the pool. This means that each NFT will have multiple pools for it.

From the LP side, they can better control of price ranges because the quotes that each pool gives is determined by a bonding curve. The bonding curve that each pool uses is determined and stated when the pool is created. There are three types of bonding curves currently used. A constant bonding curve means that the pool is always quoting the same price, a linear bonding curve means that the price quoted by the pool increases/decrease linearly with buys/sells, and the exponential curve works by increasing/decreasing price by a certain percentage depending on the buys/sells.

The main benefits of the Sudo AMM is that the market becomes more liquid, users get an instant quote on their NFTs rather than waiting for a bid and then settling by simply accepting the best offer on Opensea. It is also a good decentralized alternative. Platforms like Opensea are basically web2 platforms, they have all the control. We have seen them randomly blacklist accounts and prevent the selling of certain NFTs as they please. A problem with Sudoswap is that it is not as gas efficient as the current marketplaces. Even though the team has done all that they can to minimize gas expenditure, it will take a lot of work to reach to the point of being competitive with other NFT marketplaces from that aspect.

Personal Thoughts

Now that all the different types of AMMs are in one place, what I’ve noticed is that most of the innovation after Uniswap comes from improving the experience from the LP side. Liquidity is everything in DeFi, without it a product is pretty much rendered useless. Hence, making the experience easier and more profitable for LPs has a direct effect on the success of a protocol. It also helps improve capital efficiency which is a major o[point of focus for all new AMMs.

From the traders perspective, the mechanisms are pretty much figured out. Most DeFi users are very familiar with the swap experience. The main thing is for the UI to keep improving. It should be as easy as possible for traders to navigate and execute on these platforms. Something that should be added to improve the trading experience on DEXs is limit orders. As I outlined earlier, it is tough to do because of frontrunning but if a team can figure out a way to do it or a variation of it, they will be very successful.

Most of the AMMs mentioned here haven’t been launched as yet which is why AMMs like Uniswap still remain dominant, but when they launch it will be interesting to see whether the improvements for the user will be significant enough to take them away from an experience they are already familiar with.

That is all for this Deep-dive into AMMs.

If you enjoyed reading and are feeling generous then please consider donating to 0x43A5D9C141125Cd67B9268ef28C7c6a9dC15F3c9

Subscribe to the substack and follow the medium

Haseeb Qureshi, Jul 2020

Imagine a college friend reached out to you and said, “Hey, I have a business idea. I’m going to run a market making bot. I’ll always quote a price no matter who’s asking, and for my pricing algorithm I’ll use x * y = k. That’s pretty much it. Want to invest?”

You’d run away.

Well, turns out your friend just described Uniswap. Uniswap is the world’s simplest on-chain market making operation. Seemingly from nowhere, it has exploded in volume in the last year, crowning itself the world’s largest “DEX” by volume.

If you haven’t paid close attention to what’s happening in DeFi in the last year, you’re probably wondering: what is going on here?

For the uninitiated: Uniswap is an automated market maker (AMM). You can think of an AMM as a primitive robotic market maker that is always willing to quote prices between two assets according to a simple pricing algorithm. For Uniswap, it prices the two assets so that the number of units it holds of each asset, multiplied together, is always equal to a fixed constant.

That’s a bit of a mouthful: if Uniswap owns some units of token x and some units of token y, it prices any trade so that the final quantities of x and y it owns, multiplied together, are equal to a fixed constant, k. This is formalized as the constant product equation: x * y = k.

This might strike you as a weird and arbitrary way to price two assets. Why would maintaining some fixed multiple between your units of inventory ensure that you quote the right price?

Uniswap by example

Let’s say we fund a Uniswap pool with 50 apples (a) and 50 bananas (b), so anyone is free to pay apples for bananas or bananas for apples. Let’s assume the exchange rate between apples and bananas is exactly 1:1 on their primary market. Because the Uniswap pool holds 50 of each fruit, the constant product rule gives us a * b = 2500—for any trade, Uniswap must maintain the invariant that our inventory of fruit, multiplied together, equals 2500.

So let’s say a customer comes to our Uniswap pool to buy an apple. How many bananas will she need to pay?

If she buys an apple, our pool will be left with 49 apples, but 49 * b has to still equal 2500. Solving for b, we get 51.02 total bananas. Since we already have 50 bananas in inventory, we’ll need 1.02 extra bananas for that apple (we’ll allow fractional bananas in this universe), so the price we have to quote her is 1.02 bananas / apple for 1 apple.

Note that this is close to the natural price of 1:1! Because it’s a small order, there is only a little slippage. But what if the order is larger?

You can interpret the slope at each point as the marginal exchange rate.

If she wants to buy 10 apples, Uniswap would charge her 12.5 bananas for a unit price of 1.25 bananas / apple for 10 apples.

And if she wanted a huge order of 25 apples—half of all the apples in inventory—the unit price would be 2 bananas / apple! (You can intuit this because if one side of the pool halves, the other side needs to double.)

The important thing to realize is that Uniswap cannot deviate from this pricing curve. If someone wants to buy some apples and later someone else wants to buy some bananas, Uniswap will sweep back and forth through this pricing curve, wherever demand carries it.

Uniswap sweeping back and forth through its pricing curve after a series of trades.

Now here’s the kicker: if the true exchange rate between apples and bananas is 1:1, then after the first customer purchases 10 apples, our Uniswap pool will be left with 40 apples and 62.5 bananas. If an arbitrageur then steps in and buys 12.5 bananas, returning the pool back to its original state, Uniswap would charge them a unit price of only 0.8 apples / banana.

Uniswap would underprice the bananas! It’s as though our algorithm now realizes it’s heavy on bananas, so it prices bananas cheap to attract apples and rebalance its inventory.

Uniswap is constantly performing this dance — slightly moving off the real exchange rate, then sashaying back in line thanks to arbitrageurs.

Impermanent Loss in a Nutshell

This should give you a sense for how Uniswap pricing works. But this still begs the question — is Uniswap good at what it does? Does this thing actually generate profits? After all, any market maker can quote prices, but it’s another thing to make money.

Uniswap charges a small fee for every trade (currently 0.3%). This is in addition to the nominal price. So if apples and bananas always and forever trade at 1:1, these fees will simply accumulate over time as the market maker sweeps back and forth across the exchange rate. Compared to the baseline of just holding those 50 apples and bananas, the Uniswap pool will end up with more fruit at the end, thanks to all the fees.

But what if the real exchange rate between apples and bananas suddenly changes?

Say a drone strike takes out a banana farm, and now there’s a massive banana shortage. Bananas are like gold now. The exchange rate soars to 5 apples : 1 banana.

What happens on Uniswap?

The very next second, an arbitrageur swoops in to pick off the cheaply priced bananas in your Uniswap pool. They size their trade so that they purchase every banana that’s priced below the new exchange rate of 5:1. That means they’ll need to move the curve until it satisfies the equation: 5b * b = 2500.

Running the math out, they’d purchase 27.64 bananas for a grand total of 61.80 apples. This comes out to an average price of 2.2 apples : 1 banana, way under market, netting the equivalent of 76.4 free apples.

And where does that profit come from? Of course, it comes at the expense of the pool! And indeed, if you do the accounting, you’ll see that the Uniswap pool is now down exactly 76.4 apples worth of value compared to someone who’d held the original 50 apples and 50 bananas. Uniswap sold off its bananas too cheaply, because it had no idea bananas had become so valuable in the real world.

This phenomenon is known as impermanent loss. Whenever the exchange rate moves, this manifests as arbitrageurs sniping cheap assets until the pool is correctly priced. (These losses are “impermanent” because if the true exchange rate later reverts back to 1:1, then now it’s like you never lost that money to begin with. It’s a dumb name, but oh well.)

Pools make money through fees, and they lose money via impermanent loss. It’s all a function of demand and price divergence — demand works for you, and price divergence works against you.

In retrospect, it’s incredibly elegant, one of the simplest possible products you could have invented, and yet it arose seemingly from nowhere to dominate DeFi.

The Cambrian AMM Explosion

Since Uniswap’s rise, there has been an explosion of innovation in AMMs. A legion of Uniswap descendants have emerged, each with its own specialized features.

Different curves are better suited for certain assets, as they embed different assumptions about the price relationship between the assets being quoted. You can see in the chart above that the Stableswap curve (blue) approximates a line most of the time, meaning that in most of its trading range, the two stablecoins will be priced very close to each other. Constant product is a decent starting place if you don’t know anything about the two assets, but if we know the two assets are stablecoins and they are probably going to be worth around the same, then the Stableswap curve will produce more competitive pricing.

Seeing the growth in CFMM volume, it’s tempting to assume that they are going to take over the world — that in the future, all on-chain liquidity will be provided by CFMMs.

But not so fast!

CFMMs are dominating today. But in order to get a clear sense of how DeFi evolves from here, we need to understand when CFMMs thrive and when they do poorly.

The Correlation Spectrum

Let’s stick to Uniswap, since it’s the simplest CFMM to analyze. Let’s say you want to be a Uniswap LP (liquidity provider) in the ETH/DAI pool. By funding this pool, there are two simultaneous things you have to believe for being an LP to be better than just holding onto your original funds:

1. The ratio in value between ETH and DAI will not change too much (if it does, that will manifest as impermanent loss)

2. Lots of fees will be paid in this pool

To the extent that the pool exhibits impermanent loss, the fees need to more than make up for it. Note that for a pair that includes a stablecoin, to the extent that you’re bullish on ETH appreciating, you’re also assuming that there will be a lot of impermanent loss!

The general principle is this: the Uniswap thesis works best when the two assets are mean-reverting. Think a pool like USDC/DAI, or WBTC/TBTC — these are assets that should exhibit minimal impermanent loss and will purely accrue fees over time. Note that impermanent loss is not merely a question of volatility (actually, highly volatile mean-reverting pairs are great, because they’ll produce lots of trading fees).

We can accordingly draw a hierarchy of the most profitable Uniswap pools, all other things equal.

Mean-reverting pairs are obvious. Correlated pairs often move together, so Uniswap won’t exhibit as much impermanent loss there. Uncorrelated pairs like ETH/DAI are rough, but sometimes the fees can make up for it. And then there are the inverse correlated pairs: these are absolutely awful for Uniswap.

Imagine someone on a prediction market going long Trump, long Biden, and putting both longs in a Uniswap pool. By definition, eventually one of these two assets will be worth $1 and the other will be worth $0. At the end of the pool, an LP will have nothing but impermanent loss! (Prediction markets always stop trading before the markets resolve, but outcomes are often decided well before the market actually resolves.)

So Uniswap works really well for certain pairs and terribly for others.

But it’s hard not to notice that almost all of the top Uniswap pools so far have been profitable! In fact, even the ETH/DAI pool has been profitable since inception.

This demands explanation. Despite their flaws, CFMMs have been impressively profitable market makers. How is this possible? To answer this question, it pays to understand a bit about how market makers work.

Market making in a nutshell

Market makers are in the business of providing liquidity to a market. There are three primary ways market makers make money: designated market making arrangements (traditionally paid by asset issuers), fee rebates (traditionally paid by an exchange), and by pocketing a spread when they’re making a market (what Uniswap does).

You see, all market making is a battle against two kinds of order flow: informed flow, and uninformed flow. Say you’re quoting the BTC/USD market, and a fat BTC sell order arrives. You have to ask yourself: is this just someone looking for liquidity, or does this person know something I don’t?

If this counterparty just realized that a PlusToken cache moved, and hence selling pressure is incoming, then you’re about to trade some perfectly good USD for some not so good BTC. On the other hand, if this is some rando selling because they need to pay their rent, then it doesn’t mean anything in particular and you should charge them a small spread.

So a market maker’s principal job is to differentiate between informed and uninformed flow. The more likely the flow is informed, the higher the spread you need to charge. If the flow is definitely informed, then you should pull your bids entirely, because you’ll pretty much always lose money if informed flow is willing to trade against you.

(Another way to think about this: uninformed flow is willing to pay above true value for an asset — that’s your spread. Informed flow is only willing to pay below the true value of an asset, so when you trade against them, you’re actually the one who’s mispricing the trade. These orders know something you don’t.)

The very same principle applies to Uniswap. Some people are trading on Uniswap because they randomly want to swap some ETH for DAI today. This is your uninformed retail flow, the random walk of trading activity that just produces fees. This is awesome.

Then you have the arbitrageurs: they are your informed flow. They are picking off mispriced pools. In a sense, they are performing work for Uniswap by bringing its prices back in line. But in another sense, they are transferring money from liquidity providers to themselves.

For any market maker to make money, they need to maximize the ratio of uninformed retail flow to arbitrageur flow.

But Uniswap can’t tell the difference between the two!

Uniswap has no idea if an order is dumb retail money or an arbitrageur. It just obediently quotes x * y = k, no matter what the market conditions.

So if there’s a new player in town that offers better pricing than Uniswap, like Curve or Balancer, you should expect retail flow to migrate to whatever service offers them better pricing. Given Uniswap’s pricing model and fixed fees (0.3% on each trade), it’s hard to see it competing on the most competitive pools — Curve is both more optimized for stablecoins and charges 0.04% on each trade.

Over time, if Uniswap pools get outcompeted on slippage, they will be left with majority arbitrageur flow. Retail flow is fickle, but arbitrage opportunities continually arise as the market moves around.

This failure to compete on pricing is not just bad — its badness gets amplified. Uniswap has a network effect around liquidity on the way up, but it’s also reflexive on the way down. As Curve starts to eat the stablecoin-specific volume, the DAI/USDC pair on Uniswap will start to lose LPs, which will in turn make the pricing worse, which will attract even less volume, further disincentivizing LPs, and so on. So goes the way of network effects — it’s a rocket on the way up, but on the way down it incinerates on re-entry.

Of course, these arguments apply no less to Balancer and Curve. It will be difficult for each of them to maintain fees once they get undercut by a market maker with better pricing and lower fees. Inevitably, this will result in a race to the bottom on fees and massive margin compression. (Which is exactly what happens to normal market makers! It’s a super competitive business!)

But that still doesn’t explain: why are all of the CFMMs growing like crazy?

Why are CFMMs winning?

Let’s take stablecoins. CFMMs are clearly going to win this vertical.

Imagine a big traditional market maker like Jump Trading were to start market making stablecoins on DeFi tomorrow. First they’d need to do a lot of upfront integration work, then to continue operating they’d need to continually pay their traders, maintain their trading software, and pay for office space. They’d have significant fixed costs and operating costs.

Curve, meanwhile, has no costs at all. Once the contracts are deployed, it operates all on its own. (Even the computing cost, the gas fees, is all paid by end users!)

And what is Jump doing when quoting USDC/USDT that’s so much more complicated than what Curve is doing? Stablecoin market making is largely inventory management. There’s not as much fancy ML or proprietary knowledge that goes into it, so if Curve does 80% as well as Jump there, that’s probably good enough.

But ETH/DAI is a much more complex market. When Uniswap is quoting a price, it isn’t looking at exchange order books, modeling liquidity, or looking at historical volatility like Jump would — it’s just closing its eyes and shouting x * y = k!

Compared to normal market makers, Uniswap has the sophistication of a refrigerator. But so long as normal market makers are not on DeFi, Uniswap will monopolize the market because it has zero startup costs and zero operating expense.

Here’s another way to think about it: Uniswap is the first scrappy merchant to set up shop in this new marketplace called DeFi. Even with all its flaws, Uniswap is being served up a virtual monopoly. When you have a monopoly, you are getting all of the retail flow. And if the ratio between retail flow and arbitrageur flow is what principally determines the profitability of Uniswap, no wonder Uniswap is raking it in!

But once the retail flow starts going elsewhere, this cycle is likely to end. LPs will start to suffer and withdraw liquidity.

But this is only half of the explanation. Remember: long before we had Uniswap, we had tons of DEXes! Uniswap has decimated order book-based DEXes like IDEX or 0x. What explains why Uniswap beat out all the order book model exchanges?

From Order Books to AMMs

I believe there are four reasons why Uniswap beat out order book exchanges.

This is not a small point. Once next generation high-throughput blockchains arrive, I suspect the order book model will eventually dominate, as it does in the normal financial world. But will it be dominant on Ethereum 1.0?

The extraordinary constraints of Ethereum 1.0 select for simplicity. When you can’t do complex things, you have to do the best simple thing. Uniswap is a pretty good simple thing.

Third, it’s extremely easy to provide liquidity to Uniswap. The one-click “set it and forget it” LP experience is a lot easier than getting active market makers to provide liquidity on an order book exchange, especially before DeFi attracts serious volume.

This is critical, because much of the liquidity on Uniswap is provided by a small set of beneficent whales. These whales are not as sensitive to returns, so the one-click experience on Uniswap makes it painless for them to participate. Crypto designers have a bad habit of ignoring mental transaction costs and assuming market participants are infinitely diligent. Uniswap made liquidity provision dead simple, and that has paid off.

Recall that one of the three ways that traditional market makers make money is through designated market making agreements, paid by the asset issuer. In a sense, an incentivized pool is a designated market maker agreement, translated for DeFi: an asset issuer pays an AMM to provide liquidity for their pair, with the payment delivered via token airdrop.

But there’s an additional dimension to incentivized pools. They have allowed CFMMs to serve as more than mere market makers: they now double as marketing and distribution tools for token projects. Via incentivized pools, CFMMs create a sybil-resistant way to distribute tokens to speculators who want to accumulate the token, while simultaneously bootstrapping a liquid initial market. It also gives purchasers something to do with the token—don’t just turn it around and sell it, deposit it and get some yield! You could call this poor man’s staking. It’s a powerful marketing flywheel for an early token project, and I expect this to become integrated into the token go-to-market playbook.

That said, I don’t believe Uniswap’s success will last forever. If the constraints of Ethereum 1.0 created the conditions for CFMMs to dominate, then Ethereum 2.0 and layer 2 systems will enable more complex markets to flourish. Furthermore, DeFi’s star has been rising, and as mass users and volumes arrive, they will attract serious market makers. Over time, I expect this to cause Uniswap’s market share to contract.

Five years from now, what role will CFMMs play in DeFi?

In 2025, I don’t expect CFMMs the way they look today to be the dominant way people trade anymore. In the history of technology, transitions like this are common.

In the early days of the Internet, web portals like Yahoo were the first affordance to take off on the Web. The constrained environment of the early Web was perfectly suited to being organized by hand-crafted directories. These portals grew like crazy as mainstream users started coming online! But we now know portals were a temporary stepping stone on the path to organizing the Internet’s information.

The original Yahoo homepage and the original Google homepage

What are CFMMs a stepping stone to? Will something replace it, or will CFMMs evolve alongside DeFi? In my next post, entitled Unbundling Uniswap, I’ll try to answer this question.