Fund I/O can be used not only for digital goods, but for anything that has large fixed costs and low marginal costs. This includes physical goods that have a large social impact, such as vaccines. Here is how this might work.

Suppose a vaccine for a certain disease has already been developed. Now it needs to be produced at scale to reach millions of people world wide. The problem is vaccine production requires a substantial investment to get going. This means, if a supplier were to produce just 100,000 doses of the vaccine, each dose might be prohibitively expensive. But at a scale of 10 million doses, each dose could be very cheap. As long as the market is small, the price for the vaccine will be very high so that only few people will be able to afford it. But once the market grows, the price will drop, and many more people will be able to afford the vaccine.

This creates a chicken and egg problem: To get the price down, many people would need to buy the vaccine. But for many people to buy the vaccine, the price would need to drop. So how can we get the market started?

One approach to getting the market started are advance market commitments. A couple of large buyers of the vaccine, such as governments or charitable donors, make a joint commitment to buying a large number of doses at a specified price. The idea is that by guaranteeing a certain demand for the vaccine in advance, the market can be jump-started at a volume attractive to suppliers and at a price attractive to buyers.

There is a catch, however: Because, the price for the vaccine will drop over time, buyers have an incentive to wait. Those who start buying early will pay a premium. It you wait, you pay less, and the later you get in, the more you save. Even if everyone wants to buy the vaccine, those who manage to start buying last will pay the least. Early adopters effectively subsidize late entrants.

You can imagine that this leads to quite a lot of maneuvering over who gets to be the last. Especially because the market for governments buying vaccines has several features that exacerbate the problem: First, very large sums of money are at stake. Second, buyers (such a governments) are often frugal, by necessity. Third, implicitly subsidizing other parties may be unpopular, even if it is for a good cause such as getting more vaccines to more people. And finally, the only benefits of entering early are measured in terms of impact (saving lives) and not profit (return on investment), which often makes spending money harder instead of easier - unfortunately. Advance market commitments can help a lot in this setting: by forming a coalition of first movers, the financial disadvantage to buying early is reduced considerably. But the incentive to get in later remains, and this makes it often very hard to establish such a coalition.

Here Fund I/O can help. In fact, Fund I/O was designed to solve exactly this chicken-and-egg problem. Using Fund I/O, buyers do not have an incentive to wait. To achieve this, all you need to do is apply the refund mechanism used in the second phase of the Fund I/O model. Buyers can purchase immediately and still benefit from economies of scale as the market matures. Through the refund mechanism, they can rest assured they will benefit from decreasing prices in just the same way as a late entrant.

This solution makes jump-starting markets much easier, by offering a smooth transition from a low volume/high price market to a high volume/low price market. Moreover, it does not require complicated negotiations aimed at getting buyers to commit to making purchases now, even though it would be in their financial interest to wait. The Fund I/O mechanism guarantees the same financial benefits as an advance market commitment involving all potential buyers, present and future, but it can be set in motion by just a few interested parties right away. Agreements are not made at a negotiating table but on a platform of exchange that incentivises all participants to reveal their true valuation for the vaccine.

Of course there are many more details to be addressed here, both as far as the theoretical model and practical issues are concerned. I will deal with these as I go along. In the next post, I’ll start with a detailed example of this use-case of Fund I/O in action.

Why Funding Open Source Software is Difficult on Kickstarter

Crowdfunding has taken off, for physical goods as well as for software-related products like games. So why do we see so few open source project take advantage of crowdfunding? In fact, Kickstarter is based on the Street Performer Protocol (SPP), a method for raising funds that was originally intended as a model for financing open source software, and that had some early successes such as making Blender open source. Could it be that rewards-based crowdfunding as on Kickstarter simply does not work as well for open source software?

There is one subtle difference between open source and closed source projects on Kickstarter, and this difference may be crucial: The successful game projects on Kickstarter have no intention of giving away their core product for free. This makes pledges pre-sales instead of donations. Open source projects on the other hand announce right away that their product is going to be free. Does this influence backer behavior?

It is certainly true that many crowdfunding campaigns are successful because some altruistic backers pledge huge amounts of money to the project - much more than they would need to pledge to get a copy of the product for themselves. But projects rarely can be funded through such altruistic contributions alone. Take a look at Shrouds of the Avatar, for example: the large majority of backers pledged the minimum amount required to receive a copy of the game. These backers are obviously price conscious and don’t pledge more than they have to. I will call these backers rational, in the economic sense. This means that they are self-interested and act to maximize their own welfare.

This leads me to the following working hypothesis:

Crowdfunding projects need contributions from both rational and altruistic backers to raise large amounts of money.

This would explain why open source projects appear (so far) to be less successful than other creative projects at raising money via crowdfunding: Because open source software is (going to be) free, rational backers don’t pledge anything.

In fact, there is research in the economics literature on the optimal behavior of an entirely rational backer who is faced with the decision of backing an open source project. The amount he or she is willing to pledge is determined by the probability of their pledge being pivotal and getting the project funded. Backers correctly observe that this probability decreases with the total number of backers interested in a project. A consequence is that - if all backers were rational - the amount of money open source projects can expect to raise grows only proportional to the square root of the number of people interested in the project. In contrast, the amount a closed source project can raise grows linearly in the number of people interested in the project. This can quickly lead to a difference of several orders of magnitude!

(Now, before anyone gets a wrong impression: I do not think that people behave entirely rationally or that they should behave rationally. It is vital for our society that people act out of altruistic motives. And it is great that crowdfunding has found a way to get altruism to scale, so that we can make great things happen. I think we should look for more ways in which we can integrate altruism into our economic activities. All I am saying is that there is a reason that our entire economic system is built on the assumption that people tend to behave rationally. Our goal should be to find economic models that incorporate both altruistic and rational motives.)

Fund I/O as a Model for Funding Open Source Software

Fund I/O is a new crowdfunding model based on the Average Cost Threshold Protocol (ACTP). (If you are not yet familiar with Fund I/O, you may want to check out this example before continuing here.) When I developed the ACTP, one of my goals was to find a crowdfunding model that would work for open source software. And in many ways Fund I/O achieves precisely that:

• Most importantly, Fund I/O gets rational backers to reveal how much they would truly be willing to pay for the software.
• This allows open source project to raise as much money as closed source projects.
• Moreover a project can raise more money on Fund I/O than using the Kickstarter model, because it lets the market determine the optimal price of the product.
• At the same time, Fund I/O still allows altruistic backers to dontate as much as they want to the project. (This requires Fund I/O to be used with the option of allowing each backer to set their own minimum price level.)
• Finally, Fund I/O provides a clear path towards the relase of the software product under an open source license.

Now, you may have noticed that I did not quite say Fund I/O “works” for open source software. In fact, I do not think the Fund I/O model gives me everything I would personally want from an open source funding model. However, I do think it works much better than anything that is currently out there. So let me first explain in what way it does work for open source software. In the next section I will then go on to explain the limitations I see.

Fund I/O works great for open source projects if your project meets the following assumptions:

• Your goal is to finance the creation of version 1.0 - no more.
• You are willing to release your software under a closed license at first.

Then you just run through the three phases of the Fund I/O mechanism (with individual minimal price levels) and at the end:

• you have completed the software you wanted to write,
• you have released the software and its source code under an open source license, free for everyone to use and modify,
• your costs were covered right from the start and
• you even made a certain profit that you can determine in advance.

The release under an open source license happens if and only if there is enough altruism among your customers to finance the costs of production through voluntary donations. However, this absolutely not the same as simply using the SPP. First of all, your project gets made if there is enough rational interest in your software. It just won’t be open sourced unless there is enough altruistic interest. Second, Fund I/O achieves a smooth transition from the state where there are few users who would need to donate very large amounts to the state where there are many users who need to donate just a couple of dollars. This lessens the burden on any individual donor considerably. Moreover, the donation is a sunk cost for the customer: donating means forgoing future refunds instead of paying money now. All of these factors make it much easier to gather the required donations, than using the SPP, even if Fund I/O is used with individual minimal price levels. If a fixed minimal price level is used for everyone, then gathering the required donations gets easier still.

Caveats

Now, I believe the above could work great for developing, say, an open source game. But for open source software I see a couple of caveats.

First of all, the user base of the software has to be willing to use closed source software during the second phase of the protocol. If a substantial number of users will never run anything but free software, then you are of course out of luck. I do not think this will be a problem; it was not a problem for Light Table during its crowdfunding campaign.

Second, it has to be reasonable to make the software non-free during the second phase. This means that the software is for sale and according to the license users are not allowed to pass copies of the software along to their friends. This is a stronger restriction than just not releasing the source code and may turn off more of the free software crowd. More importantly, this limits viral marketing effects during the crowdfunding and sales phases. People have to buy the product to try it out. (Even though a demo could certainly be offered.)

Of course the whole point of Fund I/O is that because of the refund mechanism, there is less of a disincentive to buying than in a classical sales context. In particular, if a customer sets their minimal price level to zero, then they get a full refund in case the software is released as open source. But in a world where many people expect application software (as opposed to games) to be free, this may be a hurdle. I’d be optimistic and say that the compromise of making software non-free for a limited amount of time, according to a transparent mechanism, in order to cover the costs of development is convincing enough to assuage these concerns.

Finally, and most importantly from my point of view, software requires maintenance. Application software needs to be developed continually. A game may be “done” after the 7th patch or so, but application software never is. If the only goal is to fund the creation of a version 1.0, then this is not a problem. But what if you are looking for a way to finance the development of open source software throughout its entire lifecycle?

An obvious modification to Fund I/O for financing several consecutive versions would be to run a new campaign for each major version. The mechanism could easily be adjusted to make a part of the revenues from the version 2.0 sales refunds for version 1.0. And once 1.0 is free, buyers/backers of subsequent versions could stop paying refunds to version 1.0 owners. The problems I see with this are therefore not theoretical, but practical in nature. The company developing the software would have to maintain several versions of the software side-by-side, they would have to backport security patches and provide support, and they would have to finance these activities from their version 2.0 revenues. While all of these concerns appear in commercial (and open source) software development anyway, they are exacerbated by keeping different versions under different licenses around - just because of the funding scheme.

Ultimately, it will depend on the particular project which of these considerations are relevant, and which variant of Fund I/O makes the best fit. Personally, I view Fund I/O as a toolbox that can be tailored to fit many different needs.

Conclusion

I hope to have convinced you that crowdfunding open source software is possible at much bigger scale than we have seen so far. What we have to do to make this happen is to find a crowdfunding model that takes the backers’ diverse patterns of economic behavior into account. I think Fund I/O is a step in the right direction, but can certainly be improved upon with regard to funding open source software.

What do you think would be the challenges when applying Fund I/O to open source software? How would you modify Fund I/O to make it more applicable to open source software? Comments and questions are welcome!

What is the Fund I/O business model? Fund I/O is a business model for the production of goods with substantial economies of scale. In a nutshell:

• Financing is provided through presales made during a crowdfunding campaign.
• The producer commits in advance to a pricing scheme that passes savings due to economies of scale on to customers directly, by lowering prices as fast as possible.
• Early buyers are guaranteed to get a refund when the price drops, so that they have no incentive to wait with their purchase.

If you are looking for details, good places to start are fund-io.com and this example.

What makes Fund I/O different from the conventional way of doing business? Suppose you run a company and want to produce a product that benefits from economies of scale. Here is what Fund I/O means to you:

• You finance your product entirely through presales. You do not have take on a loan, you do not have to pay interest, and you do not have to maximize profit for your investors. This drastically reduces your risk and your cost of capital. The presales appear as a non-interest bearing liability on your balance sheet, and the primary interest of your financiers is to receive a great product, not to get paid.
• You place an absolute limit on the profit you can make off a product. This limit can be chosen arbitrarily, but by choosing a refund scheme you are committing to an absolute limit on your profits. Note that because you used presales from customers to finance production you are making a profit without having invested any capital yourself, so from that point of view your return on investment is infinite, even if your maximum profits are not!
• You obtain a tremendous amount of market information: By committing to a refund scheme, you give your customers an incentive to reveal exactly how much they are willing to pay for your product. This translates into higher sales, more revenue and faster growth.

Through the combination of crowdfunding, a clear price reduction scheme with a limit on profits and a refund mechanism, Fund I/O manages to align the interests of key stakeholders and it provides them with a mechnism to communicate truthfully about value: It resolves the fundamental conflict of interest between investors who want to maximize their profits and customers who want to minimize what they have to pay.

What makes Fund I/O different from other crowdfunding schemes? There are basically two different crowdfunding models out there: reward-based crowdfunding and equity crowdfunding.

Equity crowdfunding is not that different from the conventional business model. Financiers are still interested in return on investment. The key difference is that investors often have another stake in the business in addition to their profit interests. For example, they may be not only investors but customers at the same time, or they may have an interest in the social impact of the business. But still the business is obliged to maximize profits on behalf of its investors, which gives customers incentives to understate their valuation of the products the business is offering.

Reward-based crowdfunding makes customers the financiers of a product, aligning the interests of two key stakeholders. But it still does not provide a mechanism to communicate about value. In reward-based crowdfunding, creators have to fix the price a priori and often a majority of backers pledge just this price and no more. The success of reward-based crowdfunding hinges on the generosity of backers who self-select for price targeting by willingly paying a large premium for the product. This type of altruism (whether motivated by appreciation of the project, the thrill of being part of something great or the enjoyment of receiving secondary merchandise) is great, if it suffices to finance the project. Fund I/O can be tweaked to captialize on this type of gift economy as well, but the main contribution of Fund I/O lies elsewhere: Fund I/O makes crowdfunding work in cases where the capital requirements of a project are large or where the potential customers are price sensitive. Fund I/O provides the mechanism that allows producers and customers to join forces in order to create great products, even if all parties involved need to avoid paying more than they have to.

What problem does Fund I/O solve? From the point of view of a business, Fund I/O reduces risk, aligns interests of stakeholders, provides a wealth of market information and generates more sales. Moreover Fund I/O provides financing alternatives to debt and equity and conventional crowdfunding. This is particularly useful in cases where debt and equity are too expensive, too risky or simply unavailable and when altruistic contributions made through a rewards-based crowdfunding campaign do not suffice.

However, this is just one angle on the question: “What problem does Fund I/O solve?” Over the next days, I will post other answers, taking the point of view of a customer, or examining the applicability of Fund I/O to non-profit projects aimed at social impact. One very technical answer is already written up: Fund I/O provides a practical implementation of an incentive compatible, individually rational mechanism for the private provision of excludable public goods that is asymptotically optimal.

As you may have seen, I have been blogging recently about a new crowdfunding mechanism for public goods, the Average Cost Threshold Protocol. Since the last post on the topic, I have spent quite some time discussing this idea with people, and have decided to give this idea into a project dubbed Fund I/O with an own website fund-io.com and an own Twitter account.

To get updates about his project and idea, you can subscribe to a (very-low volume) mailing list on the project website, or just follow this blog. I’ll keep posting whatever I have to say on the subject of fair crowdfunding on this blog. (If and when Fund I/O gets its own blog, I will make an announcement here.)

Finally, in case you are wondering, Fund I/O is not a startup. (Not yet, anyway.) The current goal of Fund I/O is to try some variant of the ACTP in practice. If you have got just the right project in need of a revolutionary funding mechansim, let me know!

[Note: I am using Sage Cells to produce the pictures in this post. This has the advantage that you, dear reader, can interact with the pictures by modifying the Sage code. However, this will probably not run everywhere. It certainly won’t work in most RSS readers…]

Here is the graph of the GCD! First a little static preview…

… and now a proper interactive graph. Be sure to rotate the image to get a 3d impression. (Requires Java…)

What precisely are we seeing here? The gcd is a function $\mathrm{gcd}:\mathbb{N}^2\rightarrow\mathbb{N}$. Its graph is a (countable) set of points in $\mathbb{N}^3$. This graph is precisely the set of integer points contained in one of the (countably many) lines indicated in the above picture. The value of the gcd is given by the vertical axis.

Now, what are those lines? Suppose $a$ and $b$ are two natural numbers that are relatively prime, i.e., $\gcd(a,b)=1$. Then we know that $\gcd(ka,kb)=k$ for any positive integer $k$. So all (positive) integer points on the line through the origin and $(a,b,1)$ are in the graph of the gcd. Conversely, for any numbers $x,y$ with $gcd(x,y)=z$, we know that $\gcd(\frac{x}{z},\frac{y}{z})=1$, that is, the point $(x,y,z)$ lies on the line through the origin and $(\frac{x}{z},\frac{y}{z},1)$. So by drawing all these lines, we can be sure we hit all the integer points in the graph of the gcd.

What I find very beautiful about this is that when looking at this picture from the right angle, you can see the binary tree structure of this set of lines. It is precisely this binary tree that we walk along when running the Euclidean algorithm. The lines in the above picture are color coded by their depth in this “Euclidean” binary tree.

The version of the Euclidean algorithm I like best this one. Suppose you want to find $\gcd(a,b)=1$. If $a>b$, compute $\gcd(a-b,b)$. If $a<b$, compute $\gcd(a,b-a)$. If $a=b$, we are done and the number $a=b$ is the gcd. At each point in the algorithm we make a choice, depending on whether $a>b$ or $a<b$. If we keep track of these choices by writing down a 1 in the former case and a 0 in the latter, then, for any pair (a,b), we get a word of 0s and 1s that describes a path through a binary tree. As we walk from one pair $(a,b)$ to the next in this fashion, we trace out a path in the plane: For each 1 we take a horizontal step and for each 0 we take a vertical step. This path can look as follows.

Now comes the twist. We can turn the Euclidean algorithm upside down. Instead of walking from the point $(a,b)$ to a point of the form $(g,g)$ and thus finding the gcd, we can also start from the point $(1,1)$ and walk to a point of the form $(\frac{a}{g},\frac{b}{g})$ to find the gcd $g$. (Why $(\frac{a}{g},\frac{b}{g})$? Well, $(\frac{a}{g},\frac{b}{g})$ is the primitive lattice vector in the direction of $(a,b)$. In terms of the graph at the top, $(\frac{a}{g},\frac{b}{g},1)$ is the starting point of the ray through $(a,b,\gcd(a,b))$.)

How do we get there? We start out by fixing the lattice basis with “left” basis vector $(0,1)$ and “right” basis vector $(1,0)$. Their sum $(1,1)$, which I call the “center”, is a primitive lattice vector. Now we ask ourselves: is the point $(a,b)$ to the left or to the right of the line through the center? If it is to the left of that line, we continue with the basis $(0,1)$ and $(1,1)$. If it is to the right of that line, we continue with the basis $(1,1)$ and $(1,0)$ and recurse: We take the sum of our two new basis vectors as the new center and ask if $(a,b)$ lies to the left or to the right of the line through the center, and continue accordingly. If $(a,b)$ lies on the line through the center, we are done: the gcd of $a$ and $b$ is the factor by which I have to multiply the center to get to $(a,b)$.

As we follow this algorithm, we draw the path from one center to the next, until we reach $(\frac{a}{g},\frac{b}{g})$. Then we get the following picture. It looks almost like a straight line from $(1,1)$ to our goal, but it is only straight whenever we take two consecutive left turns or two consecutive right turns. Whenever we alternate between right and left, we have a proper angle, it only gets difficult to see this angle the farther out we get. If we write down a sequence of 1s and 0s, corresponding to the left and right turns we take, we get the exact same sequence as with the standard Euclidean algorithm.

The obvious next step is to draw the complete binary trees given by the above two visualizations of the Euclidean algorithm (up to a certain maximum depth). Drawing the first tree gives a big mess, because many of the edges of the tree overlap. You can clean up the picture by removing the lines and drawing only the points: In the code below comment the first definition of thisdrawing and uncomment the second. This then gives you a picture of all primitive lattice points (pairs of relatively prime numbers) in the plane at depth at most maxdepth in the tree.

The second variant of the Euclidean algorithm gives a much nicer picture, revealing the fractal nature of the set of primitive lattice points. The nodes of the tree, i.e., the points drawn in the picture are the starting points of the rays plotted in the 3d graph of the gcd given at the beginning of this post. In this way, this drawing of the tree gives the precise structure of the binary tree intuitively visible in the three-dimensional graph. One thing you may want to try is to increase the maxdepth of the tree to 10. (Warning: this may take much longer to render!) Note how far out from the origin some of the points at depths 10 are.

The reader may have observed that the binary tree described in this post gives us a very nice way of enumerating the rational numbers. Calkin and Wilf made this observation in the paper Recounting the Rationals, whence it is sometimes called the Calkin-Wilf tree, even though Euclid tree might be a better name as it is given by the Euclidean algorithm itself. As we have seen above, a look through the lens of lattice geometry tells us how to draw this tree in the plane and how to find it in the graph of the GCD.

Does the Average Cost Threshold Protocol incentivize content creators to “sue the Internet”?

Very short answer: The incentive for creators to sue the Internet is very small (much smaller than usual) but not zero.

Short answer: The Average Cost Threshold Protocol (ACTP) drastically reduces the incentive of content creators to sue people, as creators’ costs are covered from the start. Moreover it drastically reduces the incentive for people to copy content outside of what is allowed by the protocol, because buyer’s payments are used to lower the price (for themselves and others) and to eventually release the content under a copyleft license.

That said, there is a “sales phase” in the protocol where such incentives exist, even if they are limited. However, this drawback is outweighed by the protocol’s key advantages:

• Creators can expect to raise significantly more money than with funding models that employ only copyleft licenses.
• Creators and users agree from the outset on a clear path towards release of the content under a copyleft license.

Moreover the protocol can be modified so that creators have no incentive whatsoever to sue the Internet - however, in this case creators also lack an economic incentive to deliver a great product.

Long answer: Here are the key reasons why creators have very little incentive to sue the internet, if they choose to adopt the ACTP.

• The protocol guarantees that creators’ costs are covered before content is ever released. This drastically reduces the creators’ financial risk, which in turn reduces the threat unlicensed copies pose to creators.
• Costs are covered by users, not investors. This frees creators from having to worry about paying back loans or maximizing their return on equity. Thus the creators’ economic goal is to maximize user value not shareholder value.
• The ACTP provides a clear path towards releasing the content under a copyleft license. By adopting the ACTP, creators subscribe willingly to this goal.
• Creators’ profits are maximized if and only if the content is released under a copyleft license, at the end of ACTP. This means that creators’ have an incentive to work towards the goal of releasing the content for free.

That being said, the ACTP does have a “sales phase” where the content is available only under a “closed” copyright license and people have to buy access. In this sales phase, users have an economic incentive to obtain unlicensed copies.

• Creators receive a share of the revenues earned during the sales phase as profit. They thus have an incentive to sue people who avoid paying for access in the sales phase.
• However, if there are enough buyers to “free” the content and cause its release under a copyleft license, then the creators do not care how many people made unlicensed copied during the sales phase: their profits in this case are fixed from the start.
• The ACTP could be modified so that creators do not receive any profits during the sales phase. All revenues from sales could go towards lowering the price by refunding previous buyers/backers. In this case creators do not have any incentive whatsoever to sue the Internet, at any point in time. However, I do not think that this is a good idea. Because in this case, creators’ do not receive any revenue after the initial fundraising campaign and thus they have no economic incentive to make their product shine and promote it accordingly.

Moreover, the ACTP provides incentives for users to actively participate in the sales phase of the protocol by paying, instead of by making or obtaining unlicensed copies:

• In the short term, the more people pay, the faster the price will drop. As a buyer this means you yourself will get a larger refund and at the same time you will enable other people to afford access.
• In the long run, the more people pay, the sooner will the content be released freely, under a copyleft license.

The sales phase thus becomes a cooperative game among users, where they can help each other by lowering the price in the short term and ultimately “freeing” the content for everyone. This of course has the drawback that it might cause cooperative users (who buy) to sue uncooperative users (who make or obtain unlicensed copies). But the amount of refund money cooperative users stand to loose from unlicensed copies is very limited, so I hope this is not going to be an issue. Users who obtain unlicensed copies hurt themselves by making it less likely that the content is eventually going to be officially released under a copyleft license.

But why should we have a “sales phase” at all? Would it not be better to release the content under a copyleft license right away? The reason is a phenomenon that has been well-studied in the economics literature: By restricting access to content during the sales phase, it is possible to raise significantly larger amounts of funds for the creation of digital content than would otherwise be possible. In economics jargon: It is significantly easier to raise funds for the private provision of a public good with use exclusions than for the private provision of a pure public good without use exclusions. The amount of money we can raise for content that is going to published under a copyleft license from the start grows on the order of $\sqrt{n}$ where $n$ is the number of people in the economy. If we introduce use exclusions, on the other hand, the amount of money grows on the order of $n$. This is a difference of several orders of magnitude, with a huge potential impact in practice! (I will write more about these results in a future post. In the meantime I recommend this post and these two articles.) The ACTP aims to strike a balance between the conflicting goals of publishing free content and raising funds by employing use exclusions in an intermediate phase and providing a clear path towards release under a copyleft license.

Bottom-line: The ACTP does a far better job of aligning incentives than any other mechanism for funding digital content that I know. I think it goes a long way towards resolving the economic conflict over copyright and I can’t wait to put it into practice and try it out.

The reason is a conflict of economic interests. On the one hand we need to fund the creation of digital goods. On the other hand, our creation can do the most good if it is made available to literally everyone. But how can we convince people to pay for a digital good, if they know that eventually everybody can download a copy for free? What we need is a new idea that addresses both of these issues: the public welfare and the profitability of creating digital goods.

In this regard crowdfunding platforms are very promising, as they can in principle be used to fund digital goods that are then made freely available to everybody. However, a closer look reveals that in practice the digital goods financed by crowdfunding campaigns are often sold like apples - as if they could not be copied at all. Here is one idea how we can do better:

The Average Cost Threshold Protocol is a fair crowdfunding mechanism that takes economic interests of both users and creators into account.

How it works is best explained by way of a concrete example. Suppose a company wants to raise \$1,000,000 to finance the production of a computer game. They start a crowdfunding campaign on a website implementing the Average Cost Threshold Protocol. The website starts collecting pledges until a deadline is reached. Suppose on the day of the deadline the pledge look as in picture below. In particular, we find that there are 20,000 people who pledged \$50 each or more. If all of them pay exactly \$50, then the company’s costs will be covered and everybody will have paid the same amount. Moreover \$50 is the lowest price that covers costs: for example 24,000 people would be willing to pay \$40, but that yields only \$960,000. This lowest price now becomes the threshold price: everybody who pledged at least \$50 pays exactly \$50 and they become backers, who will receive a copy of the game once it is finished. All others do not pay anything and do not get a copy. The principle is simple: the costs of production are distributed equally among all those who get access. The threshold price is chosen such that it provides access to the largest number of people at the lowest possible price such that the costs of production are covered.

Now that the game is released, more people have heard about it and want to purchase access. Typically, the company would simply sell copies and keep the revenues as profit. But here comes the twist! Instead of paying a fixed price, interested buyers submit pledges on what they would be willing to pay for the game. Suppose there are another 10,000 people who each pledge \$40 or more. Then we drop the price to \$40 dollars and charge each newcomer \$40 for a copy of the game. The revenues of \$400,000 are split 50-50. One half goes as profit to the company. The other half goes to the original backers, who each get a refund of \$10. Following this principle the price keeps dropping the more people buy the game and thanks to the refunds everybody who gets access is guaranteed to pay the same price no matter how much they pledge or when they pledge.

The more people buy the game, the lower the price for everyone. And the lower the price, the more people can afford to buy the game! What if this virtuous cycle takes off and the game becomes really popular? At this point another feature of the mechanism kicks in: At the very beginning of the crowdfunding campaign a price of freedom of \$10 is announced. This means that as soon as the threshold price drops to the price of freedom of \$10, the game is made available to everyone, the whole world, no matter if they paid or not. This includes the release of the game under a copyleft license as well as the publication of all source code and assets, to allow people to freely modify the game. In the example, the price of freedom of \$10 is reached if the game becomes so popular that at least 180,000 people buy the game. In this scenario, the company has not only covered its costs, but on top of that it also made a profit of \$800,000. (This amounts to an infinite (!) return on investment as the customers contributed all the funding for the project!) The public gains a game they are free to use and modify. And the backers and buyers of the project spent only \$10 each, both to gain early access to the game and to make all of this happen.

We can even go one step further and allow every customer to set their own price of freedom, that is, the price below which they do not care to receive refunds. With this small modification, it is entirely rational for people to pledge exactly as much as they are really willing to pay for the game. This is a truly amazing property that most other methods of raising funds do not satisfy. In a retail sales context, on Kickstarter, in you-name-your-price mechanisms like the Humble Bundle, customers always have an incentive to understate how much the product is worth to them. Here, even entirely self-interested customers have an incentive to tell the truth, at every point during the process. This is not the end of the story, though! The Average Cost Threshold Protocol has a host of variations and wonderful theoretical properties.

As you can see, there is a lot of room to improve upon existing methods for funding digital goods. In a world where digital goods make up an ever increasing percentage of the global economic output this can have a profound impact on our economic lives. With new ideas and experiments we can come up with business models that can raise large amounts of money to finance the creation of high-quality digital goods and at the same time make sure that as many people as possible get access. In particular, we can provide a clear path towards releasing digital goods under a copyleft license. This way, everybody can benefit from the economic power of copying!

If you like that sort of thing, check out this video or these lecture notes.

In this post, I want to describe a different way to fund the private provision of public goods: the Average Cost Threshold Protocol.

Before we get started, let me clarify what I mean by the term public good. The term does not have the egalitarian meaning of a common good that is shared by everyone and that all members of a community are entitled to. Rather, a public good is any good that has a specific economic property: If a public good is provided to one person, it could also be provided to everybody else at no additional cost. Note that I am not saying the public good actually is provided to everyone. It may very well be that some people are excluded from using the public good. Also, I am not saying that nobody has to pay anything for the public good.

Practically all digital goods are natural public goods. This includes movies, music, ebooks, online lectures, scientific articles, software and computer games. These can, in principle, be copied at virtually no cost. Often, legal or technical restrictions are placed on copying these goods, as producers want to raise money for the provision of these goods by selling these goods as private goods (or by selling advertising tied to these goods). This post is about finding better ways to fund the creation of such digital goods. As my running example of a public good, I am going to use a computer game. Not because games are the most important type of public good, but simply because they are widely popular, require no maintenance after their release (or after, say, the tenth patch) and because a number of computer games have been very successful on Kickstarter, which shows that the audience is open to experimenting with new funding mechanisms.

The Theory of Average Cost Pricing of Public Goods

A very natural variation of threshold pledge systems like the Street Performer Protocol is a fixed fee mechanism with average cost prices. In this section I will present the theory behind this mechanism, before turning to its practical implementation (and a practical example!) in the next section.

The basic idea is very straightforward:

1. Everybody interested in a given project makes a pledge, saying they are willing to contribute a certain amount of money to funding the project. If somebody is not interested in contributing that just don’t pledge anything.
2. After all pledges have been made, we distribute the costs of the project equally among as many people as possible: Everybody pays exactly the average cost of the project, as long as they pledged at least as much. More precisely, we figure out the smallest price $p$ such that if everybody who pledged at least $p$ pays exactly the price $p$, then the costs of the project are covered exactly. Nobody pays more than $p$ and if somebody pledged less than $p$ they don’t have to pay anything.
3. If there is no such price $p$, for example because people pledged too little, then the project is not funded and nobody pays anything!
4. In case the project is funded, the money raised is used to create the public good. But only those people who paid get access to the public good! (Here we assume that the public good in question is excludable.)

Points 1 and 3 are very similar to the Street Performer Protocol (SPP) and what happens on Kickstarter. Point 2 is crucially different, as in the SPP and on Kickstarter everybody pays what they pledged and not the price $p$. Point 4 is what happens in many projects on Kickstarter, as I observed in my last post, but it is very different from the idea behind the SPP, which was intended to fund pure public goods without use exclusions. There is another vital difference to what happens on Kickstarter that will become clear in the next section.

The fixed fee mechanism with average cost prices has crucial theoretic advantages:

• Everybody has an incentive to pledge exactly what the project is actually worth to them. The technical term for this incentive compatibility. Note that the SPP is not incentive compatible: there, people have an incentive to understate the value the project has for them, in order to pay less. (This is called “free riding” in the public goods literature.)
• Nobody is forced to contribute anything if they don’t want to. In other words, no matter what happens, after we have used this mechanism, everybody will be at least as well-off as before. This is called individual rationality. Many mechanisms for funding public goods, both practical ones, such as taxation by the government, and theoretical ones, such as the Groves-Clarke mechanism, are not individually rational. But for the private provision of public goods in which people participate voluntarily, individual rationality is essential.
• If the fundraising is successful, we raise exactly as much money as is needed to fund the project. That is, the mechanism not only covers costs but moreover it balances the budget.

Of course this mechanism also has a key disadvantage: We exclude people form using the public good even though the public good could be provided to them at no extra cost. In technical terms we say the mechanism is inefficient. This, however, is unavoidable as there are theoretical results such as the Myerson-Satterthwaite Theorem which says, roughly, that there does not exist a mechanism for the provision of public goods that is incentive compatible, individually rational and efficient. This result tells us that the first best solution of giving everybody access to the public is impossible to attain. The good news is, though, that in face this impossibility result, the fixed fee mechanism with average cost prices is the best possible alternative:

• According to a result by Peter Norman the above mechanism is (asymptotically, in a large economy) the most efficient one among all mechanisms that are incentive compatible, individually rational and guarantee budget balance.
• If the cost of funding the public good is small in comparison to the total size of the economy and the number of people interested in the public good, then the average cost will approach zero. This allows us to get arbitrarily close to giving everyone access to the public good. See also this paper by Martin Hellwig.

Regarding the history of the fixed fee mechanism with average cost prices, I want to mention that average cost prices have been studied for a very long time in the context of monopoly economics and a number of authors have examined fixed fee mechanisms in the context of public goods. However, as far as I was able to find out, the paper by Peter Norman is the first instance where this exact mechanism has been studied in a public good setting.

Of course there is a lot more to say about these results and I plan to write more about the technical details in the future. But today, I want to talk about how this mechanism could work in practice and present a practical implementation which I dub the Average Cost Threshold Protocol.

The Average Cost Threshold Protocol in Practice

Suppose a company wants to create a computer game and they need money to cover their costs, which total, say, $1 million. They decide to use the above mechanism to raise the funds and so they start a project on a website like Kickstarter which provides all the necessary infrastructure. The project is open to receive pledges for 30 days and at the end of that period the pledges are tallied to see if the project can be funded according to the above rule. (Strictly speaking, setting a deadline is not necessary, but given that many crowd-funding projects raise a large part of their funds the days immediately before the deadline, it seems like a good idea to make use of this psychological effect.)

Many people chip in and pledge various amounts. There are 20,000 people who pledge \$50 or more. So setting a price of \$50 would exactly cover the costs of the game. However, there are only 23,000 people who pledge \$40 or more, so setting a price of \$40 would raise only \$0.92 million which does not cover the costs of the game. Let us assume that \$50 is the lowest price that covers the costs of the game.

Thus, the price is fixed at \$50 dollars. Everybody who pledged \$50 or more now has to pay exactly \$50 dollars. The people who pay these \$50 are now called backers. The total amount raised is \$1 million which covers the costs of the game. This money is used to create the game, and once it is finished, every backer receives a copy.

So far so good. But what about all the other people who would like to play the game? In all likelihood, there are many more people out there who would be happy to pay \$50 to get a copy! Maybe they heard about the project only after the fundraising ended, so they did not have a chance to become a backer. Or maybe they wanted to wait and see how the game turned out before committing to the purchase. Or maybe they did not have \$50 to spare back then, but they do now. Whatever their reasons, it makes perfect sense to provide these people with the public good - we just have to find a way that is consistent with our mechanism.

How do games companies currently do it on Kickstarter? Well, they just sell copies of the game. And the companies keep the revenues from these sales for their own profit. There is nothing wrong with creators making a profit from their work. The problem here is that this breaks our mechanism! Suppose there are another 20,000 people (let’s call them buyers as opposed to backers) who pay \$50 for the game and these revenues are the profits of the company. Now there are a total of 40,000 people (buyers + backers) who have paid \$50 each, which amounts to a total of \$2 million dollars. However distributing the total cost of the game (\$1 million) among 40,000 people would lead to an average cost of just \$25! So, in this scenario, if the company decides to sell the game for profit after it is finished, people would pay twice the average cost, which is very different from what our mechanism specifies.

So let’s suppose we take our mechanism seriously. What would need to happen with the profits from selling the game? Suppose we have 20,000 backers plus 20,000 buyers. As we observed above, distributing the costs of \$1 million equally among those 40,000 people would lead to an average cost of just \$25. But the backers already paid \$50! So here is what needs to happen according to the mechanism: The buyers need to pay just \$25 each. And these revenues need to be given to the backers instead of the company. This makes sure that everybody pays exactly the average cost of the public good.

But now the price of the good has dropped by 50% and the product is gaining ever more public attention. So now there are, say, 40,000 additional people out there who would be willing to pay \$25 for the game. Here is what the mechanism tells us to do: Charge each of the 40,000 newcomers \$12.50 and give the proceeds to the 40,000 people who purchased the game first. This way, everybody just paid \$12.50.

You can now see where this is going:

• As more and more people buy the public good, the average cost of the good drops.
• The price of the good drops, making the good available to more and more people.
• In this process, early buyers are refunded for the high price they paid in the beginning.
• In particular, because of the transfers from late buyers to early buyers, nobody has an incentive to wait with their purchase until prices drop.

All we need for this to work is for the website that coordinates this process to manage these transfer payments. Of course actually distributing money among many different bank accounts whenever a single purchase is made would incur way too much transaction costs. But the website could simply keep track of purchases and how revenue needs to be redistributed and allow customers to withdraw funds every once in a while. The transaction costs could be passed on to backers/buyers directly or could they could simply be financed from the interest the website earns from keeping the payments for some time.

It is also important to note that the infrastructure for pledging available on the website should still be used for sales, even after the product is finished. This way, if the current price for the game is still \$40, people who would be willing to pay \$20 for the product could submit this “bid” on the website. If enough people pledge \$20, the price of the product will actually drop and they will get a copy. The important thing is that people have an incentive to reveal the true price they are willing to pay! This is in contrast to a classical sales context, where customers have an incentive to understate their true valuation to get the company to lower their price.

The Benefits in Summary

The Average Cost Threshold Protocol is a practical mechanism for funding public goods that allow use exclusions. It is an implementation of the well-known fixed fee mechanism with average cost prices, and thus it enjoys many desirable properties:

• It is in everybody’s own best interest to pledge exactly what they are willing to pay. This is true even in hindsight!
• The protocol makes everybody better off. In particular, nobody is forced to pay anything.
• The costs of the project are guaranteed to be covered for the company.
• Among all possible mechanisms with these properties this one is optimal, that is, it creates the most value for society as a whole.

Beyond these theoretical merits, the practical protocol suggested above has a number of additional benefits:

• Prices to the public good are lowered continuously, in order to provide the widest possible audience with access to the public good as quickly as possible.
• At all times throughout the process, everybody has paid exactly the same price for access to the public good. So nobody has an incentive to wait with their purchase for prices to drop.
• In the sales-phase, it is in the interest of customers to bid what they would be willing to pay, even before prices drop. This allows the mechanism to lower prices more quickly, via optimal price targeting.
• Everybody benefits from getting more people to buy access. Even today, backers of crowd-funding projects often provide significant marketing services to their projects through word of mouth, long after fundraising has ended. Using this protocol, backers are not only personally rewarded for this engagement, but they also help to give more people access at lower prices.

Of course this is not the end of the story! There are a number of variants of this mechanism that are worth exploring.

Variations of the Protocol

There are a number of aspects of the basic version of the Average Cost Threshold Protocol presented above which can be improved further.

1) Most importantly, the public good provided by the protocol is still subject to use exclusions. We would really like to actually provide a pure public good that everybody has access to, no matter if they paid anything or not. But if we modified the mechanism so that people know the good is eventually going to be provided for free, the economic incentive to contribute something would disappear. Remember, this is the essence of the Myerson-Satterthwaite impossibility theorem!

Nonetheless, there are a number of ways the protocol could be modified to fund the creation of a pure public good without use exclusions. For example, we could set a “reserve price” of, say, \$5 dollars. If the price of the good falls below \$5, the good becomes a pure public good available to everyone for free. Now, of course, everyone who did pay \$5 has paid \$5 too much, which would destroy incentive compatibility in a strict sense. But as \$5 is a relatively small loss, backers who care about the project may very well be ready to accept this loss and gain the warm glow-effect of having made the public good available for everyone. Instead of fixing a common reserve price of \$5 for everyone, backers might also set their own individual reserve price when buying the product. (This of course would require the redistribution scheme to be adjusted.)

A completely different option would be to set a fixed “expiration date” of the use exclusions, for example, three years after the release of the finished product. Buyers would then purchase early access to the product, which is a common business model already today. The difference is that this early access would come with a guarantee that the product will become a pure public good eventually.

Of course such modifications would ruin some of the nice game theoretical properties of the mechanism. But these theorems hinge on the assumption that all backers are entirely rational anyway. And humans are not entirely rational, they are also benevolent and they tend to be tremendously enthusiastic about creative projects they like. So there is room enough for such small changes to work, even if they don’t fit into the rational framework.

2) Companies or creative individuals funding a project using the “vanilla” Average Cost Threshold Protocol as defined above enjoy the tremendous benefit of having their costs covered in advance by payments very similar to pre-purchases. They are not funded through equity and they have no liabilities, which means they have no investors that they need to satisfy through profits and they have no debt that they need to pay off. But that does not mean that all is well.

First of all, after the project is funded and the company received the payment to cover their costs, they are not going to receive any further payments, whatsoever. This means that they have no further economic incentives to make the project succeed. They may have incentives in terms of their creative ambition, their reputation as a company, their individual careers or simply their personal integrity. But the economic incentives to make the product shine, to market it well, to finish it on time and on budget or even to complete the project all - they are all gone. This is clearly not in the interest of anybody! Therefore it is a good idea to allow the company to make some profits in order to create the corresponding economic incentives.

Moreover, no matter how accurately the company projected the costs of the project at the outset, the actual development may run over budget. Projects often (always?) do. So despite the fact that the initial fundraising is expected to cover costs in advance, the company still faces financial risks. To make these financial risks worthwhile to the company, it stands to reason that the company is allowed to make some profit.

Fortunately it is straightforward to allow the company to make a profit and at the same time allow the public to enjoy decreasing prices. The rule is simple: Half of the payment a new buyer makes goes to the previous backers, the other half is profit for the company. An example. Initially, 20,000 backers paid \$50 each to raise \$1 million. Now, 10,000 additional buyers want access. According to the original protocol, everybody would have to pay \$33.33 now. But instead, we ask the newcomers to pay \$40. Half of that is the profit of the company, which amounts to \$200,000 in total. The other half goes towards refunding the original 20,000 backers so that everybody paid just \$40 in total. In this way, prices will decrease steadily with an increasing number of buyers. But still, the company stands to make an unlimited profit from creating a great product! This amounts to a reasonable compromise between the interests of the company and the social goal of giving access to as many people as possible.

The great thing about this variation is that it preserves many of the nice game-theoretic properties of the original mechanism. In particular, this modified mechanism is still incentive compatible and individually rational. It still covers costs by producing a budget surplus instead of a balanced budget. It is less efficient than before, because fewer people get access for the same amount of money. But still, as the number of buyers grows, the price goes to zero, enabling everybody to afford access if the public good becomes popular enough.

Of course, nobody says that revenues always have to be split 50-50 among the company and its backers. Any other ratio would do. The ratio could change over time. Or each backer could choose their own ratio (similar to what the Humble Bundle is doing), leading to a democratic vote on how revenues should be split. Also, this idea can be combined with the first variation presented above to create a protocol that produces both profits for the company and a public good without use exclusions, provided the good becomes popular enough. In this case the profits for the company are bounded and it is not entirely rational for buyers to reveal their true valuation, but still this promises to be an excellent compromise.

3) From a game theoretic perspective it is important (though not indispensable) that everybody pays the same price. However, from a practical perspective that may not be desirable. Some backers may want to be charged more than other backers. Maybe because they want to show how much they value the project. Maybe the company decides to offer rewards for backers who pay much. Most importantly, there is a real possibility that projects cannot be funded without backers who self-select to pay a very large premium on the average cost. The public interest in the project may not be broad enough to get the costs covered on an average cost pricing basis, but the interest may well be deep enough to cover costs if some people are allowed to pay more.

A special case is the money the company itself puts into the development. Companies and creators running crowd-funding projects often put a significant amount of personal wealth into their projects. Usually, these are funds that do not show up during the fundraising campaign. But an effective mechanism for funding public goods should explicitly incorporate a facility to account for this common practice.

Again there are several ways to allow backers to pay more than the average cost during the initial fundraising campaign. (Note that backers can always pledge as much as they want, but in the original protocol, they will never pay more than the average cost.) One way to accommodate this is via the variable reserve price mechanism mentioned above. Backers who want to pay a lot could simply set their personal reserve price to exactly the same amount as their pledge. Then, they could get charged the entire amount if the fundraising campaign is successful.

However, the above variation would also imply that these backers are not refunded anything. This may well be in the interest of very enthusiastic backers, but it would not fit the needs of the company who wants to recoup the large investment it made into the project in this way. Also there may be backers who are willing to make a very large payment, if the project can’t be funded otherwise, but would like to be refunded if the project turns out to be widely popular in the long run. To accommodate these interests, one could allow backers to specify that they want to be charged more during the fundraising campaign. Later on, the revenues earned from sales of the product could be used to refund backers in proportion to the payment they made during fundraising. This way large backers could eventually recoup their investment if the project is widely successful.

While it may generally not be rational for backers to make such large payments, the presence of this option does not change the fact that for the average backer the protocol remains incentive compatible and individually rational. As before, this variation can also be combined with the variations 1) and 2) mentioned above.

Conclusion

The Average Cost Threshold Protocol and its variations promise to yield an effective mechanism for the private funding of public goods. Even if this particular mechanism is not the ultimate answer, it shows that there is a lot of room out there for improving upon existing crowd-funding mechanisms in this regard. I hope that more people apply their creativity to invent new ways of making the private provision of public goods attractive. In a world where public goods make up an increasing share of the global economic output, such a mechanism could change the way we do business and interact with each other’s creative work.

This paper is a continuation of the work that Matthias Beck and Thomas Zaslavsky started on using inside-out polytopes to prove reciprocity results for counting polynomials and that Raman Sanyal and I adapted to the case of modular counting functions. In the present work we generalize all of this considerably, by proving a whole collection of combinatorial reciprocity theorems for flow, tension and chromatic quasipolynomials defined on cell complexes, i.e., in terms of arbitrary integer matrices. The fact that the Ehrhart theory methods developed for the graph case do generalize to cell complexes is a testament to the remarkable power of the geometric approach to combinatorics. Head over here to hear the full story!

Crowd funding really took off in 2012. From my personal perspective, the most visible embodiment of this trend was the emergence of Kickstarter as a means of funding game development projects. Double Fine Adventure was one of the first projects that brought this concept to public attention by raising $3.3 million. This was quickly followed by the success of such projects as The Banner Saga ($723k), Shadowrun Returns ($1.8m), Star Citizen ($2.1m) and Project Eternity ($4.0m). Of all of these, I think The Banner Saga stands out, as it is a very original project by a comparatively unknown team, whereas the other projects capitalized on well-established concepts and the fame of their celebrity project-leaders.

When I first encountered Kickstarter, I became very excited, not just because of the addictive warm-glow effect of making great projects happen, but also because Kickstarter struck me as the first large-scale, widely popular implementation of the Street Performer Protocol. The Street Performer Protocol is a brand name coined in the 90s by Steven Schear, John Kelsey and Bruce Schneier for a very simple and very old mechanism for fundraising: Artists announce that they will do a public performance if the audience as a whole pays a fixed total amount (or more). If enough spectators chip in to reach this threshold, the artists collect the money and perform. Otherwise, nobody pays anything and there is no performance. This is essentially how Kickstarter operates.

Now, back in the 90s, the Street Performer Protocol (SPP) was heralded as a mechanism for funding public goods. A public good is a good that, once it is created, can be enjoyed by anybody without being “used up”. (See the end of this post for more details on the term.) Software and computer games make excellent examples of public goods in that they can be copied perfectly at a vanishingly small cost.

Commercial publishers of software and games often impose legal and technical restrictions on copying. They create use-exclusions in order to be able to sell the software like a classical private good. While this is a perfectly valid business model, it seems wasteful because it prevents people from using the software who could benefit from it at no additional cost. By contrast, the open source / free software model of software development ensures that free copying is possible, both legally and technically. But if software is to be made available as a public good without use-exclusions, how is its development to be funded? People hoped the SPP might provide an answer to this question, and, sure enough, the SPP soon had important successes. For example, in 2002, Ton Roosendaal managed to raise €100,000 in order to buy the rights to the 3d software Blender (that he created) from the creditors of his bankrupt company NaN and released Blender under the GPL. But despite such isolated successes the SPP did not gain widespread adoption, and cross-subsidies remained/became the primary source of funding for open source software projects.

With all of this in mind, I became very excited when I discovered how successful Kickstarter was. Did we finally have a working way of financing the private provision of public goods directly? My enthusiasm led me to contribute funds to a couple of projects on Kickstarter, which has been a great experience all around. But. This experience also made it clear that, contrary to my first hopes, Kickstarter is not about the provision of public goods and it is not used to implement the SPP - at least judging by the way it is used in the game development community.

None of the games mentioned above are going to be released as a public good when they are completed. Instead, they are going to be sold, with the profit to be shared by the development studio, possible publishers and whatever investors they may have. The Kickstarter backers are not investors in this context. Their pledge is not a contribution towards the creation of a public good, it just buys a single license for a future product and is thus seen as, merely, a pre-order. From this point of view, customers buy a product years in advance that they know virtually nothing about. Moreover they self-select for price targeting by voluntarily paying much more than price of the pre-order. (This price targeting is significant. All aforementioned projects had individual backers pledging more than $10,000 dollars and a significant percentage of backers pledged more than double the price of the respective pre-order.) In exchange for these additional funds, they get additional merchandise of limited value and the warm-glow effect of being part of a project they care about.

To be sure, this long-term pre-order model of funding games does potentially have great positive effects. When an audience not only buys the product but funds the creation process from the start, stakeholders become the key financiers of the project, which may lead to fewer conflicts of interest. Development teams achieve greater creative independence, they can follow their instincts and have to worry less about mass market appeal and investor interests. In the most optimistic scenario, this can lead to a more satisfying experience, both for the developers and for the customers.

Nonetheless, treating a Kickstarter pledge as a “pre-order” sounds just wrong to me, for several reasons. First of all, backers take on a large amount of risk with their pledge. They have no guarantees that the product is going to be delivered, they have no influence on the creation process, they have no information about the product they are buying, the price they pay is, on average, way above the final market price of the product, and they receive none of the revenues made by selling the final product. In short, if Kickstarter is used to create software that is going to be sold for-profit, then backers make a huge investment, reap none of the profits that arise from their investment, carry a large share of the risk, and end up paying much more than customers who buy the product after it has been released.

To be clear: I do think that all of the aforementioned projects are run by development studios with the best intentions. And while contributing on Kickstarter does have a warm-glow effect that can be addictive, I am convinced that most backers are rational in their decision. The huge transfer payment from backers to developers inherent in the pre-order funding model described above is a deliberate decision to fund art that would not be created without such a payment. But for studios to dismiss this huge gift as a pre-order and then to sell the resulting product exclusively for their own profit strikes a wrong chord with me.

Now, the pre-order model is certainly not the only way Kickstarter is used. The are projects, such as Chris Granger’s wonderful Light Table project, that were created with the explicit purpose of producing open source software. The Light Table Kickstarter project raised $316,720 and makes a prime example of the use of the Street Performer Protocol for funding a public good. I do hope that other Kickstarter projects will have the courage to ask backers for money, even if they pledge to make their final product open source upon release. In my view, this would be a much fairer deal. Projects such as LightTable (and Blender ten years ago) show that this can work, despite the fact that it is not rational in a strict economic sense for an individual backer to donate money to such a development project. If this mode of using Kickstarter catches on, we will have a truly new way of financing public goods. Such a mechanism would have a huge impact, given that an ever-growing share of the global economy deals in virtual goods that could in principle be turned into public goods.

So, I do still have hopes for more public good projects on Kickstarter. But hope is not good enough!

Therefore, I will use this post to kick off a series of posts on the economics of public goods, dealing with questions such as: What other mechanisms for funding public goods are out there? What tools do we need to analyse such mechanisms? What are the theoretical limits? How would a funding platform for the private provision of public goods have to work in order to be widely successful?

Addendum: One paragraph introduction to public goods. A public good is a good that, once it is created, can be enjoyed by anybody without being “used up”. Anyone can stand by and watch a street performance, once it is happening. Anyone can walk on a street, once it is built. And anyone can make a copy of a digital movie, once it has been created, without anyone having “less” of the movie. Of course this is not quite true. A performance may become too crowded for new arrivals to see anything, and a street may become too congested for anybody to walk or drive. So, under extreme conditions there is some rivalry in the consumption of these good, but in most cases one more onlooker or one more pedestrian does not “cost” anything, which makes these goods non-rival. Now, as far as digital movies are concerned, their copying is perfectly non-rival as both copies are identical and nobody has “less” of the movie. However, there are often many legal and technological barriers in place to prevent consumers from copying. That is, the industry tries (with moderate success) to exclude consumers from obtaining a copy the movie, if they did not pay for it. When these restrictions are not in place, there is nothing preventing a consumer from making a copy, and the movie becomes a public good without use-exclusions. A pure public good is perfectly non-rival and non-excludable. In practice, public goods often have some limited rivalry, but it may still be instructive to think about them as public goods as they are non-rival in most cases. For my purposes, I do not include non-excludability in my definition of a public good, as the role of use-exclusions in the funding of public goods is precisely what I want to discuss.

However, as long as current automatic proof systems are unable to do even high school mathematics on their own, this dream will not turn into reality. Yet, we do not need to wait for automatic proof systems to get better. On the contrary, there is something that we can do now that will help both the formalization of mathematics and the improvement of automatic proof systems:

We need to create a standard file format for formal sketches of mathematical articles.

In the rest of this post, I will explain what I mean by this and why I think this is useful. Note, however, that these ideas are very much a work in progress. Also, I should say that my perspective is that of a mathematician working in discrete geometry and combinatorics. I am not a logician and I am no expert in interactive or automatic theorem proving, and I have begun to explore the world of formal proof systems only recently. Nonetheless, I think that the one thing that could make writing formal mathematical proofs more accessible to a working mathematician like me is a standard file format for formal sketches of mathematical articles.

Formal Sketches

With a mathematical article I mean an informal mathematical article on some topic of current research interest, as might be found in a journal or on the arXiv. With a formal sketch I mean a formal version of such an informal mathematical article with the following properties:

• A formal sketch contains the definitions, theorems and proofs given in the informal mathematical article.
• The formal sketch is human-readable and human-writable.
• Moreover, reading or writing a formal sketch of an informal mathematical article should not be significantly more difficult than reading or writing the informal article itself.
• Definitions and theorems in the formal sketch are given in a fixed logic without any ambiguity.
• Proofs in the formal sketch are formal proof sketches in the following sense: A formal proof sketch is not a complete formal proof and it is not a human readable representation of a given formal proof. Instead, a formal proof sketch serves as advice for constructing a formal proof by breaking the task of constructing a formal proof down into smaller, easier steps. The purpose of a formal proof sketch is to help an automatic proof system to construct a complete formal proof.
• Formal proof sketches are independent of any particular automatic proof system and any fixed library of definitions and theorems. Formal sketches of mathematical articles follow a decentralized open-world approach to formalizing mathematical theory.

I do not claim that current automatic proof systems will actually be able to construct a formal proof from a formal proof sketch. This goal may still be many years away. Current automatic proof systems will still require additional prover-specific human help to construct formal proofs. Nonetheless, I argue that it is still beneficial to create a standard file format for formal proof sketches now.

The creation of a standard file format for formal sketches of mathematical articles will, I hope, accomplish two things:

• A standard file format for formal sketches of mathematical articles will make it more attractive for everyday mathematicians to formalize their mathematical articles. This will facilitate the creation of a large body of formalized real-world mathematics. Moreover, this effect will become more pronounced as automatic proof systems become more powerful.
• A standard file format for formal sketches of mathematical articles will encourage competition between automatic theorem provers and innovation in automated reasoning. Formal sketches may serve as a benchmark for comparing the capabilities of different provers. The availability of an interesting set of formal proof sketches will also make improvements in the automated reasoning capabilities of current provers more relevant.

These two effects mutually reinforce each other. This has the potential to create a positive feedback loop that will help the large scale formalization of mathematics to take off, finally.

Of course, a large effort will be required to get this process going. One means of bootstrapping this process may be the creation of a supplemental file format for the annotation of formal proof sketches with prover-specific advice. Nonetheless, I think that the creation of such an ecosystem is possible and that the potential benefits justify the effort. As outlined above, the foundation of this system is a standard file format for formal sketches of mathematical articles.

In the remainder of this post I will discuss how such a standard file format might look like.

What file formats are out there already?

Fortunately, there is already a rough consensus on what a formal, human-readable, declarative language that resembles ordinary mathematics should look like. Many people tried to come up with their own version of such a language and, independently, they arrived at essentially the same result, a common mathematical vernacular. This argument has been made by Freek Wiedijk in a wonderful article where he compares the languages of the Hyperproof, Mizar and Isabelle/Isar systems and points out the common structure behind their differences in surface syntax. His miz3 syntax for HOL Light is another example of this mathematical vernacular.

This mathematical vernacular is a format for formal proofs with a very precise meaning. Formal sketches, on the other hand, should live at a higher level of abstraction. In particular they should be intentionally vague in specifying how, exactly, to prove “the next step” in a declarative proof. Freek has introduced the notion of a formal proof sketch and given some very nice examples. A formal proof sketch, in his sense, is an abbreviated version of a full formal proof in the Mizar language, in which some intermediate steps and justifications have been removed. It turns out that using such abbreviations, one can arrive at a document that is very close to a natural language version of the proof and that still accurately reflects the structure of the underlying formal proof. A formal proof sketch is correct if it can be extended to a formal proof in the Mizar language just by adding labels, justifications and intermediate steps.

Freek intends formal proof sketches to serve only as a means for presenting a formal proof. He expressly does not intend formal proof sketches “to be a ‘better way’ of writing formal proofs”. In this post, however, I use the terms “formal sketch” and “formal proof sketch” with the explicit intent of employing these sketches as a tool for authoring and archiving formal proofs.

In the context of this prior work, a formal sketch in the sense of this post would be the following: A document containing definitions, theorems and proofs, where the proofs are written in an abbreviated version of the mathematical vernacular. Such a document would be correct, if the proofs can be extended to full formal proofs in the mathematical vernacular by adding labels, intermediate steps and additional justifications. Such a formal document would not serve as an abbreviated representation of an underlying formal proof. Rather, it would serve as formal but ambiguous and incomplete advice to the (human or machine) reader for constructing a formal proof.

The purpose of creating a standard file format for formal sketches based on the mathematical vernacular is to share formal sketches across different proof systems. In this regard, the OpenTheory project has done vital pioneering work. The OpenTheory article format is a low-level format for encoding proofs in a fixed logic. This file format can be read, written and interpreted by several different provers. The OpenTheory article format is prover independent in the sense that in order to use the proofs contained in an OpenTheory article, a prover only has to use a version of higher order logic that can simulate the primitive inference rules defined in the OpenTheory article standard. OpenTheory demonstrates that sharing of proof documents among provers is possible. The creation of a standard for formal sketches of mathematical articles would aim to do the same at higher level of abstraction.

OpenTheory achieves prover independence by compiling prover-specific proof tactics down to elementary inferences. The only way to achieve prover independence at the high level of abstraction that formal sketches aim for, is by removing all prover-specific tactics and library-specific references from the justifications of the declarative proof steps. (Incidentally, this removal of justifications is also Freek’s key step in converting a declarative formal proof into a formal proof sketch.) In this way, the higher level of abstraction is “bought” at the cost of introducing ambiguities into the formal proof sketch.

In terms of previous work, a standard for formal sketches of mathematical articles could thus be described by the slogan “OpenTheory for formal sketches of Mizar-style articles”. In the next section, however, I want to change tack and describe the properties a format for formal sketches should have, by drawing analogies to informal mathematical articles as they are used today. In particular, I will argue that informal mathematical articles are successful precisely because they are ambiguous.

What properties should the file format have?

The research articles that mathematicians all over the world write every day have several astonishing properties. Compared to most other texts humans write, even in the sciences, mathematical articles are extremely formal. Yet, compared to formal proofs in the sense of proof theory, mathematical articles are extremely informal, ambiguous, imprecise and even erroneous. There are three aspects of this inherent ambiguity of mathematical articles that I want to call particular attention to.

• Even though mathematical articles build on a huge amount of previous results, they manage to refer to this huge body of knowledge without becoming too verbose. This feat is achieved by three means: Mathematical articles typically 1) assume background knowledge on part of the reader, 2) include brief preliminary sections that summarize the most important parts of the required background knowledge, and 3) state the most important previous results they draw on as explicit theorems.
• Mathematical proofs are written in a declarative rather than in a procedural style. They consist of a sequence of true assertions, rather than a sequence of instructions on how exactly to get from one assertion to the next. This allows readers to make use of their own background knowledge and understanding of the subject matter and avoids a commitment to any particular set of rules.
• To make sense of a mathematical article in all its details requires an often significant effort on the part of the reader. Every mathematician has been in the situation of spending hours trying to understand a step in a proof that was (apparently) obvious to the author. This is made necessary by the previous two properties of mathematical articles mentioned above.

All three of these properties make mathematical articles more successful at communicating mathematical ideas. Mathematical articles would be less readable today and completely unintelligible 50 years from now, if they were extremely verbose, required the reader to refer to a particular textbook while reading and forced the reader to follow exactly the same train of thought the author used.

I think there is something we can learn from this:

Mathematical articles have been successful at communicating mathematics in the last centuries precisely because they are ambiguous. Formal mathematical articles have to embrace this ambiguity if they are to become successful.

Concretely, this has the following implications for a formal format for mathematical articles.

• Formal mathematical articles have to avoid binding themselves to any particular library of theorems or any particular proof system.
• Formal mathematical articles should be written in a declarative style.
• Formal mathematical articles have to place the burden of working out the details on the reader, no matter if the reader is a human or an automatic proof system.

These general considerations point to a declarative format for formal proof sketches in the Mizar style in which the explicit references to external theorems and prover specific tactics have been removed as far as possible.

From a pragmatic point of view, the removal of most explicit references to libraries and provers from formal sketches has two huge advantages for mathematicians wanting to formalize their articles:

• Mathematicians do not have to learn a (huge) dictionary or “API” to get started with formalization.
• Mathematicians do not have to fear that their formal sketches will become unreadable when the current generation of provers becomes obsolete.

These two factors, the tie-in and the steeper learning curve associated with binding a formal sketch to a particular API, are the two factors that keep me personally, as an everyday mathematician, from starting to formalize my research. (The danger that formal proofs break as prover technology changes is particularly problematic.) The removal of these two deterrents would, I hope, make formalization attractive to many other mathematicians as well.

The ingredients of a specification

So far, these are blue-sky ideas and it is way to early to try to turn these into an explicit specification of a formal sketch format (FSF). But, to get the ball rolling, I want to list some ingredients for such a specification that, I think, will be important for success, based on the above considerations.

As explained above, the starting point for FSF is a Mizar-style declarative proof language in the spirit of the mathematical vernacular. Proofs in FSF are abbreviated by omitting intermediate steps and in particular justifications. Prover-specific justifications are not allowed in formal sketches at all. To make FSF work, this basic concept should be extended in the following ways.

First of all, as this whole idea revolves about creating a cross-prover file format, close integration with OpenTheory is desirable. In particular:

• FSF should use the same logical foundation as OpenTheory.
• The packaging mechanism should be essentially the same as in OpenTheory. An FSF article imports theorems as axioms and promises to produce a bunch of new theorems as a result (if the reader/prover is clever enough). Note that while the set of exports is uniquely determined by the FSF article, the set of imports is not uniquely determined by the FSF article, but depends on the reader/prover as well.
• The benchmark for whether a prover $P$ “understands” formal sketch $S$ should be this: Is $P$ able to transform $S$ into an OpenTheory article $O$ on its own? $O$ is not uniquely determined by $S$, in particular its imports may depend on $P$. However, all imports of $O$ should be available in $P$’s library.
• FSF should have built-in support for citing theorems from the OpenTheory library.

The formal sketch format should not tie itself to OpenTheory exclusively, however, as independence of any particular system is crucial for the success of FSF. Instead:

• FSF has to embrace an open-world model.
• FSF has to allow all kinds of references to mathematics outside of a given formal sketch article, including references to theorems that are ambiguous or not machine-readable. Examples include:
  • informal citations of an informal mathematical source, like “Proposition 2.1 in Herbert Wilf, generatingfunctionology, 3rd Edition”,
  • explicitly stated theorems,
  • references to open libraries such as Isabelle’s Archive of Formal Proofs, and even
  • free format URLs.

It is important to note that automatic systems processing FSF should not be required to make sense of any of these. A mathematician does not check all references cited in a given article either.

As mentioned before, current provers will probably not be able to “understand” any interesting formal sketch at this time. While the hope is that provers will achieve that goal at least for some formal sketches in the not too distant future, there is definitely going to be a transition period in which provers need additional advice to cope with a given formal sketch. To this end, a supplemental file format for annotating FSF articles should be developed.

• An annotation file $A$ for an FSF article $S$ will consist of prover-specific advice for some steps of the proofs contained in $S$.
• There can be many annotation files for any given article $S$. Different annotation files may give contradictory proof advice.
• It is important that annotations reside in different files than the sketch $S$ itself, so that annotations may be changed, extended or removed while keeping $S$ fixed. Also, this prevents $S$ from becoming unnecessarily verbose.
• As explained above, FSF articles should have explicit justifications only in places where humans would mention these justifications in informal mathematical articles. However, independently of the number of justifications, FSF articles can be made easier for provers to “understand” if they contain many small intermediate steps. While these small steps may be very useful in early FSF articles, they pose the danger of making an FSF article less readable for humans. Therefore:
  • It should be possible to move these “small steps” into annotation files.
  • In the case that these “small steps” are kept in the formal sketch itself, the file format should contain hints indicating which parts of a proof should be “hidden” or “folded” at a first reading.

The goal is of course to progressively remove annotations as provers become more powerful, until the annotations can be omitted entirely.

People tend to have very strong opinions about the surface syntax of any data format. As Freek pointed out, even the different variations of Mizar-style file formats that are out there attach different meanings to the same keywords. Therefore:

• The FSF specification should fix one data model for formal sketches.
• FSF should allow for different serialization formats of this data model.
• The FSF specification should provide at least two different serialization formats:
  • a Mizar-style plain text serialization for human editing, and
  • a plain text serialization that lends itself to easy machine processing (using JSON, XML or Lisp-style S-expressions).

The formal sketch format needs to be supported by a convenient infrastructure. In particular, it will be useful to create

• an open repository for formal proof sketches, and
• standard interfaces for processing a proof sketch plus annotations with the most popular provers.

One might even imagine facilities to “mix-and-match” provers: Use HOL to do the first step in a proof, use Isabelle for the second and have both produce low-level output in OpenTheory format in the process. But of course the realization of such fancy ideas is even further off than a basic version of FSF. Which brings me to the conclusion of this post.

Next steps

Where to go from here? Before trying to go for a formal specification, a proof-of-concept is needed. Therefore, I plan to do the following:

• Create a rough draft of a formal sketch format and an accompanying format for annotations.
• Convert a couple of Freek’s formal proof sketches into this formal sketch format.
• Implement a basic interface for compiling the example sketches plus annotations into OpenTheory articles - for two different provers.
• Create annotations so that the formal proof sketches actually compile with these two different provers.
• Begin experiments on how the provers in question can be improved to do these same proofs with fewer annotations.

At that point, we will have a clearer idea of how the formal sketch format should look and what needs to be done to get this positive feedback loop of more prover-independent formalized mathematics and more powerful automatic provers going.

Comments on these ideas and help in this process are very welcome!

I would like to recommend a few articles on the current economic crisis and the Euro crisis in particular that give a much sounder picture of what is going on than can be found in the German news media at large.

• Paul Krugman - End this depression now!
• Jospeh Stigliz - After austerity
• James Surowiecki - The fairness trap
• Mark Cliffe - Roads to survival
• George Soros - Remarks at the festival of economics, Trento Italy
• Martin Wolf - The German response
• Dani Rodrik - The end of the world as we know it
• Gustav Horn - Die Rettung in Europa ist nahe!
• The Economist - The choice and Start the engines, Angela

Here is the gist of what I take away from all this:

I can understand the reflex to say “if you have lived beyond your means, you have to tighten your belt and save”. On the level of a single household or a single company that is sound advice. But it does not make sense at the level of an entire economy. As Paul Krugman puts it: “Your spending is my income and my spending is your income.” This means that I can only save (i.e, earn more than I spend) if there is somebody else who lives beyond his means (i.e., spends more than he earns). Therefore, it is simply impossible for everybody to save at the same time. But this is exactly what people, companies and governments all over the world are trying to do right now. People and companies try to save because of bleak economic prospects, governments try to save because they think they need to be austere. This creates a vicious cycle in which everybody tries to spend less then everybody else. The result is a depression in which nobody manages to save.

We are in a depression that is caused by a lack of demand. Therefore, propping up the supply side will not help in getting us out of this depression. So while structural reforms in the European periphery are certainly important, structural reforms alone will not get us out of this crisis, as long as demand remains depressed. And the austerity politics that Germany in particular is hell-bent on pursuing only depress demand even more.

This lack of demand creates massive unemployment (which further depresses demand). This is no accident. On the contrary, mass unemployment is the official goal of austerity politics. Austerity politics in Europe seek to achieve “internal devaluation” in the European periphery: Mass unemployment is supposed to reduce wages which in turn will restore the competitiveness of the periphery. Unfortunately, this does not work because of a phenomenon called “downward nominal wage rigidity”, which basically means that wages decrease less in a depression than they would increase in a boom. Moreover, the idea of internal devaluation disregards the catastrophic effect that mass unemployment has for the people, especially when they are young. The unemployed have to cope with huge psychological and economic pain (which even results in higher suicide rates). Also, the skills of the unemployed deteriorate which has a negative long-term effect on the economy as a whole.

An alternative means of balancing the differences in productivity between the core and the periphery is to create modest inflation in the core while keeping the inflation rate in the periphery close to zero. This would devalue the euro, making European exports more competitive on the world market. It would also create less unemployment and put less pressure on the demand in the periphery, allowing structural reforms to actually translate into economic activity.

Note that so far this has little to do with the rescue packages that seem to be the only thing that European crisis management revolves around. In fact, I think it is pointless for Germany to finance the debt of peripheral countries as long as there remains the slightest chance that peripheral governments might default. We have to make government default in Europe impossible, in exactly the same way that all other countries that are in control of their own currency do it as well: The ECB has to guarantee European sovereign debt, acting as a lender of last resort. Yes, this may amount to printing money. Yes, this will create inflation, but controlled inflation helps reducing the debt burden. Yes, this devalues the Euro, but that stimulates exports. Yes, this would amount to an implicit transfer payment from Germany to the periphery, but I am not against transfers. I am all for transfers as long as they make sense. And yes, of course this introduces a moral hazard that requires further political and financial integration to be kept in check.

German public opinion about the ever-recurring bail-out packages ranges from weary to hostile, because people rightly observe that these bail-out packages do not work but cost billions. Curiously, however, the reasons why these packages fail - from the refusal to change the ECBs role to the self-defeating mechanics of austerity - are rarely discussed in German media. Thus, most people do not realize that Merkel’s austerity is the cause of the problem. Instead, they are of the opinion that the compromises that Europeans extract from Merkel are to blame. As long as this self-serving ignorance of the German media continues, Germans will keep supporting Merkel’s policy and Merkel will keep being the hardliner that Germans expect her to be, even if Germany drops into recession.

Germans feel that they are being taken advantage of. The reasons for this lie in recent German domestic policy. The hard social security and labor market reforms that Schröder introduced with his Agenda 2010 hurt. Badly. And it took years for them to bring about economic benefits. And just now, when the sharp austerity measures that Germans themselves endured are starting to bear fruit and lead Germany into this economic boom, other countries ask Germany to step up and pay the bill for them. They didn’t do their homework and now they don’t want to endure what Germany has endured. Of course this is not entirely true, but I would claim that this is what is going on in the “German subconsciousness”. It also explains the Germans’ insistence on structural reform and their focus long-term economic outlook: these are the measures that worked for them, so these are the only measures that will work for anyone else. “There is no easy way out, so do not ask us to pay for short-term relief.”

Unfortunately, what worked for Germany in the previous decade cannot work for Europe today. There are many reasons for this. The most important one is that Germany could flourish because of its export-oriented economy. It could behave like a household that tries to earn more and spend less, because there were other countries who spent more than they earned. Now, we are in a global depression, where everybody spends too little. So the only way out is to lead Europe like an economy, not like a household. And this means stimulating demand.

Of course this will have to be at the expense of Germany. Whether this is fair or not is beside the point. It is necessary, if we do not want the world to slide into a Second Great Depression. Of course, all the unfairness entailed in and all the moral hazard created in this process will have to be addressed: No, it is not fair that the 99% pay for a crisis that the 1% caused and from which they even profited. No, it does not make sense that Germans have to work until they are 67 years old or even longer, just so that the French can enjoy an early retirement at the age of 60. And no, one country should not borrow without limit at the expense of another. But these issues can only be resolved through tighter European integration and cooperation. Trying to force tighter integration now, in the middle of this self-inflicted depression, will succeed only in fostering anti-European sentiment on all sides, without accomplishing anything.

Europe is worth paying for. And even those who do not agree should realize that this crisis will cost Germany huge amounts of money, one way or another. But when we invest these resources we have to make it count. Any workable solution needs to have widespread support in Europe. To that end we have to finally start listening to each other. We need to create a true European public and start a true dialog across national boundaries.

To learn HOL light, I decided to pick a simple theorem (from the theory of formal power series) and try to formalize it, both in a procedural and in a declarative style. In this blog post, I document what I have learned during this exercise.

My point of view is that of an everyday mathematician, who would like to use formal proof systems to produce formal proofs to go along with his everyday research papers. I realize of course, that this is not practical yet, given the current state of formal proof systems. But my goal is to get an idea of just how far away this dream is from reality.

During my experiments the hol-info mailing list was a huge help. In particular, I would like to thank Freek Wiedijk, Petros Papapanagiotou and John Harrison.

I will begin by giving a short summary of the informal proof and its two formal versions. This should give you a general idea of what all of this is about. If you want more details, you can then refer to the other parts of this post. Here is an outline:

• Summary: One Theorem and Three Proofs
• Meeting HOL Light
• Formal Definitions
• A Procedural Proof
• Mizar in HOL: miz3
• A Declarative Proof
• Making Declarative Proofs more Self-Contained

So, without further ado, let’s start with a summary.

Summary: One Theorem and Three Proofs

The theorem that I want to prove formally is simply that every formal power series with non-zero constant term has a reciprocal - and this reciprocal has a straightforward recursive definition. In everyday, informal mathematical language, this theorem and its proof might be stated like this.

Theorem. Let $f=\sum_{n\geq 0} a_n x^n$ be a formal power series with $a_0\not = 0$. Then, the series $g=\sum_{n\geq 0} b_n x^n$ with \begin{eqnarray*} b_0 & = & \frac{1}{a_0} \cr b_n & = & -\frac{1}{a_0}\sum_{k = 1}^n a_k b_{n-k} \; \forall n\geq 1 \end{eqnarray*} is a reciprocal of $f$, i.e., $f\cdot g = 1$.

Proof. To see that $g$ is a reciprocal of $f$, we have to show that \[ f\cdot g = \sum_{n \geq 0} \sum_{k=0}^n a_k b_{n-k} = 1 \] which means that for all $n\in\mathbb{Z}_{\geq 0}$ we have \[ \sum_{k=0}^n a_k b_{n-k} = \left\{ \begin{array}{ll} 1 & \text{if } n=0 \cr 0 & \text{if } n\geq 1 \end{array} \right. \] To show this, we proceed by induction. If $n=0$, then we have \[ \sum_{k=0}^n a_k b_{n-k} = a_0 b_0 = a_0 \frac{1}{a_0} = 1 \] as desired. For the induction step, let $n\in\mathbb{Z}_{\geq 0}$ and assume that the identity holds for $n$. We are now going to show that it also holds for $n+1$. To this end we calculate \begin{eqnarray*} \sum_{k=0}^{n+1} a_k b_{n+1-k} & = & a_0 b_{n+1} + \sum_{k=1}^{n+1} a_k b_{n+1-k} \ &=& a_0 \left( -\frac{1}{a_0}\sum_{k = 1}^{n+1} a_k b_{n+1-k} \right) + \sum_{k=1}^{n+1} a_k b_{n+1-k} \ &=& 0. \end{eqnarray*} End of Proof.

This proof is very simple and straightforward. Therefore, many authors would try to be more terse. For example, in Wilf’s classic text book with the wonderful title generatingfunctionology this theorem is Proposition 2.1 and its proof does not even state the above calculation explicitly. Personally, I think the above proof is just about the right length.

You may have noticed that in the above proof, we did not use the induction hypothesis in the induction step. Indeed, the above proof can be done just using case analysis, without induction at all. I will nonetheless stick with the induction version of the proof, because it is more idiomatic when formalized in HOL Light.

Now, let’s see how the above proof looks when rendered in a formal proof language. In fact, we are going to do this proof in two formal proof languages: a procedural and a declarative one.

We first need to make some definitions for working with generating functions (aka formal power series). I chose to model a generating function $f$ simply as a function $f:\mathbb{N}\rightarrow\mathbb{R}$. So, if $f$ corresponds to $\sum_{n\geq 0} a_n x^n$ in the power series notation, then $f(n)=a_n$. With this approach, addition, multiplication and various other things can be defined as follows. See the section on Formal Definitions below for details.

1
2
3
4
5

let gf_plus = new_definition `gf_plus = \(f:num->real) (g:num->real) (n:num). (f n) + (g n) :real`;; 
let gf_times = new_definition `gf_times = \(f:num->real) (g:num->real) (n:num). (sum (0 .. n) (\k. (f k)*(g (n-k)))) :real`;;
let gf_one = new_definition `gf_one = \(n:num). if n = 0 then &1 else &0`;;
let gf_reciprocal = new_definition `gf_reciprocal = \(f:num->real) (g:num->real). gf_times f g = gf_one`;;
let recip = define `recip f (0:num) =  ((&1:real)/(f 0) :real) /\  recip f (n+1) = -- (&1/(f 0)) * sum (1..(n+1)) (\k. (f k) * recip f ((n + 1) - k))`;;

With these definitions, the theorem we are trying to prove is this:

1

`!(f:num->real). ~(f (0:num) = (&0:real)) ==> gf_reciprocal f (recip f)`

Note how the recursive formula for the reciprocal of $f$ has been wrapped in the definition of recip. The relation gf_reciprocal is used abbreviate the statement that recip f is a reciprocal of f.

Now we are ready to give the two formal versions of our above proof of this theorem. We start with the procedural version. It looks like this:

1
2
3
4
5
6
7

let GF_RECIP_EXPLICIT_FORM = prove (`!(f:num->real). ~(f (0:num) = (&0:real)) ==> gf_reciprocal f (recip f)`,
  GEN_TAC THEN DISCH_TAC THEN REWRITE_TAC[gf_reciprocal;gf_times;gf_one] THEN REWRITE_TAC[FUN_EQ_THM] 
  THEN INDUCT_TAC THEN ASM_SIMP_TAC[NUMSEG_SING;SUM_SING] THEN CONV_TAC NUM_REDUCE_CONV 
  THEN REWRITE_TAC[recip] THEN SIMP_TAC[REAL_ARITH `x * &1 / x = x / x`] THEN ASM_SIMP_TAC[REAL_DIV_REFL] 
  THEN REWRITE_TAC[MATCH_MP (SPEC `(\k. f k * recip f (x - k))` SUM_CLAUSES_LEFT) (SPEC_ALL LE_0);SUB_0;ADD_0;ADD_AC;ADD1;recip] 
  THEN (FIRST_ASSUM (ASSUME_TAC o (MATCH_MP (REAL_FIELD `!x. ~(x = &0) ==> !a. x * -- (&1 / x) * a = -- a`)))) 
  THEN ASM_REWRITE_TAC[] THEN SIMP_TAC[REAL_FIELD `--a + a = &0`] THEN ASM_SIMP_TAC[ARITH_RULE `~(x + 1 = 0)`]);;

I will explain this proof in detail in the section A Procedural Proof below. For now, the important thing to note here is simply this: Presented in this way, the procedural proof is completely unintelligible. To make sense of procedural proofs, they have to be run interactively. I will walk you through such an interactive run of this proof below and try to explain what is going on in the process. The gist of it is this: Procedural proofs like this one consist of a sequence of proof tactics. Tactics are essentially functions like REWRITE_TAC that are written in OCaml and that produce a part of the proof as their output. They may take pre-proved theorems like FUN_EQ_THM as their arguments.

This way of writing proofs is opposite to the way humans write informal proofs. Take for example an equational proof, i.e., a proof done by rewriting a term step by step to show an equality. Humans would write down the intermediate expressions obtained during the process. In the procedural style, you write down the rule you apply to move from one expression to the next.

While in some ways, this procedural style of carrying out a proof may in fact be closer to how humans think about proofs, this procedural notation has huge drawbacks when it comes to communicating proofs. To me, the most important drawback is that the meaning of the proof is hidden almost entirely within the library of tactics and theorems that it is built upon. Without constantly referring to a dictionary of all these ALL_CAPS_SYMBOLS it impossible to read or write these proofs at all. This is problematic insofar as it adds an additional layer of complexity to the problem of reading or writing a proof: You not only have to know the subject matter, you also have to bind yourself to a particular “API”. This not only makes proofs less human readable, it also makes procedural proofs stored in this fashion “bit-rot at an alarming rate”.

Of course, the procedural style also has its advantages, most importantly the way it lends itself to automation of proof methods. However, as I stated in the beginning, my primary concern is the translation of everyday informal mathematical articles into a formal language. Which brings us to the declarative proof style.

The declarative version of the proof that I came up with is the following:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

let GF_RECIP_EXPLICIT_FORM = thm `;
  !(f:num->real). ~(f (0:num) = (&0:real)) ==> gf_reciprocal f (recip f)
  proof
    let f be num->real;
    assume ~(f 0 = &0) [1];
    !n. sum (0..n) (\k. f k * recip f (n - k)) = (if n = 0 then &1 else &0)
    proof
      sum (0..0) (\k. f k * recip f (0 - k)) = f 0 * recip f (0 - 0) by NUMSEG_SING,SUM_SING;
        .= f 0 * recip f 0;
        .= f 0 * &1 / f 0 by recip;
        .= f 0 / f 0;
        .= &1 by REAL_DIV_REFL,1;
        .= (if 0 = 0 then &1 else &0) [2];
      !n. sum (0..n) (\k. f k * recip f (n - k)) = (if n = 0 then &1 else &0)
          ==> sum (0..SUC n) (\k. f k * recip f (SUC n - k)) = (if SUC n = 0 then &1 else &0) [3]
      proof
        let n be num;
        assume sum (0..n) (\k. f k * recip f (n - k)) = (if n = 0 then &1 else &0);
        !x. ~(x = &0) ==> !a. x * -- (&1 / x) * a = -- a [4];
        sum (0..SUC n) (\k. f k * recip f (SUC n - k)) = (\k. f k * recip f (SUC n - k)) 0 + sum (0 + 1..SUC n) (\k. f k * recip f (SUC n - k)) by SUM_CLAUSES_LEFT,LE_0;
          .= (f 0 * recip f (SUC n)) + sum (1..SUC n) (\k. f k * recip f (SUC n - k)) by ADD_0,SUB_0,ADD_AC; 
          .= (f 0 * recip f (n + 1)) + sum (1..(n + 1)) (\k. f k * recip f ((n + 1) - k));
          .= (f 0 * -- (&1/(f 0)) * sum (1..(n + 1)) (\k. (f k) * recip f ((n + 1) - k))) + sum (1..(n + 1)) (\k. f k * recip f ((n + 1) - k)) by recip;
          .= -- sum (1..(n + 1)) (\k. (f k) * recip f ((n + 1) - k)) + sum (1..(n + 1)) (\k. f k * recip f ((n + 1) - k)) by 1,4;
          .= &0;
          .= (if SUC n = 0 then &1 else &0);
      qed;
    qed by INDUCT_TAC from 2,3;
    (\n. sum (0..n) (\k. f k * recip f (n - k))) = (\n. (if n = 0 then &1 else &0)) [5] by FUN_EQ_THM;
  qed by gf_reciprocal,gf_times,gf_one,1 from 5;
`;;

To my eye, this looks wonderful! This is an entirely formal proof, that

• is very close to the informal version,
• is not that much longer than the informal version,
• is human readable,
• uses very few references to an external “dictionary” or “API” (not all steps have to be justified!),
• and all external references refer to theorems and not to tactics.

Of course there are still many aspects that can be improved. First of all, while this ASCII syntax is quite readable (much more readable than LaTeX anyway), the ability to edit a fully typeset version of the proof would be a pleasant convenience. I will definitely look into creating a custom transformer for Qute for this syntax.

More importantly, some steps in this proof seem unnecessarily small. For example, when calculating sum (0..0) (\k. f k * recip f (0 - k)), it would be great to go directly to f 0 * recip f 0 without using f 0 * recip f (0 - 0) as a stepping stone. In a similar vein, I would like to avoid stating trivial lemmas like !x. ~(x = &0) ==> !a. x * -- (&1 / x) * a = -- a explicitly, since in the line where it is used, it is easy to infer from context what lemma is needed here. Finally, I think it is very important to reduce API dependencies even further and so I would like to drop explicit references to such “trivial” library theorems as ADD_0 or ADD_AC.

More details about the declarative proof system and the construction of this proof make up the last part of this post.

On the whole, I am extremely impressed with the state of current formal proof systems. While a lot remains to be done until everyday mathematicians not specialized in formal proof can routinely produce formal versions of their papers, this dream appears to be almost within reach.

After this summary, I am now going to get my hands dirty and explain all of this in detail.

Meeting HOL Light

The proof system I used for these experiments is HOL Light. HOL stands for “higher order logic” which basically means classical mathematical logic with the additional feature that quantifiers can range over functions as well and not only over elements of the ground set. (In practice, working with formal higher order logic is not too different from the mix of “colloquial” classical logic and set theory that I, as a mathematician not specialized in logic, am used to working with.) Higher order logic is one of the standard foundations of mathematics used in the formal proof community and several major proof systems such as Isabelle/HOL, HOL4, ProofPower and HOL Light work with it.

In this post, I will not assume familiarity with HOL Light and I hope to keep things reasonably self-contained. Nonetheless, I want to recommend some references as starting points for further reading: Freek Wiedijk has done some very helpful work comparing and contrasting the different provers out there, see for example the list of the Seventeen Provers of the World which he compiled. To get started with HOL Light, I recommend the great HOL Light Tutorial by John Harrison. When doing a proof, it is useful to have the quick reference at hand.

Installing HOL Light is non-trivial. First, you need to have a working installation of OCaml. Second, you will need Camlp5, which has to be compiled with the -strict configuration option. This is often not the case for camlp5 packages that ship with major Linux distributions. Third, you need to fetch HOL Light from its SVN repository and compile it. I will not walk through these steps in detail, but I can recommend a script by Alexander Krauss that does these things for you (and that you can read if you need instructions on how to do this by hand).

After everything is installed, you run OCaml from the HOL Light source directory and then, from the OCaml toplevel, start HOL Light with the command

1

#use "hol.ml";;

In the rest of this section, I will give a very short summary of the basics of HOL Light. As an introduction to HOL Light, this whirlwind tour is of course woefully inadequate. It is mainly intended as a brief orientation for readers unfamiliar with HOL who do not want to read the tutorial first. Either way, I hope that these brief notes will help in making sense of the exercise that this post is about.

• Proofs in HOL Light are typically developed interactively at the OCaml toplevel.
• In HOL Light, mathematical expressions in higher order logic are enclosed in backquotes. These expressions have the OCaml type term. For example, if you enter `1 + 1 = 2` at the OCaml toplevel (followed by two semicolons and return) you will see:

  1 2	# `1 + 1 = 2`;; val it : term = `1 + 1 = 2`

• The HOL Light syntax for higher order logic is straightforward. For example the principle of induction on the natural numbers \[ \forall P: P(0) \wedge \left( \forall n: P(n) \Rightarrow P(S(n)) \right) \Rightarrow (\forall n. P(n)) \] would be written as

  1	!P. P 0 /\ (!n. P n ==> P (SUC n)) ==> (!n. P n)

  where SUC is a predefined constant, $\forall$ is written as ! and $\wedge$ is written as /\. Here is the notation for some other useful symbols: $\exists$ is written as ?, $\vee$ is written as \/, $\neg$ is written as ~ and $\Leftrightarrow$ is written as <=>. Lambda expressions are written using \, e.g., $\lambda x. x + 1$ is written as \x. x + 1.
• Higher order logic in HOL Light is typed. As long as HOL Light can figure out the types on its own, you don’t need to specify them. If you want to give type information explicitly, you can use the colon. For example, here is the principle of induction on lists:

  1	!P:(A)list->bool. P [] /\ (!h t. P t ==> P (CONS h t)) ==> !l. P l

• Theorems in HOL Light are objects of type thm. The only way to produce an object of type theorem in HOL Light is to prove that theorem.
• Inference rules are special OCaml functions that produce theorems. They encapsulate proof methods. For example ARITH_RULE is a function of type term->thm that uses linear arithmetic methods to prove statements about natural numbers. For example we can produce the theorem $\vdash 1 + 1 = 2$ by

  1 2	# ARITH_RULE `1 + 1 = 2`;; val it : thm = |- 1 + 1 = 2

  or we can produce the theorem $x > 0 \vdash x \geq 1$ by

  1 2	# UNDISCH (ARITH_RULE `x > 0 ==> x >= 1`);; val it : thm = x > 0 |- x >= 1

  where UNDISCH takes a theorem of the form $\vdash p \Rightarrow q$ and produces the theorem $p \vdash q$.
• A goal captures a claim of the form $p_1,\ldots,p_n\vdash q$ that we are currently trying to prove but have not proved yet. In the HOL Light documentation, the distinction is made clear using the ?- notation, e.g., the goal that $x > 0 \vdash x \geq 1$ might be written as x > 0 ?- x >= 1. On the OCaml toplevel (a stack of goals containing) this goal would be printed as:

  1 2 3 4 5

  val it : goalstack = 1 subgoal (1 total) 0 [`x > 0`] `x >= 1`

  where the expression in square brackets represents the assumption of that goal.
• A tactic is a function that (essentially) takes a goal and produces a list of subgoals, such that a proof of all subgoals produces a proof of the original goal. This allows backward proofs. These are the typical way of working with HOL Light.
• An example. Suppose we want to prove $x\not=0 \Rightarrow 1 \leq x$ using the lemma $0 < x \Rightarrow 1 \leq x$. We store the lemma in an OCaml variable.

  1 2	# let lemma = ARITH_RULE `0 < x ==> 1 <= x`;; val lemma : thm = |- 0 < x ==> 1 <= x

  Then we set up our goal with the function g.

  1 2 3 4 5

  # g `~(x=0) ==> 1 <= x`;; Warning: Free variables in goal: x val it : goalstack = 1 subgoal (1 total) `~(x = 0) ==> 1 <= x`

  Next we apply the tactic DISCH_TAC using the function e. This reduces the goal $\vdash x\not=0 \Rightarrow 1 \leq x$ to $x\not=0 \vdash 0 \Rightarrow 1$.

  1 2 3 4 5 6

  # e DISCH_TAC;; val it : goalstack = 1 subgoal (1 total) 0 [`~(x = 0)`] `1 <= x`

  Now we apply our lemma to “simplify” the conclusion of our goal, using MATCH_MP_TAC which is of type thm->tactic. MATCH_MP_TAC uses the theorem $\vdash p \Rightarrow q$ to reduce the goal $\vdash q$ to $\vdash p$, making instantiations as necessary.

  1 2 3 4 5 6

  # e(MATCH_MP_TAC(lemma));; val it : goalstack = 1 subgoal (1 total) 0 [`~(x = 0)`] `0 < x`

  What is now left to prove is the content of the theorem LT_NZ in the library.

  1 2	# LT_NZ;; val it : thm = |- !n. 0 < n <=> ~(n = 0)

  We make use of this theorem via ASM_REWRITE_TAC of type thm list -> tactic, which rewrites goals by the list of (equational) theorems it is passed, taking assumptions into account.

  1 2	# e(ASM_REWRITE_TAC[LT_NZ]);; val it : goalstack = No subgoals

  The statement that there are no subgoal left, means that the proof is complete. We can get the theorem we just proved using top_thm.

  1 2	# top_thm();; val it : thm = |- ~(x = 0) ==> 1 <= x

  (Note that this example is non-sensical insofar as we could have used ARITH_RULE to solve our goal immediately.)
• If you make a mistake during one of these goalstack-style proofs, you can use b to back up one step, by calling b();;.

After this minimal introduction to HOL Light, we can now turn to formalizing the above proof.

Formal Definitions

Before we can start with the proof proper, however, we need to make a couple of definitions.

For the purposes of this exercise a formal power series or generating function will be simply a map $f:\mathbb{N}\rar\mathbb{R}$. Of course you could also view $f$ as a sequence of real numbers or, classically, as the sequence of coefficients of a power series. Addition of formal power series is defined componentwise, i.e., $(f+g)(n)=f(n)+g(n)$, which in HOL Light looks as follows:

1

let gf_plus = new_definition `gf_plus = \(f:num->real) (g:num->real) (n:num). (f n) + (g n) :real`;;

The product of two generating functions is defined via the Cauchy product rule \[ (f\cdot g)(n) = \sum_{k=0}^n f(k)\cdot g(n-k). \] This looks much more natural when phrased in terms of formal series and this is the reason why formal power series are traditionally written in power series notation: \[ \sum_{n\geq 0}a_nx^n \cdot \sum_{n\geq 0} b_n x^n = \sum_{n\geq0} \left(\sum_{k=0}^n a_k b_{n-k}\right) x^n. \] Personally, I think of this “product” more as a “Minkowski sum”, but that will be the subject of another blog post. The HOL Light version of this definition is straightforward:

1

let gf_times = new_definition `gf_times = \(f:num->real) (g:num->real) (n:num). (sum (0 .. n) (\k. (f k)*(g (n-k)))) :real`;;

Next, we define a constant gf_one which represents the formal power series $1$, i.e., the function $f$ with $f(0)=1$ and $f(n)=0$ for all other $n$.

1

let gf_one = new_definition `gf_one = \(n:num). if n = 0 then &1 else &0`;;

Now, it is straightforward to define a binary relation gf_reciprocal that specifies when a generating function $g$ is the reciprocal of a generating function $f$, that is, $f\cdot g = 1$.

1

let gf_reciprocal = new_definition `gf_reciprocal = \(f:num->real) (g:num->real). gf_times f g = gf_one`;;

Finally, the theorem we will prove contains a recursive formula for the reciprocal of a generating function $f$, namely: \begin{eqnarray*} b_0 & = & \frac{1}{a_0} \cr b_n & = & -\frac{1}{a_0}\sum_{k = 1}^n a_k b_{n-k} \; \forall n\geq 1 \end{eqnarray*} We will use the shorthand recip f to denote the function defined by this recursive formula.

1

let recip = define `recip f (0:num) =  ((&1:real)/(f 0) :real) /\  recip f (n+1) = -- (&1/(f 0)) * sum (1..(n+1)) (\k. (f k) * recip f ((n + 1) - k))`;;

Note that these definitions all return theorems. In addition they have the side effect of modifying the OCaml variable the_definitions, see fusion.ml. This has the drawback that you cannot change existing definitions within a running HOL Light session. You will have to restart HOL Light to change definitions. (If you want to learn more about definitions and state in HOL, you may want to look at these papers.)

A Procedural Proof

The theorem that we want to prove says that if $f(0)\not= 0$ then $\text{recip}(f)$, as defined in the previous section, is a reciprocal of $f$. In HOL this statement reads simply as follows.

1

!(f:num->real). ~(f (0:num) = (&0:real)) ==> gf_reciprocal f (recip f)

Note that 0 refers to the natural number $0$, whereas &0 refers to the real number $0$ - which are distinct in HOL. Real number literals always have to be prefixed with & in HOL. The type annotations given above are not essential. We could also write simply !f. ~(f 0 = &0) ==> gf_reciprocal f (recip f).

In this section, we will go through the procedural proof given in the summary at the beginning step by step, and take a look at the goalstack at each point. If you want to expand other proofs in a similar fashion, you may want to take a look at the tool Tactician.

We start out by pushing the theorem we want to prove onto the goalstack.

1
2
3
4

# g `!(f:num->real). ~(f (0:num) = (&0:real)) ==> gf_reciprocal f (recip f)`;;
val it : goalstack = 1 subgoal (1 total)

`!f. ~(f 0 = &0) ==> gf_reciprocal f (recip f)`

Next, we eliminate the universal quantifier and the implication, turning $f(0) \not = 0$ into an assumption, and we expand the definitions.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

# e(GEN_TAC);;

val it : goalstack = 1 subgoal (1 total)

`~(f 0 = &0) ==> gf_reciprocal f (recip f)`

# e(DISCH_TAC);;

val it : goalstack = 1 subgoal (1 total)

  0 [`~(f 0 = &0)`]

`gf_reciprocal f (recip f)`

#  e(REWRITE_TAC[gf_reciprocal;gf_times;gf_one]);;

val it : goalstack = 1 subgoal (1 total)

  0 [`~(f 0 = &0)`]

`(\n. sum (0..n) (\k. f k * recip f (n - k))) =
 (\n. if n = 0 then &1 else &0)`

We have to show that two functions are equal. This boils down to showing that their values are equal for all arguments. This is the content of FUN_EQ_THM, which we apply using REWRITE_TAC.

1
2
3
4
5
6
7

# e(REWRITE_TAC[FUN_EQ_THM]);;

val it : goalstack = 1 subgoal (1 total)

  0 [`~(f 0 = &0)`]

`!x. sum (0..x) (\k. f k * recip f (x - k)) = (if x = 0 then &1 else &0)`

Now we start working at the heart of the problem. Just as in the informal proof, we are going to use induction, even though a more elementary case analysis would work as well. To use induction we apply INDUCT_TAC which leaves us with two subgoals: The base of the induction and the induction step.

1
2
3
4
5
6
7
8
9
10
11
12
13

# e(INDUCT_TAC);;

val it : goalstack = 2 subgoals (2 total)

  0 [`~(f 0 = &0)`]
  1 [`sum (0..x) (\k. f k * recip f (x - k)) = (if x = 0 then &1 else &0)`]

`sum (0..SUC x) (\k. f k * recip f (SUC x - k)) =
 (if SUC x = 0 then &1 else &0)`

  0 [`~(f 0 = &0)`]

`sum (0..0) (\k. f k * recip f (0 - k)) = (if 0 = 0 then &1 else &0)`

We start working on the base of the induction, i.e., the second subgoal. All we simplify the equality we have to prove, by evaluating the if .. then .. else .. and using the fact that the sum over an interval that contains just $0$ can be obtained by evaluating the expression we sum over at $0$. This latter fact follows from the lemmas NUMSEG_SING and SUM_SING.

1
2
3
4
5
6

# e(ASM_SIMP_TAC[NUMSEG_SING;SUM_SING]);;
val it : goalstack = 1 subgoal (2 total)

  0 [`~(f 0 = &0)`]

`f 0 * recip f (0 - 0) = &1`

The next couple of steps are about calculating with numbers and we use a couple of different methods for doing so. NUM_REDUCE_CONV is a so-called conversion, that can be used to simplify expressions as part of a bigger expression, and conversions can be turned into tactics using CONV_TAC. NUM_REDUCE_CONV evaluates expressions over natural numbers and this allows us to reduce $0-0$ to $0$, without knowing the name of a lemma in the HOL library that says so. Afterward, we expand recip according to its definition.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

# e(CONV_TAC NUM_REDUCE_CONV);;
val it : goalstack = 1 subgoal (2 total)

  0 [`~(f 0 = &0)`]

`f 0 * recip f 0 = &1`

# e(REWRITE_TAC[recip]);;

val it : goalstack = 1 subgoal (2 total)

  0 [`~(f 0 = &0)`]

`f 0 * &1 / f 0 = &1`

Next, we want to go from $f(0)\cdot \frac{1}{f(0)}$ to $\frac{f(0)}{f(0)}$. Again we don’t know the exact name of a lemma in the library that allows us to get there. However, the library does contain a general purpose inference rule REAL_ARTIH that can prove certain identities of real numbers automatically. We use this to prove the lemma we need on the fly. After this, all that is left to show is that $\frac{f(0)}{f(0)}=1$ under the assumption that $f(0)\not=0$. The appropriate lemma in the library is REAL_DIV_REFL. To apply it, we use ASM_SIMP_TAC which uses not only the theorem it is passed as argument, but also takes into account the available assumptions. This, then, completes our proof of the base of the induction, and we continue to the next goal on the stack: the induction step.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

# e(SIMP_TAC[REAL_ARITH `x * &1 / x = x / x`]);;
val it : goalstack = 1 subgoal (2 total)

  0 [`~(f 0 = &0)`]

`f 0 / f 0 = &1`

# e(ASM_SIMP_TAC[REAL_DIV_REFL]);;
val it : goalstack = 1 subgoal (1 total)

  0 [`~(f 0 = &0)`]
  1 [`sum (0..x) (\k. f k * recip f (x - k)) = (if x = 0 then &1 else &0)`]

`sum (0..SUC x) (\k. f k * recip f (SUC x - k)) =
 (if SUC x = 0 then &1 else &0)`

The first thing that we want to do here is to split off the first summand of the sum. The theorem in the library that says we can split off the first term in a sum is SUM_CLAUSES_LEFT.

1
2

# SUM_CLAUSES_LEFT;;
val it : thm = |- !f m n. m <= n ==> sum (m..n) f = f m + sum (m + 1..n) f

We want to use this theorem for $f=\lambda k. f(k)\cdot \text{recip}(f,x-k)$, $m=0$ and $n=n$. We can achieve these specializations as follows. Here LE_0 is the theorem $\forall n. 0 \leq n$ and SPEC_ALL LE_0 is simply the theorem $0 \leq n$. Note that $n$ refers to a natural number here.

1
2
3
4
5
6
7
8
9
10
11
12

# let sum_lemma = (SPEC `(\k. f k * recip f (x - k))` SUM_CLAUSES_LEFT);;
val sum_lemma : thm =
  |- !m n.
         m <= n
         ==> sum (m..n) (\k. f k * recip f (x - k)) =
             (\k. f k * recip f (x - k)) m +
             sum (m + 1..n) (\k. f k * recip f (x - k))
# let split_sum_lemma = MATCH_MP sum_lemma (SPEC_ALL LE_0);;
val split_sum_lemma : thm =
  |- sum (0..n) (\k. f k * recip f (x - k)) =
     (\k. f k * recip f (x - k)) 0 +
     sum (0 + 1..n) (\k. f k * recip f (x - k))

We use our split_sum_lemma to split the sum in question. After some simplifications, the first occurrence of $\text{recip}$ in the expression is in the exact form we need in order to be able to apply the recursive definition of $\text{recip}$.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

# e(REWRITE_TAC[split_sum_lemma]);;
val it : goalstack = 1 subgoal (1 total)

  0 [`~(f 0 = &0)`]
  1 [`sum (0..x) (\k. f k * recip f (x - k)) = (if x = 0 then &1 else &0)`]

`f 0 * recip f (SUC x - 0) +
 sum (0 + 1..SUC x) (\k. f k * recip f (SUC x - k)) =
 (if SUC x = 0 then &1 else &0)`

# e(REWRITE_TAC[SUB_0;ADD_0;ADD_AC;ADD1]);;
val it : goalstack = 1 subgoal (1 total)

  0 [`~(f 0 = &0)`]
  1 [`sum (0..x) (\k. f k * recip f (x - k)) = (if x = 0 then &1 else &0)`]

`f 0 * recip f (x + 1) + sum (1..x + 1) (\k. f k * recip f ((x + 1) - k)) =
 (if x + 1 = 0 then &1 else &0)`

# e(REWRITE_TAC[recip]);;

val it : goalstack = 1 subgoal (1 total)

  0 [`~(f 0 = &0)`]
  1 [`sum (0..x) (\k. f k * recip f (x - k)) = (if x = 0 then &1 else &0)`]

`f 0 * --(&1 / f 0) * sum (1..x + 1) (\k. f k * recip f ((x + 1) - k)) +
 sum (1..x + 1) (\k. f k * recip f ((x + 1) - k)) =
 (if x + 1 = 0 then &1 else &0)`

Now we want to cancel the two occurrences of $f(0)$ in the numerator and the denominator. Standard facilities for working with expressions of real numbers, such as REAL_RAT_REDUCE_CONV, do not suffice here, as we have to take the assumption that $f(0)\not=0$ into account. However, the lemma that we need in this situation can be proved automatically by REAL_FIELD.

1
2
3
4
5

#  let calc_lemma = (REAL_FIELD `!x. ~(x = &0) ==> !a. x * -- (&1 / x) * a = -- a`);;
2 basis elements and 0 critical pairs
3 basis elements and 1 critical pairs
3 basis elements and 0 critical pairs
val calc_lemma : thm = |- !x. ~(x = &0) ==> (!a. x * --(&1 / x) * a = --a)

We need to specialize this lemma to make use of it. The expression FIRST_ASSUM (ASSUME_TAC o (MATCH_MP calc_lemma)) is idiomatic for this purpose. FIRST_ASSUM tac tries to apply the tactic tac to each assumption in turn, until it finds one on which tac succeeds. In our case ASSUME_TAC o (MATCH_MP calc_lemma) succeeds immediately on the first assumption. MATCH_MP calc_lemma applied to the assumption ~(f 0 = &0) makes the substitution $x=f(0)$ in calc_lemma and produces the theorem !a. f 0 * --(&1 / f 0) * a = --a. This is then added as an assumption to our goal by ASSUME_TAC. Afterward, ASM_REWRITE_TAC is able to perform the simplification we intended.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

# e(FIRST_ASSUM (ASSUME_TAC o (MATCH_MP calc_lemma)));;
val it : goalstack = 1 subgoal (1 total)

  0 [`~(f 0 = &0)`]
  1 [`sum (0..x) (\k. f k * recip f (x - k)) = (if x = 0 then &1 else &0)`]
  2 [`!a. f 0 * --(&1 / f 0) * a = --a`]

`f 0 * --(&1 / f 0) * sum (1..x + 1) (\k. f k * recip f ((x + 1) - k)) +
 sum (1..x + 1) (\k. f k * recip f ((x + 1) - k)) =
 (if x + 1 = 0 then &1 else &0)`

# e(ASM_REWRITE_TAC[]);;
val it : goalstack = 1 subgoal (1 total)

  0 [`~(f 0 = &0)`]
  1 [`sum (0..x) (\k. f k * recip f (x - k)) = (if x = 0 then &1 else &0)`]
  2 [`!a. f 0 * --(&1 / f 0) * a = --a`]

`--sum (1..x + 1) (\k. f k * recip f ((x + 1) - k)) +
 sum (1..x + 1) (\k. f k * recip f ((x + 1) - k)) =
 (if x + 1 = 0 then &1 else &0)`

For the last simplification we again create the necessary theorem on the fly, without looking it up in the library.

1
2
3
4
5
6
7
8
9

# e(SIMP_TAC[REAL_FIELD `--a + a = &0`]);;
1 basis elements and 0 critical pairs
val it : goalstack = 1 subgoal (1 total)

  0 [`~(f 0 = &0)`]
  1 [`sum (0..x) (\k. f k * recip f (x - k)) = (if x = 0 then &1 else &0)`]
  2 [`!a. f 0 * --(&1 / f 0) * a = --a`]

`&0 = (if x + 1 = 0 then &1 else &0)`

The last thing we need to do is to show that the right-hand side of this equality is $0$, using the fact that $x+1\not=0$ for any natural number $x$. The proof is then complete and we can retrieve the theorem we just proved using top_thm.

1
2
3
4
5

# e(ASM_SIMP_TAC[ARITH_RULE `~(x + 1 = 0)`]);;
val it : goalstack = No subgoals

# top_thm();;
val it : thm = |- !f. ~(f 0 = &0) ==> gf_reciprocal f (recip f)

As we can see, running the procedural proof interactively is crucial to understanding what the proof does. However, working through this exercise, I have often found it excruciatingly hard to perform even simple manipulations of the goalstack in the way I intended. My understanding of HOL’s library of theorems and tactics has to improve considerably, before I can become effective at this. Fortunately, working with declarative proofs was much easier for me.

Mizar in HOL: miz3

The Mizar system pioneered declarative proof languages, and has a lasting influence on declarative proof languages in other systems, such as the Isar language that is part of Isabelle or the various variants of Mizar implementations for the HOL family of provers. The latest incarnation of Mizar for HOL Light is miz3 by Freek Wiedijk.

As you can see from the declarative version of the proof given in the introduction, Mizar-style proofs are largely self-explanatory. Of course some elements still require explanation, but I will not go through them in detail here. Instead I want to refer you directly to Freek’s paper. Instead, I will use this section to give some tips for installing and using miz3. These tips refer to the version of miz3 available at the time of publication of this post (miz3 releases do not appear to be numbered).

• You can get miz3 from here. To use miz3, all you need to do is to extract miz3.ml from the archive and place it in your HOL Light source directory.
• miz3 comes with a Vim interface. This interface is very lightweight and its use is not required: You can always paste miz3 code into your OCaml toplevel. But these notes are mainly about using miz3 via Vim. To install the Vim interface, you need to
  • place the two executables miz3 and miz3f from the bin/ subdirectory of the miz3 distribution on your path, and
  • add source /path/to/exrc to your .vimrc, where the path points to the file exrc from the miz3 distribution.
• To use miz3, you need to start the OCaml toplevel using the command ocaml unix.cma. On the toplevel, first run #use "hol.ml";; and then #use "miz3.ml";;. This automatically starts the miz3 server and you can begin using Vim.
• In Vim, you can press Ctrl-C RET to send paragraphs of code (delimited by blank lines) to the OCaml toplevel. Error reports are then displayed right in the Vim buffer. However, some errors are not. So you should always keep an eye on your toplevel!
• You should keep the paragraphs you send to miz3 minimal: Separate your miz3 proofs and your (* OCaml comments *) with a blank line. Do the same for definitions and proofs. Otherwise miz3 may throw errors that only appear on the toplevel.
• GOAL_TAC is another very useful tool to know about in a declarative proof. If you want to know, what exact goal you have to prove at any given step of you declarative proof, you can add by GOAL_TAC to the end of the line. After the declarative proof has been processed by miz3 you can inspect the goal at that point by running p();; at the toplevel. This works no matter if you proof has been successful or not. As GOAL_TAC is a tactic, it is very flexible and can be used in procedural proofs or as part of a more complicated tactic as well.

After these introductory comments, we can now go on to the declarative version of the proof.

A Declarative Proof

Using a vanilla installation of HOL Light and miz3, a declarative version of the proof might look like this.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

let GF_RECIP_EXPLICIT_FORM = thm `;
  !(f:num->real). ~(f (0:num) = (&0:real)) ==> gf_reciprocal f (recip f)
  proof
    let f be num->real;
    assume ~(f 0 = &0) [1];
    !n. sum (0..n) (\k. f k * recip f (n - k)) = (if n = 0 then &1 else &0)
    proof
      sum (0..0) (\k. f k * recip f (0 - k)) = f 0 * recip f (0 - 0) by NUMSEG_SING,SUM_SING;
        .= f 0 * recip f 0;
        .= f 0 * &1 / f 0 by recip;
        .= f 0 / f 0;
        .= &1 by REAL_DIV_REFL,1;
        .= (if 0 = 0 then &1 else &0) [2];
      !n. sum (0..n) (\k. f k * recip f (n - k)) = (if n = 0 then &1 else &0)
          ==> sum (0..SUC n) (\k. f k * recip f (SUC n - k)) = (if SUC n = 0 then &1 else &0) [3]
      proof
        let n be num;
        assume sum (0..n) (\k. f k * recip f (n - k)) = (if n = 0 then &1 else &0);
        !x. ~(x = &0) ==> !a. x * -- (&1 / x) * a = -- a [4] by REAL_FIELD;
        sum (0..SUC n) (\k. f k * recip f (SUC n - k)) = (\k. f k * recip f (SUC n - k)) 0 + sum (0 + 1..SUC n) (\k. f k * recip f (SUC n - k)) by SUM_CLAUSES_LEFT,LE_0;
          .= (f 0 * recip f (SUC n)) + sum (1..SUC n) (\k. f k * recip f (SUC n - k)) by ADD_0,SUB_0,ADD_AC; 
          .= (f 0 * recip f (n + 1)) + sum (1..(n + 1)) (\k. f k * recip f ((n + 1) - k));
          .= (f 0 * -- (&1/(f 0)) * sum (1..(n + 1)) (\k. (f k) * recip f ((n + 1) - k))) + sum (1..(n + 1)) (\k. f k * recip f ((n + 1) - k)) by recip;
          .= -- sum (1..(n + 1)) (\k. (f k) * recip f ((n + 1) - k)) + sum (1..(n + 1)) (\k. f k * recip f ((n + 1) - k)) by 1,4;
          .= &0;
          .= (if SUC n = 0 then &1 else &0);
      qed;
    qed by INDUCT_TAC from 2,3;
    (\n. sum (0..n) (\k. f k * recip f (n - k))) = (\n. (if n = 0 then &1 else &0)) [5] by REWRITE_TAC,FUN_EQ_THM;
  qed by REWRITE_TAC,gf_reciprocal,gf_times,gf_one,1 from 5;
`;;

Note that this version differs slightly from the declarative proof given at the beginning. The difference is that this version uses additional references to the HOL library, making explicit use of the tactics REAL_FIELD and REWRITE_TAC. The proof given in the introduction is a bit less verbose and it is tactic-free, i.e., it only references theorems and not tactics in the library.

Why do I care about a proof being tactic-free? After all, the availability of proof tactics and the corresponding opportunities for automation are a great strength of procedural provers such as HOL Light? Well, on the one hand this exercise is about translating human-written proofs and not about automated reasoning. On the other hand, theorems used in by clauses can be stated in higher order logic and can thus be easily included in a prover-independent library. Tactics, however, depend critically on the prover they were written for and are therefore less portable. I will write another post in the near future on why I think prover independence is so important.

Making Declarative Proofs more Self-Contained

In this section we will see how to tweak miz3 in order to eliminate the tactics REWRITE_TAC and REAL_FIELD from the above declarative proof.

First, we have to examine a bit more closely how miz3 proves each step in a declarative proof:

• If the by clause contains only theorems and no tactics (or if it is even empty altogether), then miz3 uses the tactic specified in the variable default_prover to prove this step.
• If the by clause contains a tactic, that tactic is used for the proof and the default prover is ignored.

The standard default prover in miz3 is HOL_BY, which in turn was inspired by the prover contained in the original Mizar. HOL_BY resides in the file Examples/holby.ml in the HOL Light source directory (not the miz3 source directory). It is set as the default prover of miz3 at the very top of miz3.ml.

1

let default_prover = ref ("HOL_BY", CONV_TAC o HOL_BY);;

HOL_BY is of type thm list -> term -> thm whence CONV_TAC o HOL_BY is of type thm list -> tactic, which is the type expected of a default prover.

We now turn to the task of eliminating the explicit reference to REAL_FIELD from the proof. The idea is to improve HOL_BY so that the default prover tries REAL_FIELD automatically. REAL_FIELD is an inference rule. So in order to find a suitable place for integrating REAL_FIELD into HOL_BY, we look for other inference rules in the HOL_BY source. Right at the beginning of the definition of the function HOL_BY itself, we find a bunch of conversions and inference rules. So we just include REAL_FIELD right after REAL_ARITH, like this:

1
2
3
4
5
6
7
8
9
10
11
12

let HOL_BY =
  let BETASET_CONV =
    TOP_DEPTH_CONV GEN_BETA_CONV THENC REWRITE_CONV[IN_ELIM_THM]
  and BUILTIN_CONV tm =
    try EQT_ELIM(NUM_REDUCE_CONV tm) with Failure _ ->
    try EQT_ELIM(REAL_RAT_REDUCE_CONV tm) with Failure _ ->
    try ARITH_RULE tm with Failure _ ->
    try REAL_ARITH tm with Failure _ ->
    try REAL_FIELD tm with Failure _ ->
    failwith "BUILTIN_CONV" in
  fun ths tm ->
    ... etc. ...

And indeed, with this addition, we can remove REAL_FIELD from our declarative proof (after reloading miz3.ml).

Eliminating REWRITE_TAC from our proof turns out to be a bit more subtle. If we remove the explicit occurrences of REWRITE_TAC from the proof, we get an inference timeout. Does this imply that REWRITE_TAC can do something that HOL_BY cannot? No. Even if we replace REWRITE_TAC with ALL_TAC, the proof will succeed, even though ALL_TAC is the tactic that does nothing. The answer to this riddle lies in the code that miz3 wraps around the tactics it is given. This code we find in the function tactic_of_by in miz3.ml.

In tactic_of_by we find the block:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

   (try
      0,((FILTER_ASSUMS (fun _,(x,_) -> x <> "=") THEN
        FILTER_ASSUMS
         (fun n,(x,_) ->
            mem x labs or n < hor or (n = 0 & mem "-" labs) or full_asl) THEN
        MAP_ASSUMS (fun l,th -> l,PURE_REWRITE_RULE sets th) THEN
        MIZAR_NEXT' (PURE_REWRITE_TAC sets) THEN
       (fun (asl',w' as g') ->
          let key = name,(map concl thms,map concl thms'),w' in
          try
            if grow then failwith "apply";
            let e,th = apply (!just_cache) key in
            if e = 0 then (ACCEPT_TAC th THEN NO_TAC) g' else
            if e = 2 then raise Timeout else failwith "cached by"
          with
          | Failure "apply" ->
              try
                let (_,_,just as gs) =
                 ((fun g'' ->
                    let gs' = TIMED_TAC (!timeout) (tac thms) g'' in
                    if grow then raise (Grown (steps_of_goals g gs'))
                            else gs') THEN
                  REPEAT (fun (asl'',_ as g'') ->
                    if subset asl'' asl' then NO_TAC g''
                    else FIRST_ASSUM (UNDISCH_TAC o concl) g'') THEN
                  TRY (FIRST (map ACCEPT_TAC thms'')) THEN
                  REWRITE_TAC thms'' THEN NO_TAC) g' in
                let th = just null_inst [] in
                just_cache := (key |-> (0,th)) !just_cache;
                gs
              with
              | Grown _ as x -> raise x
              | x -> if name <> "GOAL_TAC" then just_cache :=
                    (key |-> ((if x = Timeout then 2 else 1),TRUTH))
                      !just_cache;
                  raise x
        )) g)

And the crucial piece in here is the line:

1

let gs' = TIMED_TAC (!timeout) (tac thms) g'' in

Here tac refers to either the tactic that was found in the by clause or the default prover. TIMED_TAC tries the tactic it is passed for a fixed amount of time. If the tactic does not succeed in the specified interval, a Timeout exception is raised and the tactics that follow are never even tried. As a consequence a long-running tactic such as HOL_BY may achieve less than a instantaneous tactic such as ALL_TAC, because the exception thrown by the former prevents the last few tactics in tactic_of_by from being called. A workaround, therefore, is to modify the default prover so that HOL_BY is tried for a fixed amount of time strictly less than the timeout. After that HOL_BY is aborted without raising an exception and the subsequent tactics given in tactic_of_by are still executed.

To implement this workaround, we add the following definition to then end of miz3.ml right before the definition of reset_miz3:

1

let TIMEOUT_HOL_BY = fun ths g -> try TIMED_TAC 1 (CONV_TAC (HOL_BY ths)) g with Timeout -> ALL_TAC g;;

This defines a new tactic TIMEOUT_HOL_BY that tries to run HOL_BY and aborts it silently after 1 second. Next, we set the timeout after which tactics are killed and an exception is raised to 10 seconds (so that this timeout does not interfere with TIMEOUT_HOL_BY) and set the default prover accordingly to TIMEOUT_HOL_BY, by modifying reset_miz3 as follows:

1
2
3
4
5
6
7
8
9

let reset_miz3 h =
  horizon := h;
  timeout := 10;
  default_prover := ("HOL_BY", TIMEOUT_HOL_BY);
  sketch_mode := false;
  just_cache := undefined;
  by_item_cache := undefined;
  current_goalstack := [];
  server_up();;

After this, we reload miz3.ml and then run reset_miz3 !horizon;; on the toplevel. Our miz3 can now tackle the tactic-free declarative proof given at the beginning of this post!

As I already wrote at the end of the summary, I am very impressed. With current provers it is possible to write proofs that are at the same time formal, human-readable and reasonably short and self-contained. On top of that, further improvements seem to be within reach.

For me, the biggest pleasant surprise was that a declarative proof in miz3 requires relatively few explicit justifications (by clauses) in each step. I think this is very important, as this reduces the volume of the dictionary that a user has to learn to get started, it decreases the danger of bit-rot and it increases portability. I will write more on these issues in a future blog post.

I will write more about the rationale behind this feature when it is more polished. Right now, I just want to give you a little heads-up on the latest rough-around-the-edges feature in Qute 0.4.1.

To define a custom markup language, you include an OMeta grammar right inside your document. Such a grammar might look like this.

You can then tell Qute that any paragraph in your document is written in the custom language you just defined. Then Qute will transform that paragraph according to the rules specified in the grammar. In this example, we calculate the number 42 by way of OMeta.

Qute 0.4.1 even supports some rudimentary error reporting.

If you want to try this out for yourself, grab a copy of Qute 0.4.1 from here.

The next milestone is to make this do something interesting. For example, an implementation of Markdown in OMeta would allow the user to easily extend Markdown to suit his or her needs. This is something that I have been missing sorely while writing mathematics with Qute.

Moshé Feldenkrais’ work and Laozi’s classic, the Daode Jing, have both had a big effect on me and I have always been struck by how alike in spirit these (at first glance) very different philosophies are. In the summer of 2010, I heard a great talk by Steve Palmquist about parallels between the Daode Jing and Kant’s philosophy and this experience gave me the confidence to sit down and work out my own ideas on the relationship between Feldenkrais and Laozi.

The result is an article that has just been published by the Journal of Daoist Studies. You can

download a PDF of my article right here

or you can purchase an ebook or a printed copy of the entire journal from Lulu.

Not beyond Journals, beyond Papers.

I agree! And I would like to go one step further. We have to go beyond journals. We have to go beyond papers. And we have to go beyond that which we, as mathematicians, hold dearest:

We have to go beyond theorems!

In this post, I want to make an argument why, from my point of view. This is going to be a long post, but I will try not to rant too much.

The number of new results per year is increasing

The number of mathematical articles - and theorems! - published each year has increased dramatically in the last decades. To give you just one piece of evidence, here is a graph of the number of new research articles entered in the Zentralblatt MATH index each year. This figure is taken from Bernd Wegner’s article in the June 2011 newsletter of the European Mathematical Society.

This trend will continue. The human population on earth is increasing and (on average) becoming more prosperous. So, more people will pursue an academic career and more people will become research mathematicians. And, because new theorems are the currency in which mathematicians currently measure research activity, this means that the number of new (!) theorems proven each year will continue to increase dramatically - unless we change our way of thinking about research.

This increase is a loss for everyone

There are roughly 100,000 mathematical articles on the Zentralblatt index that have been published in 2010. One hundred thousand articles, of which the majority contain (supposedly) new results. I’d go so far as to say that most mathematicians cannot read and properly digest more than 100 articles per year. (For me personally, the number is certainly lower than that.)

This means that most mathematicians cannot even make use of 0.1% of the new results published each year.

This must have at least one of the following two consequences. In my opinion, it has both.

Mathematical research is becoming more and more specific.

Mathematicians have a great talent for inventing ever more particular fields and sub-fields and sub-sub-fields in which we can discover ever more new theorems. If this strategy is successful, then these theorems are truly new and they have never come up in another area before. The flip side of the coin is of course that the majority of these very particular theorems will simply not be relevant for most mathematicians (and they will be utterly irrelevant for the rest of the world). A favorite objection to this argument is that, a couple of decades down the road, these results will become relevant in other fields and that results from today will be made use of then. But how is this supposed to happen if mathematicians can only read less than 0.1% of what is published this year? It is much more likely that mathematicians who, 20 years later, arrive at an equivalent problem in another field will simply solve it again on their own. Which brings us to the other consequence.

Mathematical research is becoming more and more redundant.

Independent rediscoveries of theorems happen all the time. They may happen more or less simultaneously. Or they may happen a couple of years apart. Or they may happen in entirely different fields, in which case it may take a while for people to notice that these results are, in fact, equivalent. Whatever form these redundancies take, their number is going to increase.

One can make the argument, that this is no problem at all. Science is redundant. So if all a rediscovery achieves is to make an old result popular in a different time or a different area, then so be it! But this clear eyed view of the world goes against the grain of our current self-image as mathematicians, where only original (never-seen-before) theorems count.

Also, redundancy becomes a real problem in conjunction with increasing specialization. What is the point of doing foundational research in a narrow area of specialization today, if the people who may need this stuff twenty years down the road have no means of discovering it? The only insurance against this type of disaster is to focus on advertising our small corner of mathematics, in the hope that somebody will remember it when the time comes. But again, our mathematical community is not built on this socio-dynamic view of research.

In the end, both trends, increasing specialization and increasing redundance, have the same effect:

Mathematical research is becoming increasingly irrelevant.

Note that I am not saying that mathematics is becoming increasingly irrelevant! There will always be seminal theorems that have a far reaching impact on both mathematics and the real world. But the work of the average research mathematician is going to become increasingly irrelevant.

Unless we finally stop to believe that the only valid form of “research” in mathematics is proving new theorems.

Beyond theorems

I have tried to argue above that the business of churning out as many new theorems as possible is not in the best interest of both mathematicians and mathematics. The two causes that I see as central in this regard are the following.

• We as a community view only new theorems as research. (Thus we create incentives to produce an unnecessarily large inflation of theorems.)
• Our current methods of discovering theorems do not scale with the increase in theorem production.

Thus the best way out of this situation is to acknowledge other forms of mathematical activity as research, not only the proving of new theorems. In particular, activities that help in the discovery of mathematical ideas should be promoted. Here are a couple of suggestions:

Communicating mathematics. This includes all forms of communication, by means of talks, videos, games and of course writing in blog posts, web forums (such as mathoverflow), survey articles and books. This should also include all levels of writing, from expository work for students to references for experts. Finally, this should also include all audiences, from the interested public, to scientists outside mathematics, to mathematicians in other fields and finally specialists in your area. All of these forms of communicating mathematics are widely recognized in the mathematical community. But they are not recognized as research. I argue that mathematicians should get credit as researchers for communicating mathematics, because a larger part of every mathematician’s work will have to be about communication if we are to cope with the present inflation of new results. And we should make it possible that researchers specialize in these social activities, which will only be possible if we give them proper recognition.

Collaborative theorem proving. Instead of everyone trying to get their little theorem published, why do we not try to find new ways of working on the big open questions collaboratively? The polymath projects are very valuable efforts in this regard, but they are only a small first step. As someone on the sidelines of these projects, it appeared to me that in the end, only a relatively small number of people were able to actually contribute to any given polymath project. We have to find ways to make large projects like this asynchronous and distributed. That means, we have to find ways how people can contribute half-baked ideas to research projects, how people who might find them useful can discover them and how to give credit for half-baked ideas in the mathematical community. How to solve these problems is wide open, from my point of view.

The two topics of communicating mathematics and “crowd sourcing” proofs have been widely dicussed in the current debate about mathematical publishing. Now on to two points that have received less attention, as far as I know.

Implementing mathematical software. To make mathematical methods discoverable (and useful), it is of tremendous value to have them implemented in software. This applies to algorithms and methods for visualization as well as to such mundane tasks as the development of decent user interfaces. While these may be engineering tasks in many cases, we should recognize them as research, because they are crucial to the advancement of mathematics! Moreover turning all the useful algorithms available in the literature into usable software is a huge endeavour - it just won’t happen without the proper incentives.

Formalizing mathematics. In my opinion, formalizing mathematics is the single one most important thing that needs to happen if we want mathematics to scale. With “formalizing mathematics” I mean a huge process that includes all of the following.

• The development of the languages and tools necessary to make the formalization of mathematical definitions, theorems and proofs feasible for the average mathematician.
• This will have to go hand in hand with the development of automatic proof systems that can automatically “fill in the gaps” present in any research article.
• The creation of an infrastructure that allows formal mathematical articles to be shared and discovered across different proof systems and formal mathematical languages.
• And, finally, the formalization of (almost) all of mathematics.

It goes without saying that this is more than enough work for generations (plural!) of mathematicians. We still have a long way to go in all of the four areas indicated by the four bullet points above. And even though work is going on in all of these areas, this work is not as widely known as it should. Freek Wiedijk was kind enough to give me a brief tour last fall, and I plan to blog about this in the near future.

Here is why I think the project of formalizing mathematics is crucial. We, as humans, cannot cope with the sheer quantity of mathematical output that is going to be produced in the 21st century. So we have to teach computers how to discover theorems for us that are relevant for the questions we are interested in. Otherwise, the work of most mathematicians will disappear into obscurity faster than anyone can be comfortable with.

Communicating mathematics, collaborative theorem proving, implementing mathematical software and formalizing mathematics are of course all closely interrelated. Why should we not dream of composite electronic resources that are accessible and understandable by all kinds of different readers, that we work on collaboratively and that combine expository texts with detailed explanations and usable software with formalized mathematics? This dream is certainly not new and it is shared by many. How should this brave new world of mathematics look like in detail and how do we build it? The process of exploring these questions will be interesting indeed. My point here is that to get there we need to give up the belief that theorem proving is what mathematical research is all about.

The self-image of the mathematician

No matter what happens, proving mathematical theorems is going to become less relevant. If we succeed in changing they way mathematical research works, other activities will become more important. If we do not, the theorems most mathematicians prove will, generally, be redundant and overspecialized. So:

Theorems will become less relevant.

The social and psychological impact of this simple observation is huge. The self-image of the modern mathematician is built entirely on proofs and theorems. We, as mathematicians, grow up on stories of great “heros” who became “immortal” by proving all these great theorems we learn about. Naturally, we aspire to do as they did and so we seek to wrest these eternal truths from the world of pure, abstract ideas that we conquer using nothing but our “beautiful mind”.

Of course I am being ridiculous here, I exaggerate. But our identity as mathematicians is based on the stories we tell each other about other mathematicians - and those stories revolve around theorems and proofs. Often the only thing we know about a mathematician is what theorem they proved and we set up Millenium Prizes to celebrate the act of attaching a particular mathematician’s name to a particular theorem.

The association of “being a mathematician” with “proving a new theorem” is so tight that I have serious doubts about whether we as a community will accept other mathematical activities as research anytime soon. To do so would call into question what a mathematician is at a fundamental level. But this is precisely what we have to do.

Conclusion

The current model of doing mathematics is in need of reform. The fundamental issues with the current model cannot be resolved just by placing mathematical journals on a different economic and organizational basis. The core of the problem is our fixation as a community on theorems and proofs. As these notions are tied so closely to our self-image as mathematicians, finding a new model of doing mathematics is going to be a long and arduous process. But we have to do it, if we want our work to be relevant in a couple of decades.

The \(f^*\)- and \(h^*\)-coefficents of a polynomial \(p(k)\) are simply the coefficients of \(p(k)\) with respect to certain binomial bases of the space of polynomials. If \(p\) is of degree \(d\), then they are given by \[p(k)=\sum_{i=0}^d f^*_i {k-1 \choose i} =\sum_{i=0}^d h^*_i {k+d-i \choose d}.\]

Where do these definitions come from? The motivation is Ehrhart theory. Given a set \(X\subset\mathbb{R}^d\), the Ehrhart function \(L_X\) of \(X\) is given by \[L_X(k)=\#\mathbb{Z}^d\cap k\cdot X,\] for \(1\leq k \in \mathbb{Z}\), that is, \(L_X(k)\) counts the number of integer points in the \(k\)-th dilate of \(X\).

Now, a famous theorem by Ehrhart states that if \(X\) is a polytope with integral vertices, then \(L_X\) is a polynomial. More precisely, there exists a polynomial that coincides with \(L_X\) at all positive integers and by abuse of notation, we will denote this polynomial by \(L_X\) as well.

After this preamble you may already have guessed it. The binomials in the expansion above are Ehrhart polynomials of particularly nice polytopes, namely half-open unimodular simplices. We denote by \(\Delta^d_i\) the \(d\)-dimensional standard simplex with \(i\)-open faces, i.e., \[\Delta^d_i = \{ x\in\mathbb{R}^{d+1} \;|\; x_0,\ldots,x_{i-1} > 0, x_i,\ldots x_d\geq 0 \sum_{i=0}^d x_i = 1 \} \]

and then it is not hard to check that \[L_{\Delta^d_i}(k) = {k+d-i \choose d}\] and in particular \[L_{\Delta^d_{d+1}}(k) = {k-1\choose d}.\] Thus:

• If \(P\) is a lattice polytope with a shellable unimodular triangulation \(K\), then \(h^*_i\) counts the number of \(d\)-dimensional simplices with \(i\) open faces in a shelling of \(K\).
• If \(P\) is a lattice polytope with a unimodular triangulation \(K\), then \(f^*_i\) counts the number of \(i\)-dimensional faces in \(K\).

In particular, if \(P\) is sufficiently nice, then both the \(f^*\)- and the \(h^*\)-coefficients are non-negative. The point of my article is to show that even if \(P\) is not nice and partitioning \(P\) into unimodular simplices does not work at all, the \(f^*\)-vector is still non-negative.

Theorem. If \(P\) is a polytopal complex, then the \(f^*\)-coefficients of \(L_P\) are non-negative integers, even if \(P\) does not have a unimodular triangulation.

This becomes relevant when \(P\) is not-convex. As long as \(P\) is a polytope, its \(h^*\)-coefficients are non-negative by a famous theorem of Stanley. However, the \(h^*\)-coefficients of polytopal complexes may well be negative. For example, in recent work, Martina Kubitzke, Aaron Dall and myself have shown that there exist coloring complexes of hypergraphs with negative \(h^*\)-coefficents.

In fact, we can take this theorem one step further and use it to characterize Ehrhart polynomials of what I call partial polytopal complexes. An integral partial polytopal complex is any set in \(\mathbb{R}^d\) that can be written as the disjoint union of relatively open polytopes with integral vertices.

Theorem. A vector is the vector of \(f^*\)-coefficients of some integral partial polytopal complex if and only if it is integral and non-negative.

Behind all of this lies a combinatorial interpretation of the \(f^*\)-coefficients in a (not necessarily unimodular) lattice simplex and a new way of partitioning the set of lattice points in a simplicial cone into discrete cones.

For more results, in particular about the rational case, and more details on all of this, please check out my article! It has the id arxiv: 1202.2652.

This fall semester, I taught the Math 890 course at San Francisco State University. It was the very first course that I taught all on my own. I am currently funded by the DFG, so I have no teaching obligation (in fact, I had to ask the DFG for permission to devote my time to something other than research). But when I was offered this opportunity, I leapt at the chance and volunteered to teach this course, even though, with two 75 minute classes each week, it was a significant amount of work. I wanted the experience, and, after all the years where I was responsible only for my own research portfolio, I was looking forward to taking on some real responsibility. Also, this was a topic course for graduate students, so that I could teach anything that I wanted. What a wonderful opportunity to experiment!

In this post, I will present the idea behind the course I gave, give an outline of the topics covered, provide all my slides and notes for the lecture as well as all the exercise problems, give a list of topics for the seminar, and review how the course, as a whole, went.

Note: The material linked below is known to contain a number of typos.

The Idea

I always wanted to try a different format of a course. With “format” I do not mean the methodology for teaching - experiments in that department are certainly called for, but I wanted to stick to the classical lecture + seminar format (because I actually enjoyed it as a student). What I wanted to experiment with was the content of my course.

Math courses, the way I experienced them at the FU Berlin, typically build a theory from the ground up, in a very detailed fashion. They emphasize rigor over intuition, and have a classical definition-theorem-proof structure. Courses of this “classic” format are certainly important, especially for teaching mathematical rigor, but here, I wanted to do things differently:

• The course was to cover a large range of topics, with the aim of showing their interconnections,
• intuition was to be emphasized over rigor,
• and it was to make extensive use of visualization.

In short, I wanted to give my students a grand tour of the mathematics that I find interesting. Adding courses of this type to the curriculum is important for a number of reasons. I do not want to go into too much detail on this, so here are just a few bulletpoints, in no particular order.

• For me personally, grand tours are simply more inspiring than develop-a-theory-from-the ground-up-lectures. I want to take a grand tour of the house, instead of spending my time laying bricks for the foundations!
• Grand tours have more room to present applications of the covered material.
• Pure mathematics especially are not measured only in terms the applications they have. The beauty, insight and perspective they have to offer are of direct value to us as human beings. However beauty, insight and perspective are tied primarily to the rough ideas and much less to the technical details. So let’s cover more beautiful ideas and less details!
• Showing the connections between different fields make all of these fields more relevant to the audience!
• First, I cannot remember a theorem unless I develop some kind of intuition for it. Second, I cannot prove similar theorems on my own, without having an intuitive idea why it should be true. Third, the intuitive idea is often all I remember off the top of my head after a couple of weeks have passed. So why not focus on teaching the intuition in the first place?
• For me personally visualization is by far the best way to develop an intuition.

Outline

Using Intuitive Geometry - Using Geometry Intuitively

Adventures with Hyperplanes

• Visualizing Numbers (Gauss sum formula, lattice points in triangles, staircases, Dedekind sums)
  • Lecture 1 lecture notes
  • Lecture 2 lecture notes
• Polytopes (H- and V-description, faces, permutahedron)
  • Lecture 3 + 4 slides
• Linear Programming (Fourier-Motzkin eliminiation, Farkas lemma, LP duality, simplex algorithm, combinatorial optimization)
  • Lecture 5 slides
  • Lecture 6 slides
  • Lecture 7 slides
• Game Theory, Social Choice (minimax theorem, Nash equilibria, spatial voting, McKelvey’s theorem)
  • Lecture 8 slides
  • Lecture 9 lecture notes, slides
• Topological Combinatorics, Fair Division (Brouwer’s fixpoint theorem, Sperner lemma, ham sandwich splitting)
  • Lecture 10 slides
  • Lecture 11+12 slides
• Ehrhart Theory, Lattice Points in Polyhedra (Ehrhart’s theorem, Ehrhart-Macdonald reciprocity, Stanley non-negativity, combinatorial applications)
  • Lecture 13+14 slides
  • Lecture 15+16 slides
• Generating Functions (formal power series as lattice point sets and vice versa, enumeration of lattice points using formal power series)
  • Lecture 17+18 slides
  • Lecture 19 slides
• Machine Learning (support vector machines, kernel trick, VC dimension)

Exercises

• Exercise 1
• Exercise 2
• Exercise 3
• Exercise 4
• Exercise 5
• Exercise 6
• Exercise 7
• Exercise 8
• Exercise 9

Seminar Talks

After 10 weeks of lectures, we had 4 weeks of student talks. Some of the talks were in-depth talks on a particular subject, while others were brief introductions to large subject areas. These are the topics I suggested:

• The Mordell-Pommersheim Tetrahedron. (Source: Computing the Continuous Discretely. Beck, Robins. 2007. Chapter 8.)
• Power Indices and Spatial Power Indices (Source: Power and Stability in Politics. Straffin. In: Handbook of Game Theory Volume 2. Chapter 32. 1994.)
• LP and MIP Modelling Tricks (Source: AIMMS Optimization Modelling. Bisschop. 2011. link)
• Interior Point Methods (Source: The integer-point revolution in optimization: History, recent developments, and lasting consequences. Wright. 2004. link)
• The Borsuk-Ulam Theorem and Ham Sandwich Splitting (Source: Using the Borsuk-Ulam Theorem. Matousek. 2003. Chapters 2 and 3.)
• Integer Programming Methods (Source: Theory of Linear and Integer Programming. Schrijver. 1986. Chapters 23 and 24.)
• Order Polynomials (Source: Combinatorial Reciprocity Theorems. Beck. 2011. Chapter 6. link)
• P-Partitions (Source: Combinatorial Reciprocity Theorems. Beck. 2011. Chapter 7. link)

How did it go?

I am very happy with how the course turned out! This “grand tour” style of a course succeeded in engaging the students. From what I heard, my students not only learned something, but also took away a new perspective on many of the subjects I covered. One student even told me that they now had a much better understanding what the thesis they had been working on is about. Was will man mehr!

Nonetheless there are of course a number of things I would like to improve the next time I teach this course - which I would really like to do! Here is a list of notes that I would like to keep in mind for the next iteration.

Exercises

The exercises were what made this “intuitive tour” type of course work. Had my students not spent the time to accquaint themselves personally with the subject matter, I guess many parts of this course might have remained too vague. The great thing is, that my students did work through virtually all of the exercise problems, even though I did not have the resources to grade the correctness of their solutions. It would have been too easy to skip corners - but they didn’t! Thus, I really have to thank them for their initiative, curiosity and dilligence - they made this course work (even more than ususal)!

In hindsight, I think this large program is better suited to a course with three 75 minute sessions per week. Two sessions for lectures and one for talking about the exercise problems. Also, it would have been very valuable to provide individual corrections for the exercises and/or solutions of the exercises for the students to review on their own.

Lecture Notes

I wrote the script for my lectures as I went along. Given that the material was so diverse and scattered over so many sources and given that it was my ambition to present some material from an (AFAIK) original perspective, this was very work intensive. Originally, I had planned to use my blog to record detailed lecture notes, but this turned out to be infeasible. (Currently, it takes me at least four hours to write up notes for one 75-minute lecture in a form suitably self-contained for a blog post - even if I have prepared the material beforehand in slide form and given the lecture in front of class. Is that typical or somewhat slow?)

The points mentioned so far (publish lecture notes, publish solutions for the exercises, provide individual corrections for the exercises) all boil down to putting in more work. And when I give this course the next time, I certainly want to prepare lecture notes and solutions beforehand and work together with a teaching assistant for grading! Here are some more thoughts, though, that take a different direction:

Seminar

Combining the lecture with a seminar was an excellent choice. For two reasons. First, it gave the students the opportunity to work on a subject in depth. This was especially important as the “intuitive tour” format (by necessity) left the students wanting more details. The seminar gives students the opportunity to pick one subject they find particularly appealing and dig into the details that were only hinted at in the lecture and in the exercises.

Second, the seminar was enjoyable not only for the students, but for me as well! During preparation, it was simply fun to get together and puzzle about a problem or figure out the best picture for illustrating a theorem. (Especially after I had spent hours on end with mainly one-way communication during the lecture.) On the whole, the talks were very good and got across all the important points. Some even went above and beyond the suggested material! Next time, I might be tempted to make the seminar a much bigger part of the course!

Proofs

What place do proofs have in this type of course? After the first two weeks of the course I become worried that my intuitive style of lecturing might be too vague for my audience. This worry was not fed by any actual observations during class, but rather by my rookie-lecturer-confidence wavering. So I decided to up the level of detail in the next few weeks and include more proofs and a greater level of formality. In retrospect, after talking with my students about it and judging from the their exercise solutions, I can say that this failed completely. This “proofy” part of the lecture was the part my students enjoyed the least and it was the part they had the least grasp of, judging by the exercises. (Of course, proper lecture notes would have helped their performance in the exercises, but I think this is not the main point here.)

I was reminded here of these great quotes by Victor Klee:

• “A good talk contains no proofs; a great talk contains no theorems.”
• “Mathematical proofs should only be communicated in private and to consenting adults.”

In any case, they have no place in this type of lecture! Intuitions for why a theorem should hold, rough proof sketches, all of these certainly belong in this type of lecture. But going the extra distance from an intutive proof sketch to an actual “formal” proof is not worth it - simply because any formalization is significantly harder to understand than the intuitive idea that gave rise to it. If your goal is to teach a theory, then teaching that formalization is part of what you want to communicate. If your goal is to show connections between different fields intuitively, then that formalization is only tangential to what you want to communicate. So, adding formalism just for the sake of communicating an intuitive idea is not going help - instead of clarifying the point it will obfuscate.

Of course this does not mean that proofs are irrelevant! On the contrary, if students are to learn how to use geometric intuition, then learning how to make intuitive ideas precise is crucial. However this is an active process. Students have to do it themselves when solving exercises or preparing seminar talks. My doing it at the whiteboard does not help anybody! More precisely: my doing it at the whiteboard helps only, if students have tried to do it by themselves beforehand.

I will not try to write down here all the mental notes I made for myself on how to present proof ideas if you do not want to make them 100% precise. (Maybe I will write another blog post about this.) I want to mention two things, however: First, when explaining an algorithm/constructive proof it is much more important to make the construction precise and illustrate it with an example than to prove its correctness. Second, always give a concise summary of the key point of a theorem or proof that you just explained. That way, even those members of your audience that you may have lost will have something concrete to remember.

Drawing

Of course, when talking about geometric intuition, drawing is the most important thing! Two shorts thoughts about this:

The most important aspect of many mathematical figures is the way they are drawn and not the final image. Thus it is very important to construct a figure slowly, step by step. No fancy visualization tool beats a good drawing on a whiteboard!

In this vein, I encouraged my students to draw the objects they were talking about, both in the exercises and in the seminar - and I got everybody to do it in the end! I was very happy about this, because, in my opinion, drawing is an essential skill for any mathematician!

Conclusion

It was a great experience teaching this course! I learned a lot, both in terms of hard skills and soft skills. I would love to teach this course a second time, and the next time I would like to experiment more with teaching methodology. Also, I would really like to write up this tour through some uses of intuitive geometry in book form and smooth out all the wrinkles along the way. But, maybe, the printed page (or static PDF document) is just not the right medium for this.

I think mathematics could benefit from more courses and texts of this “intuitive tour” type: overview material with a strong emphasis on intuition and visuals, accessible to advanced undergraduates but with enough depth to satisfy graduate students.

Bibliography

• Beck, Combinatorial Reciprocity Theorems.
• Beck, Robins, Computing the Continuous Discretely.
• Bisschop, AIMMS Optimization Modelling.
• Breuer, Heymann, Staircases in $\mathbb{Z}^2$, Integers.
• de Longueville, A Course in Topological Combinatorics.
• Gaiha, Gupta, Adjacent Vertices on a Permutohedron, SIAM Journal on Applied Mathematics.
• Grötschel, Lineare und Ganzzahlige Programmierung.
• Matousek, Using the Borsuk-Ulam Theorem.
• McKelvey, Intransitivities in multidimensional voting models and some implications for agenda control, Journal of Economic Theory.
• McKelvey, General Conditions for Global Intransitivities in Formal Voting Models, Econometrica.
• Raghavan, Zero-sum two-person games, Handbook of Game Theory, Vol 2.
• Schölkopf, Smola, Learning with Kernels.
• Schrijver, Theory of Linear and Integer Programming.
• Straffin, Power and stability in politics, Handbook of Game Theory, Vol 2.
• Wright, The interior-point revolution in optimization: History, recent developments, and lasting consequences, Bulletin of the American Mathematical Society.
• Ziegler, Lectures on Polytopes.

In a democracy, people are fighting all the time over control of the political agenda. Media campaigns, protests and lobbying are all examples of people trying to shape the political discourse. They seek to get “the public” to discuss a topic they care about, they seek to shift the focus of the current political discussion towards the points they think are important or they may even want to throw up a smokescreen to divert the public from a topic they consider dangerous for them. This process is vital for a democracy to work. In fact, I want to present a mathematical argument that supports the following bold claim: This process is more important for a democracy than voting! And perhaps the most amazing thing about this observation is that this is a direct consequence of the geometry of voting.

Policies as points in space

Political choices can often be represented by points in space. By this I mean that often, all the different policies that can be adopted with regard to a given political issue can be represented by different points in space.

As an example, let us consider budget decisions. A state’s budget can be represented by three numbers:

• the state’s revenue (e.g. taxes)
• the state’s expenditures (e.g. public spending)
• the state’s deficit

deficit = spending - taxes.

This equation defines a plane in three dimensional space. All possible budgets lie in this hyperplane. To visualize how this looks consider the following three dimensional coordinate system. The axes represent spending, taxes and deficit. The budget hyperplane lies slanted in space. Its intersections with the three coordinate hyperplanes are shown as dashed lines. For example, the horizontal dashed line is the set of budgets with spending = 0. It is a diagonal in the plane spanned by the deficit and taxes axes. Similarly, the other two dashed lines are diagonals in the deficit/spending and spending/taxes coordinate hyperplanes.

As we have seen, the set of all budgets with spending = 0 forms a line in the plane of all possible budgets. Increasing or decreasing spending by, say, 1 dollar corresponds to translating this line in the plane. Taking all translates by $k$ dollars, we obatin a family of parallel lines. And if we repeat this for the lines given by taxes = 0 and deficit = 0, we obtain a triangulation of the budget plane as shown below.

This picture shows a part of this triangular lattice. It allows us to visualize what happens if we change a given budget $b$. Say $b$ is given by a spending of 3 trillion dollars, tax revenues of 2 trillion dollars and a deficit of 1 trillion dollars and suppose we increase tax revenues by 1 dollar. This corresponds to moving from $b$ to the line given by 1 trillion and one dollar of tax revenues. However, we have to make a choice here. Do we use that one extra dollar to lower the deficit or increase spending? If we go “left” in the picture, we stay on a line where the deficit remains constant but we move from one “spending line” to the next thus increasing spending. On the other hand, if we go “right” we keep spending constant and decrease the deficit.

Now that we have a means to visualize how different budget decisions relate to each other, we can use that to visualize the current positions of various political actors in the US with regard to the federal budget.

We place the 2011 federal budget in the center of our picture at the point labeled $b$. Next we draw the three lines through $b$ along which spending, taxes and deficit are constant, respectively. Both, the democrats and the republicans want to decrease the deficit. So if you asked a democrat or republican where their ideal 2012 budget would be, then they would tell you that it lies below the deficit line through $b$. Ideally, however, a democrat would like to decrease the deficit by increasing taxes, without decreasing spending. So, qualitatively, a democrat’s ideal policy would lie in the blue region. A republican, on the other hand, would want to reduce spending to finance the decrease in deficit without raising taxes. So a republican’s ideal policy would lie in the red region. There are other people, though, e.g., Keynesian economists, who think that lowering the deficit is not a priority at the moment. They think that it is much more important to stimulate the economy by lowering taxes and increasing spending, even if that means that the deficit has to increase even further. People in this comparatively small group might have an ideal budget somewhere in the green region.

We have now seen how to think geometrically about politics. Now we are going to apply this point of view to analyze the dynamics of the majority vote.

Spatial Voting and the Power of Agenda Control

So far, we have only visualized policies as points in space. Now we come to McKelvey’s theorem, the main result I want to tell you about. To be able to state this theorem, we need to formulate a model of voting, called spatial voting.

As we discussed, every policy corresponds to a point in space. Now, spatial voting simply means the following.

• Voters are asked to decide between two alternatives, the status quo which we call $x$ and a proposed new policy $z$. Voters can vote for either of these two alternatives.
• The alternative with the most votes wins, i.e., we are having a majority vote.
• Every voter has an ideal policy in mind. So the voters can be represented by a set of points in space.
• Finally, we make a very important assumption: Every voter will vote for the alternative closest to his or her ideal policy!

Consider as an example the following picture. The black points represent the ideal policies of all voters. The blue point labeled $x$ marks the status quo, while the red point labeled $z$ marks the proposed new policy. In this example, most voters are going to vote for $z$ as $z$ is closer to their ideal than $x$. As we are running a majority vote, this means that $z$ is going to be approved and will become the new policy.

Now we can state McKelvey’s theorem.

What does this mean? Suppose there were someone with complete control of the agenda. This agenda controller would have absolute control over which new policies are proposed. Whenever the agenda controllers proposes a new policy, the voters would get to vote and the agenda controller would be bound by their decision according to the majority rule. This means that an agenda controller cannot decide anything. All he can control is between which alternatives the voters get to choose.

So, suppose there is an aganda controller and the current status quo is $x$. Suppose further that the agenda controller wants voters to pass a new policy $z$ that he favors (but that everybody else may hate). McKelvey’s theorem now states that there are cleverly devised policies $y_1,\ldots,y_k$ such that the agenda controller can proceed as follows. First, he proposes $y_1$. The voters get to choose between $x$ and $y_1$ and they approve $y_1$ as the new policy because the majority of voters prefers $y_1$ to $x$. Now $y_1$ is the new status quo. Next, the agenda controller proposes $y_2$ and again, the voters prefer $y_2$ over $y_1$ so they are going to adopt $y_2$ as the new status quo of their own free will. The agenda controller continues in this fashion until the voters pass $y_k$ as their new policy. Then he makes his final move and proposes $z$, the policy that he likes but that everybody else hates. However, through his previous moves, he has managed to put the public in a position where the majority actually prefers $z$ to the current status quo $y_k$. And so they are going to approve of $z$ of their own free will.

In this way, someone in control of the political agenda is in complete power, even if he is bound by the majority vote of the population. This explains why, for example, freedom of speech and the right to protest are so important parts of democracy. Protest is a way of getting a certain item on the political agenda, trying to influence the direction that current policy should take. (See here for a recent example.) Conversely, if there is a small elite who dictates the political topic of the day, then this elite will get its way, no matter if vote or not. So the morale of the story is this:

Voting alone does not give the people any power! Just by voting, you can’t control anything. You also have to have a public discussion about political topics and this is what protest is about.

At this point, it is important to stress the difference between the provocative morale that I drew from the story and the statement of the original theorem. The theorem is provably true, but applies only to a very specific abstract model of a society. In reality, no one is in absolute control of the political agenda! And the theorem tells us that this is a very good thing!

So, while the theorem does not apply to western democracies such as the USA, it can still tell us something about the real world, by way of analogy, in just the same way that a good novel can reveal a truth about the world we live in. (See the Q&A section for more about this.)

Now that we have both a formal as well as an intuitive feeling of what McKelvey’s theorem is saying, let us try to get an idea why the theorem should be true.

The Idea behind McKelvey’s Theorem

In this section, I want to give an idea how the sequence of policies $y_1,\ldots,y_k$ are constructed. If you want, this is a proof by example, but of course it is not a complete proof of the theorem. For a thorough proof I refer the interested reader to the literature cited at the end of the Q&A section.

We work with an example in 2 dimensions. There are three voters $v_1,v_2,v_3$ whose ideal policies are given by the three black points $p_1,p_2,p_3$ in the picture. We draw the lines through any pair of these three points. The status quo is given by the blue mark in the center of this triangle formed by the voters’ ideals. So, intuitively, the current status quo $x$ seems like a pretty good compromise between the voters’ preferences.

Which policies defeat the status quo $x$ in a majority vote? Remember that the three voters will vote for the alternative closest to their ideal policy. So, for each voter, we draw a circle around that voter’s ideal policy and we choose the radius of that circle such that the cirlcle passes through the status quo $x$. Let $C_1$ be the circle around voter $v_1$’s ideal policy $p_1$. $v_1$ will prefer any policy $y$ to $x$ that lies strictly inside the circle $C_1$!

Consequently, the policies that will defeat $x$ lie in the intersection of any two of the circles shown in the picture. The area corresponding to the set of policies that defeat $x$ are shaded green in the picture. As you notice, the shaded area extends outside of the triangle spanned by the three ideal policies $p_1,p_2,p_3$. Let $L_1$ denote the line through $p_1$ and $p_2$. If we reflect $x$ at $L_1$, then we will obtain a point $x’$ at exactly the same distance to $p_1$ and $p_2$ as $x$. If we now move $x’$ inwards for a little bit, the resulting point $x”$ will lie strictly in the interior of both $C_1$ and $C_2$. Thus $x”$ will defeat $x$ in a majority vote, because both voters $v_1$ and $v_2$ will like $x”$ slightly bettern than $x$. Note, however, that for $v_3$ the new policy $x”$ is much worse than the status quo $x$. Since we are working with a majority vote, however, this is not taken into account.

As it turns out, this construction always works! We summarize this observation as follows.

For any policy $x$ and any line $L$ through two of the voters’ ideal policies, the policy $x”$ obtained by first reflecting $x$ at $L$ and then moving the reflected point back towards $L$ a tiny bit, will always defeat $x$ in a majority vote. Typically, $x”$ will be “slightly better” than $x$ for two of the voters and much worse for the third voter.

This observation is the staples of the agenda controller’s strategy for manipulating the voters. Phrased provocatively, the agenda controller’s strategy is this:

Play off different groups of voters against each other! Offer a majority of voters marginal incentives to hurt a minority really bad!

So how can an agenda controller use this strategy to get voters to vote for his favorite policy? The following figure tells it all.

We start with a status quo $x$. This we reflect $x$ along the line $L_1$ through $p_1$ and $p_2$ by playing off voters $v_1$ and $v_2$ against voter $v_3$ to get to a new policy $y_1$. Next, we reflect $y_1$ along $L_3$ by playing off $p_1$ and $p_3$ against $p_2$ to get to $y_2$. Then, we reflect $y_2$ along $L_2$, by playing off $p_2$ and $p_3$ against $p_1$. We can keep going in this fashion as long as we like. Note that in each step we give two of our voters a tiny improvement (we move towards their ideal points by a tiny bit) while at the same time making the situation much worse for the third one (we move much further away from his ideal policy). In total, we make the situation worse in every step! So if we continue this process long enough, everybody will be so unhappy with the current policy ($y_k$ will be so far away from all voters’ ideal policies) that everybody will be happy to agree to the policy $z$ that the agenda controller had in mind all along.

I find this proof striking for a number of reasons. First of all, even from this small example, you get a clear idea how the proof will proceed in the general case (even though you need a couple of technical tools like median hyperplanes). Second, for three voters, the proof requires only an elementary geometric construction that (I hope) does not scare off people who do not feel that comfortable around mathematics. Finally, and this is most remarkable, this proof gives clear instructions how someone who wants to manipulate public decision making should go about it: Play off different groups of voters against each other! The fact that many political decisions are two or higher dimensional provides the necessary room to maneuver.

(In fact, I find it striking how similar the manipulative stragtegy suggested by the geometry of voting is to how many agents on the political stage actually behave. To manipulate public opinion, they try to reduce the public discussion on an issue to a one-dimensional problem. To a choice between two alternatives. They try to hide the fact that real world problems are higher-dimensional.)

Conclusion

I hope to have convinced you that there is merit in looking at politics from a geometric point of view. A natural consequence of this point of view is McKelvey’s theorem, which states that an agenda controller can get voters to agree to anything the agenda controller wants of their own free will. Fortunately, there are no perfect agenda controllers in real world democracies! But McKelvey’s theorem reminds us that we had better exercise the freedom we have to shape the political agenda. Because without active political debate, the right to vote alone does not give the public any power at all.

Q&A

Budgets are more complicated than this!

Of course a state’s budget is much more complex than the model I suggested above. There are many more parameters to take into account! The advantage of this model is that it captures the gist of the problem while still being low-dimensional enough so that we can draw something!

There are some assumptions missing from McKelvey’s theorem!

That is true. First of all, McKelvey’s theorem only works in dimensions two and up. If different policies all lie on a line, then McKelvey’s theorem fails. The same thing can happen in higher dimensions if the voters’ individual preferences have a particular pattern. For example, if all voters have the exact same ideal, then agenda control won’t work. The technical assumption I omitted in the above statement, is that the median hyperplanes (whatever that is) of the the voters’ ideal policies do not all intersect in a single point. However, this assumption is not critical for the following reason: By slightly perturbing the voters ideal policies we can always guarantee that this assumption is satisfied. Also, the theorem assumes that the set of policies is all of $\mathbb{R}^n$. This is not true in the budget example, as negative values for taxes or spending do not make sense (a negative deficit is fine, though). But generalizations of McKelvey’s theorem take care of this issue (see below). Finally, the theorem assumes an odd number of voters. McKelvey argues, however, that this makes no difference, as the agenda controller can decide to participate in the vote if and only if his vote is required to make the total number of votes odd.

It should also be mentioned that McKelvey’s theorem assumes that all voters use the Euclidean distance to measure which policies they prefer. However, this assumption is not essential as McKelvey showed in a subsequent paper. (See below.)

What generalizations of McKelvey’s theorem are there?

McKelvey generalized his theorem considerably in subsequent work. For example, he showed that if the set of policies is an arbitrary connected subset $X$ of $\mathbb{R}^n$ and each voter measures with his or her own, private notion of distance (which has to be continuous) then, still, the top cycle of the majority rule is going to be all of $X$ (i.e., agenda control works). He concludes, in his own words:

“Regardless of other voters’ preferences, any one voter with complete information of the other voters’ preferences, control of the agenda, and the ability to cast his own vote as he chooses can always construct majority paths to get anywhere in space.”

Is protest really more important than voting?

The provocative catchphrase that “protest is more important than voting” is meant in the following sense: A political system in which people do not shape the political agenda at all but only go to vote is not democratic. In this sense, active participation in the public political discourse is at least as important as going to vote. This is not to be construed as an excuse not to go voting, however!

Is there a mathematical proof that influencing the political agenda is more important than voting?

No. Mathematics makes statements about abstract models. McKelvey’s theorem makes a statement about a particular abstract model of a society in which the people do not influence the political agenda at all and behave in a predictable way. This statement is provably true. But “mathematics” makes no claim whatsoever about what this statement about an abstract model has to do with reality.

How realistic is the model of society assumed in McKelvey’s theorem?

First of all: I am not a political scientist, and I do not know of any empirical research on this subject. My personal opinion, however, is the following: Western democracies such as the USA do not fit the model of McKelvey’s theorem at all. In particular there are a number of important differences:

1. In practice, all actors on the political stage constantly strive to control the agenda! So McKelvey’s theorem does not apply to practical politics, but it offers an unconventional explanation why control of the agenda is so strongly contested.
2. The theorem assumes that the agenda controller has complete information about the preferences off all voters. This is not true in practice. However, never before was so much data about personal opinions readily available to anyone with an internet connection. So one could argue that western societies are converging on the condition required for McKelvey’s theorem to apply.
3. Voters oppinions change over time. The theorem assumes that opinions are constant. I would guess, though, that the theorem could be extended to account for changing opinions.
4. There are few voting processes where voters only decide to aprove or reject a single proposition. Presidential election in the US fit this description but are too closely tied to personality to admit a reasonable parameterization by points in space. In most other public elections, the voters have to decide between many alternatives, so spatial voting does not fit either. In process of passing a bill in congress fits the model accurately, however there voters are constantly trying to control the agenda. Ballot propositions also fit the model, but again, voters have direct influence on the available propositions through the initiative system.
5. As mentioned above, the theorem makes a couple of additional technical assumptions. However, I do not think that these are essential, especially with regard to McKelvey generalizations of his theorem.
6. There is one technical assumption that deserves special mention: continuity. I do not think that it is that much of a constraint to assume that voters preferences are continuous. Rather, I think it is a significant departure from reality to assume that the space of all policies is a continuum in the first place. For a budget, this is simply false: Two budgets cannot differ infinitessimally - they can only differ by cents.

There is no policy that cannot be defeated by any other policy in a majority vote, right?

Exactly! But instead of looking for a policy that cannot be defeated, we can go for the policy that is least likely to be defeated! This is the policy for which the shaded green area in circle picture above is smallest. As it turns out this so-called “strong point” of the majority game lies roughly in the center of all voters’ ideal points. More precisely, it is given by a convex combination of the voters’ preferences, where the weights of the convex combination are given by spatial power indices of the voters. This means that centrist voters have more influence on the strong point than extremist voters. (Owen and Shapley 1989)

Where can I read more on this?

I recommend the excellent article by P.D. Straffin Jr. in the Handbook of Game Theory and Richard McKelvey’s original articles on the subject.

1. Straffin, P. D. J. (1994). Power and stability in politics. Handbook of Game Theory with Economic Applications Volume 2 (Vol. 2, pp. 1127-1151). Elsevier.
2. McKelvey, R. D. (1979). General Conditions for Global Intransitivities in Formal Voting Models. Econometrica, 47(5), 1085-1112.
3. McKelvey, R. D. (1976). Intransitivities in multidimensional voting models and some implications for agenda control. Journal of Economic Theory, 12, 472-482.