Coding the Commune - blog

"Alienating Ourselves: A Response to Justin Searls on 10x Developers"

2023-07-31T00:00:00-05:00

In a recent blog post, Justin Searls of Test Double claims that 10x developers are fading away, because the conditions that make one a 10x developer were more prevalent for developers born before 1990. The difference between those born before 1990 and those who come more lately is, according to Searls, enthusiasm. That is, before the huge influx of cash into the industry, programming was simply "a firmly middle class job that appealed to people who really loved computers." More simply, Searls thinks there wasn't a strong incentive to become a developer other than you were a computer enthusiast, a hobby that boomed during the personal computing revolution. Now, programming has become a "comfortably upper-middle class profession that attracts anyone who wants to secure their financial future," meaning that people who don't particularly enjoy or have an enthusiastic interest in computers are becoming developers. He doesn't see people programming for the cash to be a problem, but he does predict a looming generational conflict between the 10x enthusiasts and the mere professionals.

However, even if we grant Searls' claims about 10x developers and enthusiasm, I don't see why the field gaining careerists would drive enthusiasts, and thereby 10x developers, to extinction. The same incentives still exist for people who are enthusiastic about programming to become developers, whatever the corresponding increases in cash incentives. It just draws non-enthusiasts in greater numbers. Nor was there a singular moment in history where computers were just right to create enthusiasts, as the following facts are manifest: 1. enthusiast developers under 30 exist in great numbers 2. many enthusiast developers born before 1990 do not have a programming origin story like Searls lays out (including me!)

Given these facts, at most, the proportion of enthusiast developers in the field might decrease, but they would not become depleted. In fact, I would think that by Searls' lights, there should be numerically more enthusiast developers in the industry than ever before. So if there is a looming conflict between enthusiasts and mere professionals, there isn't strong reason to believe the shift is generational.

But Searls' blog post, by his admission elsewhere, isn't really about 10x developers. I would argue it isn't even really about the empirical claims about generational incentives (which would be a lot of work to prove, I might note). What it's really about, in my opinion, is a connection between enthusiasm and productivity, and the way we developers sometimes misuse our enthusiasm for programming in order to become more productive at work.

The Good Life and Alienation

What makes a life go well? What leads to human flourishing? What makes lives more or less meaningful? These are difficult philosophical questions to answer, and in the Western tradition of philosophy, these questions go back at least to Socrates himself. But we don't need a full theory of the good life or meaningfulness to know that some things are likely to make one's life go worse. For instance, working excessive, unpaid overtime hours on a work project only to make others richer, most of us would say, does not make one's life go better. But this is exactly what many of us enthusiast developers, including me, do on a regular basis, whether or not we're 10x developers (which I'm certainly not).

The concept of a 10x developer is fraught, and perhaps mythical, and so I think we should abandon it for a more realistic term: a hyperproductive developer. A hyperproductive developer can finish more work than the average developer (aka, a productive developer) in the same amount of work days. Perhaps the hyperproductive developer works more off hours than the average developer, or is unusually efficient, or both. Contra Searls, I don't think there's any necessary connection between enthusiasm and hyperproductivity. I do agree with Searls that many of us who are enthusiasts, people who live to program, use our enthusiasm to power their productivity at work.

In particular, hyperproductive developers regularly work extra hours when the costs clearly outweigh the benefits of just logging off, and they are able to motivate themselves to keep going through their love of code. Probably the most severe costs of hyperproductivity are burnout and workaholism. But what's more interesting to me, philosophically speaking, is that hyperproductive developers are needlessly alienating themselves from their creative forces.

Alienation

Karl Marx and Friedrich Engels famously defended their concept of alienation in The German Ideology, noting that once human societies engage in a division of labor, each person has an exclusive sphere of activity: "He is a hunter, a fisherman, a shepherd, or a cultural critic, and must remain so if he does not want to lose his means of livelihood" (The German Ideology, MECW V, 47). In tech, we are generally pigeonholed much further in what we kind of work we do. But if I am to truly flourish, Marx and Engels argue, I require freedom to pursue any branch of activity I wish, where I can "do one thing today and another tomorrow, to hunt in the morning, fish in the afternoon, rear cattle in the evening, criticise after dinner, just as I have a mind, without becoming hunter, fisherman, shepherd, or critic" (ibid.).

In terms of we enthusiast developers, the ideal world would be one where we could do (say) data science in the morning, compiler design in the afternoon, web applications in the evening, embedded systems work after dinner, just as we have a mind, without becoming committed to these as job titles. In such a situation, our work is our life, in a good way: "My work would be a free manifestation of life, hence an enjoyment of life" ("Comments on James Mill", MECW III, 228). This enjoyment comes from making something through an activity where my life, my creative energies, would be made manifest in something external to me, and I can have the pleasure of being recognized in my work, by myself and others (ibid., 227). However, when I make something for a company, a boss, a client, etc., I am alienated from the fruits of my labor: it belongs to that company, boss, client, and my work is unrecognizable in the product. As a result, "my work is an alienation of life, for I work in order to live, in order to obtain for myself the means of life" (ibid., 227). I am not living to program anymore, I am programming to live.

Obviously, as professional developers, we must program to live. But why, as enthusiast developers, do we program to live after hours? Why do we not live to program (or whatever other activity we desire) outside of work? We must resist the urge to conspire in our own alienation, and use our creative forces for better pursuits than work in our free time. Rather, we should be using our creative energies on software projects that fulfill us and will make the world closer to a place where anyone can pursue any branch of knowledge or activity, just as they please.

Bibliography

MECW: Marx, Karl and Friedrich Engels. Karl Marx, Frederick Engels: Collected Works. New York, International Publishers. 1975-1987

"Lazy Consensus and the Problem of 'Endless Meetings'"

2023-05-21T00:00:00-05:00

In the standard case, collective decisions are made by a group of people in a room together. This arrangement requires planning on the part of the participants. Rooms need to be booked - at an inn, a library, a community center, or someone's house. Schedules have to be accommodated. Refreshments have to be prepared.

As time has gone on, the need for spatial and temporal proximity between participants has lessened. The COVID-19 pandemic has forced many of us to participate in collective decision-making from afar, via tools like Zoom, Teams, and Meet. Other communication tools like e-mail, Slack, and Discord allow us to conduct decision-making processes asynchronously, through voice and text messages. Discussions can be continued over several days without the need for late-night discussions or repeated bookings and reschedulings.

Obviously, not all collective decision-making can be done this way. Some collective decision-making needs all participants together, as the decisions may need to be made rapidly and require everyone see the current situation unfold. Some people do not have access to the aforementioned tools for remote decision-making. Some discussions are too sensitive to be recorded even on encrypted services like Signal.

Luckily, generally speaking, software development doesn't face these hurdles. We can conduct collective decision-making at a distance and asynchronously. And this simplifies all kinds of decision-making procedures for software development, but especially decisions by consensus. When conducting collective decision-making by consensus, a decision is made only if every participant in the project freely consents to the decision. So, no decision can be made unless no one in the project has any objections to the decision.

But there's an obvious case that makes this arrangement difficult: emergency changes. When there's a critical bug in the code, we cannot wait for everyone to have a chance to voice any concerns. Someone needs to push the change as soon as possible. How can a project that makes decisions by consensus account for this?

Lazy Consensus

The answer is lazy consensus, a method I briefly discussed in my introduction to software by consensus. Lazy consensus allows a smaller group of participants to change the code without holding a discussion about the change first. Lazy consensus is possible with software development, because distributed version control tools, like Git or Mercurial, give us a "Time Machine" to roll back the changes if anyone disagrees after the fact. This is a fairly unique luxury that is built into software projects: most other decisions, once made, cannot be undone.

Besides allowing for quick emergency changes, lazy consensus mitigates a classic criticism of consensus decision-making - namely, that it leads to "endless meetings." If we need to have consensus on every decision, then we will need to hold a meeting for every decision. Compare this with organizations that elect officers, who can quickly make decisions in lieu of the participants of the organization's project(s). Although highly relevant to software projects, the debate between consensus and majority rule is most commonly found in the realm of leftist politics. Typically, majoritarian socialist organizations, such as the Bread and Roses caucus of the Democratic Socialists of America, make this objection against libertarian-socialist and left-anarchist competitor organizations, which are typically consensus-based.

But lazy consensus greatly reduces the number of meetings needed, as trivial changes can go ahead without discussion beforehand. For example, no discussion needs to be held for the correction of a minor typo, as it is likely no one will object. On the other hand, if a change is not trivial, one can hardly complain about the need for a meeting, as it merits discussion. Between this, and modern communication tools, the number of meetings is greatly reduced.

The Problem of Endless Meetings

However, this isn't the whole of the "endless meetings" objection, as one could accuse decision-making by consensus of being "endless" in duration. That is, they claim that reaching consensus is sometimes impossible, or takes too long to reach in many cases. One can argue that lazy consensus doesn't avoid this problem, as many discussions over significant, non-trivial issues will be interminable, or simply end in no change at all. Much more efficient, they say, is majoritarianism, which is the paradigm of decision-making that makes decisions by majority vote. When discussion begins to repeat itself, or becomes deadlocked, you put it to a vote and move on.

The majoritarian critique of consensus thus makes two claims: one is that majoritarianism is generally more efficient than consensus; the second is that consensus is prohibitively inefficient.

The first claim is certainly true: generally speaking, any time consensus is reached on a decision after protracted debate, a majority decision would have been reached just as quickly, if not faster, due to an earlier majority vote. But a mere difference in efficiency is hardly a reason to reject decisions by consensus in favor of majoritarianism. That would be to say that consensus was without value, something the majoritarian can hardly claim, as a consensus decision is surely better than a fractious, divided one for the long-term health of a project. Consensus is the goal of all collective decision-making - even majoritarian organizations only resort to a vote when there is disagreement (Blunden 2016, 5). We will discuss this more in a later post.

Rather, we have to defeat the stronger, second claim, that consensus decision-making is prohibitively inefficient, which would cast doubt on lazy consensus as a viable collective decision-making procedure.

But the inefficiency of consensus decision-making is an empirical claim about the world. Majoritarian political organizations, like Bread and Roses, often state that consensus decision-making leads to failure and breakdown of projects. However, they typically simply provide anecdotes of consensus's worst-case scenarios of long, bitter debates, without providing any evidence that these scenarios are representative.

And we too could provide anecdotal data if we wished: plenty of successful software projects operate by lazy consensus. But setting dueling anecdotes aside, research suggests consensus is hardly prohibitively inefficient: in a rare study on consensus decision-making's efficiency, it was found that the average debate time, in person, took 2 hours for the organizations surveyed (Leach 2016). Some groups are faster than others, of course, and Leach's research suggests that the time to reach consensus is improved by: familiarity with decision-making by consensus; belief in the importance of reaching consensus; meeting more frequently; and personal investment in the project (Leach 2016, 53). Interestingly, Leach's research found no correlation between number of participants and efficiency, nor did the age of the organization seem to play a role (Leach 2016, 53).

Further Questions

Exactly what lessons we can draw from Leach's research for software development projects that are done at a distance and asynchronously is hard to say. What we can conclude, though, from Leach and anecdotal evidence is that lazy consensus does not inherently lead to "endless meetings" to resolve issues. Obviously, more difficult and serious questions may take a very long time to resolve, but that is precisely because they are difficult and serious decisions. Is this a problem? That depends on how you answer a number of questions: - Ethical and value judgments - Is the process of debate and reaching a consensus inherently valuable? - Does overruling the disagreement of minority voices violate their rights of autonomy? - Practical questions - Does the value of efficiency outweigh the risks of ignoring some participants' reservations? - Does the value of efficiency outweigh the risks of long-term incoherence in a project, due to multiple different visions of the project?

These are questions I will address in the next post.

Acknowledgements

Thanks to Preston J. Werner for valuable discussion.

Bibliography

Blunden, Andy. 2016. The Origins of Collective Decision-Making. Brill, 2016 (also available at Haymarket Books)

Leach, Darcy K. 2016. "When Freedom Is Not an Endless Meeting: A New Look at Efficiency in Consensus-Based Decision Making." The Sociological Quarterly 57, no. 1 (2016): 36-70.

"Software by Consensus"

2023-05-15T00:00:00-05:00

The goal of this post is to carve out some ideal types of software development projects, where decisions are made by consensus (or rough consensus). This exercise is, in part, to lay the groundwork for other posts that imagine what software development would look like in a more equitable future society, as well as to identify some currently existing projects that prefigure such an arrangement.

We will get into what making decisions by consensus means, and compare it to other forms of collective decision-making below. But first, we must start with more primitive concepts.

Projects

Human beings work on projects. The relevant concept of a project is difficult to define, but it can be characterized. First, projects are (for this discussion) a group activity. They involve participants coming together to do something. Second, participants in a project come together in good faith to pursue their shared interests (although they can certainly disagree on much else). A group pursuit of shared interests forms a collective will to do something - in our case, create software.

This leads to an important corollary: not all groups activities are projects. For example, when a company hires someone to write some code, this does not constitute a project, because the employer and the employee do not share the same interests, and therefore do not form a collective will. The employee is at the command of the employer, and pursues the employer's interest in exchange for wages. In this sense, our use of the word project is idiosyncratic, but I hope not too much so.

Participants

In the kind of software project we are interested in, a participant is anyone who has a voice in the project. In software, participants in the project are not just the developers. Participants also include users, testers, and reviewers (where a reviewer ultimately approves the change). The role of users is particularly complex and will be set aside for another post.

Everyone in the community behind a project can play some or all the roles simultaneously. In the limit case, everyone in the community works on the project as a developer, a tester, and a reviewer -- and these same people are the only users of the software. This kind of equilibrium is uncommon simply because most of us want our software to be used widely and be put to good use. This would be impossible if we limited use of software to those who were actively developing, testing, and reviewing the software. (This kind of project need not be exclusionary: it might just so happen that no one needs the software outside the small group of people directly building it.)

(As an aside, we're assuming that all participants have access to the source code at all stages of development. Otherwise how would participants make collective decisions about changes to the code? As a result, we are not so much interested in the Cathedral versus the Bazaar distinction Eric S. Raymond draws [apologies for having to bring him up], at least for now.)

Decisions

So now we come to collective decision-making: when a change is proposed, how do participants decide whether to implement it? What I've hinted at is that such decisions could be made by consensus. But that is not the only paradigm of collective decision-making. Following the work of Andy Blunden in his The Origins of Collective Decision-Making (pp. 4-5), we will consider three paradigms (summarizing Blunden): - Majoritarianism: When a project's participants can't come to a consensus on what to do, it is put to the vote. Everyone's vote is equal, and when the votes are tallied, the majority carries the day. Those in the minority do not have the right to block the majority's will once the votes are cast, and their participation in the project binds them to this commitment. - Consensus: A decision is made only if every participant in the project freely consents to the decision. In projects that require consensus to make decisions, minority views have a great deal of weight, and the stability of the status quo is therefore favored over rapid innovation. Participants who disagree with the decision, but do not want to block progress, may "stand aside" (although participants frequently standing aside is a sign that the consensus decision-making of a project has deteriorated). - Counsel: When decisions are made by counsel, one particular person (a king, a benevolent-dictator-for-life, etc.) takes the counsel of the other participants, but ultimately it is that one person alone who makes the decision. Even though it is undemocratic, it is a case of collective decision-making. Every participant in the project has a voice, even if their voice is not equal.

The Basics of Consensus

The consensus model of decision-making requires that everyone consent to the proposed action(s). In that sense, any one participant can veto any proposal. This is radically different from majoritarianism where one "no" vote can only negate one "yes" vote, and vice versa. Under the consensus model, a "no" vote is all powerful, as no amount of "yes" votes can overcome it. Minority views are thereby "treasured" by the consensus model, while they are merely tolerated in the majoritarian model (Blunden, 5). Even so, wrongheaded or lasting disagreement is typically not tolerated in consensus, and can lead to expulsion from the project or informal sanctions. But for all this talk of voting and vetoes, it's really only a figure of speech: there is no voting in the consensus model, there is only consensus or dissent. Those are the only two states that matter. As a result, there's no sense in counting every "vote" other than to help guide discussion. Further, unlike in other contexts, silence counts as consent, requiring no "vote" to be cast in order to agree.

Consensus and Software

Consensus may seem completely implausible as a method for making decisions in a software development project, especially if, like me, you come from a corporate programming background. But there are features of software development that make it particularly suited to decision-making by consensus.

The first is that version control tools, like Git, give us a "time machine", which allows for "lazy consensus." That is, a participant of a project can make a change to the software if they believe it would have reached consensus (e.g., a critical bug-fix) because it can always be rolled back if anyone objects. This eliminates one of the most frustrating aspects of consensus decision-making: the need to discuss every decision, especially on very short notice. Lazy consensus has been implemented with great success at the Apache Foundation.

The second feature that makes consensus suited for software development is that software development decisions lend themselves to asynchronous communication. Collaborating on software is much different than collaborating on laying railroad tracks, cutting down trees, or conducting spacewalks. There is generally no need for eye contact, instantaneous communication, or spatial proximity. As a result, discussions are easily conducted using tools like mailing lists, slack, GitHub issue threads, etc. This makes meetings easier to hold, as there is no requirement for scheduling to make sure everyone can make it.

The final feature of software development that makes it suited for consensus decision-making is distributed version control tools, like Git and Mercurial. Such tools allow for decentralization of projects, as in a distributed version control system, there is no centralized codebase. Every participant of the project can have the full codebase and its history of changes locally on their machine, and can propose changes for everyone's consideration. Without the distributed nature of these systems, some subset of the project's participants would be gatekeepers to the "pristine" codebase, and could lock dissident voices out. Further, if the project devolves, participants can readily fork the codebase into different projects.

Conclusion

In the next few posts, I plan on discussing these features of software development in greater depth. The next post will discuss lazy consensus and the Apache project. After that I will turn to the history of Git and how its structure enables different kinds of collective decision-making, focusing mostly on consensus.