Academic Politics: Optimizing For Visibility and Prestige

The people in positions of authority and seniority in science got where they are because they are very good in at least one of the following things:

• their scientific domain (ideally, but even that is sadly not always the case)
• writing smart-sounding papers (i.e. convince others they found knowledge)
• getting funding for awesome-looking projects (i.e. selling big visions)

What is not measured by the sticks used in the performance evaluation of a scientist, principal investigator, professor, etc., are for example:

• the sustainability and reusability of the work that has been done
• the quality and actual practical usefulness of the produced output

What counts most in science is that there is a nice publication or project report at the end – another citation with the names of the authors to fulfil the required quota. Published text, and some nice presentations – this is the interface between the people doing the work and the people giving out the money.

As the vast majority of scientific output consists of novel ideas and prototypes, and often the topics are so deep that proper evaluation is barely feasible, the game is biased in such a way that presentation and appearance is often much more important for success than the actual merit of the work. This, of course, is not how science is supposed to function, but the current global academic environment adapted to the metrics it has been subjected to. It developed elaborate methods for producing quantity over quality, and a lot of hot air sold in huge, shiny balloons.

This is an art that some bad actors seem to have perfected in order stay afloat, and this puts everyone who tries to do actual good work at an evolutionary disadvantage in this brutal competition for so-called excellence and reputation. If appearance is enough to survive, all extra work is wasted energy, and is better used to produce more appearance.

To get funding, you usually need to frame your work as something novel, something grand and inspiring. It is much easier to get money for “innovative” projects that

• try to apply experimental approaches (not battle-tested, often questionable)
• use exotic convoluted technology (created + mindlessly promoted in academia)

than it is to get money for important but more “boring” things such as

• creating small useful tools that help people in a scientific field
  (e.g. solving a small but common problem they face in the daily work)
• maintaining and incrementally improving existing systems
  (unless the improvement is super-visible, making it worthy of a paper)

There is often not much of an incentive to find the most useful, practical and technically feasible solution, instead a lot of incentive to sell snake-oil and deliver half-assed prototypes. And if getting money for a project is not already difficult enough, you need to work around political requirements and landmines, such as

• everyone wanting to put their names and logo on the project
  (if we can’t, please reinvent the wheel for us – i.e. classic NIH syndrome)
• some big guys doing very similar things but disliking each other
  (so you need to avoid them, instead of joining forces)

Software Development In Adverse Conditions

If you are a software developer and take a position in some academic institution, unless you took a job in some central administration / infrastructure department – let us ignore this special and quite different case – then…

1. you might be the only skilled developer in an otherwise non-developer team
2. the people you report to probably have no idea how to manage software projects
3. your contract is almost certainly funded by a 3rd party project and is temporary

In the following sections I unpack these points and explain the problems they imply.

Lone Wolves of Academia, Unite!

Software development is the creative team sport of effectively building complex systems that work correctly and reliably over longer periods of time. Being the only developer working on a project in your unit, or being a too small team for the scale of the project – this is a massive problem, as it limits the kind of the projects you can successfully complete.

There is a limit to the size of a project a lone warrior developer can handle. It works well enough for small tools, but it does not scale to large, complex software – at least not if you want to get things done in reasonable time. So for bigger projects you will be often working in loose collaboration with other people, who are employed at unrelated institutions, each of them with different agendas, priorities and paths of decision making. Welcome to the nightmare of diffused responsibility and lack of accountability. ¹

Working in a distributed team of developers might have its intrinsic challenges, but those can be and are overcome – under the assumption of a coherent organization, e.g. managed within the same company or institution. But the kind of loose and promiscuous project collaboration as practiced between academic institutions, while maybe working well for doing some research together and ensuring fruitful exchange of ideas, makes it very difficult to effectively work on a complex piece of software together, unless the majority of work is managed and done at one place and only minor tasks are offloaded to motivated external contributors.

Academic software development works like (and often actually is) open source development – it is based on voluntary, unstable and barely uncoordinated investment of work and time, driven by the self-interest of contributors. But a successful open source project might have tens of thousands of stars on GitHub and dozens of enthusiastic contributors, whereas the popularity of your average work project in academia will be closer to that of your average personal toy projects, and will have similar difficulties in getting any attention. If you don’t have at least a few developers in your team you can really count on, good luck.

Software Project Management != Scientific Management

Science is pretty good in finding and walking the paths less travelled and discovering new and unexpected places. However, what science is exceptionally bad at is building a nice, straight, cemented road that others can use in order to arrive somewhere they actually wanted to go.

The people getting money and taking decisions very well understand the game of academia, but they usually have no clue about product development and team management as it is needed for larger software development efforts. Scientists, on any level, are not used to breaking down big projects into realistic milestones, distributing work and setting priorities wisely. The scale and required investment for a complex software is simply much larger than the average research project that just is expected to produce a paper or two in the end.

Because non-technical staff manages technical staff, work is often distributed ineffectively. They simply do not know what a single developer is capable of, and what the individual strengths of the available people are. And beyond vague project goals, there are rarely clear requirements – another massive problem.

If you are told to build a bridge, but not really from where to where, for which purpose, how much material you can use, how much load it needs to carry and what other constraints need to be satisfied – what kind of solution should you design? If your assignment is to “build a big bridge that can carry a lot of traffic”, what is even a proper measurement of success? A very imprecise way of stating requirements or planning projects is often even done by design, because if you do not say what “big” and “a lot” means when you apply for funding, you can define it in your progress report later in such a way that it perfectly fits what has been done so far.

So this is the cheap trick often used to ensure that almost every project in academia ends up being a success (at least on paper): You first shoot the arrow, and then you say that whatever you hit was the target all along. It really does work roughly like in that old joke. Your funding proposal basically just needs to say: “We are going to shoot an arrow into a well-chosen direction” and your final report will say that “after a lot of deliberation we settled on a promising direction, identified a suitable target and successfully hit it with our arrow, proving the efficacy of our arrow-shooting device”. Piece of cake.

So they just want you to figure something out, come up with something that sounds reasonable and can be tried out – the same way you just give a promising idea to a PhD student to let them work it out until they can write a paper or thesis about it that somehow fits the initially planned topic. Only that you often have absolutely no clue about the domain of people you try to build your solutions for, so you cannot do the requirement engineering properly, and end up building something that might, possibly, hopefully, make some sense and help the people it is intended for.

If the requirements are not made clear enough, it is impossible to fail on paper, but it is also impossible to succeed in real life and come up with a useful solution. And the truth is, your actual solution often does not really matter all that much, because nobody really cares anyway.²

There is rarely a person combining the authority, expertise enough time and the willingness to take on the “product owner” role of a larger software project in order to push and guide it towards a genuine success. Usually, your manager will already be busy…

• getting more external collaborators on board and motivating them to contribute
• writing reports that say how awesome the project is and how well it is going
• trying to secure more funding to keep you around, or get some more developers

In the meantime, feel free to build small and big bridges all over the place the way you see fit. They will just observe which bridges end up being used and try to get more funding for them. Wait – does that remind you of something? Yes, that’s right – this is how research is typically done. Invent interesting solutions, then look for problems they might solve, and find people who might care about it.

The Persistence of Volatility

Now consider all of the problems above, and multiply them by the fact that your position is not permanent, as is the position of most employees in your institution, as is the position of the majority of people doing the actual down-to-earth work in science.

A scientist can hope to eventually get a permanent position and live happily ever after. But a software developer in science – what is this? There are effectively no predictable career paths for highly qualified technical (but non-scientific) staff in science, and only few institutions afford to even have some larger technical department of people doing de-novo software development (in contrast to, e.g., maintenance of existing infrastructural software, such as digital libraries).

So most software in science is developed by small, often highly distributed groups of people, who are often not even trained to be developers or maintainers of a software project and most likely will not be around in a few years. The instability of the employment of people who write and maintain code directly affects the long-time health and viability of that code. It is a well known, still mostly unsolved problem that software projects in research tend to die soon after the one or the few critical people who were taking care of it move to a different project or leave academia altogether. Most software projects in research have a bus factor of 1.

And it is on top of this fragile house of cards consisting of barely maintained eternal prototypes where a lot of cutting-edge research is being done. If you hear this the first time and this does not give you a sense shock and existential dread, then I do not know what will. Academia simply has no tools or mechanisms in place to ensure successful and reliable development and operation of complex software on time scales going beyond 3-5 years³.

Summary

Over the last three years I went through one stage of disillusionment after the other. I guess I went through something like a quarter-life crisis, and I think that I came out as a more adult and less naive person – both in good ways and in bad ways. The first-hand glimpses I got of the plumbing and machinery of academia taught me a lot about people, large organizations and politics overall. And lately, I have been standing at the bifurcation of two paths. The key question determining my future path being – am I willing to sacrifice myself to make a tiny dent in this ugly gigantic mess called academia, could I be content by just imagining that I gave it a very little nudge into the right direction?

Do I have enough idealism, resilience and hope for science that I could learn to accept and live with the rules of the game which cannot be easily changed, to stay in this environment, trying to make the best out of it? Or do I rather accept the reality that the idealized, romantic idea of science is not at all how the real world works, probably never was, and most likely never will be?

One day I woke up and my gut feeling told me that the penny has dropped. I simply did not care anymore. And this moment I knew that it is time to move on. This environment does not really value my qualification and skills, and it does not offer me any perspectives for development or growth. The intrinsic passion for science (as a “social institution”) that I once had is now gone for good, and I know that the longer I would stay in this swamp, the more bitter, cynical and emotionally divested I would become. My technical abilities would slowly wither, lacking the right kind of challenge and motivation, and eventually I would end up being a hollow shell, either stuck forever in this cursed quicksand, or eventually spat out onto the street, emotionally drained and now also old and useless for the so-called industry, after wasting too many years too close to the ivory towers.

Once the system completely lost all the remaining magical flair of almost holy transcendence, walking the path of the idealistic martyr completely lost its appeal for me. All these causes looking like something to live and die for end up being mirages and lies. So probably it should be not very surprising to find that when all illusions are stripped away, science is not much different from the rest of the world and just as ugly – narcissistic people playing games of power and glory, while riding on the back of selfless people with the right mix of helper and Stockholm syndrome.

No, all the insanity of academia cannot be solved by any kind of software, while at the same time creating actually useful software for any purpose is nearly impossible in this dysfunctional environment⁴. I, for one, have seen enough. So academia, consider that a divorce!