Response to "too little technological detail"

[microprediction]

Brief response: The book points to an operating microprediction oracle and also describes in terms that must obviously be less precise, other ideas for microprediction oracles and micromanagers. See discussion re existing oracle.

Longer response

What detail do you want to see?

1. Documentation for other people's ideas does not exist because their implementations don't either (yet)

But they might, someday.

This book is aimed at all potential contributors to a prediction web, as it might be described. You can supply code, or launch algorithms, or create new feeds, or supply mathematical insight, or help in any number of technical ways.

You can also help by socializing the idea, or adding to the demand for explicit versus implicit microprediction - by nudging your data scientists to send their model residuals to their nearest microprediction ``oracle''. Five lines of Python won't kill anyone.

Five lines of Python might help you grok it too, in greater generality. See below.

2. Documentation for one microprediction oracle is gorgeously detailed (docs)

And if you are not happy with it, you can file a pull request. Thanks. See the doc map perhaps. There are also lots of articles on the game, strategy and so forth, an entire companion site microprediction.com and so forth.

3. I said "no whining" and yet here we are

This book is pitched at people who want to create things that don't yet exist. It is pitched at people who don't assume things are impossible because they aren't spec'd out for them already in excruciating detail. I would not need help if the code was already written. It is pitched at people who in many cases have a better idea than me as to what some or many of those details should be. It is pitched at people who might stand a chance of moving this forward.

I take the example of the parimutuel, in our ``collider'' category, and note that implementation was achieved by engineering pioneer Sir George Julius more than a century ago - many decades before the dawn of the computer era.

Thus anyone complaining about the task has been forever shamed. Julius did not have Python, javascript or web assembly at his disposal. We are not faced with the challenge of inventing the mechanical rotary shaft adder (one of Julius' pre-requisite inventions for his collider).

And unlike micro-managers, Julius needed to please humans - all those patrons at the racecourse, and the operators, and provide them visual stimulus. Early totalizator ``user interfaces'' included a mechanical array of giant thermometer-like representations of the money invested in each horse. Meanwhile, operators were connected to the machine via chains and pulleys.

Yeah, he had it hard. We do not.

Compared to building quantum computers or fully self-driving cars, the task of building a worldwide prediction network is ridiculously easy. It is so easy I shall do it on my own if it turns out that the world's technology talent is forever devoted to burning the Earth. I shall do it even if the world's visionary capitalists are obsessed only with finding uses for append-only ledgers. See if I don't.

4. Ideas and mathematics outlast technology prescriptions

As an author, I feel it is my prerogative to choose the level of abstraction at which I convey an idea. There is no right or wrong one. I choose to focus on the mathematical aspects because that kind of guidance is not likely to change as quickly. See this discussion if you feel there is not enough detail there either. But on the matter of technology:

It's a very good thing that in the coming years, the prediction web can surf the MLOps wave. It's not so great for your author just at the moment, because its incredibly difficult to provide reasonable advice that will stand the test of time.

And thus I had a primary goal in a rather brief section on implementation:

The remarks I make here take a minimalist slant, intended to be somewhat orthogonal to existing implementations. I do this not because that is necessarily the best approach, but because it provides a fast way to arrive at the conclusion I would like you to reach, namely that the task is not too hard.

It just isn't that hard.

5. There is no perfect micromanager

Let us proceed to another important aspect of my response. Now in truth, I cannot really set aside the fact that the existing oracle is already useful because I'm too hyper-rational to ignore the this fact and its logical implications for our discussion. But I will pretend that I can set this aside, or that this doesn't exist, and we shall continue the discussion anyway.

I will politely point out that the reasons for not being overly normative in the prescription of other micromanagers is already discussed in the book.

What should the micro-manager look like?

The unfortunate answer is I don't know the ``right way'', and neither does anyone else. There was a time when I hoped a universal, canonical middle-manager would spring to mind and with it, a blueprint for a prediction web.

Continuing...

Since cost-aware prediction is only a sub-set of the micro-managers' job, finding the ``master micro-manager'' feels a lot like devising a master algorithm (though paradoxically, no harder since presumably the master algorithm could be applied simultaneously to all tasks faced by the micro-manager). This isn't to discount the possibility of increasingly elegant and general approaches to the task.

Fortunately it is not necessary and middle-people don't need to be perfect, only good, as they play their role in the larger economy. Given our relatively modest goal, a micro-economy itself aspires to be the master algorithm - albeit one that only works on the domain of repeated tasks where automated assessment is possible, and can be carried out quickly.

Consequently, I emphasize what I called the vanishing overhead principle, and the importance of recursion and supply chains.

Is there a flaw in that logic? I'd like to know what it is, and why you believe I am on the hook for providing you with a prescription of the perfect router (both statistical and technological implementation, one presumes) when I think that is both impossible and unnecessary. Let us return to Chapter 3:

To state that it is essential to have depth (which is to say a contest among different contest-like mechanisms as opposed to a single contest) is merely to assert that there is no obvious best way to mechanically exploit a collection of rent-seeking prediction models.

For instance, it may seem reasonable to run a simple beauty contest between models (recent accuracy), and then select the best model, and give it a reward, and then use that model's predictions for the next batch of forecasts.

Unfortunately, as is well appreciated in a number of different areas, it is more often the case that some weighted average of prediction models outperforms the best one. This goes by many names, such as mixture modeling, boosting, hierarchical modeling, stacking and so on. We see it in the gating and pooling inside neural networks.

A single model might also be outperformed by a meta-model where weights depend on the forecast question and exogenous data (and not just the accuracy of algorithms whose outputs are being combined). There is a danger lurking here because data is being used twice - but it can be effective nonetheless.

6. Don't say something is unclear until you try to use it

It is possible to write five lines of Python to send your models residuals instructions.

For markets to work efficiently with almost no trade friction, participation by humans and inaminate autonomous algorithms should be seamless. If you are willing to cut and paste one line on bash command you’ll be on the way to being convinced of that fact, since this will initiate your participation in a stripped-down improvement of an options market (no email or ISDA required). See tldr.

Do this

It will not only help you understand one microprediction oracle, but create some hooks for you to hang your intuition on. It will improve your board vision. The potential operation of other quite different micromanagers will seem a lot less foreign.

The call to action

I'll say it again, even though I said it in the book

The book is a call to action to help create something that does not yet exist. If you don't wish to help, because you find what I have written too vague, that is of course your prerogative. By all mean haggle with me on this point, but you must also understand that I am bound by the logic expressed in the book and know that this project will need to be advanced by people taking the other side of your stance.